As Federal agencies increasingly deploy devices at the edge to collect and analyze data, information is flowing back to core infrastructures at everincreasing volumes. The data deluge presents challenges for agencies working to manage data efficiently and make it actionable – challenges and costs that are magnified when the data stream flows from satellites in space.
For data flowing from space, the National Aeronautics and Space Administration (NASA) is already taking notable steps to manage the increasingly complicated data flow, using cloud and artificial intelligence (AI), while ground stations evolve and adapt to meet emerging threats.
Data Flows for Federal Agencies
Fueled by increasing adoption of AI and machine learning, cloud-based services, and analytics, as well as increasing use of Internet of Things (IoT) and mobile devices, data is growing exponentially. Data volume is expected to grow globally at a compound annual rate of 21.2 percent over the next three years, according to IDC.
While this data surge has created vast opportunities for technological innovation and transformative changes in workflows and customer engagement, the sheer volume of data is overwhelming Federal agencies, making it difficult to draw actionable insights from this data ocean.
Agencies are deluged, and not just from structured data, the traditional data primarily seen in the data center. Instead, the increase in data volume is coming from newer and diverse data types, including IoT data, multimedia data, geospatial and location data, sensor data, and unstructured data.
Space Data Presents Special Challenges
One Federal agency with a unique set of data challenges is NASA, which collects vast quantities of data about stars, planets, and other objects, with the volume of that data seeing “ huge increases” in recent years. Satellites send data back to ground stations, where engineers and analysts turn the raw information into measurements and intelligence that are understandable and actionable.
But for more recent missions like NASA’s SWOT satellite, however, downloading these masses of data is difficult. Other challenges include ensuring the data’s accuracy, storing it, recalling it, and keeping it secure. In response, NASA is turning to the cloud, “ setting up systems in the cloud capable of processing, storing, and analyzing all of that digital information.” New AI algorithms are also helping to streamline the processing of space data, while ground stations are adapting by growing more modular, flexible, and resistant to cyberattacks.
The Data Challenge
The challenges created by this data growth, in the context of siloed repositories and outdated legacy systems, are significant. Agencies must navigate the deluge while ensuring data integrity, lineage, security, and compliance.
As data volume continues to increase, and data sources continue to explode, agencies can take a series of steps to adjust their data strategies. It is essential to establish a robust data foundation that spans all system types – block, file, and object. This will effectively decouple data infrastructure from individual data types, addressing the complexity of the data landscape.
Specifically, as edge computing advances, agencies will need to incorporate advanced data management and tiering strategies. Those include efficient data compression, smart data aggregation, and intelligent filtering techniques to ensure that only relevant data is transmitted to central servers for further processing.
Agencies also must focus on efficient management of data storage, which is crucial for seamless data integration and analytics. Properly implemented storage solutions and data fabrics can optimize decision-making, ensuring compliance operational, security, and legal policies.
The Data Foundation Solution
Agencies must move towards modern storage environments that simplify the movement and access of data wherever it is required. With data dispersed across multiple, disparate storage architectures, each with different experiences and considerations, creates storage challenges that surpass traditional issues of cost, performance, and protection.
The right data infrastructure will equip agencies to simplify the complex landscape of data storage, enabling the management of structured and unstructured data across distributed hybrid environments. This allows them to seamlessly operate and optimize applications, whether hosted onpremises, in the cloud, hybrid, or at the edge.
Whether they operate in outer space or fully on land, transformation and modernization will be key for agencies to continue utilizing the growing streams of data to support agency missions with actionable insights based on quality and timely data.
This article is included in the Space Force Association Magazine, Issue 12, December 2024.