You are here

Leveraging Commercial Big Data Solutions for ISR Applications

Operationalizing intelligence for global missions requires a novel approach to current ISR challenges. In collaboration with IBM and Exelis VIS, our innovative research leverages commercial Big Data solutions and cloud-based processing, exploitation, and dissemination (PED) systems to save time and money. This joint effort – facilitated through our independent research and development (IR&D) program – has already demonstrated exponential improvements in processing time, automation efficiencies, and reduced risk and lifecycle costs.

The Problem
Extracting actionable intelligence from massive amounts of data has always been a complex challenge for the ISR community, traditionally met by hiring more analysts. An often pursued alternative is process automation, yet the potential for long term cost savings is hampered by the very high costs of implementation. Another alternative is to collect less data by reducing missions, but once again, this is less than ideal as it puts at risk operational mission success.  None of these methods provide an optimal solution to the ISR Big Data problem.

Our Solution
In the fall of 2012, Riverside Research, IBM, and Exelis VIS began developing an innovative solution that leverages proven, commercial Big Data analytics and cloud-based exploitation services to address the ISR Big Data problem in every dimension – volume, velocity, variety, and veracity.

Not surprisingly, the ISR community and the commercial world share similar Big Data challenges when it comes to producing actionable intelligence. Businesses create Big Data treasure troves to store customer emails, help desk chats, online shopping experiences, in-store browsing and buying behavior, and other customer data with the ultimate goal of improving real-time point of sale metrics. As it is in the ISR community, the cost to automate business analytics is daunting. To address this problem, IBM developed InfoSphere Streams, a commercial Big Data solution designed to optimize software performance over a generic hardware platform as opposed to traditional methods of manually developing parallel applications that only run on the specific hardware for which they were developed. Our joint research effort was founded on the premise that this approach would work for ISR as it has for the commercial world for near real time data processing.

Exelis VIS, developer of the ENVI Services Engine (ESE), joined the partnership as a leading provider of cloud-based ISR analysis solutions. Our initial proof of concept was based on an operational airborne hyperspectral platform, addressing a real-world mission shortfall while demonstrating the value of rapidly applying IBM and Exelis capabilities to meet broader ISR data processing needs.

Initial Results
After assembling the IBM PureFlex hardware, InfoSphere Streams, and Cloud Provisioning Software, our team modified the current operational algorithms to run under ESE, which was then wrapped in InfoSphere Streams for execution. The operations baseline transformed a single data file from its native sensor format into the National Imagery Transmission Format (NITF), performed atmospheric corrections, geo-rectification, and statistical analysis to create an exploitation-ready hyperspectral data cube. In the currently deployed environment, each data cube takes approximately 6 minutes to process. Using the approach outlined above, with a functionally identical process running under InfoSphere Streams, the processing time for each data cube was reduced to 4-6 seconds.

Future Work
The next phase of this IR&D project will build on the successes of the previous work by developing specific analytic solutions to address Big Data challenges of the broader ISR community.

© 2017 Riverside Research