Measuring Resiliency of Systems of Systems Using Chaos Engineering Experiments
Chaos Engineering (CE), which Netflix introduced in 2008, is used by researchers to assess and find weaknesses in system resiliency. Such weaknesses can arise, when subsystems are individually robust, but that robustness disappears when multiple subsystems are paired together in a System of Systems (SoS). CE researchers develop methods and metrics for finding such fragilities. In this paper, we expand previous examinations of CE experimentation for SoS and introduce Security Chaos Engineering (SCE) for SoS. These SCE experiments include terminating message service, flooding multi queues/message, and injecting corrupted Service. SCE assumes compromise by adding a malicious actor to the tests that can induce adversarial failures into a SoS.
For our SoS testbed, we instantiated a virtual Unmanned Aerial Vehicle (VUAV). We use the open-source Chaos Toolkit to run consistent CE and SCE experiments on the VUAV. Chaos Toolkit with SCE exposes the VUAV attack surfaces to evaluate performance and system security. This research allows us to establish an understanding of baseline system performance and gaps in procedures, techniques, and tools from the state of the art as applied to DoD-relevant systems like SoS. We use the load placed on the Central Processing Unit (CPU) and Random-Access Memory (RAM) by the VUAV as metrics for baseline performance. The results showed that these two metrics did not provide enough fidelity in where CE/SCE creates failures. Feeding these results into the CE methodology allows for additional metrics to better pinpoint failures with CE/SCE testing.
- Year: 2022
- Category: Secure & Resilient Systems
- Tag: Cybersecurity, chaos engineering, hardware, UAV, DoD, System of Systems
- Author: Patrick Marchione, Pete Swawrtz, Raed Salih, Michael R. Clark, Robert Denz
- Released: Proceedings Volume 12117, Disruptive Technologies in Information Sciences VI
Featured Riverside Research Author(s)
Thomas Bailey
Mr. Thomas Bailey is a research manager at Riverside Research. He earned his undergraduate degree in Computer Science from the University of Kentucky in 2022, where he also earned the university's certificate in Cybersecurity. He has worked in the areas of language-theoretic security, software security, and vulnerability management and analysis. He has experience leading teams of subject matter experts in addressing highly technical problems and has worked on multiple DoD research efforts.
LinkedIN
Peter Swartz
Michael R. Clark
LinkedIN
Robert Denz
Dr. Robert Denz is Vice President of the Open Innovation Center at Riverside Research. He's an experienced technology executive and strategic leader, leading a research and development business unit focused on critical technology areas such as Cyber Security, Artificial Intelligence & Machine Learning, Radio Frequency, and Optics. These cross-functional teams drive innovation and deliver impactful solutions for national security, leveraging Riverside Research's open innovation model to foster collaboration and accelerate the development of cutting-edge technologies.
Dr. Denz earned a Ph.D. in Computer Engineering from Dartmouth College and a B.S. in Computer Engineering and Computer Science from Rensselaer Polytechnic Institute. His academic background, coupled with his industry experience, provides him with a strong foundation for leading technical teams and driving innovation.
Dr. Denz has co-authored numerous publications in security, virtualization, and operating systems. His contributions to the field of cyber security have been significant, and he continues to mentor others to make an impact on the national security research base.
LinkedIN
The above listed authors are current or former employees of Riverside Research. Authors affiliated with other institutions are listed on the full paper. It is the responsibility of the author to list material disclosures in each paper, where applicable – they are not listed here. This academic papers directory is published in accordance with federal guidance to make public and available academic research funded by the federal government.