Virtual Reality Framework for Design and Evaluation of Multispectral Computer Vision Algorithms and Augmented Reality Interfaces for Enhancing Situational Awareness in Degraded Visual Environments
Advanced fire control technologies that utilize computer vision-guided target recognition will enable dismounted soldiers with augmented reality displays, such as the integrated visual augmentation system, enhanced situational awareness. Here we describe a virtual reality framework and environment for the design and evaluation of computer vision algorithms and augmented reality interfaces intended to enhance dismounted soldier situational awareness.
For training models, synthetic image datasets of targets in virtual environments can be generated in tandem with neural network learning. To evaluate models under simulated operational environments, a dismounted soldier combat scenario was developed. Trained models are used to process input from a “virtual camera” in-line with a rifle-mounted telescopic sight. Augmented reality overlays are projected over the sight’s optics, modeling the function of current state-of-the-art holographic displays. To assess the impact of these capabilities on situational awareness, performance metrics and physiological monitoring were integrated into the system.
To investigate how sensors beyond visible wavelength optical imaging may be leveraged to enhance this capability, particularly in degraded visual environments, the virtual camera framework was extended to introduce methods for simulating multispectral infrared imaging. Thus, this virtual reality framework provides a platform for evaluating multispectral computer vision algorithms under simulated operational conditions, as well as iteratively refining the design of augmented reality displays. Improving the design of these components in virtual reality provides a rapid and cost-effective method for refining specifications and capabilities toward a field-deployable system.
- Year: 2024
- Category: Artificial Intelligence
- Tag: virtual reality, computer vision technology, machine vision, computer simulations, augmented reality, detection and tracking algorithms, artificial intelligence
- Author: Ross L. Bobb, Jeffry A. Coady, Virgil O. Barnard, Matthew A. Mueller, William D. Casebeer, Joseph P. Salisbury
- Released: SPIE Defense + Commercial Sensing 2022 - Virtual, Augmented, and Mixed Reality (XR) Technology for Multi-Domain Operations III
Featured Riverside Research Author(s)
Ross L. Bobb
Mr. Ross Bobb is a research scientist for Riverside Research in the Artificial Intelligence and Advanced Compute (AIAC) department. His main focus is on developing simulations to help with human factors research and synthetic data creation.
LinkedIN
Virgil O. Barnard
Virgil O. Barnard is a Senior Machine Learning Scientist in the Artificial Intelligence and Machine Learning group at Riverside Research. He received his bachelor’s in mathematics from University of Kentucky in 2012. He received his Ph.D. (ABD) in computer science from University of Kentucky in 2019. He has performed as a technical AI lead on many contracts for customers in the DoD & IC while at Riverside Research since 2019 spanning image, electro-optic, signals, and radar-based modalities. He is currently researching agent-based reasoning over knowledge graphs.
LinkedIN
Joseph P. Salisbury
Dr. Joseph Salisbury is a neuroscientist (Ph.D., Brandeis University, 2013) and software developer whose current research focuses on human-computer interaction, human-robot interaction, and applications of large language models.
LinkedIN
The above listed authors are current or former employees of Riverside Research. Authors affiliated with other institutions are listed on the full paper. It is the responsibility of the author to list material disclosures in each paper, where applicable – they are not listed here. This academic papers directory is published in accordance with federal guidance to make public and available academic research funded by the federal government.