Collaborative Helmet and Weapon Tracking for Augmented Reality based Training
(Room S320E)
30 Nov 17
10:30 AM
-
11:00 AM
Speaker(s):
Pat Garrity, ARL; Frank Dean, ARL STTC; Zhiwei Zhu, SRI International; Rakesh Kumar, SRI International; Taragay Oskiper, SRI International; Supun Samarasekera, SRI International; Glenn Murray, SRI International; Nicholas Vitovitch, SRI International; Kevin Kaighn, SRI International; Andy Coppock, SRI International; Aki Chaudhry, SRI International
There is a need within the military to have increased levels of warfighter proficiency and readiness by providing realistic training scenarios for complex urban combat at forward and home locations. Currently, elaborate infrastructure and supporting actors are needed to create training scenarios, and record and review training sessions. Live ammunition training is limited to Force on Target training with extremely limited scenarios (no movers, same old targets); while laser-based training does allow Force on Force training, it is limited by the scheduling of exercises, range time availability and scenarios possible with live forces.
The key emerging innovative technology that addresses these shortcomings is precision mobile Augmented Reality (AR). The AR system precisely tracks actions, locations, and head and weapon pose of each trainee in detail so the system can appropriately position virtual objects in the trainee’s field of view. Synthetic actors, objects and effects are rendered by a game engine on the eyewear display. Synthetic actors respond in realistic ways to actions of the trainee, e.g., taking cover, firing back, or milling as crowds. The AR-weapon can be used to fire simulated projectiles at real or synthetic entities.
This paper describes improvements made to a prototype AR system based on live testing with warfighters at a Military Operations in Urban Terrain (MOUT). We present a method for joint tracking of the helmet worn and the weapon attached sensors in a collaborative fashion in which the wearable unit on the helmet aids the weapon unit by sharing visual landmarks along with 3D location estimates in the scene. These are stored in a dynamic map on the weapon processor and continuously matched against to obtain weapon poses consistent with the head pose to provide accurate aiming capability. We also present solutions to miniaturize the system using mobile processors and smartphone sensors.