Measuring Team Performance and Coordination in a Mixed Human-Synthetic Team Training Environment
(Room S320E)
29 Nov 17
11:30 AM
-
12:00 PM
The Air Support Operations Center (ASOC) is a complex sociotechnical system. It manages routine events and crises by processing a massive flow of information that originates from a variety of external agencies via numerous communication channels and procedures and across a variety of classification levels. To address the challenge of training individuals and teams to face the complexities of the ASOC environment and to learn how to effectively coordinate, the Air Force has developed the Joint Air-to-Ground Simulation System (JTAGSS). JTAGSS provides ASOC operators with the opportunity to train as a team and to encounter realistic scenarios presented in operational contexts. To achieve this training opportunity, while maintaining current support of operations, work is currently underway to apply a combination of both human and synthetic agents to fill the variety of ASOC positions (Myers, et. al., 2016). Effective training in any environment, and particularly in an environment that requires close coordination across heterogeneous entities, requires effective measurement. To realize the impact of training on the operational environment, trainers must know what to measure, how to measure it, and how to communicate the results. Currently, there are no reliable measures of performance (MOP) and measures of coordination (MOC) that trainers can utilize to ensure progress in teams within the JTAGSS. The purpose of this research was to develop and validate measures of team coordination for mixed human and synthetic teams. The researchers applied the Rational Approach to Developing Systems-based Measures (RADSM) approach (Orvis, DeCostanza, & Duchon, 2013) to develop and validate the coordination measures. The coordination measures were based on virtual communications data and validation was accomplished through a Monte Carlo simulation which utilized representative training data. This study provides evidence of the use of communications-based data in measuring performance r