Standardizing Human Performance Measurement for Ease of Data Analytics
(Room S320F)
28 Nov 17
4:30 PM
-
5:00 PM
As interest grows for big data analytics within the Department of Defense (DoD), one prime opportunity to leverage existing data sources is performance assessment. Specifically, the use of quantitative performance data for determining skill levels of trainees supports diagnostic feedback, targeted remediation, and identification of opportunities to accelerate or tailor training to student learning progress. The successful implementation of automated, system-based performance measures within DoD training systems for assessment and trend analysis purposes, however, necessitates standardization in implementation to ensure success. Based on on-going efforts, the authors propose two areas for consideration: 1) adoption of standards for hardware and software simulation interoperability, and 2) an approach to measurement definition that is flexible to the military’s crawl-walk-run approach to training and conducive to trend analysis. Currently, the simulation community lacks a standardized way to represent human performance data requirements that are generalizable, scalable, interoperable, and transparent. Because of this gap in standards, developers are challenged with finding ways to implement technology in environments that lack the right type of data. The first step toward increased consistency would be an industry standard for interoperability. As such, this paper will outline a proposed human performance measurement standard under consideration by the Simulation Interoperability Standards Organization (SISO). This standard provides a framework for defining how a system can utilize available data to determine if trainees achieve desired outcomes based on the mission context. However, because not all facets of human performance measurement can be defined by a standard, researchers and developers must consider other factors during measure implementation. For example, measures may be presented in the form of raw data to inform instructor formulated assessments (e.g., num