Using Competencies to Map Performance Across Multiple Activities
(Room S320A)
29 Nov 17
5:00 PM
-
5:30 PM
When a single training system accumulates data on learner performance, the data are stored in a way determined by the system’s designers. This enables the system to access these data and to apply them to its interactions with learners. In environments such as live-virtual-constructive federations, each component may store performance data in its own way, making it difficult for one component to access and use data produced by another. To enable cross-component sharing of performance data, it is necessary to establish shared definitions of skills and outcomes; create a common language for expressing performance data; interpret data produced at differing levels of granularity; and (in some cases) satisfy a large array of security and privacy requirements.
This paper is based on work done by the US Advanced Distributed Learning (ADL) Initiative, the Credential Engine foundation, and several standards bodies. It starts by discussing the above challenges and their manifestations in use cases ranging from federations of different learning environments to more traditional online learning environments. The paper then describes a potential solution for collecting and processing assertions of competency, skills, and performance from multiple sources. Each assertion is of the form “Learner X has (or has not) achieved competency Y at level Z with confidence p based on evidence E.” “Competencies” are drawn from shared, machine-readable frameworks that can represent knowledge, skills, ability, and objectives. Assertions can be collected directly or generated by ingesting granular performance data and correlating it to competencies, enabling algorithms that use explicit rules and relationships to draw further inferences.
This paper ends with a description of a system that implements the suggested solution and its application in the context of live trials with 73 subjects run as part of a design-based research effort.