A framework for authentic assessment in room-scale virtual reality training simulations
Creators: name: Howie, Scott Ronald
Publisher: name: Zenodo
Year: 2024

description: Authentic assessment provides a rigorous method of measuring trainee performance by analysingtheir application of knowledge for solving real-world problems in a practice environment.Virtual Reality (VR) training simulations allow training scenarios to be performed remotely inan immersive life-like environment without any danger or safety concerns for the trainee. However, existing methods of conducting authentic assessment and generating feedback of room-scaleVR scenarios are inadequate as they are: (i) unable to capture all simulation data; (ii) predefinedrecordings, meaning trainee performances can be obscured, and (iii) disconnected in their approachin conducting assessment and generating contextual feedback.This work, grounded in the literature, was developed in consultation with industry trainingassessors to identify the scope, needs, and requirements of a novel authentic assessment frameworkfor room-scale VR simulations. The findings from industry inspired the design of the components,tools, and features of the framework which enable assessors to conduct authentic assessment andgenerate feedback for room-scale VR trainee performances.To evaluate the novel framework an implementation of it was developed. A comparative studyinvolving training assessors from industry and academia compared analytical data and video recording and assessment, along with the implemented framework. Results suggest that the frameworksatisfied the requirements of assessors, showing positive usefulness and long-term usability, indicating that the framework was the most effective and efficient method for conducting assessmentand generating feedback. In comparison, analytical data was found to be unsuitable for authenticassessment due to its lack of context for trainee actions and movement. Whilst, video assessmentdemonstrated satisfactory usefulness and usability results, it was seen as a passive medium, constrained by its predefined method of capturing and presenting data from a single point-of-view,reducing the interactivity and reliability of assessment and occasionally inducing motion sicknessin the assessors.This thesis original contribution to knowledge lies in the design and a validated implementationof a novel framework for authentic assessment in room-scale VR training simulations, and a novelmethod for dual-recording of real time VR training, which is patent pending. The novel framework introduced in this work overcomes existing issues of assessment and feedback in VR by:(i) structuring authentic assessment of VR room-scale simulations; (ii) utilising a dual-recordingmethodology to capture all simulation data with negligible impact on run-time performance, and;(iii) embedding assisted assessment and feedback tools to streamline the evaluation process forassessors.

The originality of this work stems from the design and development of a novel framework,which includes a patent-pending dual-recording methodology. This framework is backed by original contextual design research performed within an industrial training company, which identifiedthe necessary requirements for a framework that would enable assessors to perform authentic assessment in room-scale virtual reality training simulations