# Remote Virtual Simulation for Incident Commanders--Cognitive Aspects
## 5 Enablers for RVS Examination
### The Pilot Test, before Deciding upon Remote Virtual Simulation Examination  
In the few reported cases of using VS for assessment [1, 67], the sessions have been held on-site, not remotely, which was a pressing need. However, the learning objectives, reflecting the necessary competencies for safe and effective incident command are the same. In LS, students and instructors are in the same physical space. The instructors/assessors can watch the student move, observe (see and hear) when he/she talks, and then observe the actions of the persons the student talked to and to observe the actions taken, e.g., if the leadership and the communication is satisfactory in relation to the assessment. Testing _how_ the instructors/assessors would manage this, to reliably assess the students' performance in RVS, needed to be developed. This motivated the Pilot test, to explore the feasibility as well as test modes for transmitting to the instructors/assessors the necessary information to reliably assess the students. Five scenarios were designed and built for RVS examination, see Table 2 for a brief overview. The technical setup, where the assessor could see the student's face at all time, hear everything said, and see what the student was looking at in the virtual environment, was developed, to provide the necessary assessment conditions as in LS (where the instructor can see the student at all time), see Figure 6. The audio and radio solution were setup using mobile phones and the standard digital communication tool in Sweden, RAKEL (RAdioKommunikation for Effektiv Ledning). The instructors could act as any of the persons involved in the incidents, e.g., another firefighter or a bystander, by choosing a corresponding avatar. The objectives of the Pilot test were to check the technology setup, the required bandwidth, the ease-of-use of technology mainly at the student site (which could be any fire station or the student's home), and to validate the scenarios and the assessment conditions. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Questions** & **\% Answers** & **Average** & **Standard Deviation** \\  & **Likert 4 or 5** & **(Scale 1–5)** & **Standard Deviation** \\ \hline Experienced presence, compared to & 72\% & 3.90 & 0.83 \\ previous very high? & & & \\ Experienced presence in the & 59\% & 3.63 & 0.86 \\ simulated environment? & & & \\ Experienced being in the same env. as & & & \\ the “persons” you met? & 68\% & 3.81 & 0.99 \\ How easy was it to understand the & & & \\ training objectives? & & & \\ Would you like to perform similar & & & \\ training at your fire station? & & & \\ Would you like to perform similar & & & \\ training in your spare time? & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Students’ response (\(n=90\)) to VS training on-site, excerpt from [66]. Answers in Likert scale, 1 (low) to 5 (very high). Before the test, several of the eight expert ICs described their moderate expectations towards RVS, including concern of technical problems and difficulty in believing that RVS could be a satisfactory replacement for LS. The objectives of the pilot test were to check the technology setup, the required bandwidth, the ease-of-use of technology mainly at the student site (which could be any fire station or the student's home). They also had to give their comments on the scenarios, as well as the instructors' and assessors' role for running the scenarios. The instructors and assessors performed the counterplay and assessed these \"expert-students\" remotely. Valuable opinions regarding the setup as to what the assessor must see and hear to provide evidence-based assessment were expressed. \begin{table}
\begin{tabular}{c c c} \hline \hline
The evaluation of the pilot test showed a positive turn in attitude towards RVS. All the participants agreed that the scenarios were designed to match the learning goals IC-1s need to achieve at a suitable level and corresponded to previous IC-1 LS examinations. They also agreed on the ease of use of technology at the student site.