# A Survey on Immersive Cyber Situational Awareness Systems
## 5 Evaluation Mechanisms for ICSA Systems
\begin{table}
\begin{tabular}{p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**Paper** & **Evaluation Mechanism Mechanism** & **Users’ Demographics** & **Performance Metrics** & **Cognition Metrics** & **Results** \\ \hline [PERSON] et al. [12] & Questionnaire & & User time-on-task; User fact recall & Users’ performance is improved \\ \hline [PERSON] et al. [15] & SART Questionnaire; Analysis of variance with \(p\)-value Test & 6 participants; 3 males and 3 females & No. of completed tasks & Understanding of situational awareness depends on gender \\ \hline [PERSON] et al. [38] & Questionnaire with five-point Likert scale & 91 participants; 59\% male and 41\% female; Age between 18 and 65 & & AR-based game increases cyber situational awareness \\ \hline [PERSON] et al. [14] & Capture the flag exercise; Post-task survey; NASA TLX Caucasian Caucasian & 7 participants; 7 male; Age is between Caucasian Caucasian & Total elapsed time; Average response time; Countermeasure failure rate; Success EOIs; Failed EOIs & Mental demand; Physical demand; Temporal demand; Frustration & AR improves both performance and cognition of users \\ \hline [PERSON] et al. [39] & Questionnaire & 25 participants & Memory test & Interactive immersion in VR is beneficial for long-term memorization of cyber situational awareness \\ \hline [PERSON] et al. [40] & Questionnaire; Interview & 6 participants with no cybersecurity knowledge & Memory test & Cyber situational awareness training through VR is more engaging than video training. \\ \hline [PERSON] et al. [41] & Questionnaire with 5-point Likert scale; Analysis of variance with \(p\)-value Test & 181 participants; 123 male and 58 female & & Immersive game-based learning for cyber situational awareness is more effective for males than females \\ \hline [PERSON] et al. [33] & SUS usability questionnaire; Analysis of variance with \(p\)-value Test; Cybersevictiveness questionnaire & 30 participants with no cybersecurity knowledge & Physiological disorder & Users had good performance with no discomfort \\ \hline [PERSON] et al. [42] & Questionnaire with 5-point Likert scale & 208 participants; Age between 14 and 19 & Knowledge acquisition; Vulnerability detection; Defense preparation & Confidence in technology & AR-based games improve both performance and cognition \\ \hline [PERSON] et al. [48] & Post-task quiz and survey with statistical analysis (t-test) & 100 participants with cybersecurity knowledge & & VR cybersecurity training is more effective than video-based methods \\ \hline [PERSON] et al. [31] & SUS usability questionnaire questionnaire & 8 participants; 3 male and 5 female; Age: 23 to 30 & Task completion time; Task accuracy & Users’ task performance is enhanced in immersive environments \\ \hline [PERSON] et al. [47] & Pre- and post-task survey with 5-point Likert scale and statistical analysis & 20 participants & & AR has a positive impact on cybersecurity learning learning \\ \hline [PERSON] et al. [44] & Pre- and post-task survey & & task completion time; No. of unforced errors & Ver learning environments is beneficial for understanding cyber situational awareness \\ \hline [PERSON] et al. [20] & Pre- and post-task survey & Participants have cybersecurity knowledge & & VR environments are productive for cybersecurity education \\ \hline [PERSON] et al. [61] & Pre- and post-task survey; Satisfaction & & & AR provides insights in cybersecurity forensic education \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Evaluation mechanisms for ICSA systems. A comparative synthesis of evaluation mechanisms used across ICSA systems reveals significant variation in methodology, metrics, and study scope (Table 7). The majority of studies employed questionnaire-based assessments, including Likert scales, usability instruments such as the SUS, and cognitive load measures like NASA-TLX and SART. These were often supplemented by objective performance metrics, such as task completion time, accuracy, response rates, and fact recall.