# A Serious Mixed-Reality Game for Training Police Officers in Tagging Crime Scenes
## 4 Evaluation
### Evaluation Methods  
The testing and evaluation of the proposed system were carried out by using methods to assess (1) the ability of TraceGame to enhance novices' forensic capabilities, (2) the engagement of users in the experience provided by TraceGame, and (3) the usability of TraceGame. Indeed, it is essential that TraceGame is effective as a training tool, but also that the experience with TraceGame does not turn out to be frustrating, which would lead to the abandonment of the proposed training tool.  
For the first point, the ability of TraceGame to enhance novices' forensic capabilities was evaluated by analyzing the game scores achieved by the volunteers in the experimental session in two successive game sessions. Indeed, if higher scores were obtained in the second game session with respect to the first one, it would be an obvious demonstration of the ability of TraceGame to improve novices' forensic capabilities.  
To assess the engagement of users in the experience provided by TraceGame, similarly to the experimental study in [11], a questionnaire with the following open questions was designed:  
* What adjectives would you use to describe the TraceGame experience?
* What is your physical condition after using TraceGame? For example, do you have eye strain?
* What did you like the most?
* What would you change?  
In general, this questionnaire focuses on the device's acceptability and on the general feeling towards the TraceGame experience.  
Finally, for usability purposes, the well-known System Usability Scale (SUS) [30] was used. This method was chosen because it is a highly robust and versatile tool for usability professionals, as shown in the empirical evaluation reported in [31]. Moreover, it has been used in several studies and applied in the evaluation of several projects related to new interaction environments, such as extended-reality systems [32; 33]. Briefly, the SUS is a ten-item scale giving a global view of subjective assessments of usability (see the SUS survey in Appendix A, Table A1). The instrument uses a 5-point Likert scale rangingfrom 1 to 5, where 1 indicates that the respondent strongly disagrees with the statement and 5 indicates that they strongly agree. The SUS is used after a respondent has had an opportunity to use the system being evaluated. If an item has no answer, the center point of the scale should be marked. By analyzing respondents' answers, it is possible to compute an overall SUS score by weighting the single responses. In more detail, the procedure first sums the score contributions from each item. Each item's score contribution ranges from 0 to 4 in the following way: For items 1, 3, 5, 7, and 9, the score contribution is the scale position minus 1, whereas for items 2, 4, 6, 8, and 10, the contribution is 5 minus the scale position. Then, the procedure multiplies the sum of the scores by 2.5 to obtain the overall value of the SUS. SUS scores range from 0 to 100. According to the study in [31], within the interpretation of SUS scores for products, SUS scores less than 50 should be cause for significant concern and should be judged to be unacceptable; SUS scores between 50 and 70 are marginally acceptable and should be improved; SUS scores above 70 are adequate [32].