# Development of Low-Fidelity Virtual Replicas of Products for Usability Testing
## 4 Materials and Methods
### Simulation Development
#### 4.4.2 Data Collection and Usability Assessment  
Collecting information on user interactions with the AED is critical for usability analysis and is all recorded by one script. The data collected provide insight into possible areas of confusion that will need to be redesigned. A script was written to record user clicks and written to a CSV file. Logged interactions include: the \"current step\" that the subject is currently on in the simulation; the time at which the click was made relative to the beginning of the simulation; which object was clicked; whether a shockpad was selected at the time of the click; and the x and y coordinates of the subject's mouse. The shock pads were also monitored to identify when they were moved and to the hitbox identifier.  
In addition to interaction data, a survey collected subjective data to understand the user's perception of the simulated environment. A custom usability survey was created in Qualtrics to collect post-simulation data. Commonly used usability surveys such as the System Usability Scale [92] were not used to assess this pilot due to lack of specificity. The creation of a custom survey allowed for the collection of data that were relevant to the pilot study. Subjects were presented with 12 open-ended questions (Table 3). This survey asked subjects about aspects of the simulation that impacted performance: the effect of the CPR feature, the effect of the timer, and the assessment of simulation clarity. This feedback will be used to make future improvements to the simulation.  
\begin{table}
\begin{tabular}{p{227.6 pt}} \hline \hline \multicolumn{1}{c}{**Post-Simulation Questions**} \\ \hline
1. Were you able to set and change the CPR rate? If not, what parts of the simulation caused issue and what was the issue? \\ \hline
2. Did you understand the CPR rate feedback provided by the flashing lights? If not, what steps need to be clarified? \\ \hline
3. How did the timer make you feel? \\ \hline
4. Was the timer easy to read? If not, what aspects would you change to make it easier to read? \\ \hline
5. Was it clear what was expected of you at each stage of interaction? If not, what could be added to help users gain more clarity? \\ \hline
6. Was the information on the screen easy to read and legible? If not, what changes would help make the information easier to read? \\ \hline
7. Were you able to understand the voice in the simulation? Was the volume loud enough and the words clear? \\ \hline
8. Did the objects in the simulation respond how you expected (buttons responded immediately when clicked, objects lit up, etc.)? If not, what parts of the simulation caused issue and what was the issue? \\ \hline
9. Were you able to place the shock pads where you wanted them? If not, what issues did you encounter placing the shock pads? \\ \hline
10. Were you able to easily transition between Body and AED view in the simulation, if not what issues did you encounter? \\ \hline
11. Did anything unexpected occur in the simulation? \\ \hline
12. Was the real-world AED use realistic within the simulation environments? If not, what aspects would you have changed to make it more realistic? \\ \hline \hline \end{tabular}
\end{table}
Table 3: List of post-simulation survey questions.