# A Comparative Study of Design Evaluation with Virtual Prototypes Versus a Physical Product
## 3 Results and Discussion
### Providing Feedback  
The results from the evaluation of usability revealed that the visual virtual prototypes invoked more negative and passive emotional responses, particularly with regard to the operations related to the cleaning and dirt-offloading tasks. It was advantageous to investigate whether adding an instant sensory feedback to these tasks could enhance the user experience. Thus, we incorporated auditory, visual, and combined feedback during the operations. The same usability tests (Section 3.3) were conducted again. The objective was to determine whether various types of feedback could exert a different influence on the emotional responses and task performance. Not all feedback had a significant influence on the emotional responses or task performance with regard to the cleaning task. Table 7 indicates a significantly different arousal state for the three feedback groups with regard to their dirt-offloading task responses. The combined feedback caused the highest level of arousal. Subsequently, we gathered the data obtained by the first experiment for the same two tasks and analyzed the responses of the three feedback groups by using hypothesis testing. To facilitate the analysis, we encoded the groups with the following values: Physical group: \(+1\); visual virtual group: \(-1\); auditory feedback: \(-2\); visual feedback: \(-3\); combined feedback: \(-4\). When conducting the following multiple comparisons, we used the combined group (\(-4\)) as the base reference. * Cleaning Accuracy: There was no significant difference in the performance of the five groups. Emotional responses (valence): According to Table 8a, there was a significant difference among the five groups with regard to the valence construct. From the multiple comparisons listed in Table 8(b), we could observe that differences existed between the combined feedback group and the visual virtual group without any feedback (\(-1\)). The addition of multiple sensory feedback improved the valence construct when the two tasks were carried out by using the virtual prototype. The combined group had the most positive emotional response. Emotional responses (arousal): Table 9a indicates that there was a significant difference in the arousal construct among the groups. Table 9b shows that differences existed between the reference group and the auditory feedback group. The physical group had the highest level of arousal, while the combined feedback group had the second highest level of arousal, which was still higher than that of the visual virtual group. This indicates that the addition of combined feedback increased the arousal state to a level closer to that of the physical group. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & & **SS** & **DF** & **MS** & **F** & **P** \\ \hline \multirow{3}{*}{Correct-hit} & Between Groups & 0.067 & 1 & 0.067 & 0.051 & 0.821 \\  & Within Groups & 75.267 & 58 & 1.298 & & \\  & Total & 75.333 & 59 & & & \\ \hline \multirow{3}{*}{Valence} & Between Groups & 5.400 & 1 & 5.400 & 2.692 & 0.106 \\  & Within Groups & 116.333 & 58 & 2.006 & & \\  & Total & 121.733 & 59 & & & \\ \hline \multirow{3}{*}{Arousal} & Between Groups & 14.017 & 1 & 14.017 & 5.140 & 0.027 \\  & Within Groups & 158.167 & 58 & 2.727 & & \\  & Total & 172.183 & 59 & & & \\ \hline \multirow{3}{*}{Dominance} & Between Groups & 1.067 & 1 & 1.067 & 0.250 & 0.619 \\  & Within Groups & 247.667 & 58 & 4.270 & & \\ \cline{1-1}  & Total & 248.733 & 59 & & & \\ \hline \hline \end{tabular} SS: Sum of squares; DF: Degrees of freedom; MS: Mean square.  
\end{table} 
Table 7: ANOVA for the dirt-offloading task with different feedbacks. Emotional responses (dominance): There was no significant difference in the groups with regard to the dominance construct. \(\bullet\) Dirt-offloading  
Accuracy: In terms of accuracy, there was no significant difference in the performances of the five groups.