## 4 Statistical Results and Analysis
### Case 2 Results and Analysis
In this process, if any group has a large sample size compared to its corresponding group,  
In regards to team performance, two separate 8 (conditions) \(\times\) 2 (iteration) split-plot Analyses of Variance (ANOVA) were conducted to determine whether the team composition (i.e., VBSA-Expert, VBSA-Amateur, VBSA-Native Speaker, VBSA-Non native speaker) differed with respect to their performance and team situation awareness improvement over time (i.e., two-iteration), as depicted in Tables 6, 7, 9, 10 and Figure 10. According to the findings of the first split plot analysis, there was a significant condition main effect (\(p<0.05\)) in which there were no significant interaction effects of condition by iteration (\(p=0.046\) for \(p<0.05\)) and no iteration main effect (\(p=0.46\) for \(p<0.05\)). According to the significant condition main effect, the pairwise comparisons indicate that teams VBSA-Expert and VBSA-Native condition performed significantly better than the other three conditions (\(p<0.05\)). While teams in VBSA-Nursing and VBSA-Medical conditions performed equally (\(p=0.246\)), they performed significantly better than the VBSA-Amateur condition (\(p<0.0001\); see Table 7). Overall, these results show that when a trained human that is natural with language is paired with a VBSA, their overall performance increased. A reason for this outcome can be understood in the context of the major strengths of contemporary NLP models used to develop the VBSA agents. NLP models achieved natural conversational performance across a variety of domains. Situational awareness is measured based on two parameters in this test case, which are number of errors a human teammate made in executing a task and depth of involvement from human teammate, measured using interface time from Figures 11 and 12. A 8 \(\times\) 2 split-plot analysis was conducted and tests similar to team performance (score) were run for team situational awareness. Just as with team score, there was a significant condition main effect (F(2, 30.0159), \(p<0.0001\)), while there were no significant interaction effect of condition by iteration (F(12, 159)= 1.49, \(p=0.132\)) and no iteration main effect (F(4, 40), \(p=0.043\)). According to the significant condition main effect, the VBSA-expert and VBSA-native teams had significantly better performance (\(p<0.0001\)), which makes sense given how team situational awareness is highly related to team score (higher team situational awareness necessarily means that many resources reached events to complete a given task). Four other conditions did not differ on team situation awareness: VBSA-non-native speaker \(p=0.37\), VBSA-Armature \(p=0.69\), VBSA-medical \(p=0.43\); and VBSA-Nursing \(p=0.54\). Overall, these findings make sense given that there is virtually no gap in team situational awareness between VBSA- (nursing, medical, and amateur) teams. The two case studies enable us to draw inferences from several different perspectives. When it goes back to the research questions established in Section 1.2, for research question 1, we can use common metrics identified in [4] to measure performance of an HMT and its team members. For research question 2, in a controlled environment we are able to use VBSA, which is developed in [8] as a teammate to form a successful HMT. Based on results we observed in this section, HIL can be used to study the performance of HMTs, thereby effectively answering research question 3. Based on the data analyzed in this section, we awill propose guidelines to create standardized and well-tested HMTs for practical application. These proposed guidelines still need to tested through rigorous testing and validation before they can be accepted as norms. Figure 11: Task completion time for HMT identified groups.