## 2 Materials and Methods
### Prediction Models Workflow
#### 2.2.4 Rule-Based Prediction Model Design  
We gathered the inputs and feedback from the behavioral analyst when developing the coding scheme in Section 2.2.3 into a set of rules for each collaborative behavior based on the binary features. The rules were constructed to closely replicate the role of human annotators. The seven binary features that were discussed in Section 2.2.1 were used to drive the categorization of the collaborative behaviors defined in Table 3. In the rule-based model, we begin by checking the presence of speech. Since speech data contributes between  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \# & **Collaborative Behavior** & **Definition** & **Condition** \\ \hline \multirow{4}{*}{1} & \multirow{4}{*}{Engaged} & The participant is focused on the task, communicating, and progressing well. & Participant could be talking to their partner. Participant is using the controller and virtual object is moving closer to the target. \\  & & & _Engaged_ = _Spech Presence_\(\cup\) _(Controller Manipulation_\(\cap\) _Object Move Closer_)_ \\ \hline \multirow{4}{*}{2} & \multirow{4}{*}{Struggling} & The participant is not progressing with the task due to difficulty performing the task, not communicating with their partner, distracted, or disinterested with the task. & Participant is not talking to their partner while: i. manipulating the controller but virtual object moving away from the target, or ii. not manipulating the controller and not looking at the screen (virtual objects, focused area). \\  & & & _Straggling_ = _“Spech Presence_\(\cap\) _((Controller Manipulation_\(\cap\) _Object Move Away_\(\cup\) _(“Controller Manipulation_\(\cap\) _“Gaze))_ \\ \hline \multirow{4}{*}{3} & \multirow{4}{*}{Waiting} & The participant is on standby for their partner in a turn-taking task, not moving. & Participant is not talking to their partner, not using the controller, and not moving virtual objects, but is looking at an object or focus area. \\  & & & _Waiting_ = _“Spech Presence_\(\cap\) _“Controller Manipulation_\(\cap\) _“Object Move Away_\(\cap\) _“Object Move Closer_\(\cap\) _Gaze_ \\ \hline \hline \end{tabular}
\end{table}
Table 3: Definition of Participant’s Collaborative Behavior.  
20-30% of the entire collaborative interaction, in this initial design, we assumed that any utterances while performing the task as an indication of engagement. Further analysis of the speech in future work would allow us to categorize the behavior more accurately (i.e., positive utterances as _Engaged_, and negative utterances as _Struggling_). If speech was detected, the rule would assign the collaborative behavior as being _Engaged_. If speech was not detected, the second rule was to check for keypresses, which were based on controller manipulation features. If controller manipulation was present, it would set the keypresses as true, and move to the next rule to check the object distance based on the object move closer and object move away features. If the object move away feature was true, then it means the object was moving away from the target, and the rule would assign the collaborative behavior as _Struggling_, If the object move closer feature was true, then it means the object is moving closer to the target, the rule would assign the collaborative behavior as _Engaged_, and if neither the object move closer and object move away are true, then it means that the object was not moving, so the rule would assign the collaborative behavior as _Waiting_. In the case when no speech was present and no keypresses were present, the rule would check for eye gaze presence. If eye gaze was present, participant collaborative behavior was assigned as _Waiting_. However, if eye gaze was absent, the collaborative behavior was assigned as _Struggling_. We then consolidated these rules into a rule-based model as shown by the flow chart in Figure 6.