## 2 Materials and Methods
### Experiments  
Based on the performance metrics provided by the VR simulator, various datasets were derived to address the goals of the three experiments. Table 2 provides a statistical overview of the training trials performed by the participants. Firstly, we developed a binary classification model to differentiate the trials that are close to the Beginning (\(B\)) or the End (\(E\)) of the training progress. Hence, this dataset included 3 trials randomly selected from the Beginning and another 3 trials from the End of training of each participant (N\({}_{1}\) = 138 trials in total). The borderline between the two classes was different for each subject and it was defined by the median ('middle') training attempt, splitting the student's trials into two equal subsets.  
The second experiment aimed to classify student trials into three classes: the Beginning (\(B\)), Middle (\(M\)), and End (\(E\)) of training. For this purpose, the training trials of each subject were first split into three equal parts. Then, each of the three classes included 3 random trials from each part, so the total number of trials for this dataset was N\({}_{2}\) = 207.  
For the two classification experiments, the datasets were balanced across classes, and they are representative of the individualized training phases (classes) of a trainee on a particular surgical task. Moreover, in the context of surgical training if, for example, some student wants to start training on a particular task, the prior probability of being classified into one of the two/three training classes is essentially equal. In this context, our ML framework will be able to provide feedback to the trainee about his/her expertise level via the inferred training phase (class).  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Task** & **Metric** & **Value** \\ \hline \multirow{2}{*}{5} & Total number of movements & \(\leq\)70 \\  & Number of clipped ducts & \(\geq\)9 \\  & Number of lost clips & \(<\)1 \\ \hline \multirow{2}{*}{6} & Total number of movements & \(\leq\)50 \\  & Number of lost balls which miss the basket & \(<\)1 \\ \hline \multirow{2}{*}{7} & Total Number of cutting maneuvers & \(\leq\)20 \\ \end{tabular}
\end{table}
Table 1: Thresholds for performance metrics that determine the _successful completion_ of a training trial.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Metric (\# Trials)** & **Task 5** & **Task 6** & **Task 7** \\ \hline Minimum & 9 & 14 & 6 \\ Maximum & 52 & 57 & 34 \\ Median & 23 & 23 & 16 \\ Mean \(\pm\) SD & 25 \(\pm\) 11 & 26 \(\pm\) 10 & 17 \(\pm\) 7 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Ground-truth data statistics. Number (#) of trials performed by the subjects for each surgical task (SD: standard deviation).  
The third experiment focused on online regression to highlight the assessment process from a different perspective. In particular, the goal here was to provide a prediction for the remaining number of trials (RNT) required by a subject to _successfully_ complete a training task. The prediction was generated throughout the entire training process, starting from the very first trial of each subject. For this experiment, we utilized metrics from all available trials separately for each task (N\({}_{\text{Task5}}\) = 437, N\({}_{\text{Task6}}\) = 503, N\({}_{\text{Task7}}\) = 313).