## 3 Results
### Model Explainability  
Given the superior performance of the ML models in the classification tasks, we moved a step further toward explaining the overall models' behavior and prediction outcomes via explainable artificial intelligence (XAI). XAI refers to techniques that provide human users with the ability to comprehend and derive rational decisions about the overall performance of an AI model, the intrinsic mechanisms of which are very hard to explain (i.e., black box), as well as on individual predictions. The ultimate outcome of XAI techniques is commonly in the form of graphs due to their easily human-readable format and straightforward interpretation.  
For the sake of simplicity and easy interpretation of the XAI outcome, we focused on explaining the best performing model for each surgical task (LR for Tasks 5, 7, and SVM for Task 6) regarding the binary classification scheme (i.e., Beginning vs. End of training) using only the 'best features' (based on [PERSON]'s score).  
In this study, the SHapley Additive exPlanations (SHAP) was employed [32], which is a model-agnostic XAI method based on the game theoretically optimal Shapley values [33]. The goal of SHAP is to explain the prediction of an instance by computing the contribution of each feature to the prediction. Features with large absolute Shapley values are important. Although the technique provides a Shapley value \(\varphi^{(i)}\) for any instance \(x^{(i)}\), one can obtain the global importance of each feature by averaging the absolute Shapley values, as follows:  
\[I_{j}=\frac{\sum_{n}\left|\varphi_{j}^{(i)}\right|}{n}, \tag{3}\]  
where \(I_{j}\) essentially denotes the average impact on model output magnitude with respect to feature \(j\) (i.e., _feature importance_) and \(n\) is the total number of feature values (i.e., instances) on the dataset evaluated.  
Figure 5a-c shows a bar plot for each task where the best 6-7 features are ranked in descending order according to their impact on model decisions (i.e., global feature importance ranking). The feature names presented in Figure 5 are abbreviated, while full feature names can be found in Table 3. For Task 5, the top 3 features are Economy of movement--grasper, Number of movements of left instrument, and Relevant path length--clipper. For Task 6, the top 2 features are Economy of movement--left instrument and Number of movements of right instrument. For Task 7, the top feature is Trial Number. It is obvious that as trainees gain more experience, the variation in the performance metric values decreases. As a result, fewer features can differentiate between classes and significantly influence the classification process (i.e., Task 7: only one feature has substantial impact).  
Figure 5d-f presents a summary plot for each task, combining both the feature importance and the effects of each feature across the entire dataset. The \(x\)-axis represents the Shapley values of each feature (\(y\)-axis) along with a color bar where each dot is colored based on its value (high or low). A Shapley value less than 0, equal to 0, or greater than 0 corresponds to a negative contribution, no contribution, and positive contribution, respectively. For Task 5, a trial will most likely be classified as End of training when the trainee uses both the clipper and the grasper close to the ideal (shortest) path length (high values for EOM-G and low values for RPL-C) and when the left instrument is only moved when necessary (low values for #Move-L). In Task 6, the same kinematic patterns are followed and therefore a trial is classified as End of training when the trainee moves the left instrument along the shortest distance (high values for EOM-L) and the right instrument is moved only a handful of times (low values for #Move-R). For Task 7, high values for TN  
correspond to the End of training while other features do not present a clear separation between the two training stages.