## 5 Results and Discussion
### Comparative Model Performance Across Scenarios  
The performance of all models is evaluated across four scenarios designed to test their accuracy and robustness. Table 4 summarizes the key performance metrics (Accuracy and Weighted F1-Score) for each model across these scenarios.  
\begin{table}
\begin{tabular}{c c c c} \hline
**Scenario** & **Model** & **Accuracy** & **Weighted F1-Score** \\ \hline
**Baseline** & **MHMM (Expert System)** & **0.927** & **0.925** \\ \hline
**MHMM (Trained)** & **0.925** & **0.924** \\ \hline
**Hybrid HMM-LSTM** & **0.917** & **0.913** \\ \hline LSTM & 0.900 & 0.898 \\ \hline CRF & 0.885 & 0.884 \\ \hline Transformer & 0.793 & 0.793 \\ \hline Unimodal HMM (TCT) & 0.639 & 0.612 \\ \hline Unimodal HMM (TLX) & 0.635 & 0.627 \\ \hline Unimodal HMM (HRV) & 0.610 & 0.603 \\ \hline \end{tabular}
\end{table}
Table 4: Summary of model performance across evaluation scenarios.  
Figure 4: Distribution of multimodal data by proficiency state. Each violin plot displays the data distribution, with dashed lines indicating the median (central line), 25 th percentile (bottom line), and 75 th percentile (top line) of the data.  
Several key findings emerge from these results:
* Superiority of MHMM: In all four scenarios, the MHMM (Expert System) and MHMM (Trained) variants consistently outperformed all other models in both accuracy and F1-scores. The F1-score measures how well a model balances precision (how many predicted labels are correct) and recall (how many actual cases it finds). It is calculated as: \[\text{F1-score}=2\cdot\frac{Precision\cdot Recall}{Precision+Recall}\] This makes it useful for evaluating performance when some classes are less frequent than others. This demonstrates the inherent strength of the HMM framework for this task, especially when it can leverage either expert knowledge or learn directly from multimodal data.
* Clear Advantage of Multimodal Fusion: Unimodal HMMs performed significantly worse than all multimodal approaches, with accuracies hovering between 61% and 65% in the baseline scenario. This provides strong evidence for the central hypothesis of this study: Fusing physiological, behavioral, and subjective data streams provides a far more robust and reliable assessment of proficiency than any single indicator alone.  
Robustness to Noise and Missing Data: The MHMM variants showed remarkable resilience in the stress test scenarios. In the Missing Data scenario, the MHMM's performance barely degraded, highlighting the advantage of its native ability to handle missing observations without relying on imputation. In contrast, CRF's performance dropped from 0.885 to 0.801 accuracy, indicating its vulnerability to incomplete data.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Scenario** & **Model** & **Accuracy** & **Weighted F1-Score** \\ \hline
**Stress Test (15\% Noise)** & **MHMM (Expert System)** & **0.889** & **0.888** \\ \hline
**LSTM** & **0.887** & **0.880** \\ \hline
**MHMM (Trained)** & **0.885** & **0.884** \\ \hline Hybrid HMM-LSTM & 0.881 & 0.881 \\ \hline CRF & 0.856 & 0.852 \\ \hline
**Transformer** & 0.779 & 0.779 \\ \hline
**Missing Data (15\%)** & **MHMM (Expert System)** & **0.923** & **0.922** \\ \hline
**MHMM (Trained)** & **0.921** & **0.920** \\ \hline
**LSTM** & **0.921** & **0.898** \\ \hline Transformer & 0.805 & 0.805 \\ \hline CRF & 0.801 & 0.796 \\ \hline Hybrid HMM-LSTM & 0.422 & 0.422 \\ \hline
**Imbalanced States** & **MHMM (Expert System)** & **0.934** & **0.933** \\ \hline
**MHMM (Trained)** & **0.933** & **0.932** \\ \hline
**Hybrid HMM-LSTM** & **0.919** & **0.919** \\ \hline LSTM & 0.934 & 0.909 \\ \hline CRF & 0.897 & 0.897 \\ \hline Transformer & 0.824 & 0.824 \\ \hline \hline \end{tabular}
\end{table}
Table 4: _Cont._