# Multimodal Model for Automated Pain Assessment: Leveraging Video and fNIRS
## 4 Experiment and Results
### Test Accuracy Against Competitors  
The AI4 Pin test labels are not publicly available. To evaluate our model's performance, we submitted our predicted labels to the challenge organizers for testing. The organizers allow a maximum of two submissions for each team. In one of our submissions, our model achieved an accuracy of 51.33%. This score was the highest among all competitors in the challenge, positioning our model at the top in terms of performance. A detailed comparison of our model's results with other methods can be found in Table 3. After careful consideration, we submitted the video + fNIRS model for the final evaluation, as it achieved the highest validation accuracy in our internal tests. However, this accuracy was not significantly higher than that of the model that combined video, fNIRS, and manually engineered features. Although the simpler model performed well, the marginal difference suggests that additional manual features did not enhance its effectiveness in this case. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Methods (Modality)** & **Val Acc w/oAF** & **Val Accuracy** & **Val F1 Score** & **Test Accuracy** \\ \hline Baseline (fNIRS) & - & 43.20\% & - & 43.30\% \\ Baseline (Video) & - & 40.00\% & - & 40.10\% \\ Baseline (Video + fNIRS) & - & 40.20\% & - & 41.70\% \\ \hline Simple ANN + Voting (Video) [28] & - & - & - & 49.00\% \\ LSTM (Video) [28] & - & - & - & 43.00\% \\ Twins-PainViT [29] & - & - & - & 46.76\% \\ \hline MMAPA-Ours (fNIRS)\_B-6 & 59.00\% & 58.67\% & 41.71\% & - \\ MMAPA-Ours (Video)\_B-6 & - & **61.67\%** & 41.83\% & - \\ MMAPA-Ours (Manual)\_B-6 & - & 59.00\% & 39.88\% & - \\ MMAPA-Ours (Video + fNIRS)\_B-6 & 59.33\% & 60.33\% & 39.70\% & - \\ MMAPA-Ours (Video + Manual)\_B-6 & - & 59.33\% & 40.29\% & - \\ MMAPA-Ours (fNIRS + Manual)\_B-6 & 58.00\% & 58.33\% & 38.93\% & - \\ MMAPA-Ours (Video + fNIRS + Manual)\_B-6 & 59.33\% & 61.00\% & 41.42\% & - \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Model comparison against baseline and other methods. Based on the results presented in Table 4, our proposed method demonstrates strong performance, particularly under the 10-fold cross-validation setting. In this protocol, a larger portion of the dataset is used for training, while a smaller portion is reserved for testing in each fold. This setup contrasts with LOOCV, which typically exposes models to higher variance due to the minimal size of the test set. Notably, our multimodal attention-based fusion model (MMAPA - G + V) achieved the highest accuracy in the 10-fold setting, outperforming both unimodal baselines and the established PainAttNet benchmark. This improvement suggests that our model is better suited for scenarios where more training data are available, indicating stronger fitting capacity and better generalization under constrained test conditions. The results also reveal that the video-only modality consistently underperforms compared to the GSR-based model on the BioVid dataset. This observation indicates that physiological signals, particularly GSR, provide more reliable and discriminative features for pain detection in this context. The effectiveness of GSR likely stems from its direct association with autonomic nervous system responses to nociceptive stimuli. Furthermore, \begin{table}
\begin{tabular}{l c} \hline \hline  & (**a**) \\ \hline
**Model** & **LOOCV Accuracy** \\ \hline MLP [30] & 84.22\% \\ XGBoost [31] & 85.23\% \\ PainAttNet [25] & 85.34\% \\
**MMAPA-Ours (GSR)** & **89.22\% \(\pm\) 11.38\%** \\
**MMAPA-Ours (Video)** & 73.33\% \(\pm\) 15.71\% \\
**MMAPA-Ours (G + V)** & 87.22\% \(\pm\) 12.18\% \\ \hline  & (**b**) \\ \hline
**Model** & **10-Fold Accuracy** \\ \hline PainAttNet [25] & 84.09\% \(\pm\) 7.51\% \\
**MMAPA-Ours (GSR)** & 84.68\% \(\pm\) 5.38\% \\
**MMAPA-Ours (Video)** & 67.86\% \(\pm\) 5.29\% \\
**MMAPA-Ours (G + V)** & **85.75\% \(\pm\) 5.21\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of model performance with PainAttNet on the BioVid dataset using two evaluation protocols.