# Multimodal Model for Automated Pain Assessment: Leveraging Video and fNIRS
###### Abstract  
Pain assessment is a challenging task for clinicians due to its subjective nature, particularly in individuals with communication difficulties, cognitive impairments, or severe disabilities. Traditional methods such as the Visual Analogue Scale (VAS), Numerical Rating Scale (NRS), and Verbal Rating Scale (VRS) rely heavily on patient feedback, which can be inconsistent and subjective. To address these limitations, developing objective and reliable pain assessment tools that incorporate advanced technologies, such as multimodal data integration from video and fNIRS, is important for improving clinical outcomes. However, challenges such as noise susceptibility in fNIRS signals must be carefully addressed to realize their full potential. Recent studies have explored automatic pain assessment using machine learning and deep learning techniques, which require high-quality data that can accurately represent pain categories. In response to the introduction of a new dataset in the AI4 Pain Challenge, we proposed a multimodal neural network model utilizing attention-based fusion to improve overall accuracy (MMAPA). Our model leverages video and fNIRS modalities as well as manually extracted statistical features. We also implemented fNIRS signal preprocessing and artifact noise filtering, which significantly improved performance on both the fNIRS and statistical feature branches. On the hidden test set, our model achieved an accuracy of 51.33%, outperforming the official baseline of 43.33%. To evaluate generalizability, we further tested our method on the BioVid Heat Pain Database, where our fusion model achieved the highest accuracy in the 10-fold cross-validation setting, outperforming PainAttNet and unimodal variants. These results highlight the effectiveness of our multimodal attention-based approach in improving pain classification performance across datasets. attention-based fusion; brain-computer interface; deep learning; fNIRS; multimodal model; pain assessment 202515 accurate pain evaluation. Developing objective and reliable pain assessment tools that incorporate advanced technologies, such as multimodal data integration from video and biosignals (e.g., functional near-infrared spectroscopy (fNIRS)), is important for improving clinical outcomes and patient care. While biosignals offer valuable insights into brain and physiological responses associated with pain [2; 3], they can also be susceptible to noise and artifacts [4]. Therefore, effective preprocessing and multimodal fusion strategies are essential to fully leverage biosignal information in automated pain assessment. [PERSON] et al. attempted to scale the level of pain based on facial expression [5]. The same study also investigated methods for assessing pain in individuals with communication difficulties. The Visual Analogue Scale (VAS)/Graphic Rating Scale, Numerical Rating Scale (NRS), and Verbal Rating Scale (VRS) are examples of instruments for pain intensity measurement based on patient feedback. In the VAS assessment, a graphical line represents \"no pain at all\" to \"pain as bad as it could be\". In NRS assessment, individuals are asked to indicate which number in between two limits can represent the level of pain that the individual is having. In the VRS assessment, patients are asked to describe the level of pain they experience [2]. Although pain level assessment is possible through VAS, NRS, and VRS, these methods rely on subjective patient feedback. Numerous studies have been conducted to evaluate pain without relying on patient self-reports. In addition to self-report methods, observational scales are commonly used, and most consider facial expression, vocalization, and body language as important characteristics [6]. However, these scales are still manually performed by the observer; therefore, to overcome this drawback, [PERSON] et al. performed automatic pain assessment on facial features activity using machine learning methods [7]. In contrast, [PERSON] et. al.