# Multimodal Model for Automated Pain Assessment: Leveraging Video and fNIRS
## 4 Experiment and Results  
### Datasets  
The integration of artificial intelligence (AI) and sensing technologies offers a promising opportunity to enhance pain recognition, which is a critical aspect of healthcare and affective computing. The proposed Grand Challenge aims to leverage fNIRS and facial video analysis to advance automated pain assessment. By combining these sensing modalities, the challenge seeks to capture both neurophysiological and behavioral aspects of pain, providing a more comprehensive and objective approach that has not been previously explored. Participants are invited to develop novel algorithms for pain recognition, contributing to the creation of a multimodal sensing dataset that can serve as a benchmark and resource for future research in this field [10].  
The AI4 Pain dataset, consisting of 65 subjects, was divided into 41 for training, 12 for validation, and 12 for testing. In each experiment, transcutaneous electrical nerve stimulation (TENS) electrodes were placed on the forearm and the back of the hand to deliver controlled pain stimuli. TENS was used to induce both High Pain and Low Pain conditions in randomized order across 12 repetitions per subject. Each trial began with a 60-s baseline recording, which was followed by a 10-s stimulation phase and a 40-s rest period. The baseline segment was labeled as No Pain. The resulting class distribution was No Pain:Low Pain:High Pain = 1:12:12. The classification task involves distinguishing among these three pain levels using multimodal signals. The fNIRS modality includes 24 channels with each channel consisting a pair of signals: \(\Delta\)HboO2 and \(\Delta\)HHb.  
In addition to AI4 Pain, we also utilize the BioVid Heat Pain Database (Part A) [3; 13], which is a publicly available benchmark dataset for pain recognition. BioVid includes recordings from 87 participants exposed to calibrated heat pain stimuli administered via a thermode applied to the forearm. Each subject underwent a personalized pain calibration procedure to determine five thermal thresholds (T0-T4), where T0 represents no pain and T4 corresponds to the maximum tolerable pain level. Participants received 20 stimuli per pain level, each lasting four seconds, with randomized ordering and rest periods in between to avoid adaptation. The dataset comprises synchronized multimodal recordings, including high-resolution video of the upper body and face, electrodermal activity (EDA), electromyogram (EMG), and electrocardiogram (ECG). Following prior studies such as PainNet [25] and AI4 Pain [10], we adopt a three-class classification scheme by grouping  
Figure 4: **Attention fuse box** architecture. Our fusion method receives 2 embedded vectors as inputs. The first dense layer is the neck section, which is useful to shape. In our fusion method, 512 units of any dense layer were utilized. **Q**: query, **K**: key, **V**: value.  
the five pain levels into No Pain (T0), Low Pain (T1-T2), and High Pain (T3-T4). While both datasets are employed, our primary experiments and model evaluation are conducted using the AI4 Pain dataset due to its relevance to the Grand Challenge setting.