# Game Difficulty Prediction Based on Facial Cues and Game Performance
## 2 Related Work
### Multimodal Fusion  
Using single-modal data may be inadequate to obtain promising prediction performance, and some studies suggest combining multimodal data. For example, [PERSON] et al. [25] used facial information, head movement, and upper limb pressure to predict learners' wakefulness levels during a video course. The authors classified learner states into three categories: Awake, Drowsy, and Asleep, and achieved high classification performance with the CatBoost classifier [26]. However, the study used only one classification method and did not compare with other methods. [PERSON] et al. [27] combined three modalities, including facial, heart rate, and acoustic features, and trained multiple machine learning classifiers (SVM, RF, and MLP) to predict students' mental state. The results showed that RF obtained better performance than other methods. Besides, using acoustics and heart rate can more accurately predict concentration and confusion, while combining the three modalities could better classify frustration and boredom. [PERSON] et al. [28] measured the cognitive load of surgeons during surgery by recording multiple physiological signals, including heart rate variability, galvanic skin, and electroencephalography. They compared different classifiers, modalities, and fusion schemes, and the results also showed that the combination of multiple modalities could outperform a single modality. Compared to other methods, SVM was able to achieve better performance. Some studies also merge various modalities to achieve high accuracy in emotion recognition [29; 30; 31].  
From the above studies, it is clear that the combination of multimodal data can compensate for the shortcomings of single data. Although they are not directly related to game difficulty prediction, we can draw on the multimodal idea introduced in these studies to our research. Inspired by these studies, our study also uses multimodal data to predict user-perceived game difficulty. Unlike them, we use multiple facial cues and game performance instead of physiological signals or eye-tracking data. The reasons are as follows: (1) it is more convenient to use a camera to collect facial data without disturbing the user. (2) the economic cost is relatively low since a camera is the only device we need. (3) we can obtain multiple salient features (i.e., facial expressions, gaze directions, and head poses) from a single facial image. Among them, facial expressions can react to one's inner emotion, while the gaze directions and head poses can reflect the attention. Since game performance is an essential indicator of the player's game state, we integrate it into the features.  
In addition, the above studies have also shown the effectiveness of machine learning algorithms in a wide variety of classification tasks. Therefore, this study also draws support from different machine learning classification models to predict game difficulty. The study sought to answer the following research questions: (1) Which machine learning classifier performs better at game difficulty prediction? (2) Can multimodal facial features and game performance be more effective than single-modal features?