# Deep Analysis of Student Body Activities to Detect Engagement State in E-Learning Sessions
## 2 Related Work  
### Facial Features vs. Body Activity for Affect Recognition  
Most of the existing vision-based methods use facial features as the main cue for engagement, even when combined with another input modality. For example, Ref. [10] implemented an engagement detection model using facial expressions, full-body motion, and game events. The game was a pro-social game. It was designed in an uninteresting version and a more entertaining version. Every volunteer participated in the two versions of the game by playing for 10-15 min in the game session. Labeling was conducted using retrospective self-report and game engagement questionnaire (GEQ). Data were labeled either \"engaged\" or \"not engaged\". They presented the ANN model, which produced an accuracy of 85%. In another work [11], the authors proposed a new framework to assess engagement based on facial expressions. They used a lightweight CNN. The light network was used to reduce the effect of the diversity of the backgrounds and the resolutions on the prediction process. The proposed CNN comprised two parts: (1) feature extraction and (2) classification. The model used three public datasets. The RAF-DB was used as the source domain-training data. On the other hand, JAFFE and ck+ were used as the target domain datasets. These datasets included seven types of emotion: anger, disgust, fear, happiness, sadness, surprise, and neutral. Owing to an imbalance issue in some of these datasets, the authors applied under-sampling methods and data enhancement methods in order to balance the data. The model considers four types of emotion: \"understanding\", \"doubt\", \"neutral\", and \"disgust\". The authors reported that their model achieved better results than other competitors did, with 54% accuracy on the ck+ and 51% on the JAFFE. In another study [12], the researchers attempted to measure engagement intensity by fusing both face and body features into a single long short-term memory (LSTM) model. They used the dataset EmotiW 2018 [13], which contains 195 videos, with 147 clips for training and 48 clips for testing. Their fusion approach achieved a comparable performance to the state-of-the-art methods, with an accuracy rate of 75.47%. Another recent work [14], developed a two-stage algorithm using behavioral information [on-task and off-task] and emotional information [satisfied, confused, and bored]. They incorporated these two dimensions to detect whether the student is engaged or not. They used facial expressions for the emotional dimension to decide if the student was feeling satisfied, confused, or bored. As for the behavioral dimension, they used head-pose information to see whether the student was on-task or off-task. The algorithm was tested using five different CNN models applied to the DAiSEE dataset [15]. For the training and testing phases, they used 1500 and 300 frames of students' faces, respectively. The reported performance of the five models ranges between 76.8% and 92.5%. In [7], the proposed framework combined facial expression, eye gaze, and mouse dynamics. The data were recorded from the subjects in real time during reading sessions and classified into three levels of attention, \"low\", \"medium\", or \"high\", using SVM for classification, with an accuracy rate of 75.5%. As seen from some of the previous work, body activities are one of the main cues of human affect and a way to convey messages between people [2; 8; 16; 17], and this is a well-researched and well-established field [18]. Nevertheless, few methods consider investigating body activities combined with other modalities for detecting engagement in students. Therefore, we next discuss different works that show the significance of body activities and how to relate them to engagement detection. It has been proven by many studies that, in the same way as facial expressions, body expressions are very effective at expressing emotions and feelings [19; 20; 21; 22]. Scientists have demonstrated the effective transition through body expressions.