# Deep Analysis of Student Body Activities to Detect Engagement State in E-Learning Sessions
## 2 Related Work
### Facial Features vs. Body Activity for Affect Recognition
The recognition of body expressions is more difficult due to the shape of the human body, which has more points of freedom than the face. However, recent automatic affect recognition systems have started taking into consideration the analysis of body activities and expressions. Several recent works have had a similar aim to ours, focusing on the upper-body gestures of students using e-learning systems. For example, [PERSON]. [23] implemented a detection method for learner engagement that considered nonverbal behaviors, such as hand-over-face (HoF) gestures, along with head and eye movements and facial expressions during learning sessions. They proposed a novel dataset and detection method for HoF gestures, as they can emphasize affective cues, in addition to the effect of time duration. However, the study did not correlate the expressed emotions with the HoF gestures. Another work [24] considered the effect of emotional experience on the head and the position and motion of the participant's upper body. This was studied during participation in a serious game of financial education. The study included 70 undergraduate students. A Microsoft Kinect device was used to collect depth-image data on body gestures. Researchers found that bodily expressions changed during the session as an indicator of emotional state. The aforementioned studies related body gestures to emotional states in an e-learning environment but did not explicitly relate them to engagement levels. However, in our previous work [9], we proposed a new approach for measuring engagement levels. It is based on analyzing the body activities of the student. First, we collected a new realistic video dataset. It contains 2476 video clips (about 1240 min of recording) of undergraduate students during their attendance in real online courses during the COVID-19 lockdown. The collected clips were recorded in an uncontrolled environment. The volunteers used their built-in webcams to record the sessions. Therefore, working with the dataset was very challenging. Based on our research, we established two categories of body activity: (1) macro-body activities and (2) micro-body activities. We defined these categories as follows: macro activities or actions require major physical change and movement and are most likely to be voluntary; micro-body activities are involuntary actions and most likely do not require noticeable physical change. Accordingly, the data analysis and annotation phase was simplified using these two definitions. After the data analysis and labeling phase, we made several preprocessing pipelines to improve dataset quality by reducing redundancy and unnecessary data. This also accelerated the work by reducing the computation time. After that, we prepared the data to fit into the chosen pre-trained model for training. Finally, we generated the engagement detection model. The generated model was evaluated through several experiments. Moreover, we achieved an accuracy rate of 94%. In this paper, we aim to extend our previous work [9] in connecting e-learners' emotions and engagement states to their bodily activities by proposing two new prediction models that can detect more precise engagement levels based on the affective model in [8].