# Measuring Student Engagement through Behavioral and Emotional Features Using Deep-Learning Models
## 2 Related Work  
Intelligent systems have thrived with the rise of artificial intelligence (AI) [29]. AI-based robust solutions are now widely applied across various domains, including education [2; 30], healthcare [31], agriculture [32], security [33], online social media [34; 35], sports [36], and many others. In particular, AI has been increasingly adopted to provide more precise and effective solutions in the education sector. For instance, AI is used in decision-making processes for higher education [37], predicting student performance [38], developing computer-vision-based attendance systems [39], generating multiple-choice questions [30], and monitoring student engagement [2; 6; 7; 8; 9; 22; 23; 24; 25; 26; 27]. Such smart solutions not only enhance student engagement and learning experiences but also assist teachers in ensuring positive learning outcomes. Student engagement is a multifaceted concept, typically classified into cognitive, behavioral, and emotional engagement [1]. However, the constructs within these categories often overlap [40]. To measure student engagement, researchers estimate the cognitive, behavioral, and emotional aspects through various modalities, such as affective cues, including speech, head movements, facial expressions, or body language. Similarly, emotional engagement is often assessed based on [PERSON]'s model of emotions (e.g., surprise, happiness, fear, anger, sadness, disgust, or contempt) or [PERSON]'s model (e.g., astonishment, embarrassment, perplexity, contentment, relaxation, boredom, or unresponsiveness). Other modalities used to estimate engagement include physiological signals (e.g., heart rate, blood pressure, respiration, electroencephalograms (EEG), facial electromyograms (EMG), functional magnetic resonance imaging (fMRI), galvanic skin responses, or body temperature) and log files (e.g., number of logins, content views, forum reads and posts, number of clicks, access duration, or scores). Several publicly available datasets, such as NVIE [41], DAISEE [42], and EmotiW [43], have been instrumental in the study of student engagement. However, these datasets are often affected by challenges such as occlusion, background noise, and poor illumination. As a result, many researchers prefer to create customized datasets to address these issues. For example, studies [44; 45; 46] have utilized self-generated datasets, although the number of participants was limited to 61, 50, and 21, respectively. Given the advantages of self-generated datasets, our study is motivated to use a larger dataset involving more students compared to prior work. Incorporating multiple engagement constructs can improve the accuracy of student engagement estimation. In this regard, [PERSON] et al. estimated cognitive engagement using knowledge tests and behavioral engagement using facial features in an offline classroom setting [47]. The authors extracted facial features using OpenFace [48], measured the intraclass correlation to assign final labels, and used regression models to estimate engagement. Other studies [2; 13; 44; 49] have also incorporated behavioral and emotional features, but with smaller datasets and limited accuracy. Additionally, most of these works do not propose methods for determining the level of engagement. To address these limitations, our work integrates both behavioral and emotional features, aiming to not only detect student engagement but also assess its level. To compute student engagement, researchers have employed variants of convolutional neural network (CNN) and fine-tuning-based transfer learning (TL) techniques to detect behavioral and emotional features from image frames [40]. For example, [PERSON] et al. used hybrid CNN models to detect students' affective states, achieving a 76% accuracy [45]. Similarly, [PERSON] and [PERSON] used CNNs to classify engagement levels as low, medium, or high, with a 76.9% testing accuracy [2]. [PERSON] et al.