# Measuring Student Engagement through Behavioral and Emotional Features Using Deep-Learning Models
## 5 Discussion  
The underlying work presents a novel approach for computing students' engagement and the level of engagement in offline classroom environments (OCEs). While earlier studies have proposed methods for measuring students' engagement in an OCE or similar settings [2; 22; 23; 24; 25; 26; 27], these approaches often relied on dedicated physical devices or sensors [6; 22; 24], focused solely on behavioral features [7; 23; 25], tested limited datasets [8; 27], encountered efficiency issues [26], or considered only emotional features [2]. To address these limitations, this study first generated data in the form of facial frames extracted from recorded videos during lectures in an OCE. A self-generated dataset comprising 100 students was utilized, as publicly available datasets [41; 42; 43] exhibited noise, occlusion, or illumination issues. Additionally, existing self-generated datasets [44; 45; 46] had a limited number of participants. Furthermore, using publicly available datasets to train models could introduce facial bias when measuring local students' engagement levels [52; 53]. Following data generation, pre-processing was conducted, and four models--CNN, ResNet50, VGG16, and Inception V3--were trained, validated, and tested. Although some proposals have employed machine-learning or convolutional neural network models [2; 6; 8; 22; 23; 24; 26; 27], or individual pre-trained transfer-learning models [8; 25; 28], these works often relied on limited e-learning datasets and achieved constrained accuracy compared to the results presented in this study. To effectively measure students' engagement, behavior-reflecting feature frames were incorporated, including having closed eyes (indicating sleeping or using a mobile phone), being focused (looking forward or asking the teacher), looking away (talking to peers or diverting attention from the teacher/board), and yawning. These features were classified into the engaged and non-engaged categories, as seen in similar works [2; 7; 23; 25; 54]. The self-trained CNN demonstrated superior performance in the binary classification task, likely due to its inherent ability to learn hierarchical features, encompassing both low-level and high-level details [13]. Once engaged students were identified, their facial emotions--sad, neutral, happy, and angry--were classified, following methodologies established in  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Sample Image Frame** & **(\(\textit{MCS}_{i}\))\({}_{B}\)** & \ \begin{tabular}{c} **Detected** \\ **Emotional State** \\ \end{tabular}  & **(\(\textit{MCS}_{i}\))\({}_{E}\)** & **(\(\textit{SAS}\))\({}_{\text{ES}}\)** & **\(\textit{SEL}\)** \\ \hline \hline \multirow{4}{*}{ \begin{tabular}{c} **Semi-Supervised Learning** \\ \end{tabular}  } & \multirow{4}{*}{7.5} & \multirow{4}{*}{Happy} & \multirow{4}{*}{8.1} & \multirow{4}{*}{7.1} & \multirow{4}{*}{7.5} \\  & & & & & \\ \cline{1-1}  
previous studies [2; 13; 44; 49; 55]. Among the employed models, ResNet50 outperformed VGG16 and Inception V3 in this multiclassification task. ResNet50's effectiveness in multiclassification tasks can be attributed to its residual connections, which facilitate deeper networks in learning identity mappings more effectively, thus handling complex problems [56]. The model confidence scores from both binary and multiclass classification, alongside the average survey scores for the detected emotions, were utilized to compute the Student Engagement Level (SEL) in an OCE. This novel and practical metric holds potential for measuring individual students' engagement scores in real time during lectures. Explicit Comparison: Existing works primarily measure students' engagement in online contexts or controlled offline environments with limited participant numbers and constrained accuracy. Moreover, the exploration of students' engagement levels remains underdeveloped.