## 4 Methods and Experiment
### Keyframe Extraction
The architecture went through rigorous evaluation, particularly in the ILSVRC ImageNet classification challenge of 2012. We utilize the key success of Inception-V3 for high-level feature extraction from an image frame. The design of Inception design incorporates inception modules, which are clusters of convolutional layers with different filter sizes. This innovation allows Inception-V3 to analyze image frames across multiple scales, capturing features of varying granularities and complexities in the student behavior dataset. This aspect makes Inception-V3 well-suited for extracting features from RGB keyframes, which contain information about the student's appearance. The architecture for feature extraction from the video keyframe is depicted in Figure 6.

* **ResNet50-V2:** The ResNet50-V2 is a highly effective CNN model developed to address the problem of vanishing gradients in deep networks. It is an improved version

Figure 5: Shown in (**a–d**) are examples of RGB, DA, L*a*b*, and the optical flow keyframe, respectively. Figure 6: Pretrained architecture for the feature extraction of the RGB keyframe. Xn meeans that the same block is repeated n-time. of ResNet50 that leverages innovative techniques to improve accuracy and performance. One of the key features of ResNet50-V2 is its residual connections, which enable direct information flow between layers, thus mitigating the vanishing gradient challenge. ResNet50-V2 uses bottleneck layers that reduce network parameters while maintaining accuracy via \(1\times 1\) convolutions followed by \(3\times 3\) convolutions. These advancements are supplemented by improved batch normalization and initialization techniques [36; 37]. In the classroom, students' behavior classification needs to detect the head and face of students, such as when they are sitting or turning around. Leveraging the capabilities of ResNet50-V2, we implemented this network to extract keyframe features. The main difference between Inception-V3 and ResNet50-V2 is that the latter focuses on keyframe augmentation techniques [38] to enhance various keyframe features, whereas Inception-V3 focuses on the original RGB keyframe. The model architecture designed for the feature extraction of DA is illustrated in Figure 7.

* **VGG-16:** The VGG-16 excels at feature extraction from images [39], resulting in its popularity. With 16 layers, including 13 convolutional and 3 fully connected layers, VGG-16 processes input images, gradually reducing spatial resolution while deepening channel features. Trained on ImageNet, housing over a million images and 1000 classes [40], VGG-16 employs supervised learning to adapt its weights. This learning equips VGG-16 to recognize common patterns and features in images, making it invaluable for diverse image analysis tasks. To extract features from each video keyframe, we first apply an image transform from RGB to CIELAB or L*a*b* color space [41]. We then feed the transformed image through a VGG-16 model, which utilizes a global average pooling layer for its final pooling operation. This layer significantly reduces the spatial dimensions of the feature maps to \(1\times 1\) by computing the average values across the remaining pixels in each map. As a result, the output of the VGG-16 model for each video frame is a single vector containing 512 features that encapsulate the global characteristics of the input image. The model takes as input a video frame with dimensions of \(224\times 224\times 3\) and outputs a vector of 512 features. This pretrained model extracts the features from the keyframe based on the CIELAB color space. Figure 8 shows the architecture of the feature extraction model for Lab color space. * **EfficientNet-B7:** EfficientNet is a family of CNNs that scale their depth, width, and resolution using a compound coefficient to ensure uniform scaling. EfficientNet models have been pretrained on ImageNet, a large-scale dataset with 1.3 million images

Figure 8: Pretrained architecture extracting the feature from Lab color space in each keyframe.