# An Optimized CNN Model for Engagement Recognition in an E-Learning Environment
## 4 Method  
### Proposed Model Architecture  
As an extremely efficient CNN architecture, ShuffleNet v2 can be applied explicitly to mobile devices with limited computing power [30]. The calculation amount is significantly reduced with similar accuracy when compared with existing advanced models. Among different lightweight CNN models, ShuffleNet v2 has also verified its better performance on many direct metrics, such as speed, latency and memory access cost, rather than only approximate factors such float-point operations (FLOPs) [30]. Therefore, we considered optimizing a ShuffleNet-based model to improve its accuracy without affecting other performance parameters as much as possible. From the perspective of model practicability, this paper chooses a ShuffleNet v2 0.5x structure to minimize computational complexity. The computational complexity of ShuffleNet v2 with different sizes is shown in Table 1.  
According to the activation function mentioned in the background section, the Leaky ReLU function does not only avoid the problem of gradient disappearance, but it also solves the problem of neuron \"death\" phenomenon, which can effectively improve the accuracy of the model [47]. Therefore, this study conducts a pre-experiment to compare different activation functions trained on the selected dataset. Based on the pre-experimental results, the function with the best accuracy is selected as the activation function of the engagement-recognition model.  
Replacing the activation function with better accuracy is the first measure to optimize the engagement-recognition model, and the second to consider the attention mechanism's application. It was mentioned in the related work that the attention mechanism has been widely used in computer vision. Moreover, the attention mechanism can improve accuracy with little cost in terms of calculation requirements. Our attention mechanism module adopts the SE block mentioned in the Related Work Section. The SENet attention mechanism has two main operations: squeeze and excitation. Squeeze refers to compressing the spatial information of features through GAP, and compressing the original c \(\times\) h \(\times\) w dimension information to c \(\times\) 1 \(\times\) 1. Excitation uses two fully connected layers. The first reduces the dimension, and c \(\times\) 1 \(\times\) 1. The dimension is reduced to c/r \(\times\) 1 \(\times\) 1 (with ReLU activation), and the second fc layer remaps the features back to c \(\times\) 1 \(\times\) 1 (without ReLU activation); then, after sigmoid, the weight coefficients of each channel are obtained. Then, we multiply the weight coefficient with the original feature to get a new feature. Therefore, the network structure after the embedded attention module based on the ShuffleNet v2 0.5x architecture is shown in Table 2.  
\begin{table}
\begin{tabular}{c c} \hline
**Network Size** & **Computational Complexity(FLOPs)** \\ \hline ShuffleNet v2 0.5x & 41 M \\ \hline ShuffleNet v2 1.0x & 146 M \\ \hline ShuffleNet v2 1.5x & 299 M \\ \hline ShuffleNet v2 2.0x & 591 M \\ \hline \end{tabular}
\end{table}
Table 1: Computational complexity of ShuffleNet v2 with different sizes [30].  
\begin{table}
\begin{tabular}{c c c} \hline
**Output Size** & **ShuffleNet v2 + Attention Module** & **Repeat** \\ \hline \(112\times 112\) & Conv 3 \(\times\) 3, 24, stride 2 & 1 \\ \hline \(56\times 56\) & MaxPool 3 \(\times\) 3, stride 2 & 1 \\ \hline \multirow{6}{*}{\(28\times 28\)} & Conv 1 \(\times\) 1, 48 & \\  & DWConv 3 \(\times\) 3, 48 & \\ \cline{1-1}  & Conv 1 \(\times\) 1, 48 & 4 \\ \cline{1-1}  & Attention Module & \\ \cline{1-1}  & Shuffle & \\ \cline{1-1}  & Conv 1 \(\times\) 1, 96 & \\ \cline{1-1}  & DWConv 3 \(\times\) 3, 96 & \\ \cline{1-1}  & Conv 1 \(\times\) 1, 96 & 8 \\ \cline{1-1}  & Attention Module & \\ \cline{1-1}  & Shuffle & \\ \hline \end{tabular}
\end{table}
Table 2: Engagement-recognition model structure.