## 4 Results  
In our study, we validated our results using two models, namely, MobileNetV2 and EfficientFormer. The former is a lightweight model optimized for on-device use, while the latter is the model described in Section 2.4. The two models were applied to demonstrate the significance of the loss function for various cases. Model performance was evaluated by comparing the accuracy of four methods. Method 1 involved learning pure MobileNetV2 and EfficientFormer for expression classification. This method allowed us to examine the performance of the pure model without a teacher model. Method 2 applied knowledge distillation only to classification using the expression prediction value of Emonet (the teacher model), and predicted only the expression. This method enabled us to determine the performance when valence and arousal were not used. Method 3 used the expression label of the teacher model for knowledge distillation in the case of expression, but learned valence and arousal using only the ground truth label. This method allowed us to compare performance when no teacher bounds were applied. Finally, Method 4 applied all suggested methods. This method used the teacher's expression label as well as the valence and arousal prediction values as labels, and proceeded with knowledge distillation for various predictions. Table 1 summarizes the descriptions of the methods used. The performance of each method is displayed for the two backbone models in Table 2. In the case of expression, we used the percentage of the data answered correctly for every item as the accuracy; in the case of valence and arousal, the mean absolute error value was used. In the table, it can be seen that the best performance for both models is obtained by Method 4. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & **Use of Teacher Model (Labelled Data Used for Training)** & **Knowledge Distillation** & **Knowledge Distillation** \\  & & **Truth Label Only** & **with Teacher Model** & **and Teacher Bound with Teacher Model** \\ \hline \multirow{4}{*}{**Types of predicted output**} & Expression Classification Only & Method 1 & Method 2 & \\ \cline{2-5}  & Expression Classification and & & Method 3 & Method 4 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of proposed methods. The confusion matrix presented in Figure 5 shows the accuracy of each expression for the four methods of the two models, allowing for a more accurate comparison. The matrix displays the eight emotions (neutral, happy, sad, surprise, fear, disgust, anger, and contempt), with the true labels on the \(y\)-axis and predicted labels on the \(x\)-axis. The highest and lowest accuracy distributions are represented in yellow and dark purple, respectively. Method 1 failed most often in predicting contempt for both models. However, in the case of Method 2 the overall prediction is higher than that of Method 1. Method 3 does not significantly differ from Method 2, with MobileNetV2 exhibiting poor performance in several classes. This can be confirmed by the accuracy shown in Table 2. Finally, Method 4 predicts evenly across the board  
Figure 5: _Cont_. The emotion recognition model trained through the proposed knowledge distillation method displays SOTA performance with little computation and fast inference. Table 3 shows the performance comparison between SOTA methods for facial expression and emotion recognition tasks. The proposed method exhibits SOTA performance in accuracy, and shows very high computational efficiency compared to Emonet based on multiply-accumulate (MAC) [23], which is a metric for measuring the amount of computation. Emonet, which has an operation amount of 16.99 MAC, is capable of real-time facial expression detection in a GPU environment, but real-time recognition is not possible in a CPU environment.