# Multi-Modal Emotion Recognition for Online Education Using Emoji Prompts
## 4 Experimental Results and Analyses
### Experiments and Discussion  
In the experiment, we used the accuracy and \(F1\) values as evaluation indicators. Specifically, the calculation formulae for the accuracy and \(F1\) values are shown in Equations (4) and (5).  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN}, \tag{4}\]  
\[F1=\frac{2\cdot P\cdot R}{P+R}, \tag{5}\]  
where \(TP\)/\(TN\)/\(FP\)/\(FN\) denote true positives, true negatives, false positives, and false negatives, respectively. In the formula for \(F1\), \(P\) and \(R\) denote precision and recall, respectively.  
In the first group of experiments, we compared the performance of several models on three datasets, and the results are presented in Table 3.  
From Table 3, we can see that our model performs best in the Restaurant and Laptop datasets, but not in the Twitter dataset, where it is, however, still competitive. In addition to the comparison of different models, in the second group of experiments, the specific performances of the different models on educational learning data were compared.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Restaurant**} & \multicolumn{2}{c}{**Laptop**} & \multicolumn{2}{c}{**Twitter**} \\ \cline{2-7}  & _Accuracy_ & \(F1\) & _Accuracy_ & \(F1\) & _Accuracy_ & \(F1\) \\ \hline ATAE-LSTM ([PERSON] et al., 2016) [31] & 0.785 & 0.598 & 0.753 & 0.629 & 0.842 & 0.629 \\ GTRU ([PERSON] et al., 2018) [32] & 0.776 & 0.574 & 0.753 & 0.627 & 0.847 & 0.609 \\ AA-LSTM ([PERSON] et al., 2019) [35] & 0.777 & 0.586 & 0.762 & 0.631 & 0.856 & 0.611 \\ CapsNet ([PERSON] et al., 2019) [36] & 0.781 & 0.616 & 0.747 & 0.618 & 0.838 & 0.614 \\ AS-Capsules ([PERSON] et al., 2019) [37] & 0.793 & 0.619 & 0.752 & 0.621 & 0.845 & 0.625 \\ GIN ([PERSON] et al., 2020) [38] & 0.812 & 0.624 & 0.759 & 0.632 & 0.871 & 0.650 \\ MIMLLN ([PERSON] et al., 2020) [39] & 0.783 & 0.606 & 0.753 & 0.614 & 0.858 & 0.635 \\ BERT ([PERSON] et al., 2018) [40] & 0.824 & 0.644 & 0.816 & 0.662 & 0.886 & 0.736 \\ GIN-BERT ([PERSON] et al., 2020) [38] & 0.840 & 0.660 & 0.830 & 0.653 & 0.895 & 0.749 \\ MIMLLN-BERT ([PERSON] et al., 2020) [39] & 0.828 & 0.651 & 0.830 & 0.624 & 0.881 & 0.731 \\ Hier-GCN-BERT ([PERSON] et al., 2020) [41] & 0.843 & 0.663 & 0.833 & 0.658 & 0.897 & 0.746 \\ Ours & 0.856 & 0.682 & 0.848 & 0.684 & 0.912 & 0.785 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The _accuracy/\(F1\)_ of the compared systems on 3 benchmark datasets.  
Due to the simple initial matching method used for emoji symbols, there is a high likelihood of model misidentification. To address this, we utilized seed-word-assisted verification to correct misclassification results. Table 4 illustrates the results of the auxiliary error correction, which shows the actual performance of the model before and after the application of seed words.  
From Table 4, we can see that without the use of seed word prompts and error correction, the model's accuracy and \(F1\) score both decreased by about 3 percentage points, especially for the Twitter data, which include a richer set of emoji symbols, where the decline is even more pronounced.  
Several models based on BERT were selected for comparison: BERT [40], GIN-BERT [38], MIMLLN-BERT [39], and Hier-GCN-BERT [42]. The P-R curves are shown in Figures 5 and 6. Figure 5 shows the performance of different models on the education data, on which our model performs best, and Figure 6 shows the ablation experiment; after using the emoji expression data as the prompt, the performance of the model improved to a certain extent, which also shows that our model is effective in introducing emoji data.