# Cost-Effective CNNs for Real-Time Micro-Expression Recognition
## 4 Experiments
### Performance Evaluations  
In this subsection, we describe measuring our proposed method in three aspects: recognition accuracy, needed memory space and processing speed. Since we obtained optimal results by using the Vy field and 3-layer CNN, further evaluations concentrated on this particular configuration. **Evaluation on recognition accuracy:** We performed an accuracy comparison of five objective ME class recognition (see Table 8). Our best CNN reached a similar performance as those of other studies using the same protocol of validation. It is worth mentioning that [PERSON] et al. [40] employed a macro-to-micro transferred ResNet10 model and obtained a better result. Their work used four Macro-Expression datasets (>10 K images) and some preprocessing, such as color shift, rotation and smoothing. These additional operations make their proposed method difficult for deployment on embedded systems. After seeing the confusion matrix of our model (Figure 7), we also noticed that the distribution of correct assessments for Vy was more balanced than the ones gotten from [28] (Figure 8). The DTSCNN proposed by [PERSON] et al. in [27] opted for two optical flows computed differently from a ME sample, which made the whole network robust to different frame rates of ME videos. In detail, the first optical flow is calculated using 64 frames around the apex to adapt to the frame rate of CASME I. Similarly, the second optical flow is given by the 128 frames around the apex adapted to the frame rate of CASME II. In case the number of frames composing the ME is not sufficient, a linear interpolation method is used to normalize the video clips. Their study used two CNNs in parallel to extract two separate features before concatenating them. The resultant feature vector was then \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Happiness** & **Surprise** & **Anger** & **Disgust** & **Sadness** \\ \hline
**Happiness** & 0.5615 & 0.1700 & 0.1171 & 0.1155 & 0.1195 \\
**Surprise** & 0.1700 & 0.5831 & 0.1432 & 0.1502 & 0.1618 \\
**Anger** & 0.1171 & 0.1432 & 0.5672 & 0.1176 & 0.1503 \\
**disgust** & 0.1155 & 0.1502 & 0.1176 & 0.5447 & 0.1225 \\
**Sadness** & 0.1195 & 0.1618 & 0.1503 & 0.1225 & 0.5443 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Cosine similarity for the 3-CL CNN with double-channel inputs (Vx-Vy). \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Happiness** & **Surprise** & **Anger** & **Disgust** & **Sadness** \\ \hline
**Happiness** & 0.8464 & 0.3966 & 0.3860 & 0.3126 & 0.2960 \\
**Surprise** & 0.3966 & 0.8159 & 0.4040 & 0.3362 & 0.3324 \\
**Anger** & 0.3860 & 0.4040 & 0.8344 & 0.3654 & 0.3307 \\
**Disgust** & 0.3126 & 0.3362 & 0.3654 & 0.8598 & 0.2363 \\
**Sadness** & 0.2960 & 0.3324 & 0.3307 & 0.2363 & 0.9343 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Cosine similarity for ResNet8. \begin{table}
\begin{tabular}{c c} \hline \hline
**Method** & **Accuracy** \\ \hline _LBP_TOP_[28] & 42.29\% \\ [PERSON] et al. [28] & 57.00\% \\ [PERSON] et al. [40] & 74.70\% \\ Proposed method & 60.17\% \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Comparison between our method and those of other top-performers from literature. fed as input to an SVM to be classified. The DTSCNN was tested on four classes (positive, negative, surprise and other) from a composite dataset consisting of the CASME I and CASME II databases, and it achieved an average recognition rate of 66.67%. The STSTNet proposed by [PERSON] et al. in [29] makes use of three-dimensional CNNs which carry out three-dimensional convolutions instead of two-dimensional ones (such as ResNet, VGG, the networks presented in [27; 28; 40] and our study). It was tested on three classes: positive, negative and surprise from a composite database consisting of samples from the SMIC, CASME II and SAMM databases. It achieved an unweighted average recall rate of 76.05% and an unweighted F1-score of 73.53%.