# Continuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks
## 5 Discussion  
The experiments carried out on SEWA-DB showed that the 2D CNN-LSTM architectures (Table 2) achieved better results than i3D CNN architectures (Table 6). Notably, for the former, valence was better predicted than arousal in terms of CCC. On the contrary, for the latter, arousal was better predicted than valence, also in terms of CCC. Previous works raised the fact that, intuitively, face textures on video sequences are the primary source of information for describing the level of positivity in emotions, hence the valence values. In contrast, arousal is better predicted with voice frequencies and audio signals. However, our work with inflated networks suggests that simultaneous learning of spatiotemporal face features benefits the prediction of arousal values. Regarding the complexity of the two proposed approaches for continuous emotion recognition, we had to make some trade-off that certainly impacted the quality of the results provided by i3D CNN architectures. On the other hand, high sensitivity in training this type of architecture was also observed according to various configurations. This implies that i3D CNN architectures are very flexible, and further improvement could lie in better initialization and tuning of the number and quality of parameters regarding the potential of this model. Furthermore, inflated weights provided good initialization for \begin{table}
\begin{tabular}{l l l l l l l} \hline \hline \multirow{2}{*}{**Base**} & \multirow{2}{*}{**Dataset for**} & \multicolumn{3}{c}{**Valence**} & \multicolumn{3}{c}{**Arousal**} \\ \cline{3-8}  & & **PCC\(\uparrow\)** & **CCC\(\uparrow\)** & **MAPE(\%)\(\downarrow\)** & **PCC\(\uparrow\)** & **CCC\(\uparrow\)** & **MAPE(\%)\(\downarrow\)** \\ \hline \multirow{3}{*}{VGG-11-BN} & RAF-DB & 0.035 & 0.018 & 2.9 & 0.359 & 0.348 & 8.1 \\  & ImageNet & 0.040 & 0.025 & 4.2 & 0.342 & 0.203 & 3.0 \\ \cline{2-8}  & VGG-Face & 0.119 & 0.071 & 3.0 & 0.220 & 0.166 & 5.2 \\  & RAF-DB & 0.036 & 0.028 & 3.5 & 0.242 & 0.119 & 4.6 \\  & ImageNet & 0.209 & 0.190 & 2.4 & 0.391 & 0.189 & 3.8 \\ \cline{2-8}  & VGG-Face & 0.203 & 0.105 & 3.6 & 0.347 & 0.304 & 5.3 \\  & RAF-DB & 0.123 & 0.101 & 3.2 & 0.284 & 0.165 & 3.3 \\  & ImageNet & **0.346** & **0.304** & **5.6** & **0.382** & **0.326** & **5.3** \\ \cline{2-8}  & VGG-Face & **0.313** & **0.253** & **3.5** & **0.406** & **0.273** & **4.9** \\ ResNet50 & RAF-DB & 0.113 & 0.063 & 2.9 & 0.262 & 0.207 & 4.9 \\  & ImageNet & 0.183 & 0.164 & 6.0 & 0.323 & 0.256 & 4.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Best performance of i3D CNNs for predicting valence and arousal in terms of PCC and CCC values and MAPE according to different models and their initialization. The higher values of PCC and CCC are in boldface. \begin{table}
\begin{tabular}{l l l l l l l} \hline \hline \multirow{2}{*}{**Base**} & \multirow{2}{*}{**Dataset for**} & \multicolumn{4}{c}{**Parameters**} \\ \cline{3-6}  & & & & & **Initialization** & \\ \cline{2-6}  & **Initialization** & **Inflation** & **Dilation** & **Masking** & **Centered Weights** & **Mult** \\ \hline \multirow{3}{*}{VGG-11-BN} & RAF-DB & Centered & I & No & Random & \(\times 1\) \\  & ImageNet & Centered & VIII & No & Zero & \(\times 1\) \\ \cline{2-6}  & VGG-Face & Centered & I & No & Random & \(\times 1\) \\  & RAF-DB & Copied & I & Yes & Random & \(\times 100\) \\  & ImageNet & Centered & I & No & Random & \(\times 1\) \\ \cline{2-6}  & VGG-Face & Centered & I & Yes & Random & \(\times 1\) \\  & RAF-DB & Copied & I & No & Random & \(\times 1\) \\  & ImageNet & Centered & I & No & Zero & \(\times 1\) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The i3D CNN model configurations according to the best performance for predicting arousal. action recognition tasks, suggesting that this method could also be utilized for emotion recognition. However, the main difference is that researchers had hundreds of short videos for a classification task for action recognition. In contrast, we had relatively long videos of a few subjects for the regression task.