# Guided Spatial Transformers for Facial Expression Recognition
## 5 Results  
### Evaluation of Strategies on AffectNet  
Table 1 shows the valence recognition rates obtained on AffectNet dataset. ZeroR is the simplest classifier that relies on the target and always predicts the majority category (class). Although there is no predictability power in ZeroR, it is appropriate for determining a baseline performance as a benchmark for the other classification methods. The 'Simple CNN' experiment beats the ZeroR rates, demonstrating that the proposed architecture is suitable for solving the task and evidencing a learning process. However, this result is significantly lower than the obtained when we add the Spatial Transformer, called Baseline-STN in the Table 1, which confirms that including this mechanism improves the accuracy. The rest of the results with a single mask generator get higher accuracies than the 'Baseline-STN'. These outcomes suggest that feeding the localization network with any of the landmarks strategies (binary masks, soft masks, and dilated masks) or saliency maps helps the attention mechanism to focus on the most relevant areas, which provokes a growth in the recognition accuracy. It is especially interesting the case of the 'STN with landmarks--soft masks' because its result surpasses in a significant statistical way the accuracy achieved by the 'Baseline STN'. For this reason, we can conclude that this proposal seems to be the best option to train the Spatial Transformers, despite the misdetected faces. Figure 7: Distribution of the dataset labels. The X-axis contains the valence levels: Negative, Neutral, and Positive. The \(Y\)-axis represents the number of samples per class. Although soft masks and dilated masks pursue the target of improving the binary mask version, only the soft mask leads to competitive results, with an average accuracy of 70.72%. This rate is 0.19 points higher than the dilated masks, 0.15 points superior to the binary landmarks, and 0.35 points better than the 'Baseline STN'. For the dilated masks, the results may indicate that increasing the size of the strokes introduces redundant information that is not convenient to learn the most relevant regions on an image. This strategy seems to maintain the drawbacks of the saliency and landmarks strategies. On the one hand, it misdetected the same faces as in the landmarks strategy. On the other hand, it also loses definition on the relevant areas, similar to what happens in the saliency maps. The combination of both effects concludes with an accuracy rate comparable to the landmarks-based binary masks; however, the dilated masks require more processing efforts to be generated. The last strategy of this family to analyze is the mask-based saliency. In this case, the result also improves compared to the conventional STN by 0.23 points. This outcome implies that when landmarks or ground truth of the most relevant regions on the image are not available, we can still guide the learning of the localization network using the saliency maps and improve the performance of conventional STNs. If we compare the saliency with the 'STN with landmarks--soft masks' model, the saliency maps are 0.1 points under the accuracy of the best landmark-based strategy. Despite reaching a lower performance in this task, saliency maps are still a powerful option as current saliency predictors are trained to extract saliency maps from almost any object, not only faces. This generic knowledge offers the possibility of applying this strategy in other domains without the necessity of a tailored landmarks extractor. Regarding the method to deal with faces misdetected by the facial detector, we can see that the best option is version 3. Changing the white images (v1) by saliency maps (v2) does not improve the final accuracy, probably because for AffectNet the number of lost faces was not too large, a 1.6%, or because training the STN with landmarks and saliency maps introduce some noise during the learning process because of the use of different ways of representing the information.