# Guided Spatial Transformers for Facial Expression Recognition
## 6 Conclusions  
STN strategies have been confirmed to be more effective than CNN for solving Facial Emotion Recognition tasks. Based on these models, we appended a mask generator module into a conventional STN. This proposal reaches higher accuracy rates, enhancing the suitability of landmarks and saliency masks to improve the learning of the Localization Network, and thus the performance of the STNs. The reason for this accuracy improvement compared to existing STNs relies on introducing more specific information to the attention mechanism, extracted from powerful pre-trained models that generate the saliency maps or the landmarks. The localization network receives these filtered images that contain the most relevant regions emphasized. These images with emphasized regions help the attention mechanism to concentrate only on those relevant regions to transform the original image. As a result of this transformation, the classification branch sees a filtered image without unnecessary material to solve the Facial Emotion Recognition task. For this reason, in the results, we have observed an improvement in most of our strategies compared to traditional STNs. From the different strategies tested for Facial Emotion Recognition tasks, the landmark-based methods with soft masks achieved the best performance on the AffectNet dataset, beating to the conventional STNs. On FER-2013, the landmark-based masks also achieved good results, but the saliency maps surpassed all the other strategies, including the Baseline-STN. In this case, all the versions trained applying transfer learning from the learned weights on AffectNet demonstrated better performances. These results suggest that the models trained on AffectNet are robust enough for being used in other datasets. Notice that there is a performance difference between AffectNet and FER-2013. The main reason for this difference may be the nature of the datasets and their initial distribution. AffectNet has a ZeroR of 49.26% against the ZeroR of FER-2013 that reaches a value of 52.37%. This difference explains why the models trained with FER-2013 achieved higher accuracy rates. Another important difference between datasets is the separation of our best model compared to the baseline STNs. In the case of AffectNet, this difference is 0.35% against the 1.49% reached in FER-2013. These results could be explained by the number of samples of the datasets and the resolution of their images. AffectNet contains more images with good quality and resolution, whereas FER-2013 has fewer images taken in more challenging conditions. Results seem to indicate that our proposals are more competitive for complex scenarios where the number of samples is limited. This result is coherent as in our strategies the STN only has to learn from the filtered masks (landmarks or saliency maps) and not from the complete image that is more complex and the localization network requires more samples to discover and learn patterns. In conclusion, landmark-based strategies are the best option in the absence of ground truths of the most relevant regions, considering that landmarks represent tailored information about the regions of interest. However, when landmarks are not available or the quality of the images is low, we can also employ the saliency maps. Saliency maps have also achieved high accuracy rates in the experiments and, moreover, there are numerous general purpose saliency predictors [36; 39] that could be used directly or re-trained for a new task, without the necessity of having the landmarks. These facts suggest that the method based on saliency maps could scale well to other computer vision tasks. Regarding the analysis of errors, we detected that when the number of lost faces is close to 14%, the landmark-based versions are less efficient than the saliency-based. The misdetected faces could happen due to several factors as occlusions, illumination conditions, resolution of images, quality, presence of several faces, rotations, etc.