## 2 Related Works  
### FER Systems  
FER in an uncontrolled environment [15] is challenging. The traditional approach for FER tasks often utilizes handcrafted (HC) features. Geometric features, or facial Action Units (AUs), are widely used in FER tasks. The study [16] proposed a facial landmarks model based on AUs to classify facial emotions. In [15], an algorithm was proposed and achieved highly accurate results in recognizing AUs and emotions across multiple databases. Other effective texture features that could extract facial emotional information, such as Local Binary Patterns (LBPs) [17; 18] and Oriented Gradients (HoG) [19], are also used for FER research. Nevertheless, models relying on HC features are inefficient at recognizing complex facial expressions collected in \"real world\" environments, especially on these recent \"in-the-wild\" datasets [20].  
In the early 2000s, studies in the FER research showed that deep learning models, such as CNNs, are robust to face landmark changes and scale variations, outperforming the performance of traditional models [21; 22]. In addition, recent research works have combined CNN and AUs for FER tasks and achieved significant performance improvements [23; 24]. EmoFAN is an end-to-end model that can classify facial emotions, predict facial landmarks (AUs), and valence and arousal values [24]. In 2020, researchers proposed an ensemble-based CNN model (ESR-9) that combines nine different CNN models to predict facial expressions [25]. In video or image stream FER tasks, recurrent CNN (RNN), long short-term memory (LSTM) [26; 27; 28], and 3D convolutions [29] have been developed. For instance, in [30], a 3D convolution model was proposed to learn spatial-temporal features by training on large-scale supervised datasets.  
The development of Transformers [31], a new neural network class, improved the proficiency in extracting context and meaning within sequential data by utilizing attention or self-attention mechanisms. They were originally designed for natural language processing (NLP) tasks. In 2020, Google proposed Vision Transformer (ViT) [32] to handle various computer vision tasks, including image recognition. Vision Transformers are particularly notable for their scalability and effectiveness and have been shown to perform comparably or even superiorly to traditional convolutional neural networks (CNNs) in certain tasks, such as FER [32]. Studies [33; 34] started to use ViT as a baseline model to train FER models and achieved high accuracy in multiple datasets.  
We comparatively studied different ViTs and CNN methods regarding their accuracy (see Table 1). While ViTs have achieved significant success on some datasets, their limitations in generality and robustness, particularly with unfamiliar data like artistic portraits, require further investigation. Therefore, the integration of ViT could treat an image as a sequence of tokens to extract the local information of the image. Through our experiments in Section 4, we have substantiated this claim and showcased the superiority of our methodologies of combining CNN with ViT over these models. Overall, ViT methods demonstrate excellent performance, and combining the self-attention mechanism with CNN shows outstanding results.