# DA-FER: Domain Adaptive Facial Expression Recognition
## 3 Proposed Method
### Learning Process of Domain Adaptive Method  
Figure 1 illustrates the learning process of domain adaptation methods, focusing on the design of the loss function during training. It comprises three main components: input, training, and output [11]. Specifically, all the labels of source domain datasets are used in the input part, whereas target domain datasets are not used [12]. Only during verification the labels of the target domain are employed, and the accuracy is computed by comparing the predicted label generated by the network with the actual label of the target domain [13]. Consequently, domain transfer learning is fundamentally an unsupervised learning technique [14]. The network loads images of the source domain and target domain, and they share a Feature_extraction network [15]. In the orange part of the flow chart, the source domain image is fed into the feature_extraction network, followed by a classifier that produces a predicted label. The predicted label is then compared with the ground-truth label and the loss function Ls is calculated using the cross-entropy function. In contrast, the blue branch of the flow chart involves processing the target domain image through the feature extraction network without the classifier. Instead, the difference between the source and target domain features is calculated using the maximum mean difference (MMD) method and the loss function Lmmd is obtained. The overall objective of network learning is to minimize the values of both Ls and Lmmd to achieve effective transfer learning. That can be described as:  
\[\mathrm{L}=\mathrm{L}_{\mathrm{s}}+\lambda\mathrm{L}_{\mathrm{mmd}} \tag{1}\]  
The weight parameter, denoted by \(\lambda\), calculates the weight of the discrepancy. While Ls captures the classification loss of the source domain, Lmmd measures the difference between the source domain and the target domain. However, the two loss functions cannot be assigned the same weight [16]. This is because the true labels of the source domain are involved in the network training, and the accuracy of the source domain classification should be prioritized. Therefore, \(\lambda\) is considered a hyperparameter to control the balance between Ls and Lmmd. It is typically set to a value between 0 and 1. In the experiments conducted in this study, \(\lambda\) is set to 0.5. This setting is suitable for most datasets, and satisfactory results have been obtained in previous studies. MMD is utilized for domain adaptation, which aims to map the source and target domain variables into a high-dimensional feature space, and to measure the distribution differences between the two domains in this feature space [11] (see Figure 2). Figure 1: Learning process of domain adaptive method. It can be defined as follows:  
\[ \begin{split}\text{MMD}[\text{F},\text{P},\text{q}]:=\sup_{\text{f} \in\text{F}}\bigl{(}\text{E}_{\text{p}}[\text{f}(\text{s})]-\text{E}_{\text{q}} [\text{f}(\text{t})]\bigr{)}\\ =\sup_{\text{f}\in\text{F}}\bigl{(}\text{E}_{\text{p}}[<\text{f}, \text{q}(\text{s})>]-\text{E}_{\text{q}}[<\text{f},\text{q}(\text{t})>_{\text{H }}]\bigr{)}\\ =\sup_{\text{f}\in\text{F}}\bigl{(}<\text{f},\text{ }\mu_{\text{p}}-\mu_{\text{q}}>_{\text{H }}\bigr{)}\\ =\left\|\mu_{\text{p}}-\mu_{\text{q}}\right\|_{\text{H}}\end{split}  \tag{2}\]  
The source domain data s distribution, denoted as p, is mapped to the Reproducing Kernel Hilbert Space (RKHS) by f, and the expectation \(\mu_{\text{p}}\) of f(s) is computed. Similarly, the target domain data t distribution, denoted as q, is mapped to the RKHS by f, and the expectation \(\mu_{\text{q}}\) of f(t) is computed. The upper exact bound of the difference between \(\mu_{\text{p}}\) and \(\mu_{\text{q}}\) is calculated. However, there is a problem in calculating MMD directly. For multiple kernel functions, the MMD value cannot be estimated accurately by a limited number of samples [17].