# Change and Detection of Emotions Expressed on People's Faces in Photos
## 3 Methodology
### Learning Process
This is implemented using the cross-entropy loss between the predicted labels and the target labels:  
\[\mathcal{L}_{\text{cls}}=\frac{1}{m}\sum_{i=1}^{m}\text{CrossEntropy}\Big{(} \text{out}^{(i)}_{\text{class}},x^{(i)}_{\text{fake}\_\text{cls}}\Big{)} \tag{7}\]  
where \(\text{out}_{\text{class}}\) is the classifier's output for the generated images. The reconstruction loss encourages the generated images to resemble the real images when the target emotion matches the original emotion. This loss is masked to only include samples where the target emotion is the same as the original emotion, and is computed as the mean squared error between the real and generated images:  
\[\mathcal{L}_{\text{rec}}=\frac{\sum_{i=1}^{m}\text{MSE}\Big{(}x^{(i)}_{\text{ real}},x^{(i)}_{\text{fake}}\Big{)}\cdot\text{mask}^{(i)}}{\sum_{i=1}^{m} \text{mask}^{(i)}+\epsilon} \tag{8}\]  
where mask is a binary mask indicating samples where the target and original emotions match, and \(\epsilon\) is a small constant to prevent division by zero. The total generator loss is a weighted sum of the adversarial loss, classification loss, and reconstruction loss:  
\[\mathcal{L}_{G}=\mathcal{L}_{\text{adv}}+\mathcal{L}_{\text{cls}}+\mathcal{L }_{\text{rec}} \tag{9}\]  
The learning process of the generator consisted of 30 epochs of the learning algorithm. In this process, the Adam Optimizer [39] algorithm was used. The batch size was 64. The training set consisted of automatically and manually annotated facial images from the AffectNet [23] database, which were transformed to a resolution of 128 \(\times\) 128. In the training process, a series of data augmentation techniques are applied to the images to enhance the robustness and generalization capabilities of the model. These transformations include resizing the images to a specified size, ensuring they fit within the model's input dimensions. Additionally, the images undergo the longest max size transformation to maintain aspect ratio while fitting within a maximum size constraint. Fancy PCA [40] is applied to adjust the principal components of the image colors, followed by random gamma adjustments to vary the brightness non-linearly. Sharpening is used to enhance the clarity of the image features. Horizontal flipping is randomly applied to introduce variability and improve the model's ability to generalize to different orientations. Finally, a random crop is performed to extract fixed-size patches from the images, which helps the model learn from different parts of the image and reduces overfitting. In order to properly evaluate the effectiveness of EmoDNN, a separate additional classifier using TensorFlow machine learning library was trained. The input and output of the classifier are identical to those of the discriminator. The input of the classifier is an image, which returns a vector with the detected probability for emotions as the output. In order to present the learning process of the classifier, the following loss function relationships will be discussed:  
\[\mathcal{L}_{\text{clsTF}}=\frac{1}{m}\sum_{i=1}^{m}1.14\cdot\bigg{(}\text{ CrossEntropy}\Big{(}\text{v}_{Ref}\,\ \text{v}_{Out}\Big{)}+\bigg{(}\frac{1-\text{v}_{Out}[i]}{4}\bigg{)}\bigg{)} \tag{10}\]  
where v\({}_{Ref}\) is a reference vector containing the value 1 for the correct emotion and 0 for others; v\({}_{Out}\) is the result vector of the classifier containing the probability of individual emotions; \([i]\) is the value of the i-th element of the vector representing the emotion currently presented at the network input. The learning process of the classifier consisted of three epochs of the learning algorithm. In this process, the Adam Optimizer algorithm was used. The learning rate value was set to 10\({}^{-4}\). The batch size used was 16. The training set consisted only of manually annotated facial images from the AffectNet [23] database, which were transformed to a resolution of 128 \(\times\) 128.