## 5 Experiments
### Acted Emotional Speech Dynamic Database (AESDD)
#### 5.2.2 Facial Emotion Detection  
In this subsection, we analyze the performance of the models trained for facial emotion recognition across different neural network architectures and datasets. These experiments evaluate the accuracy and efficiency of each model in categorizing emotions correctly. Several experiments were conducted with various architectures and datasets to determine the optimal model for facial emotion detection:  
* CLIP (Contrastive Language-Image Pre-training): The CLIP model, developed by OpenAI, was tested for its capacity to recognize and classify emotions in facial images. The model was evaluated using several datasets, including RAVDESS, CREMA-D, and the Christian Mejia dataset. In these tests, the model performed well on classes such as \"angry\" and \"happy\", achieving accuracy rates between 70% and 90%, depending on the dataset. However, it struggled with \"neutral\" expressions, where the highest accuracy reached only 30%, likely due to the model's lack of training on neutral emotion labels. Accuracy for the \"sad\" class varied between 50% and 60%, depending on the dataset. We trained four predictive models: the base pretrained model and three fine-tuned models using different datasets. Three evaluation tests were performed--one for each dataset. The model fine-tuned with the [PERSON] dataset achieved the highest accuracy, with an average of 75% across four emotions on RAVDESS and 47% on CREMA-D. Although the pretrained CLIP model could accurately associate images with clear emotional cues (e.g., tears indicating sadness), it struggled with more subtle emotional cues. Fine-tuning the model with additional datasets and introducing text labels like \"A person expressing a neutral emotion\" helped improve performance with neutral images. During evaluation, each image was classified by inputting four emotion labels, and the highest predicted value determined the final classification. Visual Transformers (ViTs): Visual Transformers (ViTs) were also tested, as these models can capture complex relationships between image features more efficiently. Due to hardware constraints, we initially tested the model on the RAVDESS dataset, where it achieved a validation accuracy of 95%, comparable to the 2D CNN model trained on the same dataset. Further testing on larger datasets using more computational resources is expected to improve robustness and generalization. * 3D Convolutional Neural Networks (CNNs): A 3D CNN architecture was used to process entire video sequences instead of individual frames, utilizing the RAVDESS dataset's video components. The goal was to determine if analyzing sequences could improve emotion detection compared to static frame-by-frame analysis. However, the 3D CNN model achieved only 70% accuracy on the validation set, which is significantly lower than the 95% accuracy obtained by the 2D CNN and Visual Transformers. Due to this underperformance, the 3D CNN architecture was not pursued further. * 2D Convolutional Neural Networks (CNNs): Separate 2D CNNs were initially trained for each dataset (Crema-D, RAVDESS, and Christian Mejia). Datasets without pre-defined training and validation splits were manually segmented, with 75% of the data used for training and 25% for validation. Subsequently, data augmentation techniques were applied to the training sets to assess their impact on the model's generalization capabilities. The main objective of these experiments was to assess each model's performance across different datasets and identify which emotions were more easily detected and which presented greater challenges. Following extensive testing, the 2D CNN model emerged as the most effective. The final experiment involved combining all datasets and fine-tuning the model with personalized data. The performance graphs for this final model are shown in Figure 4.  
Although the fine-tuning was performed for a single person, achieving an accuracy close to 100% is not always possible, and there is a simple explanation for this.