## 3 Methods
### Emotion Detection Based on Voice Features
#### 3.2.1 Descriptions of Datasets  
The SER classification model integrated into the NeuroPredict platform is constructed using speech data and forms 4 different speech emotion datasets. The following different datasets are used: Ryerson Audio-Visual Database of Emotional Speech and Song (Ravdess) [55], Torinto emotional speech set (Tess) [59], Crowd-Sourced Emotional Multimodal Actors Dataset (Crema-D) [81], and Surrey Audio-Visual Expressed Emotion (Savee) [54]. The recordings from these datasets are aggregated to create a more comprehensive dataset. The purpose of this is to create a more complex dataset with more diverse recordings (regarding speakers and recording environments) so that the built model can be trained on more varied data in order to achieve better generalization and to perform  
Figure 2: Flowchart for building the emotion classification model. The base for the architecture that led to the best results for the algorithm used in the NeuroPredict platform is presented. better on new, unseen data, recorded in different conditions. The initial and newly created datasets are presented in the following paragraphs. Ravdess [55] is a multimodal dataset with voice and speech data for emotion processing. It contains recordings from 24 professional actors, who were asked to act a set of predefined phrases mimicking 7 emotions: calm, happy, sad, angry, fearful, surprised, and disgusted, with two intensities. The database also contains songs sang with mimicked emotions that were not used for the current work. The language of the recordings is English (North American). Toronto emotional speech set (TESS) [59] is a database published in 2020 that contains a set of 200 target words spoken by two actresses portraying each of the seven emotions: anger, disgust, fear, happiness, pleasant surprise, sadness, and neutral. The language of the recordings is English (North American). Crema-D [81] is another dataset containing recordings from 91 actors (48 male and 43 female actors, aged between 20 and 74) saying the same phrase while mimicking one of 7 emotions: anger, disgust, fear, happy, neutral, and sad, with four different intensities: low, medium, high, and unspecified. It contains a total of 7442 recordings. The language of the recordings is English (North American), although the speakers come from various ethnic backgrounds including Asian, European and Hispanic. Surrey Audio-Visual Expressed Emotion (Savee) [54] is another dataset for emotion processing containing recordings of 4 male speakers (not professional actors) mimicking 7 emotions: anger, disgust, fear, happiness, sadness, surprise, and neutral. They recorded themselves saying the same phrases, 15 TIMIT sentences per emotion. It contains 480 recordings. Video recordings of the faces of the speakers are also available in the dataset and not used in the current work. The language of the recordings is English (British). The 4 original datasets include the following emotions: happy, surprise, disgust, calm, angry, fear, sad, and neutral. The datasets were merged in order to create a more comprehensive dataset to be used for creating a model for identifying the targeted emotions. Disgust was excluded because it is not relevant to this specific study. The original datasets and the merged dataset are presented in Table 4. In order to recognize the three main classes of emotion (positive, negative, and neutral) the emotions are distributed as shown in Table 5.  
The process for creating the aggregated dataset involves the following steps:  
* Data upload: All data from the different datasets is stored in a data frame with its specific label. A key differentiator from other work is that our algorithm utilizes data from four different emotion datasets to develop a model that generalizes well to new data. * Data relabeling: The dataset is refined by removing entries labeled with the \"Disgust\" emotion.