# TER-CA-WGNN: Trimodel Emotion Recognition Using Cumulative Attribute-Weighted Graph Neural Network
## 3 Proposed Approach for Emotion Recognition
### Assessment Metrics for Classifier Performance  
The performance of classifiers is evaluated using key metrics: Accuracy, Macro Precision, Macro Recall, and the F1 score. These metrics are defined as follows:  
* **Accuracy**: This metric quantifies the proportion of correctly predicted instances in the dataset and is calculated using the Formula (19):  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset Name** & **CMU-MOSI** & **IEMOCAP** \\ \hline
**Model Name** & CA-WGNN & CA-WGNN \\
**No. of Layers** & 7 & 7 \\
**Batch Size** & 32 & 32 \\
**Epochs** & 100 & 100 \\
**Random State** & 42 & 42 \\
**Learning Rate** & 0.001 & 0.001 \\
**Validation Split** & 0.2 & 0.5 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of Experimental Configurations.  
\[\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{Total}} \tag{19}\]  
where TP (True Positives) and TN (True Negatives) are the numbers of correctly identified positive and negative instances, respectively, and Total represents the sum of all instances.
* **Precision**: Defined as the ratio of correctly predicted positive instances to the total predicted as positive, Precision indicates the model's ability to avoid false positives as (20): \[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}\] (20) FP (False Positives) denotes instances incorrectly classified as positive.
* **Recall**: Recall, or sensitivity, measures the proportion of actual positives that are correctly identified and is given by (21): \[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}\] (21) FN (False Negatives) are actual positives that were incorrectly classified as negative.
* **F1 Score**: The F1 score is the harmonic mean of Precision and Recall, providing a balance between them, especially important in the presence of class imbalances as (22): \[\text{F1 Score}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}}\] (22)  
This process involves collecting and merging feature vectors from every vertex and edge in the graph at the final iteration \(F\). In this study, distinct methodologies were applied to each dataset to implement our algorithm. For audio data, DNN is utilized to generate speaker embeddings. This involves extracting features from audio files using the librosa library and then processing them through a multi-layer DNN to obtain embeddings. In contrast, for video data, a CNN is employed for a similar purpose. This process includes the extraction and preprocessing of frames from video files, which are then input into the CNN to derive embeddings. The core aspect of this research lies in the classification of these embeddings from each modality. In all three datasets text, audio, and video embeddings are separated into features (\(X\)) and target values (\(y\)). These are then input into a specially designed Cumulative Attribute Weighted Graph Neural Network (CA-WGNN) model. The model's performance is evaluated using metrics such as accuracy, precision, recall, and F1 score. These metrics collectively offer a comprehensive view of the classifier's performance, taking into account both error types and class distribution in the dataset.