# TER-CA-WGNN: Trimodel Emotion Recognition Using Cumulative Attribute-Weighted Graph Neural Network
## 3 Proposed Approach for Emotion Recognition
### Speaker Embedding
#### 3.2.4 Video Speaker Embedding Using Deep Learning Architectures  
Inspired by neuronal structures in the cerebral cortex, DL architectures have demonstrated efficacy across a variety of machine learning applications. These architectures operate as feed-forward networks, enabling unidirectional information propagation from input to output layers. The design and functionality of these systems draw heavily from biological processes, particularly emulating the hierarchical organization of the visual cortex composed of simple and complex cell layers. Despite variations in specific architectural designs, a common framework exists, comprising a sequence of convolutional and pooling layers, followed by densely connected layers. This design is reminiscent of traditional neural networks. The deep structure of these networks is constructed by layering these elements, often including input, convolutional, pooling, and flattening layers, as depictedin Figure 5[46]. This hierarchical integration of data through these layers culminates in a dense representation, ultimately leading to an output layer that signifies the final processing outcome.  
A CNN or ConvNet is a deep learning architecture designed to learn directly from data. CNNs are specifically well-suited for identifying patterns in images to recognize objects, classes, and categories. Moreover, they prove to be highly effective in classifying audio, time series, and signal data. In dealing with large volumes of training data, CNN often employs a mini-batch approach. Consider a set of activation features \(X=[x_{1},\dots,x_{M}]\) from \(M\) training images, where each feature vector \(x_{i}\) has a dimensionality of \(K\), and \([x_{1},\dots,x_{M}]\in\{-1,1\}^{M}\) stores the ground truth labels. A robust classifier employs a boosting technique to make predictions. The classifier \(H(\cdot)\) is a weighted sum of weak classifiers \(h(\cdot)\), formulated as Equation (8):  
\[H(x_{1})=\sum_{j=1}^{K}\alpha_{j}h(x_{ij},\lambda_{j}); \tag{8}\]  
\[h(x_{ij},\lambda_{j})=\frac{f(x_{ij},\lambda_{j})}{\sqrt{f(x_{ij},\lambda_{j})^ {2}+\eta^{2}}}; \tag{9}\]  
where \(x_{ij}\in x_{j}\) is the \(jth\) activation feature of the \(ith\) image. Each feature \(h(x_{ij},\lambda_{j})\) is associated with a potential weak classifier, producing outputs in the range \((-1,1)\) Equation (9). The function \(\frac{f(\cdot)}{\sqrt{f(\cdot)^{2}+\eta^{2}}}\) simulates a sign method, optimizing gradient descent through derivative computation. In this framework, \(h(x_{ij},\lambda_{j})\in K\) represents a decision stump, a one-level decision tree. The parameter \(\eta\), adjustable by the distribution of \(f(\cdot)\) as \(\sigma/c\), controls the function's ramp. Here, \(\sigma\) is the standard deviation of \(f(\cdot)\), and \(c\) is a constant. In this work, \(\eta\) is empirically set to \(\sigma/v\). The weight of the \(ith\) weak classifier \(\alpha_{j}\geq 0\) follows the constraint Equation (10):  
\[\sum_{j=1}^{K}\alpha_{j}=1; \tag{10}\]  
When \(\alpha_{j}=0\), the corresponding neuron remains inactive in the neural network. The balance between the losses of strong and weak classifiers is given by Equation (11):  
\[\epsilon^{B}=\beta\epsilon^{B}_{strong}+(1-\beta)\epsilon_{weak}; \tag{11}\]  
where \(\beta\in[0,1]\). The losses for strong and weak classifiers are defined as Equations (12) and (13):  
\[\epsilon^{B}_{strong}=\frac{1}{N}\sum_{i=1}^{N}(H(x_{i})-x_{i})^{2}; \tag{12}\]  
Figure 5: Architecture of CNN method.  
\[\epsilon_{weak}=\frac{1}{MN}\sum_{i=1}^{N}\sum_{1\leq j\leq K,\alpha_{j}=0}[h(x_{ ij},\lambda_{j})-y_{i}]^{2}; \tag{13}\]  
where \(\alpha_{j}>0\) ensures that dormant neurons are excluded from the loss calculation.