## 2 Materials and Methods
### Capsule-Transformer
#### 2.3.2 Emotion Capsules  
This module, based on a dynamic routing mechanism, merges features from different brain regions and frequency bands into a single feature vector for emotion classification. The computational process of the dynamic routing mechanism is illustrated in Figure 3.  
Firstly, the output of the above Transformer module is taken as the primary capsule \({f^{*}}_{i}\in R^{D_{p}}\). Since the primary capsules do not contain information about their relative positions to each other, the matrix \(W_{ij}\in R^{Dc\times Dp}\) is needed to record the mapping relationship from the primary capsules to the emotion capsules, where \(D_{c}\) denotes the dimension of the emotion capsules, \(D_{p}\) denotes the dimension of the primary capsules, and \({f^{*}}_{j|i}\in R^{Dc}(1\leq i\leq m,1\leq j\leq n,m=16,n=2)\), as shown in Equation (7):  
\[f^{*}_{j|i}=W_{ij}f^{*}_{i} \tag{7}\]  
Figure 3: The architecture of dynamic routing mechanisms.  
Figure 2: The structure of the multi-head self-attention (MSA).  
Then, since each primary capsule contributes differently to the final emotional state classification, the weight \(b_{ij}\) assigned to each primary capsule has an initial value of 0. The sum of the weights of all the primary capsules is made to be 1 by the \(softmax\) function. Each primary capsule is represented using feature vectors, and therefore vector addition operation is used to obtain the emotion capsules, as shown in Equation (8):  
\[s_{j}=\sum_{i}softmax\big{(}b_{ij}\big{)}f_{j\mid i}^{*} \tag{8}\]  
The emotion capsules use the mode length to represent the category probability, so it is necessary to compress the mode length to between 0 and 1, which is realized by using the activation function \"squashing\", and at the same time improves the nonlinear representation of the model, as shown in Equation (9):  
\[v_{j}=\frac{\left\|s_{j}\right\|^{2}}{1+\left\|s_{j}\right\|^{2}}\frac{s_{j}}{ \left\|s_{j}\right\|} \tag{9}\]  
The weights of the primary capsules are updated by calculating the similarity of the inner product of the primary capsules to the emotion capsules, as shown in Equation (10):  
\[b_{ij}=b_{ij}+v_{j}\cdot f_{ji}^{*} \tag{10}\]  
Finally, the model is converged using an iterative approach, and the overall process is shown in Algorithm 1:  
```
1:\(f_{i}^{*}=Transformer(x_{11},x_{12},...,x_{14}),i=1,...,16\)
2:\(f_{j\mid i}^{*}=W_{ij}\cdot f_{i}^{*}\)
3:for all capsule \(i\) in layer \(l\) and capsule \(j\) in layer (\(l\)+1): \(b_{ij}=0\)
4:for r iteration do
5: for all capsule \(i\) in layer \(l\): \(c_{j}=softmax(b_{i})\)
6: for all capsule \(j\) in layer (\(l\) + 1): \(s_{j}=\sum_{i}c_{ij}f_{j\mid i}^{*}\)
7: for all capsule \(j\) in layer (\(l\) + 1): \(v_{j}=squash(s_{j})\)
8: for all capsule \(i\) in layer \(l\) and capsule \(j\) in layer (\(l\) + 1): \(b_{ij}\gets b_{ij}+f_{j\mid i}^{*}\cdot v_{j}\)
9:\(||v_{j}||\) // L2 norm representing class probability
```  
**Algorithm 1** Capsule-Transformer  
During the model training phase, margin loss is utilized as the loss function, as shown in Equation (11), where \(T_{k}\) represents the one-hot encoding of the labels. When the network predicts correctly, \(m^{+}\) is applied to stretch the output vector. In case of a prediction error, \(m^{-}\) is employed to compress the output vector. \(\lambda\) serves as the balancing coefficient, and \(||v_{k}||\) denotes the magnitude of the output vector. Additionally, in subsequent relevant experiments, cross-entropy is employed as the loss function for the Transformer model, as shown in Equation (12):  
\[L_{k}=T_{k}\max\big{(}0,m^{+}-\left\|v_{k}\right\|\big{)}+\lambda(1-T_{k})\max \big{(}0,\left\|v_{k}\right\|-m^{-}\big{)} \tag{11}\]  
\[H(p,q)=-\sum_{i=1}^{n}p(x_{i})\log(q(x_{i})) \tag{12}\]