## 2 Materials and Methods
### Classification  
Because of its good performance in natural language processing, transformer has been applied to computer vision, time series classification, and prediction tasks. In recent years, it has also shown good effects in the classification of epilepsy EEG [33; 37]. EEG signals contain abundant time, frequency, and spatial information. In this paper, we used transformer networks to analyze and classify the characteristics of epileptic EEG signals in \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Case** & **Gender** & **Age (Years)** & **Number of Channels** & **Number of Seizures** \\ \hline Chb01 & F & 11 & 23 & 7 \\ Chb02 & M & 11 & 23 & 3 \\ Chb03 & F & 14 & 23 & 7 \\ Chb04 & M & 22 & 23 & 4 \\ Chb05 & F & 7 & 23 & 5 \\ Chb06 & F & 1.5 & 23 & 10 \\ Chb07 & F & 14.5 & 23 & 3 \\ Chb08 & M & 3.5 & 23 & 5 \\ Chb09 & F & 10 & 23 & 4 \\ Chb10 & M & 3 & 23 & 7 \\ Chb11 & F & 12 & 23 & 3 \\ Chb12 & F & 2 & 18 & 40 \\ Chb13 & F & 3 & 23 & 12 \\ Chb14 & F & 9 & 18 & 8 \\ Chb15 & M & 16 & 18 & 20 \\ Chb16 & F & 7 & 18 & 10 \\ Chb17 & F & 12 & 18 & 3 \\ Chb18 & F & 18 & 18 & 6 \\ Chb19 & F & 19 & 23 & 3 \\ Chb20 & F & 6 & 23 & 8 \\ Chb21 & F & 13 & 23 & 4 \\ Chb22 & F & 9 & 23 & 3 \\ Chb23 & F & 6 & 23 & 7 \\ Chb24 & - & - & 23 & 16 \\ \hline \hline \end{tabular}  
* CHB-MIT: Children’s Hospital Boston and the Massachusetts Institute of Technology.  
\end{table} 
Table 1: Details of the CHB-MIT 1 database. three dimensions. The overall architecture of the network model proposed in this paper is shown in Figure 1.  
The traditional transformer is used in the processing of text sequence, and the text data (L, D) is generally two-dimensional, where L and D represents the length and dimension of the word vector, respectively. However, after time-frequency domain transformation, the EEG segment has a three-dimensional matrix (\(C\), \(F\), \(T\)), in which \(C\), \(F\), and \(T\) represent the channel, frequency, and time step of the EEG signal, respectively. We flatten the 3D matrix \(x\in\mathbb{R}^{C\times F\times T}\) from three different dimensions into two dimensions to obtain \(x_{c}\in\mathbb{R}^{C\times(F.T)}\), \(x_{f}\in\mathbb{R}^{F\times(C.T)}\), \(x_{s}\in\mathbb{R}^{T\times(C.F)}\). These three matrices are taken as the three inputs for the model. Because the three inputs are continuous, the embedding layer of the model is replaced by a full connection layer. To the best of our knowledge, the order of each channel in the EEG sequence has no absolute or relative correlation, so we only added positional encoding on the basis of frequency-wise and step-wise input embeddings. Then, three sets of encoders were used to capture the correlation of each dimension sequence from the aspects of step, frequency, and channel information. Each layer of the encoder has two operations, namely feed forward and multi-head attention composed of multiple self-attention. The inputs of the self-attention mechanism are \(Q\) (query), \(K\) (key), and \(V\) (values), and the calculation formula for its output matrix is shown as follows:  
\[Attention(Q,K,V)=softmax\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V \tag{2}\]  
The multi-head attention mechanism further improves the self-attention layer, expands the ability of the model to focus on different positions, and gives multiple \"representation subspaces\" in the attention layer, which is expressed as follows:  
\[MultiHead(Q,K,V)=Concat(head_{1},\dots,head_{m})\text{,}where:head_{i}=Attention \left(QW_{i}{}^{Q}KW_{i}{}^{K},VW_{i}{}^{V}\right) \tag{3}\]  
Figure 1: The model architecture of three-tower transformer model. where \(m\) is the number of attention heads, and \(W_{i}{}^{Q}\), \(W_{i}{}^{K}\) and \(W_{i}{}^{V}\) are the learned projection matrices. The output matrix of the multi-head attention layer is transferred into a feed-forward neural network to enhance feature extraction. Then, the model adds a gating mechanism to integrate the characteristics of time, frequency, and channel direction.