# Text Classification Model Based on Graph Attention Networks and Adversarial Training
## 2 Materials and Methods
### Model Framework
#### 2.1.1 Subsection Encoding Layer  
In this paper, the model vectorizes text through a word embedding layer, generating input character vectors and input word vectors. During model training, an adversarial training method is incorporated, specifically the Fast Gradient Method (FGM) proposed by [PERSON] et al. [22]. The Fast Gradient Method (FGM) was selected for our adversarial training due to its efficiency and effectiveness in generating adversarial examples. FGM is computationally less intensive compared to other methods such as Projected Gradient Descent (PGD) or Carlini and Wagner (C&W) attacks, allowing for faster training times without significantly compromising robustness. Additionally, FGM's simplicity and strong performance in improving model generalization and resilience against adversarial attacks make it a preferred choice. By perturbing the input data with the gradient of the loss function, FGM effectively exposes the model to a variety of adversarial scenarios, enhancing its ability to generalize to unseen examples. The standard format of adversarial training is as follows:  
\[\underset{\theta}{min}\ \mathbb{E}_{(x,y)\sim\mathcal{D}}\left[\underset{Ax\in \Omega}{max}\ L(x+\Delta x,y;\theta)\right], \tag{1}\]  
In the formalism, \(\mathcal{D}\) represents the training dataset, \(x\) denotes the input, \(y\) is the label, \(\theta\) is the model parameters, and \(L(x,y;\theta)\) is the loss for a single sample. \(\Delta x\) is the adversarial perturbation, and \(\Omega\) is the space of perturbations. A common constraint is \(\|\Delta x\|\leq\epsilon\), where \(\epsilon\) is a constant. For each sample, an adversarial example \(x+\Delta x\) is constructed, and the pair \((x+\Delta x,y)\) is used to minimize the loss and update the parameters \(\theta\) via gradient descent. The adversarial perturbation \(\Delta x\) used is as follows:  
\[\Delta x=\epsilon\
abla_{x}L(x,y;\theta), \tag{2}\]  
For \(\
abla_{x}L(x,y;\theta)\), standardize the following:  
\[\Delta x=\epsilon\frac{\
abla_{x}L(x,y;\theta)}{\|\
abla_{x}L(x,y;\theta)\ \|}, \tag{3}\]  
In our model, TextCNN is employed to extract features from the input text. TextCNN utilizes three convolutional layers with kernel sizes of three, five, and seven, respectively. Each convolutional layer considers a domain of the same size as its kernel, allowing the output to perceive a broader dimension of the input. The model's input is a sentence along with all the vocabulary of the contiguous sub-sequences matching the current sentence. We represent the input sentence as \(s=\{c_{1},c_{2},\dots,c_{n}\}\), where \(c_{i}\) denotes the i-th character, and the vocabulary matched to the sentence is represented as \(l=\{l_{1},l_{2},\dots,l_{m}\}\). Each character \(c_{i}\) is represented by a vector, denoted as \(\mathbf{x_{i}}\), obtained by looking up the pre-trained character embedding matrix, where \(e^{c}\) is a lookup table for character embeddings. \[\mathbf{x_{i}}=e^{c}(c_{i}) \tag{4}\]  
By employing TextCNN on the sequence \(\{\chi_{1},\chi_{2},\dots,\chi_{n}\}\), we extract features, resulting in a sentence representation \(H=\{\text{h}_{1},\text{h}_{2},\dots,\text{h}_{n}\}\), where each \(h_{i}\) is a feature vector corresponding to the i-th position in the input sequence. To represent the semantic information of characters, we retrieve word embeddings from a pre-trained word embedding matrix. Each vocabulary item \(l_{i}\) is represented as a semantic vector, denoted by \(wv_{i}\), where \(e^{w}\) is a lookup table for word embeddings. This representation facilitates a richer semantic understanding of the text, integrating both character-level and word-level information. \[wv_{i}=e^{w}(l_{i}) \tag{5}\]  
The sentence representation matrix \(H\) and the word embedding matrix \(wv_{i}\) are concatenated to form the output of the encoding layer. This combined matrix is denoted as \(Node_{f}\), serving as a comprehensive feature set that captures both the contextual cues fromthe sentence and the semantic attributes of the individual words.