# Text Classification Model Based on Graph Attention Networks and Adversarial Training
## 2 Materials and Methods
### Model Framework
#### 2.1.2 Graph Attention Layer
This assignment explicitly connects characters that are adjacent within the text, facilitating the capture of local contextual information. Additionally, if a character n matches the starting or ending character of a vocabulary item m, the adjacency matrix L entry (n, m) is also assigned a value of 1. This linkage captures not just the adjacency but also the boundary alignment between characters and vocabulary terms. By mapping these relationships, the WCLGN can effectively integrate and represent both the direct context and the boundary information of vocabulary items, enhancing the model's capability to understand and process textual data comprehensively. This dual capture mechanism ensures that the text classification model can leverage both local and broader semantic cues efficiently. Figure 3: Word–Character Transition Graph Network. Figure 2: Word–Character Containing Graph Network. In the construction of the model using three graph attention network layers, each j-th layer of a graph attention network (GAT) receives an input set of node features \(NF^{j}=\{\mathsf{f}_{1},\mathsf{f}_{2},\ldots,\mathsf{f}_{N}\}\) along with an adjacency matrix \(A\in\mathbb{R}^{N\times N}\), where N denotes the number of nodes and F represents the dimensionality of the features at the j-th layer. The output of the j-th layer is a new set of node features \(NF^{(j+1)}=\{\mathsf{f}^{\prime}_{1},\mathsf{f}^{\prime}_{2},\ldots,\mathsf{f }^{\prime}_{N}\}\). A GAT employs K independent attention mechanisms, each of which computes the importance of node i's features to node j's features. The attention mechanism in GAT is used to weigh the influence of each node's features on each other based on the structure specified by the adjacency matrix \(A\). The computation of the new feature for each node involves aggregating features from its neighborhood, weighted by attention scores. The attention scores are computed as follows:  
1. Linear Transformation: Each node feature \(f_{j}\) is first transformed by a weight matrix \(W\), which is commonly shared across the network but specific to each attention head. This step projects the features into a space where attention coefficients can be more effectively learned:  
\[f^{\prime}_{i}=\parallel^{K}_{k=1}\sigma\Big{(}\sum_{j\in\mathcal{N}_{i}}\ \alpha^{k}_{ij}W^{k}f_{j}\Big{)}, \tag{7}\]  
2. Attention Coefficient Calculation: The attention coefficients that indicate the importance of node \(j\)'s features for node \(i\) are calculated using a pairwise attention mechanism on the transformed features. Typically, this involves a nonlinear transformation such as the softmax function applied to a linear combination of features:  
\[\alpha^{k}_{ij}=\frac{exp\Big{(}LeakyReLU\big{(}a^{T}\Big{[}W^{k}f_{i}\parallel W ^{K}f_{j}\Big{]}\Big{)}\Big{)}}{\sum_{k\in\mathcal{N}_{i}}exp\big{(}LeakyReLU \big{(}a^{T}\big{[}W^{k}f_{i}\parallel W^{K}f_{k}\big{]}\big{)}\big{)}}, \tag{8}\]  
where \(a\) is a weight vector, \(\parallel\) denotes concatenation, and \(\mathcal{N}_{i}\) includes node \(i\) and its neighbors as specified by A. 3. Feature Update: The new features for each node are then computed as a weighted sum of the transformed features of the neighboring nodes, scaled by the computed attention coefficients:  
\[f^{final}_{i}=\sigma\bigg{(}\frac{1}{K}{\sum_{k=1}^{K}\ \sum_{j\in\mathcal{N}_{i}} \alpha^{k}_{ij}W^{k}f_{j}\bigg{)}}, \tag{9}\]  
where \(\sigma\) is an activation function. Figure 4: Word–Character Lattice Graph Network. This framework enables each node to dynamically adjust the influence of its neighboring nodes' features based on the overall structure of the graph, thereby effectively capturing both local and global structural information in the feature updates. This process is repeated for \(K\) attention heads, and the results can be averaged, depending on the specific architecture, to form the final output features for each node.