# Go Source Code Vulnerability Detection Method Based on Graph Neural Network
## 4 Methodology
### Semantic Feature Extraction  
(1) Code Representation. To extract contextual semantic features, we first apply lexical analysis to convert the preprocessed Go source code into a sequence of tokens, providing a more fine-grained representation of the source code, as illustrated in Figure 5. We then utilize SpanBERT to embed the tokens extracted from the code slices.  
SpanBERT is an extension and optimization of the BERT model specifically designed to enhance the modeling of span-level semantic information in text, making it particularly suitable for extracting semantic features from complex program code. The model learns relationships between different words in the text and maps each word to a high-dimensional vector that captures both its semantic meaning and contextual information. Words with similar meanings are located closer together in the embedding space. As a self-supervised pre-trained model, SpanBERT introduces a novel span boundary objective (SBO), which strengthens the model's ability to represent span boundaries. This is especially beneficial for identifying structural elements in code, such as function calls and control blocks, which are critical for understanding code semantics. The inclusion of SBO also enables more efficient access to span-level information during fine-tuning, allowing for more comprehensive extraction of both local and global structural features in Go source code. Figure 6 illustrates how SpanBERT extracts and represents features from Go code.  
Figure 5: Go code lexical analysis.  
Figure 6: Examples of SpanBERT pre-trained model extraction and representation.  
Formally, a line of code is decomposed into a token sequence \(X=[x_{1},x_{2},\ldots,x_{n}]\), where each token \(x_{i}\) represents a lexical unit. For each token \(x_{i}\), we integrate positional embeddings and segment embeddings to obtain the final input representation of Go code. The position embedding \(P\) helps the model understand structural elements such as loops and conditional statements, while the segment embedding \(S\) enhances the model's ability to distinguish different code blocks and maintain strong contextual awareness in complex programs:  
\[H^{0}=[x_{1}\oplus P_{1}\oplus S,x_{2}\oplus P_{2}\oplus S,\ldots,x_{n}\oplus P _{n}\oplus S] \tag{4}\]  
Next, the SpanBERT model learns code representations through two pre-training tasks: masked language modeling (MLM) and span boundary objectives (SBOs). After pre-training, the model produces deep bidirectional representations. During forward propagation, the model generates a series of hidden layer states \(H^{l}\). Notably, the Transformer model integrates a self-attention mechanism, allowing it to capture long-range dependencies between tokens.  
\[H^{l}=Transformer(H^{l-1}),l=1,\ldots,L \tag{5}\]  
Ultimately, the final layer output representation of the code line is denoted as \(H^{L}\). After passing through \(L\) layers of Transformer networks, it aggregates the contextual semantic information of the entire Go code slice. The same process is applied to other preprocessed code slices, as described in Section 3.3.  
(2) Semantic Feature Extraction. While various neural network architectures are available for natural language processing (NLP), many suffer from the vanishing gradient (VG) problem, which can lead to ineffective training.To better capture the contextual semantic information in taint propagation chains, we take the final representation \(H^{L}\) of the token sequence X embedded by SpanBERT. We use multiple \(H^{L}\) vectors as input, and employ a Transformer model with a multi-head self-attention mechanism to extract contextual semantic features from the code slice.  
The Transformer model's self-attention mechanism addresses the challenge of long-range dependencies in code while mitigating the vanishing gradient and exploding gradient problems. By attending to different parts of the token sequence simultaneously, the model can effectively learn complex relationships between different code components, improving vulnerability detection accuracy.