# Enhancing Small Language Models for Graph Tasks Through Graph Encoder Integration
## 3 Proposed Method
### SLM Integration for Node Representation  
After encoding the graph structure through the graph encoder, GraphPrompter [6] processes the textual information associated with each node. This stage leverages small language models (SLMs), applying different approaches depending on the tuning method. In prompt tuning, the SLM's parameters remain frozen to preserve its pretrained knowledge, and only the graph and text embeddings are updated. In fine-tuning, the SLM's parameters are fully trainable, allowing the model to adapt its entire architecture to both graph and textual data.  
For each node \(v_{i}\), given its corresponding textual attribute \(T_{i}\) (e.g., the title and abstract of a paper in a citation graph), the text is tokenized using the SLM's tokenizer:  
\[T_{\text{tokens}}=\text{Tokenizer}(T_{i}) \tag{6}\]  
The tokenizer converts raw text into discrete tokens aligned with the SLM's vocabulary. These tokens are then embedded into a continuous vector space through the SLM's embedding layer:  
\[T_{\text{emb}}=\text{Embed}(T_{\text{tokens}})\in\mathbb{R}^{M\times d_{l}} \tag{7}\]  
where \(M\) represents the sequence length and \(d_{l}\) is the hidden dimension of the SLM. This text embedding captures the semantic meaning and contextual information of the node's textual attributes.  
The node embeddings \(\hat{X}^{i}\) produced by the graph encoder are concatenated with the text embeddings \(T_{\text{emb}}\) to form the final representation:  
\[Z^{i}=[\hat{X}^{i},T_{\text{emb}}] \tag{8}\]  
This fusion of graph and textual data enables the model to integrate structural features with semantic context, resulting in richer and more expressive node representations. In prompt tuning, this combination leverages the frozen knowledge of the SLM for rapid adaptation, while in fine-tuning, the entire model is refined to better capture complex node relationships and adapt to new domains.  
The combined embeddings are processed through the self-attention layers of the SLM, allowing the model to simultaneously learn node-to-node similarities, textual correlations, and graph structural relationships. This multi-faceted learning approach enhances the model's ability to capture subtle semantic nuances which may not be evident from the graph structure alone. Nodes with similar textual attributes are naturally clustered, improving the accuracy of node classification and link prediction tasks.  
This approach is particularly effective for heterogeneous data, such as citation networks, where textual attributes are critical for understanding node characteristics. By integrating structural information from the graph with external knowledge embedded in text, the model achieves a deeper understanding of interactions and dependencies, resulting in stronger predictive performance.  
A key distinction between node classification and link prediction lies in the type of information processed. Node classification evaluates the features of individual nodes, while link prediction models the attributes and relationships between pairs of nodes, such as source and target nodes. By embedding both structural and textual data, the proposed framework bridges the gap between semantic and structural understanding, enhancing performance across a wide range of graph-based tasks.