# Enhancing Small Language Models for Graph Tasks Through Graph Encoder Integration
## 4 Experiments
### Main Results  
The experimental results for node classification and link prediction tasks on the Citeseer and Ogbn-arxiv datasets are summarized in Table 1. We evaluated SmolLM2-360M-Instruct using four tuning methods: prompt tuning, fine-tuning, prompt tuning (with GCN-T), and fine-tuning (with GCN-T). Additionally, we compared these resultswith those of the larger Llama-2-7B model, including its prompt tuning (with GCN) and fine-tuning (with GCN) settings. The results show that integrating GCN-T significantly improved the performance of the small language models (SLMs). For node classification, smolLM2-360M-Instruct with fine-tuning (with GCN-T) achieved an accuracy of 73.15% on Citeseer and 68.12% on Ogbn-arxiv, outperforming standard fine-tuning by 2.26% and 0.38%, respectively. Similarly, prompt tuning (with GCN-T) improved upon standard prompt tuning, yielding a 19.01% increase on Citeseer and a 25.84% increase on Ogbn-arxiv. For link prediction, the impact of integrating GCN-T was even more evident. Fine-tuning (with GCN-T) achieved an accuracy of 84.16% on Citeseer and 80.19% on Ogbn-arxiv, significantly surpassing both the standard fine-tuning and prompt tuning results. Similarly, prompt tuning (with GCN-T) showed substantial improvements over baseline prompt tuning, highlighting the importance of graph-aware encoding for modeling node relationships and improving performance. The larger Llama-2-7B model consistently outperformed SmolLM2-360M-Instruct across all tasks, leveraging its higher parameter count. Notably, Llama-2-7B with prompt tuning (with a GCN) achieved the highest link prediction accuracy, reaching 93.49% on Citeseer and 75.28% on Ogbn-arxiv. Fine-tuning (with a GCN) also delivered strong results, reaching 91.67% on Citeseer and 73.21% on Ogbn-arxiv. These results confirm the effectiveness of graph integration in enhancing the performance of larger models. However, SmolLM2-360M-Instruct with GCN-T integration narrowed the performance gap with Llama-2-7B, especially for tasks requiring efficient parameter usage. For example, the link prediction accuracy of SmolLM2-360M-Instruct with fine-tuning (with GCN-T) on Citeseer was within 7.5% of Llama-2-7B's fine-tuning (with GCN) result, demonstrating the competitiveness of smaller models with structural enhancements. These findings highlight the effectiveness of the proposed graph encoder in bridging the performance gap between small and large language models. By enabling small models to achieve competitive results with fewer parameters, this approach provides a scalable and efficient solution for graph-related tasks, particularly in resource-constrained environments. \begin{table}
\begin{tabular}{l l c c c c} \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Method**} & \multicolumn{3}{c}{**Node Classification**} & \multicolumn{2}{c}{**Link Prediction**} \\ \cline{3-6}  & & **Citeseer** & **Ogbn-Arxiv** & **Citeseer** & **Ogbn-Arxiv** \\ \hline \multirow{3}{*}{smolLM2-360M-Instruct} & Prompt tuning & 50.67 & 26.53 & 46.00 & 40.36 \\  & Fine-tuning & 70.89 & 67.74 & 66.52 & 76.30 \\  & Prompt tuning (with GCN-T) & 69.68 & 52.37 & 64.86 & 70.08 \\  & Fine-tuning (with GCN-T) & 73.15 & 68.12 & 84.16 & 80.19 \\ \hline \multirow{3}{*}{Llama-2-7B} & Prompt tuning & 70.97 & 71.99 & 88.87 & 69.13 \\  & Fine-tuning & 73.45 & 74.58 & 87.91 & 70.10 \\ \cline{1-1}  & Prompt tuning (with GCN) & 72.29 & 75.04 & 93.49 & 75.28 \\ \cline{1-1}  & Fine-tuning (with GCN) & 73.61 & 75.61 & 91.67 & 73.21 \\ \hline \end{tabular}
\end{table} 
Table 1: Performance comparison of SmolLM2-360M-Instruct and Llama-2-7B in node classification and link prediction tasks using Citeseer and Ogbn-arxiv datasets. SmolLM2-360M-Instruct showed significant improvements in both the prompt tuning and fine-tuning scenarios when GCN-T was applied, narrowing the performance gap with Llama-2-7B.