# Enhancing Small Language Models for Graph Tasks Through Graph Encoder Integration
## 5 Conclusions  
This paper proposed a novel approach based on a graph encoder to address the performance limitations of small language models (SLMs) for graph-related tasks. On-device LMs, which rely heavily on auto-regressive learning methods, face significant challenges when processing structured graph data. Our method enables these models to effectively capture the relational and structural characteristics of graph data while adhering to the computational constraints of on-device environments. The experimental results demonstrated improved performance in graph-related tasks, validating the effectiveness of our approach. However, the proposed method has certain limitations. The graph encoder primarily focuses on relationships with adjacent nodes, limiting its ability to perform multi-hop reasoning across distant nodes in the graph. Additionally, the learning and inference processes lack sufficient interpretability, making it challenging to understand how the model leverages graph structures to reach its decisions. To address this, we plan to explore techniques like attention maps or feature attribution methods, which can show what parts of the graph the model focuses on when making predictions. This would help make the model's reasoning more understandable and transparent. In future work, we aim to enhance the multi-hop reasoning capabilities of the graph encoder to support more complex relational inference across the graph. We also plan to generalize our method by testing it in a broader range of tasks, such as node classification on unseen datasets and link prediction under varying levels of noise, to evaluate its adaptability. To provide a comprehensive evaluation of our model, we will incorporate additional metrics, such as the F1 score and model robustness, alongside accuracy and computational efficiency, offering deeper insights into its strengths and trade-offs. Furthermore, we will extend our experiments beyond synthetic graph tasks by incorporating real-world graphs into our evaluation. This would allow us to explore the potential of the proposed method in practical applications, such as answering user questions in multi-hop QA and dialog tasks. By addressing these limitations and expanding the scope of our experiments, we hope to extend the applicability of on-device models to a wider range of graph-based applications while ensuring they remain efficient, accurate, and interpretable in resource-constrained environments. Conceptualization, D.O. ([PERSON]) and D.O. ([PERSON]); methodology, D.O. ([PERSON]) and D.O. ([PERSON]); software, D.O. ([PERSON]) and D.O. ([PERSON]); validation, S.K. and H.K. ; formal analysis, S.K. and H.K. ; investigation, D.O. ([PERSON]), S.K. and H.K. ; resources, S.K. and H.K. ; data curation, D.O. ([PERSON]); writing--original draft preparation, D.O. ([PERSON]), S.K., H.K. and D.O. ([PERSON]); writing--review and editing, D.O. ([PERSON]), S.K., H.K. and D.O. ([PERSON]); visualization, D.O. ([PERSON]); supervision, D.O. ([PERSON]); project administration, D.O. ([PERSON]); funding acquisition, S.K., H.K. and D.O. ([PERSON]). All authors have read and agreed to the published version of the manuscript. This research was supported by the Kyungpook National University Research Fund, 2022. **Institutional Review Board Statement**: Not applicable. **Informed Consent Statement**: The requirement for individual informed consent was waived as all data were fully anonymized, with no identifiable personal health data. **Data Availability Statement**: We conducted our experiments using the data provided in [PERSON] et al. [6]. For detailed information regarding the dataset and its usage, please refer to the original publication. **Acknowledgments:** This research was conducted as part of a collaborative effort between humanities and engineering scholars, reflecting the interdisciplinary nature of the project. **Conflicts of Interest:** [PERSON] is an employee of the Hyundai Motors Company.