Inductive Learning on Heterogeneous Graphs Enhanced by LLM s for Software Mention Detection (2025)

Abstract: This paper explores the synergy between Knowledge Graphs (KGs), Graph Machine Learning (Graph ML), and Large Language Models (LLMs) for multilingual Named Entity Recognition (NER) and Relation Extraction (RE), specifically targeting software mentions within the SOMD 2025 challenge. We propose a methodology where documents are first transformed into heterogeneous KGs enriched with linguistic features (Universal Dependencies) and external knowledge (entity linking). An inductive GraphSAGE model, operating on PyTorch Geometric’s ‘HeteroData‘ structure with dynamically generated multilingual embeddings, performs node classification tasks. For NER, Graph ML identifies candidate entities and types, with an LLM (DeepSeek v3) acting as a validation layer. For RE, Graph ML predicts dependency path convergence points indicative of relations, while the LLM classifies the relation type and direction based on entity context. Our results demonstrate the potential of this hybrid approach, showing significant performance gains post-competition (NER Phase 2 Macro F1 improved to 0.4364 from 0.2953, RE Phase 1 0.3355 Macro F1), which are already described in this paper, and highlighting the benefits of integrating structured graph learning with LLM reasoning for information extraction.

## introduction advancing the capabilities of natural language processing (nlp) often requires moving beyond the surface level of plain text to leverage richer, more structured information. while raw text provides the foundation, incorporating details like semantic relationships, external world knowledge, or task-specific metadata can significantly boost performance on complex understanding @xcite . this necessity, however, introduces a significant challenge: developing frameworks capable of seamlessly handling diverse data formats and integrating multiple layers of annotations -ranging from word-level tags (like partof-speech) to sentence-level labels (like sentiment) and document-level classifications (like topic). kgs provide a notably flexible and powerful paradigm to address this complexity. by representing information as nodes (entities, concepts) and edges (relationships), kgs offer an inherently structured way to capture intricate connections within and beyond the text. this structure facilitates the coherent integration of various annotation types across different textual granularities, ensuring that, for instance, word-level syntactic information can coexist and relate to document-level semantic themes. crucially, kgs excel at maintaining the explicit connections between linguistic units and associated knowledge, preserving context that might be lost in purely sequential models. furthermore, the kg paradigm benefits from a mature and growing ecosystem of established standards, databases, and software tools for creation, querying, and reasoning. despite the clear advantages offered by kgs for representing rich, multi-level information, the recent trajectory of mainstream nlp research has largely centered on models that process raw text sequences directly. this past decade has been marked by significant breakthroughs: the fundamental contribution of distributional representations via word embeddings @xcite , the development of powerful sequential models like bi-lstms @xcite , arguably the most impactful release with the transformer architecture @xcite , followed by large pre-trained models such as bert @xcite and bart @xcite , and more recently, generative ai and llms such as chatgpt @xcite . although these methods have pushed the state-ofthe-art by learning complex patterns from vast textual data, their primary focus on sequential text input means the potential synergies of explicitly integrating structured knowledge, as offered by kgs, remain relatively underexplored in many application areas. the use of graphs has been previously demonstrated in adjacent fields, such as open information extraction, particularly for the chinese language, where graph-based approaches have yielded favorable results @xcite . initial research has also investigated the potential of integrating graphs and llms. the study by @xcite examines the role of graphs as both enhancers and predictors. an example of the combined capabilities of graphs and llms is microsoft graphrag @xcite , which aims to leverage their synergistic effects. the research presented here is part of the 2025 software mention detection (somd) competition. the primary contribution of this work is a multilingual ner and re system that utilizes kgs, graph ml, and llms. ## method in this section, the methodology employed is described. we will outline the overall approach, describe the processing of the dataset and finally how to generate our predictions for both ner and re. ## results this section presents the results of our training on the validation set and the performance on the test set for both phases of the competition. every run was documented using the weights & biases platform @xcite . ## conclusion although the results in the competition were not optimal compared to those of other participants, we have made significant improvements to the model during the ongoing open phase which are here presented, enhancing its competitiveness on the test set. in the ner task, the phase 2 results yielded a macro f1 score of 29.5% and a micro f1 score of 37.4%. in this open submission phase, the scores have improved to 43.6% and 58%, respectively, increasing their competitiveness. unfortunately, for re, we were unable to obtain results for phase 2 in a timely manner. thus, the phase 1 results were as follows: 33.6% f1-score, 38.4% precision and 32.1% recall. with the improvements to the entity model, these numbers are expected to be even better. our model possesses several advantages, including being lightweight, easily adaptable to other problems (as no specific code or preprocessing was conducted for this dataset), and supporting multiple languages (as long as a parser is available). despite the promising methodology and its positive aspects, considerable work remains to enhance these results. the work here developed is available at: https://github.com/ gabrielrsilva11/somd2025 . future work -incorporate additional external knowledge into our graph to enhance its ability to generalize knowledge. furthermore, integrate dbpedia @xcite into our kg. section prompt details target activity you are an intelligent assistant that helps a human analyst to identify relations between entities in a sentence.