Out-of-Distribution Detection via LLM -Guided Outlier Generation for Text-attributed Graph (2025)

Abstract: Text-Attributed Graphs (TAGs), which are characterized with text attributes, are widely used in the real world. When evaluating fully trained models designed for TAG predictions, they may perform significantly unsatisfactory on samples outside the In-Distribution (ID) data, which may raise serious security issues. To tackle it, Out-Of-Distribution (OOD) detection is introduced to the TAGs field, which aims to utilize a detector to classify OOD and ID samples. Recent studies attempt to introduce extra OOD datasets to regularize the detection model. However, due to the vastness of the OOD data space, high-quality OOD samples for training the detector are scarce and difficult to obtain in the real world. Thus, we utilize Large Language Models (LLMs) to generate the OOD training samples with high quality. There are two issues in this process: (1) LLMs tend to generate OOD-node samples significantly different from ID ones, with a limited learning value for OOD and ID relations. (2) Due to the inherent structure of TAGs, obtained OOD nodes need to be integrated with existing nodes by generating edges using LLMs. However, the large number of nodes makes reasoning over each node pair computationally unbearable. Toward these issues, we introduce LLMGuard with challenging OOD-node generation and lightweight edge predictors. Extensive experiments prove the effectiveness of LLMGuard. The source code is available.

## introduction text-attributed graphs (tags), whose graph structures are characterized with text attributes, are commonly applied in various real-world scenarios @xcite . within text-attributed graphs, graph nodes capture entities figure 1 : differences between our method and traditional methods to detect node-level ood data. our key idea is to leverage llms to generate challenging ood nodes and efficiently structure them to facilitate the formation of a tighter decision boundary. with textual information and graph edges encapsulate relationships between different entities, such as social graphs where each user node is attributed with a textual description @xcite and paper citation graphs in which language content is linked to each paper node @xcite . despite significant progress, directly transferring fully trained tag models to samples from unknown distributions may lead to a sharp drop in performance, thereby reducing the prediction reliability and raising security issues @xcite . to tackle this issue, the outof-distribution (ood) detection @xcite is introduced to the field of text-attributed graphs. it aims to utilize a detector to identify graph ood data and separate it with in distribution (id) data to reduce the risk of unreliable predictions. the widely used solution @xcite for distinguishing ood and id text-attributed graphs predominantly relies on training the detector with id samples and naturally collected ood ones. however, due to the vast diversity of the ood text-attributed graphs @xcite , there may be an extremely low per-centage of the naturally collected graphs exactly located near the classification boundary. the textual node properties and topological graph structures of these collected samples often exhibit substantial semantic differences from existing id text-attributed graphs. consequently, these ood samples are considered low-quality because they are too easy to distinguish and offer limited value in enhancing the graph detector's ability to identify more challenging ood graphs. to establish a more precise decision boundary, it's crucial to focus on highquality ood graphs sharing similar patterns with id ones yet differ fundamentally. to acquire more ood text-attributed graphs with high value, we attempt to utilize the pretrained large language models (llms) to generate the qualified samples. in this synthesis process, two major issues need to be considered: (1) low-quality ood node generation. the ood nodes directly generated by large language models typically belong to classes that are noticeably distinct from those of the id nodes. as a result, these ood nodes are positioned far from the decision boundary separating ood and id nodes, thereby diminishing their values for training the detection model. even when ood nodes belong to id-related classes, the specific textual descriptions associated with these directly generated ood nodes remain disconnected from the existing id nodes, which is another serious damage to their training value. it is crucial to consider strategies that can effectively prompt large language models to generate ood nodes of genuinely high quality. (2) costly edge completion. due to the node interdependence of graphs, newly generated ood nodes need to be associated with existing id nodes. the inherent differences between ood and id nodes make it impractical to directly infer ood edges using a predictor trained solely on id nodes. one possible approach is to leverage pre-trained large language models for direct edge prediction between node pairs. however, due to the presence of numerous nodes in text-attributed graphs, this approach often requires an excessive number of calls to large language models for each pair of nodes, leading to an unbearable computational burden. to address these issues, we propose an effective and efficient framework via llm-guided graph outlier generation for ood detection in textattributed graphs, called llmguard. the key idea is to leverage large language models to generate challenging ood nodes that facilitate learning a clear decision boundary while distilling the edge prediction capabilities of large language models into lightweight modules for efficient edge generation. this consists of two key processes. process 1: we employ pre-trained large language models, guided by potential id-related classes of ood nodes and textual content of id subgraphs, to generate high-quality ood nodes. these obtained ood nodes may be highly correlated with existing id nodes while still preserving ood characteristics. process 2: we first construct training data for edge prediction using a limited number of llm calls to identify relationships between generated ood nodes and id nodes. to better capture semantic differences between id and ood nodes, we fine-tune a local llm on the node distinction task to obtain informative node representations. leveraging both the edge training data and node representations from the local llm, we further train two lightweight edge predictors to distill the llm's ability for edge prediction. ## experiments to assess the effectiveness of our proposed llm-guard model, we conduct extensive experiments aimed at addressing the following key research questions: ## limitations in this section, we analyze the current limitations of our approach. as an early exploration of leveraging llms to generate high-quality ood graph data, certain modules in our framework can be further refined. in ood node generation, for efficiency reasons, we primarily use direct node information as background knowledge, which may limit contextual richness. additionally, in constructing the ood-ood strength dataset, exploring more advanced retrieval methods could further improve the quality of ood-ood strength dataset.