HEAL : Hybrid Enhancement with LLM -based Agents for Text-attributed Hypergraph Self-supervised Representation Learning (2025)

Abstract: This paper studies the problem of text-attributed hypergraph self-supervised representation learning, which aims to generate discriminative representations of hypergraphs without any annotations for downstream tasks. However, real-world hypergraphs could contain incomplete signals, which could deteriorate the representation learning procedure, especially under label scarcity. Towards this end, we introduce a new perspective that leverages large language models to enhance hypergraph self-supervised learning and propose a novel data-centric approach named Hybrid Hypergraph Enhancement with LLM-based Agents (HEAL). The core of our HEAL is to generate informative nodes and hyperedges through multi-round interaction with LLM-based agents. In particular, we first retrieve similar samples for each node to facilitate the node expansion agent for different views. To generate challenging samples, we measure the gradients for each augmented view and select the most informative one using an evaluation agent. From the structural view, we adopt a topology refinement agent to incorporate new hyperedges for the recovery of missing structural signals. The enhanced hypergraphs would be incorporated into a self-supervised learning framework for discriminative representations. Extensive experiments on several datasets validate the effectiveness of our HEAL in comparison with extensive baselines.

## introduction in recent years, large language models (llms) have raised a surge of interest in natural language processing (nlp) @xcite @xcite , and have demonstrated remarkable capabilities in multiple tasks. due to extensive pre-training, llms could serve as a powerful tool to capture comprehensive linguistic † corresponding authors. patterns and real-world knowledge. subseqently, llms have promoted the development of several domains, such as text generation, information extraction, and semantic understanding, through fewshot and zero-shot learning. in the relation mining field, these advantages enable llms to excel at automatically identifying and extracting implicit relationships between entities from unstructured texts @xcite . unlike traditional rule-based and machine learning approaches, llms have the capability of comprehensively combining in-context information with real-world knowledge to infer latent relationships, significantly enhancing accuracy and robustness. consequently, researchers widely employ llms for relation mining, facilitating downstream tasks such as recommender systems and financial analysis. hypergraphs, as an effective data structure, are ubiquitous across various domains, including social network analysis, recommender systems, and bioinformatics @xcite . unlike traditional graphs, hypergraphs are characterized by representing complex higher-order dependencies and asymmetric relationships among nodes. with the prevalence of graph neural networks (gnns) on graph data processing, hypergraph neural networks (hypergnns) @xcite have emerged as an essential focus for hypergraph representation learning. hypergnns could effectively learn node representations through propagating information in the topological structure. especially, in the selfsupervised learning field, hypergraph contrastive learning (hypergcl) @xcite has been proven to be a powerful framework via constructing contrastive views to maximize the similarity of homogeneous nodes and minimize the distance between heterogeneous nodes' representations. despite the remarkable achievements of existing hypergraph self-supervised representation learning, they exhibit notable limitations when employed to process hypergraphs with textual description, often referred to as text-attributed hypergraphs (tahgs) @xcite , as depicted in figure 1 . firstly, it matters to balance the relationship between textual information and hypergraph topological structure. hypergraphs are often applied to model complicated high-order relationships, whereas textual data typically exhibits sequential or hierarchical structures. due to different data structures, simply utilizing textual information to derive hyperedges is prone to destroying the original hypergraph structure, resulting in a counterproductive situation. secondly, augmenting nodes' text properly constitutes another crucial barrier. although llms excel at text generation, they are prone to hallucination @xcite , generating plausible yet nonfactual content. these shortcomings would directly impact the quality of constructed hyperedges. typically, when it comes to excessively large hypergraphs, text processing for each node would incur overwhelming time consumption and computational costs. thirdly, how to construct optimal hyperedges remains a difficult challenge. given the particularly severe hyperedge sparsity problem, recent works have started to investigate effective frameworks to tackle it, such as k-hyperedge @xcite and cash @xcite . however, all of these approaches, without exception, focus solely on initial embeddings to derive better hyperedges. although they may reasonably perform well, their effectiveness is inherently deteriorated by the failure to harness available textual resources. to address these challenges, in this paper, we propose heal, a novel llm-based hybrid hypergraph enhancement framework for selfunsupervised representation learning on hypergraphs. unlike existing approaches, such as kgc with llms that complete predefined pairwise relations, our heal framework discovers higher-order structures in open-world settings. it also goes beyond local textual enhancement in llm-gnns by reconstructing global graph topology in a semanticaware manner, enabling both fine-grained node understanding and holistic structural modeling. to address the first challenge, we provide llms with connectivity-based signals of the target node to improve hyperedge prediction. for large hypergraphs, we adopt a two-stage sampling strategy: nodes are first grouped by label, then high-degree nodes are prioritized. for the second challenge, we use knn to retrieve similar texts for the central node, enriching its textual input with semantically aligned yet diverse augmentations. to select the most informative one, we apply a gcn with contrastive loss to guide the llm via gradient-based evaluation. to tackle the last challenge, we design a topology refinement agent that recovers hyperedges with minimal structural noise. instead of modeling full hyperedge sets, the llm predicts pairwise connectivity between the current node and selected candidates. overall, our main contributions are as follows: figure 2 : high-level pipeline of our proposed heal.this pipeline comprises two major processes: text augmentation and edge augmentation. initially, we employ a bert-based llm as a text encoder to provide informative node texts for augmentation preprocessing. (1) text augmentation: for more precise text augmentation with llm, we utilize k-nearest neighbors (knn) to retrieve n similar texts as supplementary knowledge to alleviate the hallucination issue. for each node, to evaluate n augmented texts generated by llm, we employ a graph convolutional network (gcn) to generate n+1 corresponding gradient messages(including one original text and m augmented ones). (2) edge augmentation: in order to refine hyperedges simply yet effectively, we leverage llm to judge the connectivity of pairwise edges first, then reconstruct an optimized hypergraph. ## conclusion in this work, we propose a novel framework, heal, a hybrid llm-based agents to boost hypergraph representation learning in a self-supervised domain. by leveraging the text generation capability and pre-trained knowledge of llm, we ingeniously refine the topological structure of hypergraphs at minimal cost. in particular, we introduce three specific agents for textual-level augmentation and structural-level refinement, namely the retrieval-augmented node expansion agent, the gradient-based evaluation agent, and the topology refinement agent. our evaluation demonstrates that the refined hypergraph structure could advance hypergraph representation learning effectively. these findings indicate the potential of llm-based agents to tackle severe data sparsity issue in hypergraphs.