To Answer or Not to Answer ( TAONA ): A Robust Textual Graph Understanding and Question Answering Approach (2025)

Abstract: Recently, textual graph-based retrieval-augmented generation (GraphRAG) has gained popularity for addressing hallucinations in large language models when answering domain-specific questions. Most existing studies assume that generated answers should comprehensively integrate all relevant information from the textual graph. However, this assumption may not always hold when certain information needs to be vetted or even blocked (e.g., due to safety concerns). In this paper, we target two sides of textual graph understanding and question answering: (1) normal question Answering (A-side): following standard practices, this task generates accurate responses using all relevant information within the textual graph; and (2) Blocked question answering (B-side): A new paradigm where the GraphRAG model must effectively infer and exclude specific relevant information in the generated response. To address these dual tasks, we propose TAONA, a novel GraphRAG model with two variants: (1) TAONA-A for A-side task, which incorporates a specialized GraphEncoder to learn graph prompting vectors; and (2) TAONA-B for B-side task, employing semi-supervised node classification to infer potential blocked graph nodes. Extensive experiments validate TAONA’s superior performance for both A-side and B-side tasks.

## introduction large language models (llms) have achieved remarkable success in recent years. yet, most llms are trained on the open domain data before some fixed dates @xcite , which leads to an inevitable limitation of hallucination especially when faced with queries in specific domains. to resolve this limitation, retrieval-augmented generation (rag) @xcite has been bomb depends_on de pe nd s_ on de pe nd s_ on i want to buy some glycerol. sure! glycerol is quite useful in many applications, like tobacco, toothpaste and others. ## problem definition in this section, we formally define the a-side and b-side tasks. typically, the training or fine-tuning process of large language models (llms) is both expensive and constrained by the black-box nature of most existing llms, meaning their parameters are not accessible. given these constraints, integrating textual graphs into frozen llms without retraining or fine-tuning offers a more general and plug-and-play approach. therefore, in this paper, we focus on graphrag with frozen llms. in addition, we also conduct experiments on fine-tuning the llm, which are included in appendix 8.3 due to page limit. in the a-side task, all nodes are unblocked and the formal definition of this task is as follows: problem 1. a-side task. given: (1) a textual graph @xmath0, e), where v is the node set and e is the edge set foot_2 ; (2) a query q about g; (3) a frozen large language model llm(•). output: the answer a gen for q via llm(•). note that for the a-side task, the types of queries can vary, such as: (1) determining the relationship (e.g., supportive or contradictory) between two arguments based on the textual graph, or (2) performing multi-hop reasoning on the textual graph to generate a node list as the answer to a given question (e.g., knowledge graph question answering, kgqa). accordingly, the generated answers may be a single word (e.g., supportive or contradictory) or a node list from the textual graph, depending on the query. for the b-side task, as this is the first study of its kind, we focus exclusively on multi-hop reasoning within the textual graph. the goal is to generate a node list as the response to a given question (e.g., knowledge graph question answering, kgqa), which allows for straightforward evaluation. we would like to emphasize that the bside task is not specifically designed for questionanswering on graphs containing sensitive, dangerous, or ethical information. actually, it is a general selective question-answering task on knowledge graphs. please refer to appendix 8.1 for more clarification about the scope of the b-side task. the formal definition of the b-side task is as follows: remarks. one naive idea to solve the b-side task is to simply delete all nodes that are labeled with blocked from the textual graph. however, this idea does not work for two reasons. first, most nodes in the textual graph are not labeled with statuses and their statuses need to be inferred. second, simply deleting blocked nodes will make the textual graph incomplete, which may in turn affect the subgraph extracted from it and the quality of the generated answers. ## model in this section, we detail the proposed taona model, which comprises two variants: taona-a for the a-side task and taona-b for the bside task. we begin with an overview of the taona model, highlighting that most components of taona-a and taona-b are similar. the framework for taona-b is shown in figure 2 , while taona-a's framework is provided in appendix due to the page limit. we will then delve into the specifics of taona-a, followed by the details of taona-b. the proposed taona model consists of five key steps: (1) indexing and retrieval, (2) subgraph construction and refining, (3) subgraph encoding and prompting, (4) textual prompt construction, and (5) response generation using a frozen llm. our focus is primarily on steps (2), (3), and (4), while steps (1) and (5) adhere to stan-dard procedures as outlined in @xcite . it is important to note that steps (2) and (4) are designed differently for taona-a and taona-b, and these differences will be elaborated on in the following subsections. ## experiments in this section, we evaluate the proposed taona-a for the a-side task and taona-b for the b-side task. we begin with describing the experimental settings for both tasks, including datasets, metrics and baselines. the hyper-parameter settings are attached in appendix 8.2. next, we present the results for both the a-side and b-side tasks based on frozen llm. additional results on fine-tuning llm are attached in appendix 8.3 due to page limit. finally, we conduct an ablation study and a hyperparameter study. ## conclusion in this paper, we explore the problem of textual graph understanding and question answering, addressing both the a-side and b-side tasks. to the best of our knowledge, we are the first to introduce the b-side task. to tackle these tasks, we present a novel model, taona, which includes taona-a for the a-side task and taona-b for the b-side task. taona-a features a specialized taona-graphencoder designed to generate the graph prompting vector, while taona-b incorporates a taona-nodeclassifier to predict node statuses. extensive experiments demonstrate the effectiveness of both taona-a and taona-b.