Hybrid Graphs for Table-and-Text based Question Answering using LLM s (2025)

Abstract: Answering questions that require reasoning and aggregation across both structured (tables) and unstructured (raw text) data sources presents significant challenges. Current methods rely on fine-tuning and high-quality, human-curated data, which is difficult to obtain. Recent advances in Large Language Models (LLMs) have shown promising results for multi-hop question answering (QA) over single-source text data in a zero-shot setting, yet exploration into multi-source Table-Text QA remains limited. In this paper, we present a novel Hybrid Graph-based approach for Table-Text QA that leverages LLMs without fine-tuning. Our method constructs a unified Hybrid Graph from textual and tabular data, pruning information based on the input question to provide the LLM with relevant context concisely. We evaluate our approach on the challenging Hybrid-QA and OTT-QA datasets using state-of-the-art LLMs, including GPT-3.5, GPT-4, and LLaMA-3. Our method achieves the best zero-shot performance on both datasets, improving Exact Match scores by up to 10% on Hybrid-QA and 5.4% on OTT-QA. Moreover, our approach reduces token usage by up to 53% compared to the original context.

## introduction in today's data-rich world, information is scattered across various sources, which can be broadly divided into two types: structured (tables, databases) and unstructured (raw text from various files). the ability to effectively answer questions that require reasoning and aggregation across diverse data sources has become increasingly crucial. such questions are often referred to as hybrid questions and the task is often referred to as @xcite . for instance, consider the question: "what place was achieved by the person who finished the berlin marathon in 2:13.32 in 2011, the first time he competed in a marathon?", answering this question requires extracting the name of the person who finished the berlin marathon in 2011 from raw unstructured text and linking it with a structured data source like a database to determine their place. the ability to answer hybrid questions has immense real-world value, as it enables the combination of relevant information from multiple sources, leading to more comprehensive and capable qa systems. existing qa methods primarily focus on single data sources, either structured or unstructured. these approaches are limited when answering questions that require reasoning and aggregation across both structured and unstructured data. datasets like hybrid-qa @xcite and ott-qa @xcite necessitate combining information from tables and text, however current methods applied to these datasets @xcite @xcite rely on the availability of training data with pre-established connections between the structured and unstructured data. in real-world applications, such training data may not always be available and is often expensive to collect. for example, even for general data sources like wikipedia, there are only two publicly available datasets (hybrid-qa and ott-qa) that contain hybrid questions. given sufficient context, large language models (llms) @xcite can effectively extract answers from the provided context for a wide range of question categories and domains. recently, llm-based zero-shot qa has emerged as a robust alternative to traditional finetuning-based qa methods. for the task of table -text qa, a straightforward approach is to provide the entire text and table as input to the llm alongside the question. however, this method is not a scalable approach due to token cost and the limited context length of llms. approaches such as context truncation @xcite and summarization @xcite using off-the-shelf techniques like those in langchain foot_0 have been proposed. nevertheless, indiscriminate truncation or summarization can lead to significant performance degradation, as critical information may be omitted, resulting in less accurate or incomplete answers (as shown in section-5, table -3 ). to enhance the ability of llms to answer hybrid questions and efficiently retrieve and leverage relevant context, we propose odyssey foot_1 . our approach operates in a zero-shot setting, aiming to filter out noise and provide the llm with the most relevant context in a concise manner. at a high level, our method consists of two main steps: i) constructing a single hybrid graph from both textual and tabular data, and ii) pruning information from the graph based on the question. to demonstrate the effectiveness of our method compared to existing approaches, we present a case study on the hybrid-qa dataset @xcite in figure 2 . we evaluate odyssey on hybrid-qa chen et al. (2020b) and ott-qa chen et al. (2020a) datasets using three state-of-the-art llms for qa: gpt-3.5, gpt-4 and llama-3. we compare our method against relevant baselines across five metrics: exact match (em) @xcite , f1-score @xcite , precision, recall, and bertscore-f1 @xcite , demonstrating that constructing and leveraging a hybrid-graph by combining structured and unstructured information sources results in significant performance gains. our key contributions can be summarized as follows: our hybrid graph based approach uses up to 45% and 53% fewer tokens than the original table and text for the hybrid-qa and ott-qa datasets respectively (see table-text qa systems, i.e., questions that require reasoning and aggregation of information from both structured and unstructured data sources simultaneously. table-text question answering: the task of table-text qa based on text-and-tabular data has recently started gaining traction because of the creation and availability of datasets like hybrid-qa (chen et al., 2020b) and ott-qa (chen et al., 2020a), both of which are based on wikipedia as the primary data source containing both text and tabular data. to tackle challenges associated with table-text release title system developer notes mar 22 resident evil ps1 capcom survival horror mar 29 dbz: hyper dimension snes bandai third x installment table : video games in 1996 document title: capcom capcom co. , ltd.is a japanese video game developer and publisher known for creating numerous multi-million selling game franchises , including mega man , street fighter , resident evil , devil may cry, dino crisis , dead rising , sengoku basara , ghosts ' n goblins , monster hunter , breath of fire , and ace attorney as well as games based on disney animated properties . document title: bandai bandai co. , ltd. ( kabushiki-gaisha bandai ) is a japanese toy maker and a producer of many plastic model kits as well as a former video game company . it was the world 's third-largest producer of toys in 2008 after mattel and hasbro. its headquarters is located in tokyo. it produced video games for systems such as ps1, snes, xbox 360, etc. context summarized context gpt-4 question: what system was the most recent game to be released by the company known for the street fighter franchise released on ? predicted answer: snes predicted answer: xbox 360 predicted answer: ps1 gpt-4 gpt-4 hybrid graph (our method) capcom bandai street fighter ps1 mar 22 xbox 360 mar 29 snes figure 2: case study on hybrid-qa: comparison of our method (odyssey) against various baselines on an example from the hybrid-qa dataset. baselines: (i) question + context: providing the llm only the question without any additional context (ii) question + summarized context: passing the question along with the summarized documents and table. our method delivers accurate answer because the hybrid graph efficiently connects "street fighter" from the document "capcom" with the relevant table, guiding gpt-4 in generating the correct response, i.e., "ps1" from the "system" table column. qa, several approaches have been proposed @xcite @xcite . while these methods perform well, they rely on supervised fine-tuning data to learn and aggregate information across text and tables for question answering. collecting supervised data for the task of table -text qa is not only resource intensive @xcite but may not always be possible. fine-tuning-free table -text qa: the emergence of large-scaled pre-trained llms @xcite has revolutionized the field of nlp and especially has opened up ways to address the task of qa in a fine-tuning-free manner. based on a given question, the relevant context is retrieved from the data corpus using state-of-the-art retrievers @xcite @xcite and this context is passed along with the question to the llm. recent work @xcite tackles this issue by generating and executing code in a few-shot setting, similar to natural language-to-sql conversion, but it does not account for noise in the context. odyssey is a fine-tuning-free zero-shot approach for table-text qa. our method efficiently prunes relevant information from both tabular and textual sources and performs the table-text qa task. furthermore, odyssey effectively addresses the unique challenges associated with table-text qa, such as integrating information from heterogeneous sources and performing multi-hop reasoning across tables and text. ## methodology in this section, we explain the key parts of our approach for solving hybrid-qa in a fine-tuningfree manner. we begin by describing the problem setting, followed by a detailed explanation of our method, which includes question analysis, graph construction and graph traversal. finally, we discuss the retrieval and the information that is passed into the llm. ## experimental setup this section details our experimental setup, including the datasets used to evaluate our method, the baselines for comparison, evaluation metrics, and implementation specifics. ## results and analysis in this section, we present the results of our method and compare them with the baselines discussed in section 4.2. we show results using llama3-8b, gpt-3.5, and gpt-4, along with fine-tuned models on the hybrid-qa and ott-qa datasets (see tables 3, 4, and 5) across various evaluation metrics discussed in section 4.3. additionally, we analyze token efficiency, i.e., the number of tokens passed as input to the llm (see ## conclusion in this paper, we introduce odyssey, a zero-shot fine-tuning-free approach for table -text qa. by leveraging a novel hybrid graph-based approach, odyssey effectively navigates the complexities of multi-hop reasoning across structured and unstructured data sources. our method achieves significant improvements of 10% and 4.5% in exact match (em) scores using gpt-3.5 on the hybrid-qa and ott-qa datasets, respectively, compared to the baseline, while reducing the input token size for the llm reader by 45.5% on hybrid-qa and 53% on ott-qa, demonstrating its efficiency in representing relevant information concisely. we believe the insights gained from our method can pave the way for more advanced and efficient qa systems capable of navigating the ever-growing landscape of heterogeneous data sources.