S 3 HQA : A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering (2023)

Abstract: Answering multi-hop questions over hybrid factual knowledge from the given text and table (TextTableQA) is a challenging task. Existing models mainly adopt a retriever-reader framework, which have several deficiencies, such as noisy labeling in training retriever, insufficient utilization of heterogeneous information over text and table, and deficient ability for different reasoning operations. In this paper, we propose a three-stage TextTableQA framework S3HQA, which comprises of retriever, selector, and reasoner. We use a retriever with refinement training to solve the noisy labeling problem. Then, a hybrid selector considers the linked relationships between heterogeneous data to select the most relevant factual knowledge. For the final stage, instead of adapting a reading comprehension module like in previous methods, we employ a generation-based reasoner to obtain answers. This includes two approaches: a row-wise generator and an LLM prompting generator (first time used in this task). The experimental results demonstrate that our method achieves competitive results in the few-shot setting. When trained on the full dataset, our approach outperforms all baseline methods, ranking first on the HybridQA leaderboard.

## introduction question answering systems devote to answering various questions with the evidence located in the structured knowledge base (e.g., table) @xcite or unstructured texts @xcite . considering that many questions need to utilize multiple sources of knowledge jointly in real-world applications, the hybrid form of question answering over texts and tables (texttableqa) has been proposed and attracted more and more attention @xcite ., 1 https://codalab.lisn.upsaclay.fr/ competitions/7979 . year score athlete place 1960 8,683 rafer johnson eugene 1960 8,709 philip mulkey memphis 1963 8,089 chuan-kwang yang walnut q1: who is the athlete in a city located on the mississippi river? a1: philip mulkey q2: in which year did walnut-born athletes participate in the rome olympics? a2: 1960 q3: who is the higher scoring athlete from the cities of eugene and walnut? comparison a3: rafer johnson 2020b,a; @xcite @xcite . fact reasoning @xcite is a critical question type of texttableqa. it requires jointly using multiple evidence from tables and texts to reasoning the answers with different operations, such as correlation (e.g., multi-hop) and aggregation (e.g., comparison). hyperlinks among some table cells and linked passages are essential resources to establish their relationship and support the retrieval and reasoning for multi-hop questions. as shown in figure 1 , answering a complex question q1 requires jointly reasoning from textual evidence (p1) to table evidence ([r2, place]) and then to other table evidence ([r2, athlete]). existing methods consist of two main stages: retriever and reader @xcite . the retriever filters out the cells and passages with high relevance to the question, and then the reader extracts a span from the retrieval results as the final answer. however, current methods with two stages still have three limitations as follows. d d 1 d 2 divided by |g i | row-based first step retrieval θ 1 passage-based generate pseudo label d 3 d 1 second step retrieval special token table token passage token hybrid selector r 1 r 2 r 3 r 4 p 1 p 2 p 3 p 4 p 5 p 6 p 7 q1:who is the athlete in a city located on the mississippi river? q3:who is the higher scoring athlete from the cities of eugene and walnut? row ranks r 2 r 1 r 3 r 4 …… passage ranks p 3 p 2 p 5 p 4 p 6 …… row ranks r 1 r 4 r 2 r 3 …… passage ranks p 6 p 1 p 2 p 7 …… p 3 r 2 p 5 p 4 p 6 r 4 …… …… p 1 r 1 r 2 answer answer in r 2 answer in p 1 delete low relevance passages row-based passage-based generation-based reasonor θ 2 figure 2: an overview of s 3 hqa framework. the retrieval stage is divided into two steps. the hybrid selector considers the linked relationships between heterogeneous data to select the most relevant factual knowledge. in this paper, we propose a three-stage approach s 3 hqa to solve the above problems. (1) retriever with refinement training, we propose a two-step training method, splitting the training data into two parts, so that the noise in the retrieval phase can be alleviated. (2) hybrid selector has been proposed and selects supporting facts with different granularity and resources depending on the question type. by considering the hybrid data of tables and text, this paper proposes a hybrid selection algorithm that can effectively utilize the heterogeneous information of tables and passages. (3) generationbased reasoner utilizes a generation-based model for addressing different question types. the model allows better aggregation of information on the input side, which not only have better multi-hop reasoning capabilities but also be able to handle comparison and counting questions. furthermore, we are the first to use the llm in-context learning approach for table-text hybrid question-answering tasks. we evaluate our proposed model on the challenging texttableqa benchmark hybridqa. the empirical results show that our approach outperforms all the existing models foot_0 . ## related work the texttableqa task @xcite has attracted more and more attention. as for multi-hop type dataset, previous work used pipeline approach @xcite , unsupervised approach @xcite , multigranularity @xcite , table pre-trained language model @xcite , multiinstance learning @xcite and graph neural network @xcite to solve this task. as for numerical reasoning task, which is quite different from multi-hop type dataset, there is also a lot of work @xcite @xcite @xcite to look at these types of questions. unlike these methods, our proposed three-stage model s 3 hqa can alleviate noises from weakly supervised and solve different types of multi-hop texttableqa questions by handling the relationship between tables and text. ## conclusion this paper proposes a three-stage model consisting of retriever, selector, and reasoner, which can effectively address multi-hop texttableqa. the proposed method solves three drawbacks of the previous methods: noisy labeling for training retriever, insufficient utilization of heterogeneous information, and deficient ability for reasoning. it achieves new state-of-the-art performance on the widely used benchmark hybridqa. in future work, we will design more interpretable texttableqa models to predict the explicit reasoning path.