Enhancing Free-Form Table Question Answering Models by Distilling Relevant-Cell-Based Rationales (2024)

Abstract: “Free-form table question answering is a challenging task since tables contain structured contentscompared to plain texts, which requires high-level reasoning abilities to effectively identify cellsthat are relevant to the question and produce a correct and faithful answer based on their relations.Large language models (LLMs) have exhibited remarkable reasoning capabilities in numerousNLP applications. However, in some specific tasks, specially-trained small models can still out-perform LLMs. Furthermore, small models require extremely less computation costs comparedto LLMs. To leverage the strengths of both types of models, we propose a Relevant-Cell-basedKnowledge Distillation with inference-time Teacher Guidance (RCKD-TG) method. This ap-proach aims to combine small free-form table question answering models’ abilities to learn fromhuman annotations and large language models’ abilities to effectively reason from table contents,via applying Relevant-Cell-based rationales distilled from LLMs to small models’ training andinference stages. Our experiments demonstrate the superiority of our method over vanilla smallmodels in correctness, faithfulness, adequacy and fluency, also over general LLMs in adheringto the style of human annotations. We achieve state-of-the-art performance on FeTaQA, a rep-resentative free-form table question answering benchmark. Our result of a 41.3 BLEU scoredemonstrates the feasibility of effectively using small models’ task-specific abilities and LLMs’reasoning capabilities at the same time. Additionally, our method exhibits high computation ef-ficiency and data efficiency. Compared to strong baselines, we achieve better performance withsignificantly less training data.”

## introduction tables are a prevalent form of structured data commonly found in databases and on the internet. traditionally, tables were grounded using semantic parsing or by converting natural language queries into sql language to obtain desired outputs. representative benchmarks, like wtq @xcite , sqa @xcite , wikisql @xcite , spider @xcite , tabfact @xcite , have been established for evaluating table grounding methods. recent efforts have leveraged pretraining methods for table grounding tasks, with models like tapas @xcite , tabert @xcite , tapex @xcite , and tableformer @xcite achieving promising results on existing benchmarks. however, these benchmarks primarily focus on providing short-form answers to queries, which are not challenging enough for models to perform complex reasoning. with the emergence of large generative models, users increasingly prefer interacting with nlp systems through natural language queries and receiving responses in free-form answers @xcite @xcite . as a result, traditional short-form table parsing benchmarks and methods fall short of meeting modern users' demands. to address this limitation, @xcite introduced fetaqa, a free-form table question answering benchmark, aiming to bridge the gap. fetaqa challenges models to retrieve multiple relevant entities and reason over them to produce correct and faithful answers given more abstract and ambiguous queries. this benchmark is thus more challenging than previous short-form counterparts. researchers have continued to explore table pre-training methods to tackle free-form table question answering. for example, gentap @xcite leverages large-scale synthesized training data with more than 500k training samples. however, such methods may not fully enhance a model's reasoning capabilities. recently, large language models (llms) have exhibited remarkable reasoning capabilities across various nlp tasks. @xcite have demonstrated llms' ability to quickly generalize to unseen tasks and perform reasonably well given only a few prompts. however, developing and deploying llms for each specific scenario is computationally prohibitive. to address this challenge, the latest works @xcite ) have adopted knowledge distillation, using llms as teacher models and small models as student models. these methods train small models on llms' output to distill their knowledge and reasoning capabilities, and in the meantime, outperform their teacher models. while successful in classification and multiple-choice qa tasks, their effectiveness in generation tasks and structured data remains rather unexplored. it is harder to distill knowledge from larger teacher models in text generation settings because of inadequate supervision signal. automatic evaluation metrics for language generation tasks are not robust enough to assess all important aspects pertinent to the quality of generated text. they succeed in evaluating the surface similarities between the generated text and the reference text, but fails in evaluating the accuracy, faithfulness and fluency of generated text. therefore, training a small model using these metrics as supervision would constrain to the small model's output to be lexically and syntactically more similar to human annotations, but unable to guide the small model to generate semantically correct and grammatically fluent answers. meanwhile, llms could provide mostly correct answers but fail to conform to human annotated preferences, and it is computationally prohibitive to fine-tune llms on human annotated data to mitigate this problem. in light of this, we propose a simple method for free-form table question answering, relevant-cellbased knowledge distillation with inference-time teacher guidance (rckd-tg), to tackle the challenges of reasoning on table contents and generating free-from answers that are more preferred towards human annotations. we devise a specific training method, so that at inference time, llm's abilities to reason and small model's abilities to produce answers more similar to human annotations can be combined effectively. firstly, on the training data, we prompt the llm to generate information in cells that are relevant to answering the question. then we use these rationales as references to teach the small model to produce crucial information autonomously. a multi-task learning framework is employed, training the small model on both the llm's output and the qa data. secondly, during inference, we augment the student model's generation process with llm's relevant-cell-based rationale, yielding results surpassing those using small models or llms alone. with less than 10k training data, our method achieves a state-of-theart 41.3 bleu score on fetaqa, demonstrating the feasibility of combining llms' powerful reasoning and small models' task-specific capabilities while maintaining high data and computation efficiency compared to previous table pre-training methods. in summary, the contributions of this paper are: ## experiments in the upcoming experiments, we present a comprehensive analysis of our proposed rckd-tg method. firstly, we showcase the impressive performance of our method in comparison to previous works and the current state-of-the-art, highlighting its competitiveness across various evaluation metrics. secondly, we explore the impact of dataset scale and model size, demonstrating our approach's remarkable data and computational efficiency. thirdly, we conduct an in-depth evaluation of the answers generated by our method compared to those produced by llms. also, we assess the quality of the relevant-cellbased rationales generated by both teacher and student model. lastly, we delve into the capabilities of instruction-tuned models to effectively follow rationales and explore the impact of different types of instructions. ## conclusion in conclusion, this paper introduces a novel relevant-cell-based knowledge distillation with inferencetime teacher guidance method to combine small models' capabilities of learning from human annotation and llms' capabilities of reasoning effectively. through a multi-task learning on instruction-tuned models and inferencing with teacher model's guidance, we achieved a state-of-the-art bleu score of 41.3 on the fetaqa benchmark. this approach combines llms' robust reasoning strength with smaller models' task-specific capabilities, while maintaining high data and computational efficiency. our work advances free-form table question answering and demonstrates a promising path of fusing general llms and taskspecific models on downstream tasks. future works could explore on how to more effectively combine and communicate between the student model and teacher model. future works could also explore on extending our method to diverse downstream tasks, leveraging general llms as strong reasoners, and task-specific small models as response generators aligned with human annotated preferences.