Learning Relational Decomposition of Queries for Question Answering from Tables (2024)

Abstract: Table Question-Answering involves both understanding the natural language query and grounding it in the context of the input table to extract relevant information. In this context, many methods have highlighted the benefits of intermediate pre-training using SQL queries. However, while most approaches aim at generating final answers directly from inputs, we claim that there is better to do with SQL queries during training.By learning to imitate a restricted subset of SQL-like algebraic operations, we demonstrate that their execution flow provides intermediate supervision steps that allow for increased generalization and structural reasoning compared to classical approaches. Our method, bridges the gap between semantic parsing and direct answering methods, offering valuable insights into which types of operations should be predicted by a generative architecture and which should be executed by an external algorithm. Our code can be found at https://github.com/RaphaelMouravieff/Partial-Exec.

## introduction the field of table question answering (qa), which encompasses complex content manipulation tasks like projection, sorting, grouping, and aggregation, presents considerable challenges for natural language processing (nlp). its complexity and growing relevance across diverse sectors, from business to academic research, have attracted widespread attention. this domain has evolved quickly with the rise of pretrained language models (plms), but this field remains challenging for current models @xcite . former studies focused on semantic parsing (sp) techniques tailored for well-structured and clean table data, as highlighted in @xcite . however, real-world scenarios often involve heterogeneous resources, for example combining both numerical and textual content in some cells, like in wikitablequestions @xcite . among the proposed solutions, @xcite tried to generate directly the answer and therefore bypass the generation of logical forms. despite this advantage, these methods exhibit limitations, particularly when executing numerical operations (e.g., computing a mean, counting). to cope with this, a natural solution is to propose hybrids that stand as intermediary solutions between semantic parsing and direct generation. for instance, @xcite have combined basic table selection methods (e.g., selecting rows and columns, or cells) before computing aggregations or performing basic numerical operations. however, they often fail to address intricate queries necessitating the synthesis of diverse table views and interactions because of the limited expressivity of their underlying algebra. in this work, we propose to study the continuum between semantic parsing-based and direct generative methods, to leverage the strengths of both. going beyond previous works, we propose a novel framework that facilitates reasoning over heterogeneous table resources. this framework relies on the definition of an algebra over tables inspired by relational algebra. based on this algebra, each question in natural language and its corresponding table can be translated into a computational graph. by varying a cut-off criterion that specifies which part of the graph should be computed directly by the model (i.e. direct generation) and which part should be computed outside of it (i.e. semantic parsing), we can study different trade-offs and their effect in terms of effectiveness. beyond providing a stronger interpretation of the user query in the context of the table compared to semantic parsing-based (sp) methods, our framework addresses the common execution challenges associated with sp methods, which require clean tables to allow full sql execution. our approach predicts operators with associated "clean" operands from the input, thanks to the generative ability of the transformer architectures. to train our model, we leverage a pre-training procedure @xcite @xcite ) that helps neural architectures manipulate tabular data before dealing with complex table qa tasks. this is done by first learning to generate from sql queries rather than from natural language. on top of that, our main contribution focuses on the fine-tuning of such models when dealing with natural language questions, through the development of a flexible framework that allows partial execution of corresponding sql queries, enabling effective handling of real-world tables while still obtaining good performance on numerical reasoning questions (figure 1 ). our research addresses the following key questions : qa that bridges the gap between semantic parsing and direct answer generation, thereby enhancing generalization and structural reasoning capabilities ( through our experiments, we demonstrate that, for some intermediate cut-off levels, our approach generalizes better and is more robust compared to direct answer methods, which are usually limited in their structural reasoning capacities. this framework provides insights into the trade-off between different levels of supervision, paving the way for future research and innovation in the field of table qa . ## model the goal of table qa is to find the answer a given a natural language question q posed on a table t . we consider the setting where, at train time, queries in natural language are associated with corresponding sql queries. in this section, we describe 1) the manipulated data structures; 2) the algebraic operators that correspond to nodes of sql execution graphs we consider; 3) our way to build intermediate supervision through partial execution of that graph; and 4) our way for linearizing the resulting graph to form the target of our transformer architecture. ## conclusion we explored the realm between semantic parsing and direct output generation for table qa, showing that plms can leverage an appropriate level of granularity where basic table manipulations (cleanwe showed that a model, appropriately trained, achieves high performance compared to state-ofthe-art, and that, more importantly, most plm baselines are prone to overfitting (by using a simple permutation of table cells), while our method is much less affected and outperforms the best baseline, omnitab, by a wide margin. future works will include more sophisticated training procedures, a sparse attention mechanism to cope with long tables, and more in-depth error analysis.