T a C ube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data (2022)

Abstract: Existing auto-regressive pre-trained language models (PLMs) like T5 and BART, have been well applied to table question answering by UNIFIEDSKG and TAPEX, respectively, and demonstrated state-of-the-art results on multiple benchmarks. However, auto-regressive PLMs are challenged by recent emerging numerical reasoning datasets, such as TAT-QA, due to the error-prone implicit calculation. In this paper, we present TaCube, to pre-compute aggregation/arithmetic results for the table in advance, so that they are handy and readily available for PLMs to answer numerical reasoning questions. TaCube systematically and comprehensively covers a collection of computational operations over table segments. By simply concatenating TaCube to the input sequence of PLMs, it shows significant experimental effectiveness. TaCube promotes the F1 score from 49.6% to 66.2% on TAT-QA and achieves new state-of-the-art results on WikiTQ (59.6% denotation accuracy). TaCube’s improvements on numerical reasoning cases are even more notable: on TAT-QA, TaCube promotes the exact match accuracy of BART-large by 39.6% on sum, 52.5% on average, 36.6% on substraction, and 22.2% on division. We believe that TaCube is a general and portable pre-computation solution that can be potentially integrated to various numerical reasoning frameworks

## introduction there are numerous recent works on table-text joint reasoning, e.g., answering questions over tables @xcite @xcite . meanwhile, pre-trained language models (plms), which have demonstrated table linearization great success on various natural language (nl) tasks, are also well applied to table-text joint reasoning and show great effectiveness @xcite @xcite @xcite . recently, numerical reasoning (nr) over tabular data has raised increasing attention and a bunch of table qa datasets targeting numerical reasoning have been collected @xcite @xcite , promoting nr to be a hot challenge in table qa. however, faced with flexible calculations such as addition, comparison and aggregation over semistructured context, plms encounter great obstacles @xcite , especially when calculation skills have not been fully exploited during large-scale pre-training on nl corpora. existing approaches to mitigate this gap can be mainly concluded into two families. one is logical-form-based method, which formulates table qa as a semantic parsing task. the method first generates the logical form and then applies a post-execution on the logical form to produce the final answer. the answer generation process is much more trackable and explainable than directly decoding answers, but it also has deficiencies. (1) human annotations of logical forms are expensive and error-prone @xcite . (2) there lacks a dominant formulation of logical forms for table reasoning: @xcite @xcite @xcite design their own logical forms and semantic parsers; @xcite simplify the logical form design and use specific classification heads for cell selection, operator prediction and unit prediction, to produce the answer. (3) due to the representation limitation of logical forms, it lacks sufficient flexibility to generate free-form answers or answers matching special conditions, e.g., "2 million" that augments a numeric value "2" with its textual scale "million" and "2022" that is a part of a cell string "2022/05/20". the other popular way is directly generating the answer using auto-regressive plms. unified-skg @xcite leverages t5 @xcite and achieves promising and even state-ofthe-art performance on a series of datasets, showing the power of directly fine-tuning large lms on table-text joint reasoning. tapex @xcite , which is based on bart @xcite and pre-trained by learning a neural sql executor, surprisingly outperforms prior works by a large margin. tapex provides a new way of using clean and synthetic data for efficient pre-training and shows its promising capacity on several numerical reasoning types, e.g., count and comparison. however, when it comes to numerical calculations involving sum, subtraction, or division, the accuracy falls to the bottom, showing that implicit numerical calculation is brittle and error-prone. to make things worse, when auto-regressive plms produce a wrong number, e.g., "1.7" in figure 1 , it's hard to uncover where the number comes from and improve the model accordingly, because all calculations are implicitly done in transformers. to address the above shortcomings, we propose to pre-compute aggregation/arithmetic results for the target table in advance, so that they are handy and readily available for answering numerical reasoning questions. this idea is inspired by an impor-tant data cube @xcite concept in the data mining field to facilitate the online analytical processing of multi-dimensional data, so we name our pre-computed aggregation/arithmetic results as tacube. tacube systematically covers a collection of computational operations over table segments in a comprehensive way. tacube can not only be fully materialized, but also be partially materialized for efficiently application to existing models. we propose rule-based and neural-based methods to produce efficient tacube while maintaining high coverage on ground truth numerical reasoning. in this paper, we applied tacube to auto-regressive plms, for their flexibility in decoding answers by leveraging both the original table and tacube (as fast access) to avoid most error-prone implicit calculations. we believe that tacube is a general pre-computation solution that can be helpful to various numerical reasoning frameworks. our experiments show that, by directly augmenting the original table with sequenced tacube: ## tacube in this section, we first discuss how pre-computed data cubes help answer numerical reasoning questions over tables. secondly, we point out that plain implementation via brute force is not favored due to explosive increasing cube size and time consumption. conventional cube techniques such as iceberg cube and cube shell @xcite are mainly designed for data mining and data analysis and are not applicable to the current qa tasks on numerical reasoning. thus, we need to design new efficient methods to generate cubes. following such observation, we present our approach for cube generation and cube candidate selection. first, we introduce cube, which only contains aggregation operators. second, we further propose cube ext , which extends the operators to non-aggregation operators. furthermore, based on cube ext , we develop cube k to reduce the cube size when feeding to the plms. we adopt two ranking methods to perform effective filtering to pick out the most likely candidate cube items. the selected cube items of a cube make up cube k which will be finally fed to the model. we will show that the generated cube can cover a large portion of numerical reasoning cases and remains efficient in search space at the same time. ## experimental results and analysis in this section, we describe the details of tacube and evaluate the effectiveness of tacube on tablerelated questions answering benchmarks. ## conclusion in this paper, we focus on numerical reasoning problems over tabular question answering. we propose our method, namely tacube, which automatically extracts pre-computed results for a given question and a table following the designed cube generation rules. the pre-computed cube is serialized and fed to the model in the input phase, mitigating the gap in numerical reasoning skills for plms. tacube is tested over multiple table qa benchmarks using an encoder-decoder architecture and achieves new sota performance on each of them. further analysis shows the performance boost mainly comes from the numerical reasoning examples in the benchmarks.