Table Question Answering for Low-resourced I ndic Languages (2024)

Abstract: TableQA is the task of answering questions over tables of structured information, returning individual cells or tables as output. TableQA research has focused primarily on high-resource languages, leaving medium- and low-resource languages with little progress due to scarcity of annotated data and neural models. We address this gap by introducing a fully automatic large-scale tableQA data generation process for low-resource languages with limited budget. We incorporate our data generation method on two Indic languages, Bengali and Hindi, which have no tableQA datasets or models. TableQA models trained on our large-scale datasets outperform state-of-the-art LLMs. We further study the trained models on different aspects, including mathematical reasoning capabilities and zero-shot cross-lingual transfer. Our work is the first on low-resource tableQA focusing on scalable data generation and evaluation procedures. Our proposed data generation method can be applied to any low-resource language with a web presence. We release datasets, models, and code (https://github.com/kolk/Low-Resource-TableQA-Indic-languages).

## introduction tables are ubiquitous for storing information across domains and data sources such as relational databases, web articles, wikipedia pages, etc. @xcite . tables introduce new challenges in machine comprehension not present in text as they are are not well-formed sentences but a semi-structured collection of facts (numbers, long-tail named entities, etc.) @xcite @xcite @xcite @xcite . additionally, tables make position (rows/columns) bias @xcite and entity popularity bias @xcite severe. the tableqa task introduces novel challenges compared to text-based question answering (text-qa) @xcite @xcite @xcite . in addition to the semi-structured nature of tables, a tabular context leads to a high frequency of factbased questions, mathematical and logical operations such as arithmetic @xcite , set, relational @xcite , and table operations such as table joins @xcite . effective tableqa systems not only have machine comprehension skills, but also numeracy understanding @xcite @xcite , table reasoning @xcite , table summarization @xcite and answer table generation ability @xcite . low-resource tableqa aims to answer questions over semi-structured tables storing cultural and region-specific facts in a low-resource language. @xcite show that most languages struggle to be represented and are deprived of advances in nlp research. as manual data collection is slow and expensive, low-resource languages struggle with large-scale, annotated data for effective transfer learning solutions. the low-resource setting @xcite exacerbates the challenges of tableqa with challenges of data sparsity, annotated data costs, and lack of trained models. in contrast to textqa, syntactico-semantic variations such as agreement and morphology are not exhibited in tables, but high presence of culturally significant yet long-tail entities makes adapting existing high resource datasets and trained models challenging. research on low-resource table inference @xcite shows that standard approaches of translating english datasets for low-resource data creation are infeasible for tables due to high translation error as tables are not wellformed sentences. challenges. our work focuses on studying the following core challenges of low-resource tableqa: our main contributions are as follows: (1) we introduce low-resource tableqa task. table question answering (banglatabqa)) and hindi (hindi table question answering (hinditabqa)). banglatabqa contains 19k wikipedia tables, 2m training, 2k validation and 165 test samples. hinditabqa contains 2k wikipedia tables, 643k training, 645 validation and 125 test samples. ## related work tableqa aims to answer a user question from semistructured input tables. prior work on tableqa in english can be classified as extractive @xcite or abstractive @xcite @xcite . while extractive tableqa focuses on row and cell selection @xcite , abstractive tableqa generates various types of answers such as factoid answers @xcite , summaries @xcite ), or answer tables @xcite . low-resource setting poses challenges for various nlp tasks. the low-resource corpus creation @xcite @xcite has used automatic annotation efforts by synthesizing a large-scale dataset. das and saha (2022) train a bengali qa system by developing a synthetic dataset translated from standard english qa datasets. @xcite ; hasan et al. however, these methods are unsuitable due to the semi-structured ungrammatical sequential representation of tables. ## task definition we formulate low-resource tableqa as a sequence generation task. given a question q of k tokens q 1 , q 2 , . . . , q k , and table t comprising of m rows and n columns {h 1 , . . . , h n , t 1,1 , t 1,2 , . . . , t 1,n , . . . , t m,1 , t m,2 , . . . , t m,n } where t i,j is value of the cell at the i-th row and j-th column and h j is the j-th column header; the lowresource tableqa model generates an answer table t out . the input sequence is the concatenated question q, and linearized input table t separated by special sentinel tokens. the answer, t out , is also a linearized sequence. henceforth, for concreteness, we will use bengali as the example low-resource language. the input to such a model is: the answer table, t out , is a linearized sequence: ## methodology for dataset generation effective training of low-resourced tableqa requires creation of large-scale datasets of questions, input and answers tables, to align a language model to the low-resource language and adapt it to semistructured tables and qa task. we address challenge 1 by designing an automatic data generation process to generate a large-scale low resource tableqa corpus of training and validation samples. we follow a 3-step pipeline as follows: (i) table extraction, (ii) question generation, and (iii) answer table extraction. this pipeline applied on bengali, as depicted in figure 1 , generates the banglatabqa dataset. ## experimental setup we address challenge 2 by studying the effectiveness of state-of-the-art models (baselines) in bengali table qa. experimental results (section 6) show the need for a large-scale banglatabqa dataset and model training. we analyze several models' effectiveness in bengali language, mathematical/table operations and generalizability, thus providing a measure of the dataset quality and consequently the dataset creation methodology. baselines. we perform 2-shot in-context learning (icl) to adapt large language model (llm)s to banglatabqa task. we further fine-tune an encoder-decoder model. the demonstrations are the concatenated question and flattened input table with the flattened answer table. we use the following models as baselines: banglatabqa dataset with parameter-efficient fine-tuning (peft) on lora adapters. we train bntqa-mbart and bntqa-m2m with 128 batch size and bntqa-llama with 16 batch size and 4-bit quantization. all models are trained with 1e-4 learning rate on a single a6000 48gb gpu for 5 epochs with 1024 maximum sequence length. ## conclusion our work introduces tableqa for the low-resource languages. we propose a methodology for largescale dataset development on limited budget and automatic quality control which can be applied over any low-resource language with a web-presence. we discuss in detail the application of the methodology with an indic language, bengali, for which we release a large-scale dataset, banglatabqa. we further demonstrate generalizability of the process with another language, hindi. we