Large Language Models are Complex Table Parsers (2023)

Abstract: With the Generative Pre-trained Transformer 3.5 (GPT-3.5) exhibiting remarkable reasoning and comprehension abilities in Natural Language Processing (NLP), most Question Answering (QA) research has primarily centered around general QA tasks based on GPT, neglecting the specific challenges posed by Complex Table QA. In this paper, we propose to incorporate GPT-3.5 to address such challenges, in which complex tables are reconstructed into tuples and specific prompt designs are employed for dialogues. Specifically, we encode each cell’s hierarchical structure, position information, and content as a tuple. By enhancing the prompt template with an explanatory description of the meaning of each tuple and the logical reasoning process of the task, we effectively improve the hierarchical structure awareness capability of GPT-3.5 to better parse the complex tables. Extensive experiments and results on Complex Table QA datasets, i.e., the open-domain dataset HiTAB and the aviation domain dataset AIT-QA show that our approach significantly outperforms previous work on both datasets, leading to state-of-the-art (SOTA) performance.

## introduction complex tables, characterized by multi-level structure headers and numerous merged cells, are a prevalent data format that includes diverse data types and intricate associations among cells @xcite @xcite . in this context, the complex table qa task emerges as an important and challenging problem within the field of nlp. in traditional table qa tasks, the majority of research efforts focused on simple flat tables (i.e., tables with single-level column headers and no merged cells). earlier, a substantial number of * corresponding author figure 1 : the overall architecture of complex table parsing. it achieves through table structure reconstruction and prompt template designing based on gpt-3.5. texts in the pink background are prompts, where the bold text can be replaced; texts in the yellow background mean the format of the tables in the original dataset; and texts in the orange background represent the tables after refactoring. gpt-3.5 indicates the model text-davinci-003. ool indicates that the number of tokens we requested exceeds the length limit of gpt-3.5. code represents a piece of python code we insert to assist multi-turn qa. while significant attainment has been made in the studies above-mentioned, their primary emphasis has been on the development of simple flat tables, overlooking the ubiquitous complex tables. although @xcite endeavored to construct qa datasets, specifically tailored for complex tables, and evaluated the performance of the sota table qa models, the outcomes have not met expectations. most recently, with the advent of chatgpt 1 , an advanced nlp model derived from gpt-3, has showcased remarkable capabilities in generation @xcite @xcite , contextual understanding @xcite @xcite , and reasoning @xcite , has profoundly impacted on the field of nlp. a multitude of research actively explores and leverages the comprehension and generation abilities of chatgpt for qa tasks @xcite @xcite . however, we have not come across any relevant work that harnesses the competencies of gpt-3.5 for complex table qa task as yet. in this paper we first attempt to incorporate gpt-3.5 to achieve complex table parsing, in which complex tables are reconstructed into tuples and specific prompt designs are employed for dialogues. as illustrated in figure 1 , we reconstruct the table stored in json format into tuples, incorporating the hierarchical information, position information, and the value of each cell into different elements of the tuple. subsequently, we fill in and calculate the length using the designed single-turn dialogue prompt templates. if the token count does not exceed the limit of gpt-3.5, we adopt a single-turn dialogue scheme for qa task. otherwise, we utilize multi-turn dialogue scheme, in which we break down the question into sub-questions based on the logic of the single-turn prompt and introduce a 1 https://openai.com/blog/chatgpt code snippet to facilitate the answering process. through careful study and meticulous experimental analysis, we conclude that gpt-3.5 can be a great parser for complex tables. to sum up, the contributions of this paper are: figure 2 : the illustration of table reconstruction. first, we rewrite the rows and column headers of the input json-formatted table into a tree structure, where, except for the root node, all nodes have a unique precursor node. moreover, we encode the "hierarchical-index" of row and column headers according to the hierarchy sequence number of the tree. for example, "col 1" indicates that 2018, 2017, 2016, the three headers are all located at the first layer of column header, and the encoding is 1. in addition, we encode the row and column position information of the header, for example, the "column position index" of 2018 is 0, the "column position index" of 2017 is 1, etc. finally, we encode each cell in the table based on "hierarchical-index", "row position index", and "column position index". in the refactored table, the "non-header" represents the cell other than the "column header" and "row header", and the "title" represents the table title. different from them, we propose to reconstruct the table format and design suitable prompt templates to fully unleash the comprehension and reasoning power of gpt-3.5, which exhibits outstanding performance on both complex table datasets. ## method in this section, we reconstruct the table from jsonformatted to tuples (section 3.1). to better leverage the reasoning and generation capabilities of gpt-3.5, we introduce the single-turn prompts designed for tables that do not exceed the api input limit on tokens (section 3.2) and the multi-turn prompts for tables that exceed the limit. moreover, we add a piece of code to assist question answering in multiturn qa (section 3.3). here, we use python for coding and calling api. ## analysis the results show that our method proposed for complex table parsing based on gpt-3.5 effectively outperforms the optimal methods on the existing two complex table datasets. however, compared to the significant performance with single-turn dialogue, the accuracy with multi-turn dialogue, especially on the hitab dataset, performs slightly worse. we further analyze the results in this section. ## conclusion inspired by the powerful reasoning and generation capabilities of llms, we propose to leverage gpt-3.5 as a parser for complex tables with table reformat and prompt design. extensive experiments show that our method achieves the best performance on hitab and ait-qa. however, limited by the input length and inability to store historical information of gpt-3.5, how to utilize multi-turn dialogue to obtain more valuable context for complex table qa remains to be explored.