A Sequence-to-Sequence&Set Model for Text-to-Table Generation (2023)

Abstract: Recently, the text-to-table generation task has attracted increasing attention due to its wide applications. In this aspect, the dominant model formalizes this task as a sequence-to-sequence generation task and serializes each table into a token sequence during training by concatenating all rows in a top-down order. However, it suffers from two serious defects: 1) the predefined order introduces a wrong bias during training, which highly penalizes shifts in the order between rows; 2) the error propagation problem becomes serious when the model outputs a long token sequence. In this paper, we first conduct a preliminary study to demonstrate the generation of most rows is order-insensitive. Furthermore, we propose a novel sequence-to-sequence&set text-to-table generation model. Specifically, in addition to a text encoder encoding the input text, our model is equipped with a table header generator to first output a table header, i.e., the first row of the table, in the manner of sequence generation. Then we use a table body generator with learnable row embeddings and column embeddings to generate a set of table body rows in parallel. Particularly, to deal with the issue that there is no correspondence between each generated table body row and target during training, we propose a target assignment strategy based on the bipartite matching between the first cells of generated table body rows and targets. Experiment results show that our model significantly surpasses the baselines, achieving state-of-the-art performance on commonly-used datasets.

## related work information extraction (ie) refers to the automatic extraction of structured information such as entities, relationships between entities, and attributes describing entities from unstructured sources @xcite . the common ie tasks include named entity recognition (ner), relation extraction (re), event extraction (ee), etc. to achieve high-quality ie, researchers have proposed various task-specific ie methods. with the development of deep learning, researchers mainly focus on neural network based generation models, which are often seq2seq pre-trained models gener-ating serialized structured information. compared with traditional ie methods, these methods have achieved comparable or even superior results in re @xcite , ner @xcite , ee @xcite . along this line, researchers resort to unified models @xcite that model multiple ie tasks as the generation of sequences in a uniform format. in this work, we mainly focus on text-to-table generation that aims to generate structured tables from natural language descriptions. note that textto-table generation can be considered as the dual task of table-to-text generation, which intends to generate a textual description conditioned on the input structured data. usually, these structured data are represented as tables @xcite @xcite or sets of table cells @xcite . compared with traditional ie tasks, this task does not rely on predefined schemas. in this regard, @xcite first explore this task as a seq2seq generation task by fine-tuning a bart model @xcite for generation. by contrast, we model the generation of each table as a table header and then a set of table body rows. to this end, we propose a seq2seq&set text-to-table model, which can alleviate the defects caused by the sequence generation in the conventional seq2seq models. ## preliminary study we first conduct a preliminary study to inspect the effect of row generation order on the seq2seq model @xcite . since the table header is the first row of a table containing column names which should be generated first, so we only randomly reorder table body rows to construct different training datasets. then, we individually train the seq2seq models using these datasets with the same setting as the original model, and compare their performance on the same dataset @xcite . from ## our model in this section, we describe our model in detail. as shown in figure 2 , our model is composed of three modules: text encoder, table header generator and table body generator. then, we give a detailed description of the model training. ## conclusion in this paper, we propose a seq2seq&set model for text-to-table generation, which first outputs a table header, and then table body rows. most importantly, unlike the previous study @xcite , we model the generation of table body rows as a set generation task, which is able to alleviate not only the wrong bias caused by the predefined order during training, but also the problem of error propagation during inference. experimental results show that our model gains significant performance improvements over the existing sota.