L o FT : Enhancing Faithfulness and Diversity for Table-to-Text Generation via Logic Form Control (2023)

Abstract: Logical Table-to-Text (LT2T) generation is tasked with generating logically faithful sentences from tables. There currently exists two challenges in the field: 1) Faithfulness: how to generate sentences that are factually correct given the table content; 2) Diversity: how to generate multiple sentences that offer different perspectives on the table. This work proposes LoFT, which utilizes logic forms as fact verifiers and content planners to control LT2T generation. Experimental results on the LogicNLG dataset demonstrate that LoFT is the first model that addresses unfaithfulness and lack of diversity issues simultaneously. Our code is publicly available at https://github.com/Yale-LILY/LoFT .

## introduction table-to-text (t2t) generation aims to produce natural language descriptions from structured tables. a statement generated from tabular data can be inferred based on different levels of information (e.g., value of a specific cell, logical operation result across multiple cells). although current t2t models @xcite @xcite have shown remarkable progress in fluency and coherence, they mainly focus on surface-level realizations without much logical inference. recently, @xcite proposed log-icnlg, which is tasked with generating textual descriptions that require logical reasoning over tabular data (i.e., lt2t generation). lt2t generation is challenging as it requires a model to learn the logical inference knowledge from table-text pairs and generate multiple factually correct sentences. another challenge for lt2t generation is the diversity of generated text. natural language generation (nlg) encourages the diverse output of statements over a single input, as it provides -to-text generation. (a) statements generated by previous models @xcite : the generation suffers from 1) lack of diversity, as three of the generated statements are focused on the same table regions (i.e., "hale irwin" and "gil morgan"), and three of them use the similar reasoning operations (i.e., comparative); 2) unfaithfulness, as one of the generated statements is factually incorrect given the table content. (b) statements generated by loft: by utilizing logic forms to control the generation, our method can generate multiple factually correct sentences that each use a different reasoning operation to offer various perspectives on the tabular data. in this work, we attribute unfaithfulness and lack of diversity to the absence of controllability over generation. specifically, due to the large number of combinations of different logical operations and table regions, the space of factually correct statements is exponentially large. however, logic-nlg uses the whole table as the input, without providing annotations related to any other explicit control attribute. as a result, it is hard and uncontrollable for neural models to decide a favorable choice of logical selections solely based on the table input. we believe such uncontrollability leads to unfaithfulness and lack of diversity issues. this work proposes loft, a framework that utilizes logic forms as mediators to enable controllable lt2t generation. logic forms @xcite are widely used to retrieve evidence and explain the reasons behind table fact verification @xcite @xcite . in this work, logic forms are used as: 1) fact verifiers to ensure the factual correctness of each generated sentence; and 2) content planners to control which logical operation and table region to use during the generation. experimental results show that loft surpasses previous methods in faithfulness and diversity simultaneously. ## related work logical table -to-text (lt2t) generation logicnlg @xcite is tasked with generating logically faithful sentences from tables. to improve the faithfulness of generated statements, @xcite trained a system both as a generator and a faithfulness discriminator with additional replacement detection and unlikelihood learning tasks. @xcite al. (2022a) pre-trained a model on a synthetic corpus of table-to-logic-form generation. zhao et al. (2022b) demonstrated that faithfulness of lt2t can be improved by pre-training a generative language model over synthetic table qa examples. however, these methods overlook the importance of diversity in t2t generation, and might generate multiple statements that cover the same table regions or reasoning operations. previous methods in nlg proposed to improve diversity by modifying the decoding techniques @xcite . however, these approaches degrade faithfulness as measured against baselines @xcite . to enable controllable generation and improve diversity, @xcite used logical types of statements as a control. however, such methods still suffer from problems related to unfaithfulness, and may generate statements covering limited table regions. this work proposes to leverage the logic form as a fact checker and content planner to control lt2t generation, which tackles the challenges about faithfulness and diversity at the same time. table fact verification via logic form logic forms are widely used in table fact verification @xcite . specifically, given an input statement, the model @xcite @xcite will first translate it into logic form. then the logic form will be executed over the table, and return true/false as the entailment label for a given statement. while several works @xcite @xcite focused on generating fluent statements from logic forms, the utilization of logic forms to benefit lt2t generation is still unexplored. ## loft this section first introduces the logic form utilized, and then delves into the training and inference process of loft. we also explain how the use of logic forms can enhance both faithfulness and textdiversity in lt2t generation. ## experimental setup we next discuss the evaluation metrics, baselines, and implementation details for the experiments. ## experimental results this section discusses automated and human evaluation results of different systems. ## conclusions this work proposes loft, which utilizes logic forms as fact verifiers and content planners to enable controllable lt2t generation. experimental results on logicnlg demonstrate that loft delivers a great improvement in both diversity and faithfulness of lt2t generation.