Original: Search and Learn: Improving Semantic Coverage for Data-to-Text Generation
Revised: Search and Learn: Improving Semantic Coverage for Data-to-Text Generation

Original: Anonymous EMNLP submission

Original: Abstract
Revised: Abstract

Original: Data-to-text generation aims to generate textdescription based on input data.
Revised: Data-to-text generation aims to generate text description based on input data.

Original: Modern approaches are largely based on neural networks,which often require massive parallel data fortraining.
Revised: Modern approaches are largely based on neural networks, which often require massive parallel data for training.

Original: Recently, researchers address fewshot data-to-text generation by ﬁne-tuning pretrained language models.
Revised: Recently, researchers address fewshot data-to-text generation by ﬁne-tuning pretrained language models.

Original: In our work, we observe that such few-shot models suffer fromthe problem of low semantic coverage , i e.,important input slots are missing in the generated text.
Revised: In our work, we observe that such few-shot models suffer from the problem of low semantic coverage , i.e., important input slots are missing in the generated text.

Original: We therefore propose a searchand-learning approach that inserts the missing slots in a greedy manner and learns fromthe search results.
Revised: We therefore propose a searchand-learning approach that inserts the missing slots in a greedy manner and learns from the search results.

Original: Results show that ourmodel achieves high performance on E2E and
Revised: Results show that our model achieves high performance on E2E and

Original: WikiBio datasets.
Revised: WikiBio datasets.

Original: Especially, we cover 98.5%of input slots on the E2E dataset, largely alleviating the low coverage problem.
Revised: Especially, we cover 98.5% of input slots on the E2E dataset, largely alleviating the low coverage problem.

Original:  Introduction
Revised:  Introduction

Original: Data-to-text generation is a task converting structured information into human-readable text descriptions, illustrated in Figure 1.
Revised: Data-to-text generation is a task converting structured information into human-readable text descriptions, illustrated in Figure 1.

Original: Data-to-text generation has gained much attention in the ﬁeld ofnatural language processing, with applications torestaurant descriptions (Novikova et al., 2017), biographies (Lebret et al., 2016), and weather forecasts (Liang et al., 2009).
Revised: Data-to-text generation has gained much attention in the ﬁeld of natural language processing, with applications to restaurant descriptions (Novikova et al., 2017), biographies (Lebret et al., 2016), and weather forecasts (Liang et al., 2009).

Original: Traditional approaches to natural language generation (NLG) use handcrafted rules with statistics(Langkilde and Knight, 1998;
Revised: Traditional approaches to natural language generation (NLG) use handcrafted rules with statistics (Langkilde and Knight, 1998;

Original: Stent et al., 2004;
Revised: Stent et al., 2004;

Original: Rieser and Lemon, 2009), usually lacking ﬂexibility and diversity of the outputs.
Revised: Rieser and Lemon, 2009), usually lacking ﬂexibility and diversity of the outputs.

Original: Recently, data-to-text generation is generallyaccomplished by modern neural networks, suchas sequence-to-sequence recurrent neural networks (Lebret et al., 2016; Liu et al., 2018).
Revised: Recently, data-to-text generation is generally accomplished by modern neural networks, such as sequence-to-sequence recurrent neural networks (Lebret et al., 2016; Liu et al., 2018).

Original: Thesemodels use massive parallel training datasets, forexample, 42K table–text training pairs in the E2Edataset (Novikova et al., 2017).
Revised: These models use massive parallel training datasets, for example, 42K table–text training pairs in the E2E dataset (Novikova et al., 2017).

Original: This data-hungrynature of neural models makes data-to-text generation an expensive and time-consuming affair and restricts its real-world applications.
Revised: This data-hungry nature of neural models makes data-to-text generation an expensive and time-consuming affair and restricts its real-world applications.

Original: Chen et al.
Revised: Chen et al.

Original: (2020b) present a few-shot learning approach for data-to-text generation by ﬁne-tuning pre-trained language models (LMs) with a copy mechanism.
Revised: (2020b) present a few-shot learning approach for data-to-text generation by ﬁne-tuning pre-trained language models (LMs) with a copy mechanism.

Original: We observe that, in the few-shot setting, the ﬁne-tuned LMs fail to fully learn the correspondence between input and output.
Revised: We observe that, in the few-shot setting, the ﬁne-tuned LMs fail to fully learn the correspondence between input and output.

Original: They suffer from the problem of low semantic coverage , that is, important information slots are often missing in the generated text.
Revised: They suffer from the problem of low semantic coverage , that is, important information slots are often missing in the generated text.

Original: In our work, we propose a search-and-learning (S&L) approach to address the low coverage problem for few-shot data-to-text generation.
Revised: In our work, we propose a search-and-learning (S&L) approach to address the low coverage problem for few-shot data-to-text generation.

Original: We ﬁrst ﬁne-tune the pretrained T5 language model
Revised: We ﬁrst ﬁne-tune the pretrained T5 language model

Original: (Raffel et al., 2020), similar to previous work (Chen et al., 2020b).
Revised: (Raffel et al., 2020), similar to previous work (Chen et al., 2020b).

Original: To address the low coverage problem, we iteratively insert a missing slot into the generated sentence at all possible positions, and pick the most appropriate candidate sentence based on T5 probability.
Revised: To address the low coverage problem, we iteratively insert a missing slot into the generated sentence at all possible positions, and pick the most appropriate candidate sentence based on T5 probability.

Original: This can be thought of as greedy search that ﬁnds a sentence containing all slots.
Revised: This can be thought of as greedy search that ﬁnds a sentence containing all slots.

Original: Inspired by Li et al.
Revised: Inspired by Li et al.

Original: (2020), we then treat the search results as pseudo-groundtruth and further ﬁne-tune our T language model.
Revised: (2020), we then treat the search results as pseudo-groundtruth and further ﬁne-tune our T language model.

Original: In this way, our model achieves high performance, especially with a high coverage of the input slots.
Revised: In this way, our model achieves high performance, especially with a high coverage of the input slots.

Original: Also, our model is efﬁcient for generating sentences and does not increase the inference complexity, compared with search-based inference (Liu et al., 2020).
Revised: Also, our model is efﬁcient for generating sentences and does not increase the inference complexity, compared with search-based inference (Liu et al., 2020).

Original: In summary, our main contributions include:
Revised: In summary, our main contributions include:

Original: • We address the low semantic coverage prob- lem in few-shot data-to-text generation.
Revised: • We address the low semantic coverage problem in few-shot data-to-text generation.

Original: • We propose a search-and-learning approach by inserting missing slots and learning from the search results.
Revised: • We propose a search-and-learning approach by inserting missing slots and learning from the search results.

Original: • We conduct extensive experiments on E2E and WikiBio datasets.
Revised: • We conduct extensive experiments on E2E and WikiBio datasets.

Original: Our model outperforms previous few-shot models in various metrics, largely closing the gap between few-shot and fully supervised learning.
Revised: Our model outperforms previous few-shot models in various metrics, largely closing the gap between few-shot and fully supervised learning.

Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original:  Related Work 
Revised:  Related Work 2. Data-to-Text Generation

Original: 2. Data-to-Text Generation

Original: Generating human-understandable sentences fromtabular data, commonly known as data-to-text generation, is a persistent problem from early NLPresearch.
Revised: Generating human-understandable sentences from tabular data, commonly known as data-to-text generation, is a persistent problem from early NLP research.

Original: Traditional work typically follows apipeline approach of sentence/content planning andsurface realization (Dale and Reiter, 1997), usinghand-engineered rules (Kukich, 1983; McKeown,1992) and statistical induction (Liang et al., 2009;
Revised: Traditional work typically follows a pipeline approach of sentence/content planning and surface realization (Dale and Reiter, 1997), using hand-engineered rules (Kukich, 1983; McKeown, 1992) and statistical induction (Liang et al., 2009;

Original: Koncel-Kedziorski et al., 2014).
Revised: Koncel-Kedziorski et al., 2014).

Original: However, the generated text by such systems usually lacks ﬂexibilityand diversity.
Revised: However, the generated text by such systems usually lacks ﬂexibility and diversity.

Original: With the rise of deep learning, neural modelshave become a prevailing approach to data-to-textgeneration (Lebret et al., 2016; Liu et al., 2018;
Revised: With the rise of deep learning, neural models have become a prevailing approach to data-to-text generation (Lebret et al., 2016; Liu et al., 2018;

Original: Wiseman et al., 2018; Liu et al., 2019).
Revised: Wiseman et al., 2018; Liu et al., 2019).

Original: Typically, these systems require massive parallel datafor training the text generator.
Revised: Typically, these systems require massive parallel data for training the text generator.

Original: Very recently, Chen et al.
Revised: Very recently, Chen et al.

Original: (2020b) addressed fewshot learning for data-to-text generation, wherethey assume a very small parallel corpus is available.
Revised: (2020b) addressed fewshot learning for data-to-text generation, where they assume a very small parallel corpus is available.

Original: They propose to ﬁne-tune pre-trained language models (LMs) with a copy mechanism.
Revised: They propose to ﬁne-tune pre-trained language models (LMs) with a copy mechanism.

Original: Weobserve that, even such ﬁne-tuned LMs generateﬂuent sentences, they suffer from the problem oflow semantic coverage.
Revised: We observe that, even such ﬁne-tuned LMs generate ﬂuent sentences, they suffer from the problem of low semantic coverage.

Original: In fact, Dhingra et al.
Revised: In fact, Dhingra et al.

Original: (2019) have pointed out thelow semantic coverage problem of human-writtenreferences.
Revised: (2019) have pointed out the low semantic coverage problem of human-written references.

Original: They propose a new metric for betterevaluating data-to-text models.
Revised: They propose a new metric for better evaluating data-to-text models.

Original: In this paper, weemphasize the low coverage problem of text generators, and propose a search-and-learning approachto overcome it.
Revised: In this paper, we emphasize the low coverage problem of text generators, and propose a search-and-learning approach to overcome it.

Original: Gong et al.
Revised: Gong et al.

Original: (2020) improve the ﬁdelity of data-totext generators by table reconstruction and content matching along with ﬁne-tuning GPT-2.
Revised: (2020) improve the ﬁdelity of data-totext generators by table reconstruction and content matching along with ﬁne-tuning GPT-2.

Original: Our preliminary analysis during development suggests that ﬁne-tuned T5 does not generate wrong information, but may miss important input slots.
Revised: Our preliminary analysis during development suggests that ﬁne-tuned T5 does not generate wrong information, but may miss important input slots.

Original: 2. Search-Based Text Generation
Revised: 2. Search-Based Text Generation

Original: Previous work has addressed unsupervised text generation by various search approaches, such as simulated annealing (Liu et al., 2020) and hillclimbing (Schumann et al., 2020).
Revised: Previous work has addressed unsupervised text generation by various search approaches, such as simulated annealing (Liu et al., 2020) and hillclimbing (Schumann et al., 2020).

Original: The basic idea is to deﬁne a heuristic objective function (typically involving language ﬂuency, semantic coherence, and other task-speciﬁc scores) and generate text by word-level editing towards the objective.
Revised: The basic idea is to deﬁne a heuristic objective function (typically involving language ﬂuency, semantic coherence, and other task-speciﬁc scores) and generate text by word-level editing towards the objective.

Original: Li et al.
Revised: Li et al.

Original: (2020) further propose a search-and-learning approach to improve performance and inference efﬁciency.
Revised: (2020) further propose a search-and-learning approach to improve performance and inference efﬁciency.

Original: Our paper adopts the framework of Li et al.
Revised: Our paper adopts the framework of Li et al.

Original: (2020), but differs from previous search-based approaches in several signiﬁcant ways: 1)
Revised: (2020), but differs from previous search-based approaches in several signiﬁcant ways: 1)

Original: We address the few-shot setting.
Revised: We address the few-shot setting.

Original: Instead of a heuristically deﬁned objective, we use a ﬁne-tuned language model to evaluate candidate sentences.
Revised: Instead of a heuristically deﬁned objective, we use a ﬁne-tuned language model to evaluate candidate sentences.

Original: The goal of our search is for a higher semantic coverage, rather than a generic ﬂuent sentence.
Revised: The goal of our search is for a higher semantic coverage, rather than a generic ﬂuent sentence.

Original: Our search space is relatively simpler than the entire sentence space, and thus, the main focus of our work is not the search algorithm.
Revised: Our search space is relatively simpler than the entire sentence space, and thus, the main focus of our work is not the search algorithm.

Original: We adopt greedy search over multiple missing slots, which turns out to work well empirically.
Revised: We adopt greedy search over multiple missing slots, which turns out to work well empirically.

Original: To the best of our knowledge, we are the ﬁrst to address few-shot data-to-text generation by search and learning.
Revised: To the best of our knowledge, we are the ﬁrst to address few-shot data-to-text generation by search and learning.

Original: Other work learns to perform word editing in a supervised way (Dong et al., 2019), or treat rulebased editing as input (Li et al., 2018).
Revised: Other work learns to perform word editing in a supervised way (Dong et al., 2019), or treat rulebased editing as input (Li et al., 2018).

Original: We instead perform editing to search for high semantic coverage and further learn from the edit results.
Revised: We instead perform editing to search for high semantic coverage and further learn from the edit results.

Original:  Problem Formulation
Revised:  Problem Formulation

Original: Data-to-text generation aims to generate a naturallanguage description for some structured input data;we consider a common setting, where the inputis tabular data.
Revised: Data-to-text generation aims to generate a natural language description for some structured input data; we consider a common setting, where the input is tabular data.

Original: For one data sample, the inputtable is a set of slot name–value pairs, denoted by
Revised: For one data sample, the input table is a set of slot name–value pairs, denoted by

Original: T = { ( s i , v i ) }
Revised: T = { ( s i , v i ) }

Original: Si =1 , where s i is the name of the i thslot, v i is the value, and S is the number of slots.
Revised: Si =1 , where s i is the name of the i th slot, v i is the value, and S is the number of slots.

Original: The output is a sentence y
Revised: The output is a sentence y

Original: =
Revised: =

Original: ( y 1 , y 2 , · · · , y m ) thatdescribes the given input T .
Revised: ( y 1 , y 2 , · · · , y m ) that describes the given input T .

Original: Notice that the table
Revised: Notice that the table

Original: T is different for every data sample, but we omitthe sample index for simplicity.
Revised: T is different for every data sample, but we omit the sample index for simplicity.

Original: In this work, we consider the few-shot setting,where we have a small parallel corpus D p ={ ( T ( m ) , y ( m ) ) } M m =1 and another small unlabeledcorpus D u = { T ( n ) u } Nn =1 , where T ( n ) u is a differenttable than T ( m ) .
Revised: In this work, we consider the few-shot setting, where we have a small parallel corpus D p = { ( T ( m ) , y ( m ) ) } M m =1 and another small unlabeled corpus D u = { T ( n ) u } Nn =1 , where T ( n ) u is a different table than T ( m ) .

Original: Here, both M and N are smallnumbers in our few-shot setting.
Revised: Here, both M and N are small numbers in our few-shot setting.

Original: Few-shot learning is important to NLG, as itsaves human annotation labors and also helps toalleviate the cold-start problem of an NLG task.
Revised: Few-shot learning is important to NLG, as it saves human annotation labors and also helps to alleviate the cold-start problem of an NLG task.

Original: Inour work, we assume a small unlabeled corpus D uis available in addition to D p .
Revised: In our work, we assume a small unlabeled corpus D u is available in addition to D p .

Original: This is a realisticsetting for few-shot learning, because unlabeleddata are easier to obtain than labeled pairs withhuman-written sentences, and sometimes D u maybe synthesized by recombining the slots of D p fordata-to-text generation.
Revised: This is a realistic setting for few-shot learning, because unlabeled data are easier to obtain than labeled pairs with human-written sentences, and sometimes D u may be synthesized by recombining the slots of D p for data-to-text generation.

Original:  Proposed Model
Revised:  Proposed Model

Original: In our approach, we ﬁrst ﬁne-tune a pre-trained language model (LM) for conditional text generationbased on an input table (Section 4.1).
Revised: In our approach, we ﬁrst ﬁne-tune a pre-trained language model (LM) for conditional text generation based on an input table (Section 4.1).

Original: This allowsus to generate ﬂuent sentences in the few-shot setting, due to the large model capacity and extensivepre-training of the LM.
Revised: This allows us to generate ﬂuent sentences in the few-shot setting, due to the large model capacity and extensive pre-training of the LM.

Original: However, ﬁne-tuned LMs may not fully learnthe correspondence between input slots and outputtext, and have the problem of low semantic coverage .
Revised: However, ﬁne-tuned LMs may not fully learn the correspondence between input slots and output text, and have the problem of low semantic coverage .

Original: Thus, we iteratively insert a missing slotinto the generated sentence in a greedy manner,so as to improve the semantic coverage (Section4.2).
Revised: Thus, we iteratively insert a missing slot into the generated sentence in a greedy manner, so as to improve the semantic coverage (Section 4.2).

Original: Finally, these search results are treated aspseudo-groundtruth for further ﬁne-tuning our LM,which not only improves inference efﬁciency, butalso yields better sentences (Section 4.3).
Revised: Finally, these search results are treated as pseudo-groundtruth for further ﬁne-tuning our LM, which not only improves inference efﬁciency, but also yields better sentences (Section 4.3).

Original: 4. First-Stage Fine-Tuning T 
Revised: 4. First-Stage Fine-Tuning T

Original: We use the T5 model (Raffel et al., 2020) for datato-text conditional generation.
Revised: We use the T5 model (Raffel et al., 2020) for datato-text conditional generation.

Original: T5 is a text-to-text
Revised: T5 is a text-to-text

Original: Transformer model (Vaswani et al., 2017), which is pre-trained on multiple NLP tasks and achieves state-of-the-art performance on question answering, document summarization, sentiment classiﬁcation, etc.
Revised: Transformer model (Vaswani et al., 2017), which is pre-trained on multiple NLP tasks and achieves state-of-the-art performance on question answering, document summarization, sentiment classiﬁcation, etc.

Original: In fact, T5 can be thought of as an alternative for BERT (Devlin et al., 2019) and
Revised: In fact, T5 can be thought of as an alternative for BERT (Devlin et al., 2019) and

Original: GPT-2 (Radford et al., 2019).
Revised: GPT-2 (Radford et al., 2019).

Original: It is worth noting that T5 is never pre-trained on any tasks related to data-to-text generation.
Revised: It is worth noting that T5 is never pre-trained on any tasks related to data-to-text generation.

Original: Therefore, our experiments are indeed in the few-shot setting even if we use pre-trained T5.
Revised: Therefore, our experiments are indeed in the few-shot setting even if we use pre-trained T5.

Original: We linearized the input table by concatenating all slots in the format of “ name[value] ”.
Revised: We linearized the input table by concatenating all slots in the format of “ name[value] ”.

Original: In other words, a special token “ [ ” separates the name and value of a slot, and another special token “ ] ” separates different slots.
Revised: In other words, a special token “ [ ” separates the name and value of a slot, and another special token “ ] ” separates different slots.

Original: In our few-shot setting, we ﬁne-tune T5 using several hundred data–text pairs, which is considerably smaller than a usual NLG training set.
Revised: In our few-shot setting, we ﬁne-tune T5 using several hundred data–text pairs, which is considerably smaller than a usual NLG training set.

Original: The model learns to estimate the conditional probability
Revised: The model learns to estimate the conditional probability

Original: P ( y | T ) in an auto-regressive way:
Revised: P ( y | T ) in an auto-regressive way:

Original: [Equation]
Revised: [Equation]

Original: where y is the output with length n , T is the input table, and θ represents model parameters.
Revised: where y is the output with length n , T is the input table, and θ represents model parameters.

Original: We ﬁne-tune T5 with the cross-entropy loss
Revised: We ﬁne-tune T5 with the cross-entropy loss

Original: [Equation]
Revised: [Equation]

Original: 4. Search to Improve Semantic Coverage
Revised: 4. Search to Improve Semantic Coverage

Original: We observe that T5 indeed generates ﬂuent sentences, but it has a low coverage of the input slots.
Revised: We observe that T5 indeed generates ﬂuent sentences, but it has a low coverage of the input slots.

Original: In Figure 1c, for example, the slots “riverside” and “family friendly” are not mentioned in the generated text, although they are present in the groundtruth.
Revised: In Figure 1c, for example, the slots “riverside” and “family friendly” are not mentioned in the generated text, although they are present in the groundtruth.

Original: The low semantic coverage is because the fewshot parallel corpus cannot fully support T5 learning the correspondence between input and output.
Revised: The low semantic coverage is because the fewshot parallel corpus cannot fully support T5 learning the correspondence between input and output.

Original: This is evidenced by analyzing the coverage percentage and the training size: T5 ﬁne-tuned on 1% of E2E data has a coverage of 84.46% input slots, whereas it has a coverage of 97.74% if ﬁne-tuned on the whole dataset.
Revised: This is evidenced by analyzing the coverage percentage and the training size: T5 ﬁne-tuned on 1% of E2E data has a coverage of 84.46% input slots, whereas it has a coverage of 97.74% if ﬁne-tuned on the whole dataset.

Original: To this end, we propose a simple yet effective search approach that explicitly inserts the missing slots into the generated text.
Revised: To this end, we propose a simple yet effective search approach that explicitly inserts the missing slots into the generated text.

Original: We start by checking the occurrence of each slotvalue v i in T5’s output.
Revised: We start by checking the occurrence of each slot value v i in T5’s output.

Original: If a slot value v i doesnot appear in the output, we insert a phrase ˜ v ibased on manually designed templates.
Revised: If a slot value v i does not appear in the output, we insert a phrase ˜ v i based on manually designed templates.

Original: For example, if the slot “ area[riverside] ” is missing, we insert the phrase ˜ v i
Revised: For example, if the slot “ area[riverside] ” is missing, we insert the phrase ˜ v i

Original: = “ in riversidearea ”.
Revised: = “ in riverside area ”.

Original: The E2E dataset has a boolean slot“ familyFriendly[yes/no] ”, and we design the phrase template as either “ familyfriendly ” or “ not family friendly ”.
Revised: The E2E dataset has a boolean slot “ familyFriendly[yes/no] ”, and we design the phrase template as either “ family friendly ” or “ not family friendly ”.

Original: The complete list of our templates is shown in Appendix A.
Revised: The complete list of our templates is shown in Appendix A.

Original: For every missing slot, we determine the mostappropriate position for inserting the slot template.
Revised: For every missing slot, we determine the most appropriate position for inserting the slot template.

Original: This is given by an enumeration of all possiblepositions within a sentence, and we select the candidate that has the highest T5 probability p ( y | T )as ﬁne-tuned by Eqn (1), shown in Figure 1.
Revised: This is given by an enumeration of all possible positions within a sentence, and we select the candidate that has the highest T5 probability p ( y | T ) as ﬁne-tuned by Eqn (1), shown in Figure 1.

Original: This process is repeated for all missing slots, andevery missing slot is inserted greedily according tothe above criterion.
Revised: This process is repeated for all missing slots, and every missing slot is inserted greedily according to the above criterion.

Original: Our approach can be thoughtof as an optimization towards
Revised: Our approach can be thought of as an optimization towards

Original: [Equation]
Revised: [Equation]

Original: subject to [Equation]
Revised: subject to v i ∈ y , ∀ i

Original: Speciﬁcally, we optimize the T5 conditional probability for data-to-text generation by starting froman infeasible solution (i.e., an output that violatesthe constraint).
Revised: Speciﬁcally, we optimize the T5 conditional probability for data-to-text generation by starting from an infeasible solution (i.e., an output that violates the constraint).

Original: We then project the solution intothe feasible set by satisfying each constraint in agreedy fashion.
Revised: We then project the solution into the feasible set by satisfying each constraint in a greedy fashion.

Original: Our search method is inspired by recent development of search-based unsupervised text generation, such us simulated annealing for paraphrasing (Liu et al., 2020) and hill-climbing for summarization (Schumann et al., 2020).
Revised: Our search method is inspired by recent development of search-based unsupervised text generation, such us simulated annealing for paraphrasing (Liu et al., 2020) and hill-climbing for summarization (Schumann et al., 2020).

Original: However, oursearch effort is mainly devoted to projecting aninfeasible solution to the feasible set, instead ofsearching for a generic sentence that maximizesthe objective.
Revised: However, our search effort is mainly devoted to projecting an infeasible solution to the feasible set, instead of searching for a generic sentence that maximizes the objective.

Original: Our approach also resembles rule-based text generation systems in the early age (Langkilde and
Revised: Our approach also resembles rule-based text generation systems in the early age (Langkilde and

Original: Knight, 1998; Stent et al., 2004), as we also designrules and templates.
Revised: Knight, 1998; Stent et al., 2004), as we also design rules and templates.

Original: However, our work differsfrom them signiﬁcantly, as we use rules only forrevision not for generation.
Revised: However, our work differs from them signiﬁcantly, as we use rules only for revision not for generation.

Original: Further, we will havea learning component that learns from the searchresults, introduced in the next subsection.
Revised: Further, we will have a learning component that learns from the search results, introduced in the next subsection.

Original: 4. Second-Stage Fine-Tuning T5 with 
Revised: 4. Second-Stage Fine-Tuning T5 with Search Results

Original: Search Results

Original: The search approach in Section 4.2 ensures a high semantic coverage in the output sentence, but it has two major drawbacks: 1) the edited sentence may not be ﬂuent due to the ﬁxed template, and 2) it has a low inference efﬁciency as we have to evaluate multiple candidate outputs given a data sample.
Revised: The search approach in Section 4.2 ensures a high semantic coverage in the output sentence, but it has two major drawbacks: 1) the edited sentence may not be ﬂuent due to the ﬁxed template, and 2) it has a low inference efﬁciency as we have to evaluate multiple candidate outputs given a data sample.

Original: To address the above drawbacks, we further ﬁnetune T5 that learns from the search results, inspired by Li et al.
Revised: To address the above drawbacks, we further ﬁnetune T5 that learns from the search results, inspired by Li et al.

Original: In our few-shot setting, we assume there is small unlabeled corpus D u containing input tables only.
Revised: In our few-shot setting, we assume there is small unlabeled corpus D u containing input tables only.

Original: In practice, D u can either be obtained inexpensively or can be synthesized by recombining the table slots and values in D p .
Revised: In practice, D u can either be obtained inexpensively or can be synthesized by recombining the table slots and values in D p .

Original: For a given input table T ( i ) u ∈ D u , we use
Revised: For a given input table T ( i ) u ∈ D u , we use

Original: T5 to generate a candidate output (Section 4.1) and perform search for higher semantic coverage (Section 4.2).
Revised: T5 to generate a candidate output (Section 4.1) and perform search for higher semantic coverage (Section 4.2).

Original: The search result is treated as a pseudo-groundtruth, denoted by ˆ y ( i ) search .
Revised: The search result is treated as a pseudo-groundtruth, denoted by ˆ y ( i ) search .

Original: It is mixed with the original parallel corpus for further ﬁnetuning T5.
Revised: It is mixed with the original parallel corpus for further ﬁnetuning T5.

Original: In other words, our dataset becomes
Revised: In other words, our dataset becomes

Original: D p (cid:83) { ( T ( i ) u , ˆ y ( i ) search ) :
Revised: D p (cid:83) { ( T ( i ) u , ˆ y ( i ) search ) :

Original: T ( i ) u ∈ D u } .
Revised: T ( i ) u ∈ D u } .

Original: T5 is ﬁnetuned by the same cross-entropy loss as Equation (2).
Revised: T5 is ﬁnetuned by the same cross-entropy loss as Equation (2).

Original: Details are not repeated.
Revised: Details are not repeated.

Original: 4. Inference
Revised: 4. Inference

Original: For inference on the test set, we only use the twostage ﬁne-tuned T5 (i.e., ﬁne-tuned on search results in Section 4.3) to predict the output.
Revised: For inference on the test set, we only use the twostage ﬁne-tuned T5 (i.e., ﬁne-tuned on search results in Section 4.3) to predict the output.

Original: We do not have the search procedure during inference.
Revised: We do not have the search procedure during inference.

Original: In this way, our inference efﬁciency is improved than the search approach, because we do not have to evaluate multiple candidate sentences during prediction.
Revised: In this way, our inference efﬁciency is improved than the search approach, because we do not have to evaluate multiple candidate sentences during prediction.

Original: More importantly, we leverage the power of pre-trained language models, and are able to generate more ﬂuent sentences than the search itself.
Revised: More importantly, we leverage the power of pre-trained language models, and are able to generate more ﬂuent sentences than the search itself.

Original: Compared with one-stage ﬁne-tuning (Section 4.1), T5 this time is explicitly trained with pseudo-groundtruth that has high semantic coverage.
Revised: Compared with one-stage ﬁne-tuning (Section 4.1), T5 this time is explicitly trained with pseudo-groundtruth that has high semantic coverage.

Original: We will show in Section 5.1 that, after search and learning, T5 achieves near-perfect semantic coverage on the E2E dataset.
Revised: We will show in Section 5.1 that, after search and learning, T5 achieves near-perfect semantic coverage on the E2E dataset.

Original: [Table]
Revised: [Table]

Original: Dhingra

Original: (2019), we consider PARENT as our main metrics.

Original:  Experiments 
Revised:  Experiments 5. Experiment I: E2E Dataset

Original: Experiment I: E2E Dataset

Original: Dataset.
Revised: Dataset.

Original: In this experiment, we used the E2Edataset 1 (Novikova et al., 2017) to evaluate our approach.
Revised: In this experiment, we used the E2E dataset 1 (Novikova et al., 2017) to evaluate our approach.

Original: E2E is a crowdsourced dataset for data-totext generation and contains more than 50K table–text pairs for the restaurant domain.
Revised: E2E is a crowdsourced dataset for data-totext generation and contains more than 50K table– text pairs for the restaurant domain.

Original: For each datasample, the input contains 3–8 slots, and the reference contains one or a few sentences as the output.
Revised: For each data sample, the input contains 3–8 slots, and the reference contains one or a few sentences as the output.

Original: We followed the standard train/val/test split.
Revised: We followed the standard train/val/test split.

Original: Implementation details.
Revised: Implementation details.

Original: We used the T5-smallmodel (Raffel et al., 2020), which comprises 6layers in the encoder and the decoder.
Revised: We used the T5-small model (Raffel et al., 2020), which compriseslayers in the encoder and the decoder.

Original: We trainedmodels using the AdamW (Loshchilov and Hutter,2018) optimizer, with an initial learning rate of3e-4 and a batch size of 64.
Revised: We trained models using the AdamW (Loshchilov and Hutter, 2018) optimizer, with an initial learning rate of 3e-4 and a batch size of 64.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: We used the standardevaluation scripts accompanied with the E2Edataset (Novikova et al., 2017), including BLEU(Papineni et al., 2002), NIST (Doddington, 2002),
Revised: We used the standard evaluation scripts accompanied with the E2E dataset (Novikova et al., 2017), including BLEU (Papineni et al., 2002), NIST (Doddington, 2002),

Original: METEOR (Lavie and Agarwal, 2007), ROUGE-L(Lin, 2004), and CIDEr (Vedantam et al., 2015).
Revised: METEOR (Lavie and Agarwal, 2007), ROUGE-L (Lin, 2004), and CIDEr (Vedantam et al., 2015).

Original: Recently, Dhingra et al.
Revised: Recently, Dhingra et al.

Original: (2019) observe that
Revised: (2019) observe that

Original: BLEU correlates poorly to human satisfaction fordata-to-text generation, as we have diverse references.
Revised: BLEU correlates poorly to human satisfaction for data-to-text generation, as we have diverse references.

Original: They propose a set of PARENT metrics (including precision, recall, and the F-score) againstboth the references and the input data.
Revised: They propose a set of PARENT metrics (including precision, recall, and the F-score) against both the references and the input data.

Original: They show
Revised: They show

Original: PARENT metrics have high correlation with human judgment.
Revised: PARENT metrics have high correlation with human judgment.

Original: Based on such evidence, we do notrely on BLEU for evaluating our approaches, andwe adopt PARENT as the main metric.
Revised: Based on such evidence, we do not rely on BLEU for evaluating our approaches, and we adopt PARENT as the main metric.

Original: In addition, we use GPT-2 perplexity (withoutﬁne-tuning) to estimate the ﬂuency of generatedtext, and present the average sentence length (Av- gLen) for reference.
Revised: In addition, we use GPT-2 perplexity (without ﬁne-tuning) to estimate the ﬂuency of generated text, and present the average sentence length (Av- gLen) for reference.

Original: We also computed semantic coverage ratio, which is the fraction of input slots that appear verbatim in the output.
Revised: We also computed semantic coverage ratio, which is the fraction of input slots that appear verbatim in the output.

Original: This requirement appears to be strict, but is actually a good approximation, because most slots contain only one or a few words and some slots are proper nouns that should not be changed.
Revised: This requirement appears to be strict, but is actually a good approximation, because most slots contain only one or a few words and some slots are proper nouns that should not be changed.

Original: We also conducted human evaluation on a randomly selected subset of test samples, and the coverage percentage (Table 3) is close to this automatic metric.
Revised: We also conducted human evaluation on a randomly selected subset of test samples, and the coverage percentage (Table 3) is close to this automatic metric.

Original: Results.
Revised: Results.

Original: Table 1 shows the results on the E2E dataset.
Revised: Table 1 shows the results on the E2E dataset.

Original: We consider a few-shot setting, where we have 1% parallel samples as D p , and another 4% samples as D u with input tables only.
Revised: We consider a few-shot setting, where we have 1% parallel samples as D p , and another 4% samples as D u with input tables only.

Original: Before few-shot learning, we ﬁne-tuned T5 with 100% samples (Line 4) and 5% samples (Line 5), respectively, being an “upper bound” performance of our few-shot learning.
Revised: Before few-shot learning, we ﬁne-tuned T5 with 100% samples (Line 4) and 5% samples (Line 5), respectively, being an “upper bound” performance of our few-shot learning.

Original: We see that, with the entire dataset, ﬁne-tuning T5 achieves similar scores to previous state-of-the-art models, including TGEN (Novikova et al., 2017), SLUG (Juraska et al., 2018), and the S R 1 model (Shen et al., 2019).
Revised: We see that, with the entire dataset, ﬁne-tuning T5 achieves similar scores to previous state-of-the-art models, including TGEN (Novikova et al., 2017), SLUG (Juraska et al., 2018), and the S R 1 model (Shen et al., 2019).

Original: This shows that the use of T5 sets up a solid foundation for our study.
Revised: This shows that the use of T5 sets up a solid foundation for our study.

Original: We started few-shot learning by directly ﬁnetuning T5 on the small parallel training set.
Revised: We started few-shot learning by directly ﬁnetuning T5 on the small parallel training set.

Original: We observe that the performance worsens in all metrics (comparing Lines 4–6), especially the coverage drops quickly from 97.74% to 84.46%.
Revised: We observe that the performance worsens in all metrics (comparing Lines 4–6), especially the coverage drops quickly from 97.74% to 84.46%.

Original: We would like to see if a small unlabeled dataset
Revised: We would like to see if a small unlabeled dataset

Original: D u that contains tables only could help the performance.
Revised: D u that contains tables only could help the performance.

Original: We experimented self-training (Zhu and
Revised: We experimented self-training (Zhu and

Original: Goldberg, 2009), which is a common strategy for semi-supervised machine learning.
Revised: Goldberg, 2009), which is a common strategy for semi-supervised machine learning.

Original: In this competing method, we ﬁrst ﬁne-tune T5 on the parallel corpus D p , and use it to predict the output on
Revised: In this competing method, we ﬁrst ﬁne-tune T5 on the parallel corpus D p , and use it to predict the output on

Original: D u .
Revised: D u .

Original: The predicted sentences are treated as pseudogroundtruth for further ﬁne-tuning.
Revised: The predicted sentences are treated as pseudogroundtruth for further ﬁne-tuning.

Original: Unfortunately, we observe from Lines 6 and 7 that such strategydoes not help the performance much.
Revised: Unfortunately, we observe from Lines 6 and 7 that such strategy does not help the performance much.

Original: Finally, we applied our S&L approach, shown in
Revised: Finally, we applied our S&L approach, shown in

Original: Line 8.
Revised: Line 8.

Original: Results show that our approach achievesthe highest performance in the few-shot settingin terms of most metrics.
Revised: Results show that our approach achieves the highest performance in the few-shot setting in terms of most metrics.

Original: Especially, we achievea 98.50% coverage of input slots, which mostlysolves the low coverage problem.
Revised: Especially, we achieve a 98.50% coverage of input slots, which mostly solves the low coverage problem.

Original: Based on the numerical results, it appears thatour model generate less ﬂuent sentences, givenby high perplexity (PPL) scores.
Revised: Based on the numerical results, it appears that our model generate less ﬂuent sentences, given by high perplexity (PPL) scores.

Original: However, wenotice that our sentences are longer and containmore input slots, which are oftentimes very speciﬁcinformation such as the restaurant name (in the
Revised: However, we notice that our sentences are longer and contain more input slots, which are oftentimes very speciﬁc information such as the restaurant name (in the

Original: E2E dataset) as a proper noun.
Revised: E2E dataset) as a proper noun.

Original: Therefore, it isunderstandable that our PPL is slightly higher, butin general, all models are in the same ballpark interms of ﬂuency.
Revised: Therefore, it is understandable that our PPL is slightly higher, but in general, all models are in the same ballpark in terms of ﬂuency.

Original: This will be further analyzed byhuman evaluation (Table 3).
Revised: This will be further analyzed by human evaluation (Table 3).

Original: Generally, our S&L approach (Line 8) achievescomparable results to T5 trained with 4 times moreparallel data (Line 5) in several metrics, such as
Revised: Generally, our S&L approach (Line 8) achieves comparable results to T5 trained with 4 times more parallel data (Line 5) in several metrics, such as

Original: M ETEOR and CIDEr.
Revised: M ETEOR and CIDEr.

Original: In terms of PARENT metricsthat are speciﬁcally designed for data-to-text generation, we observe our S&L approach outperforms
Revised: In terms of PARENT metrics that are speciﬁcally designed for data-to-text generation, we observe our S&L approach outperforms

Original: Line 5 with a reasonable margin.
Revised: Line 5 with a reasonable margin.

Original: It even achievesclose PARENT scores to the fully-supervised setting (Line 4).
Revised: It even achieves close PARENT scores to the fully-supervised setting (Line 4).

Original: Since we only used 1% parallel dataand 4% unlabeled input, this convincingly showthat our approach achieves high performance forfew-shot data-to-text generation.
Revised: Since we only used 1% parallel data and 4% unlabeled input, this convincingly show that our approach achieves high performance for few-shot data-to-text generation.

Original: Experiment II: WikiBio Dataset
Revised: Experiment II : WikiBio Dataset

Original: Dataset.
Revised: Dataset.

Original: We further evaluate our approach on the
Revised: We further evaluate our approach on the

Original: Humans domain of the WikiBio dataset 3 (Lebretet{+
 (Lebret et+} al., 2016).
Revised: Humans domain of the WikiBio dataset {+
 (Lebret et+} al., 2016).

Original: WikiBio contains 700K English biographies from Wikipedia, associated with a tabular infobox.
Revised: WikiBio contains 700K English biographies from Wikipedia, associated with a tabular infobox.

Original: For each biography, the ﬁrst sentenceof the article is treated as the reference.
Revised: For each biography, the ﬁrst sentence of the article is treated as the reference.

Original: In our few-shot setting, we used 100 parallelsamples as the training set D p , following one of the settings in Chen et al. (2020b).
Revised: In our few-shot setting, we used 100 parallel samples as the training set D p , following one of the settings in Chen et al. (2020b).

Original: In accordance with our assumption, we included another 400 samples of unlabeled input tables as D u .
Revised: In accordance with our assumption, we included another 400 samples of unlabeled input tables as D u .

Original: When comparing with Chen et al. (2020b), however, we did not use
Revised: When comparing with Chen et al. (2020b), however, we did not use

Original: D u , but synthesized 400 samples by recombining the table slots in D p .
Revised: D u , but synthesized 400 samples by recombining the table slots in D p .

Original: This sets up a fair comparison, as we did not include any new data.
Revised: This sets up a fair comparison, as we did not include any new data.

Original: We validated our approach on 1000 samples, and tested it on the standard split.
Revised: We validated our approach on 1000 samples, and tested it on the standard split.

Original: Implementation details.
Revised: Implementation details.

Original: We used the T5-base model, which consists of a 12-layer Transformer encoder and decoder.
Revised: We used the T5-base model, which consists of a 12-layer Transformer encoder and decoder.

Original: This sets up a fair comparison with the prior work for few-shot data-to-text generation (Chen et al., 2020b), which uses a 12-layer
Revised: This sets up a fair comparison with the prior work for few-shot data-to-text generation (Chen et al., 2020b), which uses a 12-layer

Original: GPT-2 model.
Revised: GPT-2 model.

Original: Due to GPU memory constraints, we use a batch size of 20 during training and accumulate gradients for 3 steps, which results in an actual batch size of 60.
Revised: Due to GPU memory constraints, we use a batch size of 20 during training and accumulate gradients for 3 steps, which results in an actual batch size of 60.

Original: Other implementation details are mostly adopted from Section 5.1.
Revised: Other implementation details are mostly adopted from Section 5.1.

Original: In WikiBio, we followed a different strategy to add missing slots.
Revised: In WikiBio, we followed a different strategy to add missing slots.

Original: Unlike E2E, WikiBio contains longer input tables and not all input information is present in the references, i.e., the ﬁrst sentence of the Wiki article.
Revised: Unlike E2E, WikiBio contains longer input tables and not all input information is present in the references, i.e., the ﬁrst sentence of the Wiki article.

Original: Therefore, our search algorithm inserts a subset of input slots, determined by cooccurrence statistics on the validation dataset.
Revised: Therefore, our search algorithm inserts a subset of input slots, determined by cooccurrence statistics on the validation dataset.

Original: As a heuristic, we select slots which occur at least in 10% tables of the dataset and are present in at least 10% output references.
Revised: As a heuristic, we select slots which occur at least in 10% tables of the dataset and are present in at least 10% output references.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: We included BLEU for reference, as it is the metric in Chen et al.
Revised: We included BLEU for reference, as it is the metric in Chen et al.

Original: (2020b).
Revised: (2020b).

Original: However, we still consider the PARENT scores as our main metric in our study, due to the evidence Dhingra et al.
Revised: However, we still consider the PARENT scores as our main metric in our study, due to the evidence Dhingra et al.

Original: We did not compute semantic coverage ratio for
Revised: We did not compute semantic coverage ratio for

Original: WikiBio, since it is known that the reference (the ﬁrst sentence of a Wiki article) would not cover all the slots in the infobox.
Revised: WikiBio, since it is known that the reference (the ﬁrst sentence of a Wiki article) would not cover all the slots in the infobox.

Original: However, PARENT is able to measure the coverage of input slots as it considers both input and reference when computing precision, recall, and the F-score.
Revised: However, PARENT is able to measure the coverage of input slots as it considers both input and reference when computing precision, recall, and the F-score.

Original: Results.
Revised: Results.

Original: Table 2 shows the result on WikiBio dataset.
Revised: Table 2 shows the result on WikiBio dataset.

Original: Since Chen et al.
Revised: Since Chen et al.

Original: (2020b) do not report
Revised: (2020b) do not report

Original: PARENT metrics for their ﬁne-tuned GPT-2 model, we replicated the model by using their released code.
Revised: PARENT metrics for their ﬁne-tuned GPT-2 model, we replicated the model by using their released code.

Original: 4 As seen from Lines 1–2, we achieved a sim-https: calls papers/#
Revised: 4 As seen from Lines 1–2, we achieved a sim-

Original: [Table]
Revised: [Table]

Original: ilar BLEU score to Chen et al. (2020b), showingthat our replication is fair.
Revised: ilar BLEU score to Chen et al. (2020b), showing that our replication is fair.

Original: We apply our S&L to the WikiBio dataset.
Revised: We apply our S&L to the WikiBio dataset.

Original: Wesee that, with 400 unlabeled tables, we improve the
Revised: We see that, with 400 unlabeled tables, we improve the

Original: T5 model by 2–3 points in PARENT Recall and F1(Lines 4, 7–8).
Revised: T5 model by 2–3 points in PARENT Recall and F (Lines 4, 7–8).

Original: This suggests that our model notonly generates high-quality sentences in general forthe data-to-text task, but also has a higher coverageof input slots due to the nature of PARENT metrics.
Revised: This suggests that our model not only generates high-quality sentences in general for the data-to-text task, but also has a higher coverage of input slots due to the nature of PARENT metrics.

Original: We also obtain ﬂuent sentences for this dataset,shown by relatively low PPL.
Revised: We also obtain ﬂuent sentences for this dataset, shown by relatively low PPL.

Original: These results aregenerally consistent with E2E experiments.
Revised: These results are generally consistent with E2E experiments.

Original: Compared with previous state-of-the-art fewshot learning (Chen et al., 2020b), our setting usesextra 400 tables.
Revised: Compared with previous state-of-the-art fewshot learning (Chen et al., 2020b), our setting uses extra 400 tables.

Original: To make a fair comparison, wesynthesized 400 tables by recombining the slotswithout using any additional data.
Revised: To make a fair comparison, we synthesized 400 tables by recombining the slots without using any additional data.

Original: Comparing Line6 and Line 2 suggests that, even without an unlabeled corpus, our approach still outperforms theprevious state-of-the-art model in all metrics.
Revised: Comparing Line 6 and Line 2 suggests that, even without an unlabeled corpus, our approach still outperforms the previous state-of-the-art model in all metrics.

Original: We also admit that recombining table slot doesnot give as good performance as using additionalunlabeled tables (Line 6 vs. Line 8).
Revised: We also admit that recombining table slot does not give as good performance as using additional unlabeled tables (Line 6 vs. Line 8).

Original: A plausiblereason is that new tables are able to train T5 withmore slot values, and this is especially useful forfew-shot data-to-text generation, where we onlyhave a few hundred parallel samples.
Revised: A plausible reason is that new tables are able to train T5 with more slot values, and this is especially useful for few-shot data-to-text generation, where we only have a few hundred parallel samples.

Original: Recombining table slots cannot serve for this goal.
Revised: Recombining table slots cannot serve for this goal.

Original: Futureresearch can be addressed here on effective dataaugmentation for data-to-text generation.
Revised: Future research can be addressed here on effective data augmentation for data-to-text generation.

Original: 5. Analysis
Revised: 5. Analysis

Original: In this part, we provide detailed analysis of our approach.
Revised: In this part, we provide detailed analysis of our approach.

Original: Due to the limitation of space and availableresources, we chose E2E as our testbed.
Revised: Due to the limitation of space and available resources, we chose E2E as our testbed.

Original: Human evaluation.
Revised: Human evaluation.

Original: We would like to conduct human evaluation for our model, as automatic metrics may not fully reﬂect the performance of a text generator.
Revised: We would like to conduct human evaluation for our model, as automatic metrics may not fully reﬂect the performance of a text generator.

Original: We selected a random subset of 50 samples and obtained the outputs from T5 self-train and
Revised: We selected a random subset of 50 samples and obtained the outputs from T5 self-train and

Original: T5 S&L.
Revised: T5 S&L.

Original: While the subset appears to be small, we will compute statistical signiﬁcance to demonstrate that it sufﬁces to draw a conclusion.
Revised: While the subset appears to be small, we will compute statistical signiﬁcance to demonstrate that it sufﬁces to draw a conclusion.

Original: We asked three annotators to evaluate each table– text pair on three criteria: coverage , ﬂuency , and overall quality .
Revised: We asked three annotators to evaluate each table– text pair on three criteria: coverage , ﬂuency , and overall quality .

Original: Coverage measures the number of input table slots present in the text divided by total number of input slots.
Revised: Coverage measures the number of input table slots present in the text divided by total number of input slots.

Original: Fluency measures if the sentence is clear, natural, and grammatically correct (3: Fluent, natural and grammatically correct; 2: Mostly ﬂuent, with minor errors; 1: Not ﬂuent, multiple grammatical errors).
Revised: Fluency measures if the sentence is clear, natural, and grammatically correct (3: Fluent, natural and grammatically correct; 2: Mostly ﬂuent, with minor errors; 1: Not ﬂuent, multiple grammatical errors).

Original: The annotators were also asked to assign an overall quality to each sentence (3: good quality; 2: average quality; 1: poor quality).
Revised: The annotators were also asked to assign an overall quality to each sentence (3: good quality; 2: average quality; 1: poor quality).

Original: Our human annotation was conducted in a strict blind fashion, i.e., samples were randomly shufﬂed and the annotator did not know the model of a generated sentence.
Revised: Our human annotation was conducted in a strict blind fashion, i.e., samples were randomly shufﬂed and the annotator did not know the model of a generated sentence.

Original: Table 3 presents human evaluation results.
Revised: Table 3 presents human evaluation results.

Original: We see that the human-annotated coverage ratio is similar to automatic counting in Table 1.
Revised: We see that the human-annotated coverage ratio is similar to automatic counting in Table 1.

Original: Our S&L achieves near-perfect semantic coverage, whereas a ﬁne-tuned T5 with self-training only achieves 81.66% coverage.
Revised: Our S&L achieves near-perfect semantic coverage, whereas a ﬁne-tuned T5 with self-training only achieves 81.66% coverage.

Original: In terms of ﬂuency, S&L is slightly worse than T5 self-training.
Revised: In terms of ﬂuency, S&L is slightly worse than T5 self-training.

Original: However, the difference is 1/3 standard deviation, which is relatively small compared with our improvements in other aspects.
Revised: However, the difference is 1/3 standard deviation, which is relatively small compared with our improvements in other aspects.

Original: The overall quality of our approach is considerably higher than the competing method by more than two standard deviations, showing the effectiveness of our approach.
Revised: The overall quality of our approach is considerably higher than the competing method by more than two standard deviations, showing the effectiveness of our approach.

Original: The human annoation results are generally consistent with our automatic measures (except BLEU).
Revised: The human annoation results are generally consistent with our automatic measures (except BLEU).

Original: Search and learning vs. Search for inference.
Revised: Search and learning vs. Search for inference.

Original: An interesting analysis of our approach is to see
Revised: An interesting analysis of our approach is to see

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: The p -values are given by two-sided Wilcoxon paired test.
Revised: The p -values are given by two-sided Wilcoxon paired test.

Original: It only shows whether our annotated subset has collected enough evidence for drawing a conclusion, instead of how different two models are.
Revised: It only shows whether our annotated subset has collected enough evidence for drawing a conclusion, instead of how different two models are.

Original: We show the standard deviation, which roughly estimates if the gap is relatively large or not.
Revised: We show the standard deviation, which roughly estimates if the gap is relatively large or not.

Original: [Table]
Revised: [Table]

Original: Inference time was obtained by predicting the test set on a single V100 GPU.
Revised: Inference time was obtained by predicting the test set on a single V100 GPU.

Original: how search and learning (S&L) improves the searchitself.
Revised: how search and learning (S&L) improves the search itself.

Original: This can be seen by performing search forinference on the test set.
Revised: This can be seen by performing search for inference on the test set.

Original: From Table 4, we observethat S&L largely improves the search results interms of all metrics.
Revised: From Table 4, we observe that S&L largely improves the search results in terms of all metrics.

Original: Especially, the PPL of S&Lis considerably smaller than search for inference.
Revised: Especially, the PPL of S&L is considerably smaller than search for inference.

Original: This shows that the second-stage ﬁne-tuning (Section 4.3) not only learns from the search results forhigher semantic coverage, but also smooths out thesearch noise and yields better sentences in general.
Revised: This shows that the second-stage ﬁne-tuning (Section 4.3) not only learns from the search results for higher semantic coverage, but also smooths out the search noise and yields better sentences in general.

Original: In addition, S&L has a better inference efﬁciency.
Revised: In addition, S&L has a better inference efﬁciency.

Original: Despite our batch implementation and the [-V
-]
Revised: Despite our batch implementation and the [-V
-]V

Original: GPU device, search for inference takes 45% moretime than S&L in inference.
Revised: GPU device, search for inference takes 45% more time than S&L in inference.

Original: This shows that ourapproach is efﬁcient in practice.
Revised: This shows that our approach is efﬁcient in practice.

Original: Case Study.
Revised: Case Study.

Original: We conduct a case study in Figure 2.
Revised: We conduct a case study in Figure 2.

Original: There are 7 references for this data sample,and we present two references to illustrate that the input slots may be missing even in references.
Revised: There are 7 references for this data sample, and we present two references to illustrate that the input slots may be missing even in references.

Original: We see that T5 (ﬁne-tuned with few-shot D p or further self-trained with D u ) yields ﬂuent sentences, and does not generate wrong information as addressed in (Gong et al., 2020).
Revised: We see that T5 (ﬁne-tuned with few-shot D p or further self-trained with D u ) yields ﬂuent sentences, and does not generate wrong information as addressed in (Gong et al., 2020).

Original: However, a few input slots are missing in T5’s output.
Revised: However, a few input slots are missing in T5’s output.

Original: If we perform search for inference, we are guaranteed to have perfect slot coverage, but the sentence may not be ﬂuent, such as “ it is near crowne plaza high customer rating in riverside area not family friendly hotel ”.
Revised: If we perform search for inference, we are guaranteed to have perfect slot coverage, but the sentence may not be ﬂuent, such as “ it is near crowne plaza high customer rating in riverside area not family friendly hotel ”.

Original: Our
Revised: Our

Original: S&L approach yields a ﬂuent sentence with a high semantic coverage.
Revised: S&L approach yields a ﬂuent sentence with a high semantic coverage.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: In this work, we present a search-and-learning approach to address the low coverage problem for few-shot data-to-text generation.
Revised: In this work, we present a search-and-learning approach to address the low coverage problem for few-shot data-to-text generation.

Original: We ﬁrst ﬁne-tune the pre-trained T5 language model based on a small parallel corpus.
Revised: We ﬁrst ﬁne-tune the pre-trained T5 language model based on a small parallel corpus.

Original: Then, we use the T5 to predict on an unlabeled corpus, and search for higher semantic coverage.
Revised: Then, we use the T5 to predict on an unlabeled corpus, and search for higher semantic coverage.

Original: The T5 is further ﬁne-tuned with search results.
Revised: The T5 is further ﬁne-tuned with search results.

Original: Experiments on E2E and WikiBio datasets show that our model achieves high performance than previous approaches to few-shot data-to-text generation, largely closing the gap between fewshot and fully supervised learning
Revised: Experiments on E2E and WikiBio datasets show that our model achieves high performance than previous approaches to few-shot data-to-text generation, largely closing the gap between fewshot and fully supervised learning

Original: In future work, we would apply our S&L framework to ﬁdelity-oriented text generation by revising counterfactual text and further learning from the edit results.
Revised: In future work, we would apply our S&L framework to ﬁdelity-oriented text generation by revising counterfactual text and further learning from the edit results.

Original: This would beneﬁt various NLG tasks, such as paraphrasing and summarization.
Revised: This would beneﬁt various NLG tasks, such as paraphrasing and summarization.

Original: Acknowledgements
Revised: Acknowledgements

Original: Shailza Jolly was supported by the TU Kaiserslautern CS Ph.D. scholarship program, the BMBFproject XAINES (Grant 01IW20005), and
Revised: Shailza Jolly was supported by the TU Kaiserslautern CS Ph.D. scholarship program, the BMBF project XAINES (Grant 01IW20005), and

Original: the
Revised: the

Original: NVIDIA AI Lab (NVAIL) program.
Revised: NVIDIA AI Lab (NVAIL) program.

Original: A Complete List of Rules
Revised: A Complete List of Rules

Original: As explained in Section 4.2, we insert a template-based phrase if a slot value does not appear in the output text.
Revised: As explained in Section 4.2, we insert a template-based phrase if a slot value does not appear in the output text.

Original: Below is the complete list of the rules we used in our experiments.
Revised: Below is the complete list of the rules we used in our experiments.

Original: SN refers to the slot name [Table] and
Revised: SN refers to the slot name, and

Original: SV refers to the slot value.
Revised: SV refers to the slot value.

Original: Despite the simplicity of the rules, our search-and-learning (S&L) approach improves the semantic coverage to a large extent for few-shot data-to-text generation.[Table]
Revised: Despite the simplicity of the rules, our search-and-learning (S&L) approach improves the semantic coverage to a large extent for few-shot data-to-text generation.

Original: [Table]
Revised: [Table]

Original: Then the phrase template is

Original: [Table]
Revised: [Table]