Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning
Year: 2023

Abstract:
We present a novel approach for structured datato-text generation that addresses the limitations of existing methods that primarily focus on specific types of structured data. Our proposed method aims to improve performance in multitask training, zero-shot and few-shot scenarios by providing a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. We demonstrate that our proposed approach can effectively adapt to new structured forms, and can improve performance in comparison to current methods. For example, our method resulted in a 66% improvement in zero-shot BLEU scores when transferring models trained on table inputs to a knowledge graph dataset. Our proposed method is an important step towards a more general data-to-text generation framework. 1

heading: 1 Introduction
text: Data-to-text generation is the task of converting structured data into natural language text that can be easily understood by humans. Previous methods for data-to-text generation have been limited to specific structured forms. For example, graph neural networks (GNNs) have been used to encode knowledge graph input (Rik Koncel-Kedziorski and Hajishirzi, 2019; Ribeiro et al., 2020; Guo et al., 2020; Li et al., 2021), while table-specific encoders have been proposed for tables (Liu et al., 2017; Bao et al., 2018; Nema et al., 2018; Jain et al., 2018; Wang et al., 2022). However, these methods are not easily transferable to other structured forms, creating a barrier for scientific development and preventing models from learning across tasks. Recent work has attempted to address the problem of limited structured form applicability by using pretrained language models (PLMs) as a single text-to-text framework for all data structures, by
1Our code will be open-sourced at anonymous-link.
linearizing the data as text sequences. As shown by Kale and Rastogi (2020); Xie et al. (2022), these methods achieve state-of-the-art performance on a wide range of data-to-text tasks.
Despite the advancements made in the field, there are still unresolved questions regarding the relationship between various structured forms, particularly in the context of zero-shot or few-shot settings, where models are required to rapidly adapt to new structured forms. This is particularly pertinent in cases of data scarcity, when structured forms vary across different domains and there is a limited amount of data available for a specific structured form, but a single model is needed to operate on all of them. Such an example is to adapt a knowledge-graph-to-text model to a new domain with data in table format. Even when there is an abundance of data, developing a universal model that can handle all structured forms remains a challenging task. As seen in Xie et al. (2022), a multi-task trained model may perform worse than a single-task model on table inputs. One important reason for such performance drop is because previous research has not fully examined the impact of various linearization methods on these tasks and their effect on cross-task generalization. Despite the use of text-to-text transformers, linearization methods for various structured forms remain diverse, and even within one structured form, linearization can vary across studies. For example, the linearization of KG triples differs in Nan et al. (2021) and Xie et al. (2022), highlighting the need for further research on the relationship between data formats and data-to-text tasks.
In this paper, we address the unresolved questions surrounding the relationship between various structured forms by introducing a unified representation for knowledge graphs, tables, and meaning representations. We demonstrate that our method allows for the conversion of knowledge graph triples and meaning representations into virtual ta-
16171
bles, which can then be linearized in a consistent manner. Through evaluating our approach on five representative data-to-text tasks across the aforementioned formats, we show that our method not only achieves competitive performance compared to other data-specific linearizations for individual tasks, but also leads to significant improvements in transfer learning scenarios across structured forms, particularly in zero-shot or few-shot settings. For example, using the unified representation improves the zero-shot BLEU score by relatively 66% when transferring from ToTTo (Parikh et al., 2020) to DART (Nan et al., 2021). Additionally, our approach results in improved performance when used in multi-task settings compared to models trained with varied linearizations. These results provide a clear indication of the effectiveness of our proposed unified representation in enhancing cross-task generalization.

heading: 2 Related Work
text: Data-Type Specific Knowledge Encoding Research has been conducted to encode structured knowledge using various models and approaches, including Graph Neural Networks (GNNs) (Rik Koncel-Kedziorski and Hajishirzi, 2019; Ribeiro et al., 2020; Guo et al., 2020; Li et al., 2021; Song et al., 2018; Ribeiro et al., 2019; Cai and Lam, 2020; Zhang et al., 2020; Ribeiro et al., 2021b; Schmitt et al., 2021) and neural encoder-decoder models based on Gated Recurrent Units (GRUs) and Transformers (Gehrmann et al., 2018; Ferreira et al., 2019). These models have been used to assist in encoding knowledge graph inputs and meaning representations. Additionally, several models have been proposed for table-to-text generation, including approaches that combine content selection or entity memory in a Long Short-Term Memory (LSTM) model (Puduppully et al., 2018, 2019), and others that focus on table-specific encoders (Liu et al., 2017; Bao et al., 2018; Nema et al., 2018; Jain et al., 2018). More recent studies have utilized the capabilities of pre-trained language models in their designs, but have also incorporated specialized encoder structures or attention mechanisms specifically for table inputs. These include encoder-only models (Arik and Pfister, 2019; Yin et al., 2020; Herzig et al., 2020; Huang et al., 2020; Wang et al., 2021; Iida et al., 2021; Eisenschlos et al., 2021; Yang et al., 2022), as well as encoder-decoder models (Cao, 2020; Andrejczuk et al., 2022; Wang et al.,
2022). However, it should be noted that the encoder structures of these works are specifically tailored for table input and cannot be directly applied to other types of data.
Structured Data Linearization Recent developments in pretrained language models (Devlin et al., 2019; Radford et al., 2019; Lewis et al., 2020; Raffel et al., 2020) have made it possible to use a single text-to-text framework for various types of data by linearizing them as text sequences. Studies have been conducted on finetuning PLMs on table input (Parikh et al., 2020) and knowledge graph input (Kasner and Dušek, 2020; Ribeiro et al., 2021a), single-task and multi-task training on a collection of structured data grounding tasks (Xie et al., 2022), and the effectiveness of pretraining and fine-tuning strategies for data-to-text tasks (Kale and Rastogi, 2020) and table-based question answering tasks (Shi et al., 2022). These studies have consistently found that linearizing structured data as a sequence of tokens without modifying the model structure, is a simple yet effective strategy that outperforms pipelined neural architectures specifically tailored to particular data types.
Zero/Few-Shot Data-to-Text Generation The studies such as Chen et al. (2020b) and Ke et al. (2021) have evaluated the zero and few-shot performance of PLMs on knowledge graph input, highlighting the benefits of a joint pretraining strategy on knowledge graphs and texts for learning better KG representations. Keymanesh et al. (2022) studied the prompt-tuning method for KG-to-text generation and found it to be effective in a fewshot setting. Chen et al. (2020d) combines PLM with a table content selector using a switch policy. Other researchers have also explored methods such as data augmentation (Chang et al., 2021) and retrieval-based input augmentation (Su et al., 2021) to aid in few-shot data-to-text generation. Kasner and Dusek (2022) proposes a pipeline approach involving a sequence of operations, such as ordering and aggregation, and only finetunes the PLMs of these modules to make the pipeline more domainindependent.

heading: 3 Unified Representation
text: In this section, we demonstrate that structured data, such as tables, highlighted cells, knowledge graph triples, and meaning representations, can be linearized in a consistent manner. We begin by show-
ing in Section 3.1 how knowledge graph triples and meaning representations can be mapped to a virtual table and subsequently linearized in the same way as tables. Next, in Section 3.2, we demonstrate the process of linearizing a table or highlighted cells. The entire method is illustrated in Figure 1.

heading: 3.1 Virtual Table
text: KG Triple The method for converting triples from a connected sub-graph into a virtual table involves using the tail node of each triple as a cell value and the relation as the column header. Nodes that do not appear as tail nodes are not assigned a column header. An example is provided in Figure 1. "William Wasmund" does not have a column header assigned since it never appears as a tail node. If a set of knowledge graph triples contains multiple connected components, each component is converted into a separate table.
Meaning Representation We focus on textual MRs that appear as a list of comma-separated attribute-value pairs (Dušek et al., 2020). These MRs can be treated as virtual tables by associating each Attribute[Value] with a cell value, represented by the "Value", and the "Attribute" as its corresponding column header. An example of this can be seen in Figure 1.

heading: 3.2 Linearization of Tables
text: After converting both KGs and MRs into virtual tables, we end up with only table inputs that need to be linearized. In this section, we discuss one choice of such a linearization method, motivated by ToTTo linearization (Parikh et al., 2020). Additionally, we will provide a specific example of how to linearize Table 1 in the following sections.
Basic Units The basic units for linearization are presented in Table 2. Each unit is defined by a start
symbol, <xx>, and an end symbol, </xx>.
Linearization of Highlighted Cells To linearize the highlighted cells, we proceed in a left-to-right, top-to-bottom order. For instance, in Table 1, the linearization of the highlighted cells (in yellow background) appears as follows: 2
1 <title> Alma Jodorowsky </title> 2 <sub_title > Filmography </sub_title > 3 <table> 4 <cell> 2016 5 <col_header > Year </col_header > 6 </cell> 7 <cell> Kids in Love 8 <col_header > Title </col_header > 9 </cell>
10 <cell> Evelyn 11 <col_header > Role </col_header > 12 </cell> 13 </table>
2Indentation is used for clarity in this example, but it is not present in the actual input.
Linearization of (Sub)Table A row-wise linearization of the entire Table 1 is: 1 <title > Alma Jodorowsky </title> 2 <sub\_title > Filmography </sub\_title > 3 <table > 4 <row> 5 <cell> 2014 6 <col_header > Year </col_header > 7 </cell> 8 <cell> La Vie devant elles [fr] 9 <col_header > Title </col_header > 10 </cell> 11 <cell> Solana 12 <col_header > Role </col_header > 13 </cell> 14 </row> 15 ...( other rows)... 16 </table >
Such a linearization method can also be applied to column-wise. An example is provided in the Appendix B.

heading: 4 Experiments
text: Datasets We test our method on five data-totext datasets: The ToTTo dataset (Parikh et al., 2020) poses the challenge of generating a onesentence description, given highlighted cells from a Wikipedia table. Our models are evaluated on the validation set, as the annotations for the test set are not publicly available. The DART corpus (Nan et al., 2021) is an open-domain structured data-to-text resource, consisting of entity-relation triples. The LogicNLG dataset (Chen et al., 2020a) investigates the ability to generate logical inferences from table contents to implicit insights, as the target sentences. The WebNLG dataset (Gardent et al., 2017) includes triples from 15 DBpedia categories, which are mapped to their verbalization. Results are reported on the Seen (S), Unseen (U), and All (A) subsets of the data. The E2E clean dataset (Dušek et al., 2019) consists of meaning representations (MRs) from the restaurant domain. The task is to generate a sentence that verbalizes the useful information from the MR. Dataset statistics are summarized in Table 7 in the appendix.
Evaluation Metrics We evaluate the quality of generated texts using several widely accepted metrics. BLEU (Papineni et al., 2002) measures the similarity between generated text and references in terms of n-gram overlap. METEOR (Banerjee and Lavie, 2005) assesses the quality of generated text by comparing unigram matches between the text and references, including exact, stem, synonym, and paraphrase matches. TER (Snover et al., 2006) is a measure of the number of edits required to
change the generated text into one of the references. PARENT (Dhingra et al., 2019) takes into account the table input when evaluating generated text. NIST (Doddington, 2002) is similar to BLEU, but also considers the informativeness of each ngram. CIDEr (Vedantam et al., 2015) uses TF-IDF to lower the weights of common n-grams that appear in all references when calculating uni-gram to 4-gram overlaps between generated and reference sentences. We also use the NLI score (Chen et al., 2020a) on the LogicNLG dataset to evaluate the logical fidelity, which is a model-based evaluation using the BERT model trained on the TabFact (Chen et al., 2020c) dataset.
Comparing Linearizations We compare our proposed unified representation to other linearization methods from previous papers. Specifically, on DART, WebNLG, and E2E datasets, we compare our method to the linearization used in UnifiedSKG (Xie et al., 2022).3 On ToTTo and LogicNLG datasets, we use the linearization from their original papers (Parikh et al., 2020; Chen et al., 2020a) for comparison. Examples of their linearization methods can be found in the appendix.

heading: 4.1 Zero and Few-Shot Experiments
text: Our hypothesis is that a model trained on one structured form will transfer better to other forms under zero or few-shot settings when using our unified method of representation. We test this by focusing on transferring from ToTTo data (table input) to other types and from WebNLG (KGs) to ToTTo in this section. Results for other transfers can be found in the appendix.
As shown in Table 3, for each experiment, we compare three settings: (i) Only on tgt – In fewshot experiments, we only train the model on the target task using the linearization from other papers. In zero-shot experiments, we use the foundational
3The E2E dataset is not studied in the paper, but the linearization is included in their official repository.
model without any training. (ii) Src to tgt, unified – First, train the model on the source task and then fine-tune it on k-shot4 target-task data, using our unified representation for both. (iii) Src to tgt, varied – Similar to (ii), but we use the linearization from other papers for each task, as described in 4. We refer to this as the varied setting because the source and target-task linearizations are different.
During inference, we apply the same linearization method utilized during training to each target task. More implementation details are presented in the appendix.

heading: 4.1.1 Zero-Shot Performance
text: The zero-shot results are summarized in Table 4. We compare our results to recent works GPT2XL (Keymanesh et al., 2022), KGPT (Chen et al., 2020b), JointGT (Ke et al., 2021) and HTLM (Aghajanyan et al., 2022). Both KGPT and JointGT models are pretrained on large amounts of aligned knowledge graph and text data. HTLM is a hypertext language model pre-trained on a large-scale web crawl. It allows for structured prompting in the HTML format.
From the results, we make several observations. (1) The Only on tgt performance is very low as expected, as the T5-base model has not been trained on any data. However, surprisingly the NLI score on LogicNLG is the highest under this setting. We observe that this NLI score is very unstable and might not be a good metric for judging the entailment of generated text. (2) The performance of Src to tgt, unified consistently and significantly surpasses that of Src to tgt, varied, even though both models are trained using the same source-task data, but with different representations. This demonstrates that representing source and target tasks in the same format is crucial for successful zero-shot transfer, as a common representation facilitates the transfer of knowledge learned on the source data to other structured forms and tasks. (3) The zero-shot performance of the "unified" model is even better than few-shot results of the baseline models. On DART, the "unified" model’s BLEU score is 43% higher than that of HTLM. The improvement on WebNLG is particularly noteworthy for unseen categories. Utilizing a unified representation results in a zero-shot BLEU score of 39.82, surpassing the few-shot results of 37.18 by Ke et al. (2021) and 18.5 by Aghajanyan et al. (2022).
4k = 0 means no training on target task at all.

heading: 4.1.2 Few-Shot Results
text: Figure 2 shows the few-shot results for sample sizes 8, 16, 32, 64, and 128. We repeat the experiments 5 times for each sample size and report the mean and 95% confidence intervals.
Table −→ KG Triples From Figure 2a, 2b and 2c, we have identified three key observations: (1) Both the models Src to tgt, unified and Src to tgt, varied, which were initially trained on ToTTo, perform significantly better than the model Only on tgt, which was only trained on target tasks. This indicates that these two structured forms share common knowledge and that training the model on tabular input can greatly enhance its understanding of KG triples. (2) Furthermore, Src to tgt, unified (represented by the red curve) outperforms Src to tgt, varied (represented by the blue curve) by a substantial margin. This observation aligns with our previous findings in the zero-shot setting (as seen in Table 4) and highlights the importance of our unified representation approach in transferring knowledge learned from tables to KG triples. (3) Additionally, on the task of WebNLG, the improvement on unseen categories is particularly notable, further reinforcing our zero-shot findings.
Table −→ Meaning Representations Based on Figure 2d, similar observations can be made for the E2E dataset. The improvement in terms of CIDEr is particularly significant when using fewer than 64 samples, indicating that the unified model generates more informative text compared to the varied and vanilla models.
Table Description −→ Table Insights The LogicNLG task is distinct from the ToTTo task in that it requires the model to generate insights by analyzing the contents of a table, rather than generating surface-form descriptions based on highlighted cells. As shown in Figure 2e, when using only 8 samples, the Src to tgt, varied model performs better than the Src to tgt, unified model. This may be due to the fact that both tasks involve generating text from tables, and that the unified model is more proficient at transferring knowledge learned on the source task to the target task, which may lead to the generation of table descriptions rather than insights when provided with a limited number of samples. However, as the number of samples increases, the performance of the unified model improves, and it surpasses the varied model when k=128. A concrete example is provided in the case study section
(a) ToTTo (table) to DART (KG): BLEU
(b) ToTTo (table) to WebNLG (KG): BLEU (Unseen)
(c) ToTTo (table) to WebNLG (KG): BLEU (Seen)
(d) ToTTo (table) to E2E (MR): CIDEr
(e) ToTTo to LogicNLG (table): BLEU
(f) WebNLG (KG) to ToTTo (table): PARENT
Figure 2: Results of few-shot experiments transferring models between two structured forms. Each figure shows three curves, the green curve "only on tgt" is the performance of the T5-base model fine-tuned directly on the target task, the red curve "src to tgt, unified" is the performance of the model fine-tuned on both tasks using our proposed unified representation, and the blue curve "src to tgt, varied" is the performance of the model fine-tuned on both tasks using linearization from other papers, resulting in varied linearization for source and target tasks. The LogicNLG task differs from ToTTo by requiring the model to generate insights from analyzing a table rather than generating descriptions from highlighted cells.
4.3 to further illustrate our observation.
KG Triples −→ Table The benefits of utilizing unified representation are particularly substantial when transferring models that have been trained on knowledge graphs to table inputs. In Figure 2f, the PARENT gap between unified and varied models is consistently greater than 2 points. In fact, the performance of "varied" and "only on tgt" models converge when utilizing 128 samples, and is only slightly superior to that of the "unified"
model when provided with only 8 samples. This suggests that the use of unified representation is highly efficient in terms of sample utilization.

heading: 4.2 Full-Set Finetuning Results
text: In this section, we train the models on full training sets, in either single-task or multi-task settings. Additional experimental results are presented in the appendix.
Single-Task Training From the "single-task training" results in Table 5, a key finding is that the proposed unified representation method results in performance comparable to other linearization techniques studied in previous research. This is particularly evident on the DART, WebNLG, and E2E tasks, where the data was first converted into virtual tables, and the results from both methods are similar, indicating that this conversion does not result in a significant loss of information.
Multi-Task Training The performance of multitask models is summarized in Table 5 under the "multi-task training" section, revealing several key findings: (1) Overall, multi-task training using different linearizations for each dataset results in a worse performance compared to single-task training. BLEU scores for T5-base models decrease from 49.2 to 48.5 on ToTTo, from 49.0 to 48.1 on DART, and from 65.9 to 64.1 on seen categories of WebNLG. This confirms the findings of UnifiedSKG (Xie et al., 2022), which found that singletask model performance was higher than multitask performance on ToTTo dataset. However, it is unclear if this drop in performance was due to task differences, as their study included other tasks. Our results provide further insight into data-to-text tasks alone and show that multi-task performance can still be inferior if input formats are not unified. (2) In contrast, multi-task trained "unified" models consistently outperform single-task models,
with the only exception of the base model on the WebNLG dataset. This demonstrates that utilizing a unified representation approach helps models learn common knowledge across various tasks without negatively impacting performance. (3) The "unified" models consistently demonstrate superior performance compared to "varied" models in multitask training, with a larger margin of improvement observed in base-sized models.

heading: 4.3 Qualitative Study
text: We conduct a qualitative case study to compare the texts generated by the Src to tgt, unified and Src to tgt, varied models. The results are illustrated in Table 6, which displays the model’s generations for different sample sizes.
For the WebNLG example, the input contains 5 KG triples. When k = 8, the "varied" model only covers one KG triple fact, while the "unified" model includes many more nodes and relations from the input. As the sample size increases to 128, the "unified" model’s generation covers all facts accurately, while the "varied" model’s generation still misses the "funk and disco" origin of pop music.
In the E2E example, the "unified" model output is consistent and accurate with both 8 and 128 samples. In contrast, the "varied" model produces "Sorrento" twice. This serves as additional evidence that using a unified representation enhances the transfer of the generation style learned on table input to meaning representations.
The results of the LogicNLG input generation offer validation for our hypothesis that the "unified" model performs less effectively than the "varied" model when the sample size is small, due to its persistent focus on generating descriptions of the table input, as it has been trained to do on the ToTTo data. Indeed, the descriptions generated by the "unified" model when sample size is 8, are accurate reflections of the table’s content. When the sample size is increased to 128, both models generate sentences that are more akin to insights. It is noteworthy that the "unified" model generates "world golf championship" even though it is not present in the table, which pertains to the golf championship. We posit that this information is carried over from the ToTTo data, and the "unified" model is able to retain this information while the "varied" model does not.

heading: 5 Conclusion and Future Work
text: We have introduced a unified representation approach for data-to-text tasks, which effectively converts table contents, knowledge graph triples, and meaning representations into a single representa-
tion. Our experiments demonstrate that this unified representation significantly improves generalization across different structured forms, especially in zero-shot or few-shot settings. Our method is particularly beneficial in situations where data is scarce. Additionally, by using the unified representation, our multi-task-trained models consistently outperform single-task models, which is in contrast to previous findings that mixing different data types can negatively impact overall performance.
One future direction is to apply our method to other tasks that involve heterogeneous inputs, such as question answering over knowledge bases, where knowledge can be stored in both tables and knowledge graphs. It would also be interesting to investigate whether a model pre-trained on large knowledge graphs can more effectively transfer learned commonsense knowledge to table QA tasks, when using our unified representation approach.
Limitations
It is important to note that the unified representation proposed in our study is just one option among many. Other linearization methods may potentially yield better results. For example, research by Yin et al. (2022) and Aghajanyan et al. (2022) has explored using code generation with Jupyter notebooks and a hyper-text language model with structured prompting, respectively. Further research in these areas, such as converting all structured forms to markdown language or hyper-texts, may yield alternative unified representations.
Ethics Statement
We acknowledge the importance of the ACL Ethics Policy and agree with it. This study addresses the problem of data-to-text generation and explores whether a unified representation can enhance cross-task performance on various structured forms. Since our input comes from knowledge bases, a potential concern is that biases or fairness issues may be present in the KB, which could also be reflected in the generated text. Therefore, it is crucial to use the model with caution in practice. We believe this work can contribute to the field of data-to-text generation, particularly in situations where data is scarce.

heading: A Data Statistics
text: We summarize the input type and number of examples in each dataset.

heading: B Column-wise Linearization of (Sub)Table
text: A column-wise linearization of Table 1 is: 1 <title > Alma Jodorowsky </title> 2 <sub\_title > Filmography </sub\_title > 3 <table > 4 <column > 5 <col_header > Year </col_header > 6 <cell> 2014 </cell> 7 <cell> 2016 </cell> 8 <cell> 2017 </cell> 9 </column > 10 ... other columns ... 11 </table >

heading: C Other Linearizations Used in Previous Papers
text: Table highlights : Our unified representation is motivated by ToTTo linearization, and hence they are very similar. The only difference is ToTTo uses <page_title> instead of <title> and <section_title> instead of <sub_title>.
KG triples : Given a set of triples {(William Wasmund, FIELD_GOALS, 0), (William Wasmund, EXTRA_POINTS, 0)}, an alternative linearization used in UnifiedSKG (Xie et al., 2022) is William Wasmund : field goals : 0 | William Wasmund : extra points : 0
Entire table : The alternative linearization used in LogicNLG (Chen et al., 2020a) for Table 1 is: Given the table title of Alma Jodorowsky, Filmograph. In row 1 , the Year is 2014 , the Title is La ..., the Role is Solana, the Notes is TV ... In row 2 , ...
Mearning representation : The alternative linearization we use for the example in Figure 1 is simply concatenating all the MRs: name[Cocum], eatType[coffee shop], food[Italian], priceRange[cheap], familyFriendly[yes].
D Implementation Details
In the zero- and few-shot experiments, we employ the T5-base model as the base model and train it for 30 epochs for both the source and target tasks. For the source task, we use a learning rate of 5e-5 and a batch size of 32, and for the target task, we use a learning rate of 2e-5 and a batch size of 8.

heading: E More Multi-Task Results
text: We present more detailed multi-task results on each of the dataset in this section. The results are summarized in Table 8, 9, 10 and 11.

heading: F More Few-shot Results
text: We present other few-shot results using more metrics in Figure 3, 4 and 5.

heading: G Human Evaluation
text: We conducted a human evaluation on the fewshot ToTTo to WebNLG transferring experiment. Specifically, we randomly selected 50 WebNLG test data from the unseen schema and compared the performance of the 8-shot src to tgt, unified and src to tgt, varied models.
For each of the 50 samples, we generated texts using both models and asked three annotators to choose the better option based on factuality, coverage of the triples, and fluency. We received only two annotations for two of the samples as one of the annotators did not respond. For the remaining 48 samples, all three annotators reached a consensus on 21 of them (43.75%). Out of these 21 samples, the "unified" model received unanimous preference from the annotators in 15 cases (71.43%). If we consider the majority vote among the three annotators, then 75% of the results favored the "unified" model. The Fleiss Kappa value, which measures agreement among the three annotators, is around 0.23 (fair agreement).

heading: H More Qualitative Study
text: We present additional few-shot predictions for models transferred from ToTTo to WebNLG and LogicNLG in Tables 12 and 13, respectively. We also provide error analysis under each example.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: In the section "Limitations" after Section 5.

heading: 3 A2. Did you discuss any potential risks of your work?
text: In the section "Ethics Statement" after Section 5.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.

heading: B Did you use or create scientific artifacts?
text: Not applicable. Left blank.
B1. Did you cite the creators of artifacts you used? Not applicable. Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Not applicable. Left blank.
C 3 Did you run computational experiments?

heading: In Section 4.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? In Section 4 and in Appendix D.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? In Section 4.

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: One future direction is to apply our method to other tasks that involve heterogeneous inputs, such as question answering over knowledge bases, where knowledge can be stored in both tables and knowledge graphs. It would also be interesting to investigate whether a model pre-trained on large knowledge graphs can more effectively transfer learned commonsense knowledge to table QA tasks, when using our unified representation approach. Limitations
It is important to note that the unified representation proposed in our study is just one option among many. Other linearization methods may potentially yield better results. For example, research by Yin et al. (2022) and Aghajanyan et al. (2022) has explored using code generation with Jupyter notebooks and a hyper-text language model with structured prompting, respectively. Further research in these areas, such as converting all structured forms to markdown language or hyper-texts, may yield alternative unified representations