Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations
Year: 2023

Abstract:
There has been great progress in unifying various table-to-text tasks using a single encoderdecoder model trained via multi-task learning (Xie et al., 2022). However, existing methods typically encode task information with a simple dataset name as a prefix to the encoder. This not only limits the effectiveness of multitask learning, but also hinders the model’s ability to generalize to new domains or tasks that were not seen during training, which is crucial for real-world applications. In this paper, we propose compositional task configurations, a set of prompts prepended to the encoder to improve cross-task generalization of unified models. We design the task configurations to explicitly specify the task type, dataset name, as well as its input and output types. We show that this not only allows the model to better learn shared knowledge across different tasks at training, but also allows us to control the model by composing new configurations that apply novel input-output combinations in a zero-shot manner. We demonstrate via experiments over ten table-to-text tasks that our method outperforms the UnifiedSKG baseline by noticeable margins in both in-domain and zero-shot settings, with average improvements of +0.5 and +12.6 from using a T5-large backbone, respectively.

heading: 1 Introduction
text: Table-to-text tasks, such as table-based question answering (Pasupat and Liang, 2015; Herzig et al., 2020), summarization (Parikh et al., 2020), or fact verification (Chen et al., 2019), are of high interest to the NLP community and have been applied in many real-world applications. Traditionally, these tasks have been studied individually, with methods commonly optimized for one or a few tasks (Liu et al., 2021; Shi et al., 2021). However, with the recent popularity of pre-trained transformer models (Raffel et al., 2020; Lewis et al., 2020; Xue et al.,
*Work done during an internship at AWS AI Labs.
2021), there has been a paradigm towards unifying multiple NLP tasks with a single encoder-decoder model (Khashabi et al., 2020; Sanh et al., 2022). More recently, UnifiedSKG (Xie et al., 2022) extended this paradigm to table-to-text tasks by flattening the structured input (e.g., tables) into text format, and unifying all tasks with a T5 model (Raffel et al., 2020). By training the model over 21 datasets with structured input, it has established new state-of-the-art results for most of these tasks.
Despite the success, existing work often rely on a simple trick to encode task information: the name of the dataset is often used as a prefix to the encoder at both training and test time. We argue that this overly simplified design has at least two major limitations. First, since no detailed information about the task is provided, any sharable knowledge between tasks is learned in a latent manner. Second, with this design, models are trained and evaluated on their abilities to solve specific datasets, rather than tasks. As a result, we may see substantial performance degradation when we apply the model to an unseen task at test time.
5523
Figure 1 illustrates the aforementioned limitations: a unified model (such as UnifiedSKG) is trained on short-form table QA (Zhong et al., 2017) and table-based summarization (Parikh et al., 2020), and we want to test the trained model on long-form table QA (Nan et al., 2022a), where the model should take a question and a table as input and output an abstractive sentence as the answer. As there is no way to instruct the model about the information of the new task, the model can only make an educated guess by generating the most plausible text “Best Vocals” according to the training datasets, which fails to serve as a good longform answer. We therefore argue that it is critical to test a table-to-text model’s cross-task generalizability, which is captured in neither the training methods nor the evaluation setup in existing work.
In this paper, we propose the use of compositional task configurations, a set of text prompts prepended to the encoder to improve the cross-task generalizability of unified table-to-text models. For a given task, we design its configuration prompt to be compositional, describing the task type, dataset name, input type, and output type. This design offers at least two key advantages. First, the task configurations explicitly inform the model what is shared between different tasks. For example, the model is able to learn from the configurations that table-based fact checking and table-based QA share the same inputs but different outputs. Second and more importantly, using task configurations allows us to have explicit control over the model’s behaviors. For the example in Figure 1, we can now compose a new configuration for long-form table QA at test time to instruct the model to first produce a set of relevant cells and synthesize them to produce a long-form answer, which is within the capabilities of the two training tasks. We discuss this further in the next section.
Our evaluation focuses model’s cross-task generalizability. Specifically, we train our model on 5 table-to-text datasets and test it on an additional set of 5 new datasets that cover either a new domain of an existing task or a new task of which the capabilities can be composed by the ones learned through the 5 training datasets. Our main findings can be summarized as follows:
• Our method not only outperforms the strong UnifiedSKG baseline consistently on the 5 indomain datasets, but also demonstrates much stronger cross-task generalization.
• In zero-shot evaluation on the 5 test-only tasks, our model outperforms UnifiedSKG by a substantial margin of +6.5 and +12.6 average scores from using T5-base and T5-large, respectively. Notably, we find that in zero-shot evaluation on FETAQA (Nan et al., 2022a), a long-form table QA task, while the baseline completely fails with a 0.6 F1 score, our method leads to much better generalization, achieving a 21.2 F1 score.
• We also show that using the compositional task configurations allows the model to output supporting table cells that supplement its final prediction in a zero-shot manner. Human evaluation of the generated supporting cells for the TABFACT dataset reveals that more than 80% of the generated cells have high relevance to the task.

heading: 2 Method & Tasks
text: Prompting is a natural and feasible way to impose explicit control over the behaviors of pre-trained language models (Wei et al., 2021; Chung et al., 2022; Sanh et al., 2022). In this work, we implement the task configurations as prompts of an encoder-decoder model. Each task configuration contains the following four aspects: task type, input type, output type, and dataset name. The task type is the end goal of a task, e.g., QA and summarization, as shown in Figure 2. Input and output types specify the inputs of the encoder and the outputs of the decoder of table-to-text models, respectively. These types can be compositional, for example, both long-form and short-form table QA in Figure 2 require the decoder to output a set of relevant cells and the final answer. The dataset name specifies the dataset used for training. As different datasets can share the same task type, input and output types, we assume that the model is able to learn the shared and the unique knowledge across different datasets by adding the dataset names as configurations. When testing the model on a new dataset, we can simply omit the dataset name since it is not trained.
One of the major advantages of having explicit task configurations is that it enables the model to learn the mapping between a configuration and its behavior. At test time, we can compose a new set of configurations which suits best for an unseen task using the trained configurations. Figure 2 demon-

heading: Short-form Table QA (Train)

heading: Summarization Table Cell Long answer
text: strates that by training on short-form table QA, the model learns the ability to generate a set of relevant table cells according to the question and then derive the answer based on those cells. By training on table summarization, the model learns to produce a summary based on a set of table cells.1 At test time, by reformulating the long-form table QA as a query-based summarization task, our model is able to first generate a set of relevant cells (learned through short-form table QA) and then synthesize those cells to yield a long-form answer (learned from table summarization).
Note that our method is an efficient extension to the original UnifiedSKG model. It only requires a small input prefix, comprising less than 5% of the total sequence length, making it flexible for generalization to more tasks and datasets.

heading: 2.1 Datasets and Task Configurations
text: A detailed list of our datasets with their task information is shown in Table 1. We consider 5 datasets as in-domain datasets for both train-
1Conventionally, when using ToTTo, it is common to feed the cells to the encoder and decode the summary. To make the task more compatible with other table-to-text tasks, we feed the relevant cells as starting inputs to the decoder and the model’s loss is not calculated on those cells.
ing and testing: WIKISQL (Zhong et al., 2017), WIKITQ (Pasupat and Liang, 2015), SQUAD (Rajpurkar et al., 2016), TOTTO (Parikh et al., 2020) and TABFACT (Chen et al., 2019). For SQUAD and TOTTO, since no official test set is released, we follow UnifiedSKG (Xie et al., 2022) and report results on the official development sets. In addition, we consider 5 datasets for test only: NQ-TABLES (Kwiatkowski et al., 2019; Herzig et al., 2021), HYBRIDQA (Chen et al., 2020), TATQA (Zhu et al., 2021), FETAQA (Nan et al., 2022b) and FEVEROUS (Aly et al., 2021).
The test-only evaluation setup aims to assess the effectiveness of our method in enabling the model to generalize to unseen tasks with new compositional configurations, as well as to a new dataset with existing configurations. Specifically, we test if the model can benefit from a combination of input configurations for tasks such as HYBRIDQA, TAT-QA, and FEVROUS, which involve both passages and tables as inputs, despite the model is only trained on one or the other during training. Similarly, we examine if the configuration for FETAQA, a combination of WIKISQL and TOTTO as shown in Figure 2, allows for explicit control over the model’s behaviors, resulting in improved generalization. Finally, we assess the
model’s ability to generalize to a new dataset, NQTABLES, which has the same configurations as WIKISQL and WIKITQ.
For all of these datasets, we linearize the tables following the strategy used in UnifiedSKG (Xie et al., 2022). By inserting several special tokens like vertical bars to indicates the boundaries between cells and rows, a table can be linearized as: “Headers: h1|...|hm, Row 1: c11|...|c1m ... Row n: cn1|...|cnm”. Here, hi denotes the ith header of a table and cij denotes the cell content in the ith row and the jth column. For simplicity, we fix the order of the task configurations to be task type, dataset name, input type, and output type. We prepend the task configuration to the original input of a dataset and feed it to the model. To make our input and output better aligned with the configurations, we also introduce some special markups to separate different parts of inputs and outputs: Figure 3 illustrates the actual model’s input and output of the short-form table QA example from Figure 2. See Appendix B for inputs and outputs constructed for all of the datasets and Appendix A for preprocessing details of each dataset.

heading: 3 Experiments
text: Experimental settings We evaluate our method by following the experimental setup shown in Table 1. We follow the experimental settings of UnifiedSKG (Xie et al., 2022) and use T5 (Raffel et al., 2020) as the backbone of our table-to-text model. Our implementation is based on the publicly released code of UnifiedSKG which is developed based on the transformer library (Wolf et al., 2019). To balance the size of different datasets during training, we use the temperature upsampling method proposed in the original T5 paper
Encoder Input

heading: Decoder Output
text: and set the temperature to 2. For all experiments, we use a batch size of 128 and AdamW (Loshchilov and Hutter, 2018) as the optimizer with the initial learning rate set to 5e-5. We limit the maximum length of the input, including task configuration and the actual inputs, to be 1080 sentence-piece tokens. We train both the T5-base and T5-large models on the training set for 20 epochs and we use early stopping with the patience set to 2. We use deepspeed (Rasley et al., 2020) to reduce the GPU memory loads when training the T5-large model. The approximate GPU hours for T5-base and T5-large are 250 and 650 respectively on A100 GPUs with 40G memory.
Baseline We mainly compare our method against UnifiedSKG (Xie et al., 2022), a strong baseline that was shown to achieve state-of-the-art results on many table-to-text tasks via multi-task training. In UnifiedSKG, for each task, a dataset name is prepended to the encoder during multi-task fine-
tuning as a pseudo-task configuration. For fair comparisons, we re-trained UnifiedSKG models on the five in-domain datasets by using the authors’ implementation.2
Evaluation For the in-domain tasks, we simply train on their training sets and evaluate on their test sets. For the test-only tasks, we evaluate our method in two settings: 1) a zero-shot setting, where we directly apply the model trained on indomain datasets and use a new set of task configs designed for each test dataset; 2) a few-shot setting, where for each test dataset, we further fine-tune the model using n randomly sampled training examples (where n is small). Since we observed that the few-shot training is unstable and heavily depends on the sampled examples, we report average performance from 5 different random seeds (each with a different set of few-shot examples).
2We in fact found that our version of the UnifiedSKG model fine-tuned over the five in-domain datasets outperforms the original authors’ version on several datasets, establishing a more competitive baseline. For example, our version achieves 82.9 on WikiSQL and 77.1 on TabFact, whereas the original model by Xie et al. (2022) achieves 81.9 and 71.2, respectively.

heading: 4 Results

heading: 4.1 Main Results
text: We present the in-domain and zero-shot evaluation results for all datasets in Table 2 and the few-shot evaluation results for OOD datasets in Figure 4. We have the following observations:
First, using compositional task configs shows much stronger performance on zero-shot datasets unseen at training time (Table 2). For example, the UnifiedSKG baseline fails to generalize at test time to FETAQA, a long-form table QA task where the input is a question-table pair and the output is a long-form abstractive answer. This is due to the baseline model having no clue of what format of output should be produced and what knowledge learned through the training datasets should be leveraged for this task. In contrast, by reformulating the long-form table QA as a querybased summarization task and composing the input configurations to be table and query as well as the output task configs to be relevant cell and summary, our method notably improves the zero-shot performance and closes the gap between zero-shot and single-task finetuning results. Note that among the zero-shot datasets, NQ-TABLES represents a new dataset for an existing task (short-form table QA), whereas others represent new tasks unseen at train-
ing. Nevertheless, we found the improvements to be consistent for all zero-shot datasets, with average improvements of +6.5 and +12.6 for base and large models, respectively.
Second, in most cases, using compositional task configs consistently improves the in-domain performance over the UnifiedSKG baseline and single-task training (Table 2). The observation is consistent for both base and large model sizes, with average improvements of +0.6 and +0.5 over UnifiedSKG, respectively. The improvement over single-task fine-tuning is even greater for all datasets. One explanation to this improvement is that by adding task configs we explicitly encourage the model to learn the shared knowledge between different tasks and datasets.
Last but not the least, in few-shot evaluation (as depicted in Figure 4), we find that using task configurations has improved few-shot learning performance for most test-time tasks. Overall the difference between our method and the UnifiedSKG baseline is particularly notable when the number of supervised examples (n) is small; and the performance gap diminishes for HYBRIDQA, TAT-QA, and FEVEROUS as n gets larger. One possible explanation is that the prior captured by the task configurations during training is not closely aligned with these three datasets, when n getting larger, the prior introduced by the task configurations is gradually overridden by knowledge learned from the supervised data.

heading: 4.2 Ablation of Task Configs at Training time
text: The impact of individual configurations on model performance was evaluated by removing one configuration at a time during training. The results, presented in Table 3, indicate that the removal of the output type resulted in the largest performance drop, as the model was only able to guess the de-
sired output type based on learned parameters. The removal of the input type had the least impact on performance. This is likely due to the fact that learning the representation of the two input types was not difficult for the model, and explicitly informing the model about the input type does not provide significant benefit, as observed in the previous section. The removal of the dataset name also results in a performance drop, particularly on the NQ-TABLES dataset, indicating that even when the task type, input, and output are the same, including the dataset name helps the model learn dataset-independent knowledge more effectively. The removal of the task type results in a complete failure on the FETAQA dataset, demonstrating that all configurations are needed to produce the correct form of output. A more detailed discussion of these findings can be found in Section 7.

heading: 4.3 Ablation of Task Configs at Test time
text: While our method demonstrates much stronger zero-shot task performance, it is crucial to understand the extent to which input and output configurations contribute to this success, particularly for tasks involving hybrid input or output types that are not present during training. To examine the contributions of input configurations, we remove each configuration from the hybrid tasks (HYBRIDQA,
TAT-QA and FEVEROUS) at test time, with results shown in Table 4. We found that deleting either of the input configurations results in a performance drop in most cases, and the drop is quite notable when the table and passage input configurations are removed together. This suggests that the input configuration captures useful priors about the input during training, and different configurations can be combined to yield better performance in the zero-shot transfer to hybrid tasks. We also observe a similar trend in Figure 5 where we test the model performance by removing the cell output configurations for FETAQA (thereby skipping cell generation). We see that in both zero-shot and fewshot settings, model performance drops by a large margin. This shows not only that the model can generate different outputs by combining the output configurations, but also that it can better utilize the prior captured by the configurations to improve task performance.

heading: 4.4 Human Evaluation of Generated Cells
text: In addition to the strong task generalizability, a key advantage of applying the proposed task configurations to table-to-text tasks is that we can modify the task configurations to output more results for improved explainability, even when such a configuration combination is never seen at training time. An example of this is for the table-based fact verification task, TABFACT, instead of only generating a binary label, we can extend the output configuration to include a cell component that can serve as supporting evidence of the binary prediction. We include two examples of this setting in Figure 6.
To understand how well our model can generate supporting cells without ever being trained for it, we conduct a human evaluation over 50 randomly sampled outputs from the TABFACT dataset. We ask annotators to manually evaluate the generated cells based on their level of relevance and completeness. Relevance denotes the usefulness of the generated cells in verifying a claim (precision) and completeness refers to the extent to which all of the relevant cells are generated (recall). Detailed annotation instructions are shown in Appendix C. For each aspect, we ask the annotators to select between three labels that characterize its degree: “full”, “partial” or “none”. Three of the authors conduct the annotations, achieving 0.72 and 0.80 Fleiss Kappa (Fleiss, 1971) for relevance and completeness, respectively. We conduct majority vote to get
the consensus label and the results are shown in Table 5. Overall we found that the model is able to generate cells with high relevance (with 72% examples being fully relevant generations), but struggle with full completeness (with 34% fully complete).

heading: 5 Related Work
text: Table-to-text tasks Table-based tasks, including table-based question answering (Pasupat and Liang, 2015; Zhong et al., 2017; Chen et al., 2020; Cheng et al., 2022; Zhao et al., 2022), tablebased fact-checking (Chen et al., 2019; Aly et al., 2021), table summarization (Parikh et al., 2020; Suadaa et al., 2021; Moosavi et al., 2021), have gained increasing attention in recent years. A flurry of work using transformer-based structure explored modeling table structure via pretraining, for example, TabTransformer (Huang et al., 2020), VIME (Yoon et al., 2020), TABBIE (Iida et al., 2021), TaBERT (Yin et al., 2020), TUTA (Wang et al., 2021), TabT5 (Andrejczuk et al., 2022), and TableFormer (Yang et al., 2022).
Our work mainly focuses on table-to-text tasks but the ideal neural architecture for encoding table structures is not our focus. Instead, we emphasize multi-task knowledge-sharing (similar to UnifiedSKG (Xie et al., 2022)) and cross-task gen-
Caption: 1976 world junior figure skating championships
rank name nation points places
1 sherri baier / robin cowan
canada 128.39 9
2 lorene mitchell / donald mitchell united states 124.94 16
3 elizabeth cain /
peter cain australia 116.67 33
4 jana bláhová /
ludek czechoslovakia 113.74 36
Input claim: the top 2 ranked team be in north America
Task configs: [Task: Fact-checking] [Input: table] [Output: cells] [Output: binary answer]
Caption: Charl Schwartzel
tournament wins top - 5
…
top - 25 events
masters tournament 1 1 2 4
us open 0 0 3 7
the open championship 0 0 4 9
pga championship 0 0 2 8
totals 1 1 11 28

heading: Input claim:
text: there be a total of 28 event that make up the 4 tournament
Task configs: [Task: Fact-checking] [Input: table] [Output: cells] [Output: binary answer]
eralization in table-to-text tasks. Also, the proposed framework is capable of generalizing to a broader range of tasks and datasets.
Task unification There have been a vein of work that tries to solve various NLP tasks using a single model. This includes encoder-decoder models like T5 (Raffel et al., 2020), UnifiedQA (Khashabi et al., 2020), UnifiedQA2 (Khashabi et al., 2022), UnifiedSKG (Xie et al., 2022); decoderonly models driven by prompts, for example, GPT3 (Brown et al., 2020), Codex (Chen et al., 2021), PaLM (Chowdhery et al., 2022). Our work extends UnifiedSKG by using an encoder-decoder model as the backbone and designing prompts to encourage better knowledge sharing between different tasks and enable control over the model’s behaviors.
Cross-task generalization with pretrained models Various efforts have been made to improve the ability of unified models to generalize to new tasks and datasets, including instruction-tuning using a wide range of natural language instructions (Chung et al., 2022; Sanh et al., 2022; Wei et al., 2021;
Zhong et al., 2021), better design of prompts in zero-shot and few-shot setting (Wei et al., 2022; Zhou et al., 2022; Kojima et al., 2022). Our proposed method differs from instruction-tuning models like FLAN (Wei et al., 2021) in that we use a more symbolized prompt structure and it is possible to attribute cross-task generalizability to specific tasks and configurations. Also, instructiontuning models like FLAN achieve behavior control and cross-task generalizability through costly largescale instruction tuning. In contrast, our approach demonstrates that within a specific task domain with limited datasets like table-to-text, this can be achieved by utilizing a compositional prompt structure.
Our method is also relevant to Macaw (Tafjord and Clark, 2021), ProQA (Zhong et al., 2022b), and SchemaPro (Zhong et al., 2022a), which also utilize explicit task descriptions to facilitate knowledge sharing between various NLP tasks. Our work differs in two main aspects: (1) Our work focuses on compositional generalization at test time, examining whether the model can combine different configurations from multiple tasks during training to generalize to unseen tasks at test time. (2) Our work focuses on table-to-text tasks.

heading: 6 Conclusion
text: We introduced compositional task configurations for unified table-to-text models. Compared to existing unified encoder-decoder models that sim-
ply use dataset names as input prefix, compositional task configurations allow us to specify the task type, input, and output types at a finer level, which improve multi-task learning effectiveness and cross-task generalization. Further, we showed that our method allows fine-grained control over the model’s generation at test time, enabling the model to generalize to unseen tasks and improving explainability via generating high-quality supporting table cells.

heading: 7 Limitations
text: Task configurations are entangled with the full model parameters. In our ablation study of task configurations at training time (Table 3), we see that when training without task type, the model fails to generalize to FETAQA. Upon examining the model output, we find that although we change the output configuration to “long answer”, the model still produces a short-form answer. This indicates that model behaviors are not always aligned with a single configuration, leading us to question the extent to which each individual configuration influences the model. In order to have better and more interpretable control over the models, one potential avenue for future research is to develop pluggable task configurations, where each configuration controls a more atomic function of the model and can be plugged, unplugged, and combined to yield different model behaviors.
Our exploration scope is limited to table-to-text tasks. Due to the constraints of the computational resources, we haven’t explored joint training with a broader range of other NLP tasks. We think with some modifications, such as the inclusion of dataset domains in the configuration set, it would be possible to extend our approach to additional datasets and tasks.

heading: 8 Ethics Statement
text: The authors of this paper are committed to conducting research ethically. Data used in this work has been collected from public sources and used in accordance with all applicable laws and regulations. The only area of work that involves human annotation of data is described in Section 4.4, where authors of this paper annotated a group of samples for analyzing models’ behaviors. We ensure that no external human subject was involved or harmed. In addition, this work uses language models, for
which the risks and potential harms are discussed in numerous previous works (Bender et al., 2021; Weidinger et al., 2021). The authors strive to ensure that the research and its results do not cause harm.

heading: 9 Acknowledgement
text: We would like to thanks the anonymous reviewers for their insightful comments and suggestions.

heading: A Dataset Statistics and Preprocessing
text: The statistics and the licenses of the datasets we used in this paper are shown in Table 6. All datasets are English-based and all of them are based on Wikipedia except for TAT-QA which is based on
financial reports. To fit the data into the encoder, for all datasets, we limit the max length of each cell to be 15 sentence-piece tokens. If the length (measured by sentence-piece tokens) of the linearized table is longer than 1024, we truncate random rows to reduce the table size.
The original annotation of WIKISQL dataset does not include the relevant cells. We extract the relevant cells by executing the accompanied SQL query annotations. In most cases, the relevant cells equal to the final answer annotations; for the rest of the cases, aggregations or numerical operations need to be run to obtain the final answer. During training, we also create another version of the WIKISQL dataset, in which we exclude the relevant cells and only use the final answer as supervision to improve output diversity. We use both versions at training time.
NQ-TABLES is a table-based QA dataset derived from the NaturalQuestions dataset (Kwiatkowski et al., 2019) and was originally released by Herzig et al. (2021). The original test set of NQ-TABLES contains 966 unique examples. In our experiments, to make the dataset more compatible with other table-based QA tasks, we evaluate on a customized version of NQ-TABLES where we only include an example if the answer is uniquely locatable as one or more table cells. This filtering step results in 549 unique triples of table, question and answers.
To make TOTTO more compatible with other table-to-text tasks, we feed the selected cells as inputs to the decoder, as we mentioned in Section 2. Also, we find it helpful to create a reversed version of the TOTTO dataset, where we treat the annotated summary and the table as input and let the model predict the relevant cells. We add both versions of TOTTO to the training of all models, including the baseline.

heading: B Task Configurations Applied for Each Dataset
text: Below we list the task configurations applied to all datasets. For each dataset, we present input to the encoder and output from the decoder separately. For encoder, we include the template of the full input, including task configurations as well as how the dataset input is structured (with actual data replaced by “...”). For decoder, we include how we structure the annotated output during training and how we parse the output during testing.

heading: B.1 WikiSQL

heading: Encoder:
text: [Task: QA] [Dataset: WikiSQL] [Input: query] [Input: table] [Output: cells] [ Output: short answer] [query] ... [/ query] [table] ... [/table]

heading: Decoder:
text: [cell] ... [/cell] [answer] ... [/answer ]

heading: B.2 WikiTQ

heading: Encoder:
text: [Task: QA] [Dataset: WikiTQ] [Input: query] [Input: table] [Output: short answer] [query] ... [/query] [table] ... [/table]

heading: Decoder:
text: [answer] ... [/answer]

heading: B.3 SQuAD

heading: Encoder:
text: [Task: QA] [Dataset: SQuAD] [Input: query] [Input: passage] [Output: short answer] [query] ... [/query] [passage] ... [/passage]

heading: Decoder:
text: [answer] ... [/answer]

heading: B.4 ToTTo

heading: Encoder:
text: [Task: Summarization] [Dataset: ToTTo] [ Output: cells] [Output: long answer]

heading: Decoder:
text: [cell] ... [/cell] [answer] ... [/answer ]

heading: B.5 TabFact

heading: Encoder:
text: [Task: Fact-checking] [Dataset: TabFact] [Input: query] [Input: table] [Output: binary answer] [query] ... [/query] [ table] ... [/table]

heading: Decoder:
text: [answer] ... [/answer]

heading: B.6 NQ-Tables

heading: Encoder:
text: [Task: QA] [Input: query] [Input: table] [Output: short answer] [query] ... [/ query] [table] ... [/table]

heading: Decoder:
text: [answer] ... [/answer]

heading: B.7 HybridQA

heading: Encoder:
text: [Task: QA] [Input: query] [Input: table] [Input: passage] [Output: short answer] [query] ... [/query] [table] ... [/ table] [passage] ... [/passage]

heading: Decoder:
text: [answer] ... [/answer]

heading: B.8 TAT-QA

heading: Encoder:
text: [Task: QA] [Input: query] [Input: table] [Input: passage] [Output: short answer] [query] ... [/query] [table] ... [/ table] [passage] ... [/passage]

heading: Decoder:
text: [answer] ... [/answer]

heading: B.9 FeTaQA

heading: Encoder:
text: [Task: Summarization] [Input: query] [ Input: table] [Output: cells] [Output: long answer] [query] ... [/query] [table ] ... [/table]

heading: Decoder:
text: [cell] ... [/cell] [answer] ... [/answer ]

heading: B.10 FEVEROUS

heading: Encoder:
text: [Task: Fact-checking] [Input: query] [ Input: table] [Input: passage] [Output: binary answer] [query] ... [/query] [ table] ... [/table] [passage] ... [/ passage]

heading: Decoder:
text: [answer] ... [/answer]

heading: B.11 WikiSQL-Answer-only

heading: Encoder:
text: [Task: QA] [Dataset: WikiSQL] [Input: query] [Input: table] [Output: short answer] [query] ... [/query] [table] ... [/table]

heading: Decoder:
text: [answer] ... [/answer]

heading: B.12 ToTTo-Reverse

heading: Encoder:
text: [Task: Cell-generation] [Input: query] [ Input: table] [Output: cell] [query] ... [/query] [table] ... [/table]
Decoder: [cell] ... [/cell]

heading: C Annotation Interface
text: The annotation interface we used for our human study in this paper is shown in Figure 7
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 7

heading: 3 A2. Did you discuss any potential risks of your work?
text: Section 8

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Last paragraph of Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 2.1

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 2.1

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Appendix A

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Appendix A
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? We checked the data-collecting procedure of the datasets we used in this paper and found no specific offensive content is included. However, we did not take further action to conduct a more comprehensive review of each dataset, as they are widely used within the community and the large volume of datasets we employed made this impractical.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Appendix A
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments? Section 3

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 3

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 3

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 3

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 3
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 4.4

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Section 4.4 Appendix C
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: In order to have better and more interpretable control over the models, one potential avenue for future research is to develop pluggable task configurations, where each configuration controls a more atomic function of the model and can be plugged, unplugged, and combined to yield different model behaviors. Our exploration scope is limited to table-to-text tasks. Due to the constraints of the computational resources, we haven’t explored joint training with a broader range of other NLP tasks. We think with some modifications, such as the inclusion of dataset domains in the configuration set, it would be possible to extend our approach to additional datasets and tasks