SlowGuess/NAACL_2022_OCR huggingface dataset:
What allows models to adapt to these complex tasks? We hypothesize that their adaptability is (in part) compositional—when they can adapt to simple tasks, they can also adapt to compositions of those tasks. For each training paradigm  $\mathcal{T}$  and each composition function  $C$ , we run linear regression to estimate the Pearson correlation coefficient  $r^2$  between adaptability to a compositional task  $C(f_1,\dots f_n)$ ,
$$
\operatorname {a d a p t} _ {\text {g e n}} \left(\mathcal {M}, \mathcal {T}, C \left(f _ {1}, \dots , f _ {n}\right)\right), \tag {3}
$$
and average adaptability to the task's atomic components,
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} \operatorname {a d a p t} _ {\text {g e n}} (\mathcal {M}, \mathcal {T}, f _ {i}). \tag {4}
$$
Figure 3 depicts the procedure graphically.
Can language models learn compositional tasks? The average model adaptability to compositional and atomic tasks, under each training paradigm, is reported in Table 1. We observe that the gap between full-data prompt-tuned models and full-data fine-tuned ones is much larger on compositional tasks than atomic ones. Thus, prompt-tuned models can only generalize comparably to finetuned ones for sufficiently "simple" tasks.
Interestingly, this distinction disappears under few-shot learning. Though both adaptation paradigms generalize much worse in the few-shot
3We focus only on compositional functions  $C$  which have at least 20 compositional tasks  $C(f_{1},\dots ,f_{n})$  in TASKBENCH500, so that we have at least 20 points to obtain a statistically significant correlation coefficient.
Figure 3: Left: Overview of the composition experiment. We evaluate how well the adaptability on a compositional task can be predicted by the (averaged) adaptabilities of the atomic constituent tasks. Right: Correlation between compositional adaptability vs. averaged atomic adaptabilities, for the chaining, union, and map composition types, under each training paradigm. On word-level chaining and union compositions, compositional adaptability is observed: composed task performance is highly correlated with atomic task performance ( $r^2 > 0.5$ ) under all training paradigms. However, on sequential map compositions, all models perform poorly, and thus non-compositionally. This results from challenges in segmenting input sequences; if token boundaries are explicitly marked (map (+separators)), compositional adaptability is again observed.
setting compared to the full setting, they appear to be comparable to each other in the few-shot setting, even on compositional tasks. This may simply imply that few examples are insufficient to learn the nuances of complex tasks, and that simply learning a few prompt tokens is sufficient to capture what can be learned from the limited data samples.
Do language models adapt compositionally?
We visualize each regression model in Fig. 3. Higher  $r^2$  indicates higher correlation between atomic and compositional versions of tasks. Note that all model training paradigms demonstrate some degree of word-level compositionality ( $r^2 > 0.5$ )—when they succeed at word-level compositional tasks (union, chaining), they succeed at the atomic constituents to those tasks, and vice versa. However, this does not appear to be the case for sequential map. In the full-data regime, both fine-tuning and prompt-tuning have near-zero  $r^2$  values. In the few-shot regime, the  $r^2$  value, while nontrivial, is also quite low. Note the slopes of the learned regression lines—the model appears to be unable to learn the sequential versions of tasks, despite succeeding at their atomic versions. To explain this result, we hypothesize that a major obstacle to sequence-level compositional adaptabil
ity is segmentation of sequences into atomic units. This is especially the case for factual tasks: for example, the sequence Pauline Payne Whitney Charles Lloyd could be segmented as [Pauline Payne Whitney] [Charles Lloyd] or [Pauline Payne] [Whitney Charles Lloyd], etc. To test whether segmentation is a bottleneck, we train on a version of sequential tasks where we give the language model explicit markers of word/entity boundaries (e.g. the language model is given Pauline Payne Whitney # Charles Lloyd as input). We found that, with separators, performance on the map tasks increases substantially and the model demonstrates compositional adaptability  $(r^2 > 0.5)$  to these tasks in 3 of the 4 adaptation paradigms. This setting is plotted in Fig. 3 as Map (+separators).
Under this setting, full fine-tuning is the only training paradigm that does not demonstrate compositional adaptability. To better understand this phenomenon, we exclusively plot points from the Map (+separators) setting in Appendix Fig. 5. We find that the distribution of points in the full fine-tuning case shows that points tend to fall within the upper-left triangle. This indicates that for a significant number of tasks, models adapt to their sequential versions despite failing at atomic versions. In these cases, the model does not simply adapt com
Figure 4: Left: Overview of the task prediction distribution experiments (Section 7). We train a model on a balanced dataset of two tasks, and check whether the prediction distribution over tasks on novel examples matches the (balanced) empirical distribution. Right: Probability mass, across all pairs of tasks, assigned to all answers corresponding to the easier vs. harder task, when trained on a balanced dataset and evaluated novel examples. We report the average across all task pairs and held-out examples, as well as standard errors for each task pair. Note that the model tends to assign more probability to the easier task, despite the task training set being balanced.
Prediction distribution across easier/harder task
positionally, but can take advantage of additional information present in sequences (e.g., seeing more tokens, more examples of the word-level function) to outperform compositional adaptation.
7 Learning new distributions
Previous sections investigated the degree to which models could fit particular tasks using a binary metric that assigned credit to any acceptable answer. Our final set of experiments explores a finer-grained notion of correctness: when there are multiple acceptable answers, as is often the case in real NLP tasks, when does the output distribution of a model match the distribution empirically observed during adaptation?
Method We specifically investigate whether models are biased towards predicting "easy" labels, in the sense measured in Section 5. We consider all possible pairs of atomic tasks  $f_{1}, f_{2}$  (for which  $f_{1}$  and  $f_{2}$  take in overlapping sets of inputs). Let  $f_{e}$  to be the easier task in this pair and  $f_{h}$  be the harder task, relative to a model  $\mathcal{M}$  and training paradigm  $\mathcal{T}$ , in the sense that  $\mathrm{adapt}_{\mathrm{gen}}(\mathcal{M}, \mathcal{T}, f_{e}) > \mathrm{adapt}_{\mathrm{gen}}(\mathcal{M}, \mathcal{T}, f_{h})$ . We compose  $f_{e}$  and  $f_{h}$  using union to create compositional task  $\cup (f_{e}, f_{h})$ , and construct the training dataset for this task to be balanced — such that the model sees an equal number of examples of form  $(x, f_{e}(x))$  as  $(x, f_{h}(x))$ . Now let  $\mathcal{M}_{\cup (f_{e}, f_{h})}$  denote a model adapted to this task. During test-time, we
provide  $\mathcal{M}_{\cup (f_e,f_h)}$  with novel inputs  $x^{\prime}$  from the domain of both  $f_{e}$  and  $f_{h}$ , and record the average probability mass it assigns to all  $y_{e}^{i}\in f_{e}(x^{\prime})$  versus all  $y_{h}^{i}\in f_{h}(x^{\prime})$ . Finally, we average these datasetwide probabilities over all pairs of tasks, to get an aggregated probability mass assigned to all easier tasks and all harder tasks in a task pair, invariant of the actual underlying task identity. More details on this procedure can be found in Appendix D.
Results Overall, as seen in Fig. 4, across all tasks and training paradigms, the model tends to assign a higher probability to the easier relation. As a concrete example, when trained to predict either antonyms or lexical entailments, the average probability mass placed on the antonyms of a word from the held-out set (easier relation) is  $13\%$ , while the average probability mass placed on the entailments of a word (harder relation) is  $8\%$ .
Thus, despite having a perfectly balanced finetuning set, pretrained models still predict label distributions in a way that align with their inductive biases (measured via the "intrinsic difficulty" of individual labels). This holds for all task adaptation methods, including full fine-tuning, meaning even paradigms and models that can fit more complex tasks still have residual biases from pretraining that affect their predictions. This also suggests wider-reaching consequences for model fairness
4Note that the model may (and often does) assign mass to answers outside of these sets.
and equity: simply debiasing a fine-tuning dataset is insufficient to overcome biases from pretraining.
In this paper, we construct TASKBENCH500, a synthetic task set which serves as a testbed for task adaptability. We focus on three axes of adaptability: ability to memorize, ability to (compositionally) generalize, and ability to fit to novel distributions. We study two adaptation paradigms: fine-tuning and prompt-tuning, finding that: 1. unlike fine-tuning, prompt-tuning cannot memorize completely arbitrary tasks beyond a small number of examples, 2. all adaptation paradigms demonstrate compositional adaptation to word-level compositions, but not sequence-level compositions, and 3. no paradigm is able to perfectly replicate the downstream distribution—all paradigms learn output distributions that align with its inductive biases.
In future work, TASKBENCH500 can be used to study other factors that may affect adaptability, such as length of the prompt in prompt-tuning, similarity between the task distribution and the pretraining distribution, or finer-grained distinctions between tasks (beyond lexical/factual/random, or composition type) that predict task adaptability. TASKBENCH500 can also be used to explore the limitations of prompt engineering on a GPT3-scale model. Finally, the current set of tasks and primitives in TASKBENCH500 are by no means complete. Future work can expand on these primitives and study the relationships between the tasks put forth here and real NLP tasks.
This work was supported by the MIT-IBM Watson AI lab. Part of the work was done using computing resources provided by a hardware donation from NVIDIA under the NVAIL program, and by the Lincoln Laboratory Supercloud. BZL is supported by an NDSEG Fellowship.
This paper introduces a new procedure for defining task suites. This procedure is then used to create a 500-task benchmark, which measures the adaptability of pre-trained language models to new tasks. Because the benchmark is created procedurally from databases of words and entities, we anticipate that there should be little to no identifying information or toxic and hateful content. Our datasets should
also contain less social bias compared to natural datasets.
However, like with all benchmarks, overfitting to static datasets can inhibit progress in NLP. Moreover, even though this dataset is procedurally generated, we cannot avoid all biases. The resources upon we build our benchmark are themselves biased—for example, lexical databases (like WordNet) are much richer for certain languages (like English) than others, and WikiData currently features many more men than women. Our benchmark currently only features English and Spanish tasks, with a heavy bias towards standard English. This can encourage development of methods that underserve non-standard-English-speaking communities.
We hope to mitigate the aforementioned issues by releasing the code to procedurally generate task suites. We emphasize that the benchmark is dynamic: consisting of not just the static task suite that we are currently releasing, but more importantly the procedure for creating new tasks suites. We encourage future researchers to develop analogous task suites for low-resource languages, non-standard English dialects, and more balanced sets of entities.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Andrea Esuli and Fabrizio Sebastiani. 2006. SENTIWORDNET: A publicly available lexical resource for opinion mining. In Proceedings of the Fifth International Conference on Language Resources and
Evaluation (LREC'06), Genoa, Italy. European Language Resources Association (ELRA).
Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. Bradford Books.
W. N. Francis and H. Kucera. 1979. Brown corpus manual. Technical report, Department of Linguistics, Brown University, Providence, Rhode Island, US.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, page 1321-1330. JMLR.org.
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In ICML, pages 4411-4421.
Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 2021-2031. Association for Computational Linguistics.
Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, and Pratyush Kumar. 2020. IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4948-4961, Online. Association for Computational Linguistics.
Daniel Keysers, Nathanael Scharli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc van Zee, and Olivier Bousquet. 2020. Measuring compositional generalization: A comprehensive method on realistic data. In International Conference on Learning Representations.
Najoung Kim and Tal Linzen. 2020. COGS: A compositional generalization challenge based on semantic interpretation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9087-9105, Online. Association for Computational Linguistics.
Brenden Lake and Marco Baroni. 2018. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In International conference on machine learning, pages 2873-2882. PMLR.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on
Empirical Methods in Natural Language Processing, pages 3045-3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582-4597, Online. Association for Computational Linguistics.
Nelson F. Liu, Tony Lee, Robin Jia, and Percy Liang. 2021a. Can small and synthetic benchmarks drive modeling innovation? a retrospective study of question answering modeling approaches.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021b. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
M. Antonia Martí, MarionaTaulé, Lluis Márquez, and Manuel Bertran. 2007. Cess-ece: A multilingual and multilevel annotated corpus.
Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192-1202, Brussels, Belgium. Association for Computational Linguistics.
Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428-3448, Florence, Italy. Association for Computational Linguistics.
Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340-2353, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Sungjoon Park, Jihyung Moon, Sungdong Kim, Won Ik Cho, Ji Yoon Han, Jangwon Park, Chisung Song, Junseong Kim, Youngsook Song, Taehwan Oh, Joohong Lee, Juhyun Oh, Sungwon Lyu, Younghoon Jeong, Inkwon Lee, Sangwoo Seo, Dongjun Lee, Hyunwoo Kim, Myeonghwa Lee, Seongbo Jang, Seungwon Do, Sunkyoung Kim, Kyungtae Lim, Jongwon Lee, Kyumin Park, Jamin
Shin, Seonghyun Kim, Lucy Park, Alice Oh, Jung-Woo Ha, and Kyunghyun Cho. 2021. KLUE: Korean language understanding evaluation. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
Fabio Petroni, Tim Rocttäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463-2473, Hong Kong, China. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Kyle Richardson, Hai Hu, Lawrence Moss, and Ashish Sabharwal. 2020. Probing natural language inference models through semantic fragments. Proceedings of the AAAI Conference on Artificial Intelligence, 34:8713-8721.
Laura Ruis, Jacob Andreas, Marco Baroni, Diane Bouchacourt, and Brenden M Lake. 2020. A benchmark for systematic generalization in grounded language understanding. In Advances in Neural Information Processing Systems, volume 33, pages 19861-19872. Curran Associates, Inc.
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. 2019. Analysing mathematical reasoning abilities of neural models. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222-4235, Online. Association for Computational Linguistics.
Daniil Sorokin and Iryna Gurevych. 2018. Modeling semantics with gated graph neural networks for knowledge base question answering. In Proceedings
of the 27th International Conference on Computational Linguistics, pages 3306-3317, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Denny Vrandecic and Markus Krötzsch. 2014. Wiki-data: A free collaborative knowledgebase. Commun. ACM, 57(10):78-85.
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. In NeurIPS.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353-355, Brussels, Belgium. Association for Computational Linguistics.
Jason Weston, Antoine Bordes, Sumit Chopra, and Tomás Mikolov. 2016. Towards ai-complete question answering: A set of prerequisite toy tasks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings.
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaowehua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, and Zhenzhong Lan. 2020. CLUE: A Chinese language understanding evaluation benchmark. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4762-4772, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
A More details on TASKBENCH500 creation procedure
A.1 Task creation details
For atomic lexical tasks, we take a subset of relations specified in either Wordnet (Fellbaum, 1998) or SentiWordNet (Esuli and Sebastiani, 2006). For atomic factual tasks, we take a subset of tasks from Wikidata (Vrandecic and Krötzsch, 2014). We also have 3 broad categories of composition functions: set operations, logical operations, and sequential operations. The full list of atomic tasks can be found in Table 4 and the list of composition functions can be found in Table 5.
We enumerate all possible depth-2 word level compositions of each task, and the sequential versions of them (i.e. if the task is a relation, inserting it into a map, or if the task is a predicate, inserting it into a filter), up to 500 tasks. We also apply some basic heuristics to filter identical tasks: for example, we filter symmetric relations, e.g. union(B,A) is identical to union(A,B), or avoid the use of logical operations alongside set operations, e.g.  $\text{lor}(\text{in}(x,A), \text{in}(x,B))$  is identical to  $\text{in}(x,\text{union}(A,B))$ . Our full list of tasks can be found in Tables 4, 6, 7 and 8.