Transcending Scaling Laws with 0.1% Extra Compute (2023)

Abstract: Scaling language models improves performance but comes with significant computational costs. This paper proposes UL2R, a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. The key idea is to continue training a state-of-the-art large language model on a few more steps with UL2’s mixture-of-denoiser objective. We show that, with almost negligible extra computational costs and no new sources of data, we are able to substantially improve the scaling properties of large language models on downstream metrics. In this paper, we continue training a baseline language model, PaLM, with ULR2, introducing a new set of models at 8B, 62B, and 540B scale which we call U-PaLM. Impressively, at 540B scale, we show an approximately 2x computational savings rate where U-PaLM achieves the same performance as the final PaLM 540B model at around half its computational budget (i.e., saving ~4.4 million TPUv4 hours). We further show that this improved scaling curve leads to “emergent abilities” on challenging BIG-Bench tasks—for instance, U-PaLM does much better on some tasks or demonstrates better quality at much smaller scale (62B as opposed to 540B). Overall, we show that U-PaLM outperforms PaLM on many few-shot setups, including reasoning tasks with chain-of-thought (e.g., GSM8K), multilingual tasks (MGSM, TydiQA), MMLU and challenging BIG-Bench tasks.

## introduction there has been significant interest in scaling of language models @xcite @xcite . scaling has inspired new research across multiple fronts, e.g., scaling laws @xcite @xcite , emergent abilities [wei † work completed while at google. this paper proposes a new method to dramatically improve the scaling curves of large language models on downstream performance with a relatively tiny amount of additional computation cost. the key idea is to continue training an existing causal language model @xcite with a mixture of new objectives-specifically, the ul2 training objective mixture @xcite . this restoration is expected to only cost roughly 0.1% to 1% of the original training flops and requires no new data sources, making it highly efficient and convenient. we call this approach ul2r or ul2restore. the ul2 objective combines prefix language modeling and long-short span corruption (e.g., infill-ing) tasks @xcite that can be controlled at inference time using a mode switching prompt. training a large language model with ul2 can be interpreted as teaching it to leverage bidirectional attention (i.e., prefixlm) or leverage infilling-style pretraining that have been the foundation of language understanding (e.g., t5 @xcite ). to this end, we postulate that imbuing a state-of-theart large language model such as palm @xcite with these diverse pretraining schemes as a complement to the original language model objective, enables the model to perform significantly better. moreover, the ul2 objective enables new prompting capabilities in palm which allows it to perform infilling based prompting. we show that adapting palm with ul2r not only results in significantly better scaling laws on well-established few-shot nlp tasks, but also, in our scaling experiments on downstream few-shot tasks, we show that ul2r is two times more efficient (computation savings of approximately 2x) at 540b scale -reaching the performance of the final palm 540b model with only half the computation, saving up to 4.4 million tpuv4 hours. in addition to competitive performance across a range of well-established nlp @xcite , multilingual @xcite , and reasoning @xcite benchmarks, we also study the impact of ul2r on a suite of challenging bigbench tasks from @xcite . notably, a subset of tasks are described as 'emergent' because palm's performance remains flat up to model scale of 62b and only becomes better than non-random at 540b scale. on these set of tasks, we find that ul2r enables (1) doing significantly better at tasks that palm struggles at (e.g., navigate, geometric shapes, hyperbaton) and (2) elicits emergent behavior at a smaller scale such as 62b or 8b (e.g., crass ai, vitaminc fact verification). on top of that, u-palm strongly outperforms palm on some challenging bigbench tasks. emergence within the context of large language models is a nascent research area. as the nobel prize-winning physicist philip anderson put it, 'more is different.' @xcite which describes unpredictable phenomena at different scales. in our context and with mixture-of-denoisers in ul2, we would like to think of this phenomena as 'more is different, but different can also more' since different pretraining objectives can improve language model quality or elicit new emergent abilities. this work shows that diversity and richer training paradigms can be key to learning new capabilities that were previously hard to acquire with only causal language modeling. finally, in addition to emergent task performance and overall improved scaling curves, we show that u-palm is also practically more useful since it is equipped with a secondary mode of prompting, i.e., bidirectional infilling. specifically, ul2r enables a secondary capability for prompting u-palm which can be used to fill in more than one blanks in the input prompt. interestingly, we find that only a small amount of ul2r (e.g., 0.1% tokens or flops) is sufficient to imbue the model with this new capability. ## u-palm this section introduces the technical details of u-palm (i.e., palm + ul2r). u-palm is initialized from palm and leverages the same architecture. this section describes the training procedures of ul2r and how they are applied to continue training palm. we refer the reader to section 10 in the appendix for a comprehensive review of related work. ## conclusion we proposed ul2r for continued training of palm. we show that with only ≈ 0.1% additional flops (or compute), we are able to improve the scaling curve and properties of palm on many downstream tasks and metrics. notably, ul2r enables a 4.4 million tpuv4 savings at 540b scale. the resulting model which we call u-palm outperforms palm on english nlp tasks (e.g., commonsense reasoning and closed-book question answering), reasoning tasks with chain-of-thought, multilingual reasoning, mmlu and a suite of challenging big-bench tasks.