Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
Authors: name: Hamish Ivison; name: Yizhong Wang; name: Valentina Pyatkin; name: Nathan Lambert; name: Matthew Peters; name: Pradeep Dasigi; name: Joel Jang; name: David Wadden; name: Noah A. Smith; name: Iz Beltagy; name: Hannaneh Hajishirzi
Published: 2023-11-17 18:45:45

Abstract: Since the release of T\"ULU [Wang et al., 2023b], open resources for
instruction tuning have developed quickly, from better base models to new
finetuning techniques. We test and incorporate a number of these advances into
T\"ULU, resulting in T\"ULU 2, a suite of improved T\"ULU models for advancing
the understanding and best practices of adapting pretrained language models to
downstream tasks and user preferences. Concretely, we release: (1)
T\"ULU-V2-mix, an improved collection of high-quality instruction datasets; (2)
T\"ULU 2, LLAMA-2 models finetuned on the V2 mixture; (3) T\"ULU 2+DPO, T\"ULU
2 models trained with direct preference optimization (DPO), including the
largest DPO-trained model to date (T\"ULU 2+DPO 70B); (4) CODE T\"ULU 2, CODE
LLAMA models finetuned on our V2 mix that outperform CODE LLAMA and its
instruction-tuned variant, CODE LLAMA-Instruct. Our evaluation from multiple
perspectives shows that the T\"ULU 2 suite achieves state-of-the-art
performance among open models and matches or exceeds the performance of
GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data,
training and evaluation code to facilitate future open efforts on adapting
large language models.