Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023)

Abstract: Fine-tuning on instruction data has been widely validated as an effective practice for implementing chat language models like ChatGPT. Scaling the diversity and quality of such data, although straightforward, stands a great chance of leading to improved performance. This paper aims to push the upper bound of open-source models further. We first provide a systematically designed, diverse, informative, large-scale dataset of instructional conversations, UltraChat, which does not involve human queries. Our objective is to capture the breadth of interactions between a human user and an AI assistant and employs a comprehensive framework to generate multi-turn conversation iteratively. UltraChat contains 1.5 million high-quality multi-turn dialogues and covers a wide range of topics and instructions. Our statistical analysis of UltraChat reveals its superiority in various key metrics, including scale, average length, diversity, coherence, etc., solidifying its position as a leading open-source dataset. Building upon UltraChat, we fine-tune a LLaMA model to create a powerful conversational model, UltraLM. Our evaluations indicate that UltraLM consistently outperforms other open-source models, including WizardLM and Vicuna, the previously recognized state-of-the-art open-source models.

## introduction large language models @xcite @xcite (llms) have demonstrated exceptional generalization capability on a variety of language-related tasks. notably, @xcite , an optimized version of gpt-3 @xcite for conversation, along with @xcite , takes the user experience to another level via excelling in comprehending and generating responses in a natural and interactive manner. the introduction of ## data construction llms are believed to be better annotators than human-being in many scenarios @xcite . however, employing llms such as chat-gpt directly for generating multi-turn conversations may yield satisfactory but less informative results, as it cannot enjoy the benefit of reinforce- meta topics sub topics questions about conceptions representative entities meta questions detailed questions associated questions material types material generation instructions detailed instructions ## experiments we developed ultralm, an enhanced variant of the llama-13b @xcite model, by training it on the ultrachat dataset. to improve the model's comprehension of dialogue context, we break down each dialogue into smaller sequences, limiting them to a maximum length of 2048 tokens. during the training process, we only calculate the loss for the model's responses. this approach ensured that the model had access to the relevant information from earlier parts of the conversation, enabling a more comprehensive understanding of the ongoing dialogue. by incorporating the preceding context, ultralm was equipped to generate more contextually appropriate and coherent responses. we use standard cross-entropy loss to finetune the model. the model is trained with 128 a100 gpus and the total batch size is 512. the evaluation of the trained model is conducted in two folds. we first evaluate ultralm on traditional benchmark datasets to delineate the knowledge scope and the multiple abilities of the language model. to better demonstrate the chat ability of language models, an automatic response quality evaluation is performed to showcase the model's proficiency in delivering accurate and informative content during chat interactions. note that ultralm is solely trained on ultrachat dataset without further finetuning on task specific datasets.