D o G -Instruct: Towards Premium Instruction-Tuning Data via Text-Grounded Instruction Wrapping (2024)

Abstract: The improvement of LLMs’ instruction-following capabilities relies heavily on the availability of high-quality instruction-response pairs. Unfortunately, the current methods used to collect the pairs suffer from either unaffordable labor costs or severe hallucinations in the self-generation of LLM.To tackle these challenges, this paper proposes a scalable solution.It involves training LLMs to generate instruction-response pairs based on human-written documents, rather than relying solely on self-generation without context.Our proposed method not only exploits the advantages of human-written documents in reducing hallucinations but also utilizes an LLM to wrap the expression of documents, which enables us to bridge the gap between various document styles and the standard AI response.Experiments demonstrate that our method outperforms existing typical methods on multiple benchmarks.In particular, compared to the best-performing baseline, the LLM trained using our generated dataset exhibits a 10% relative improvement in performance on AlpacaEval, despite utilizing only 1/5 of its training data.Furthermore, a comprehensive manual evaluation validates the quality of the data we generated.

## introduction recent efforts in the nlp community have focused on instruction-tuning @xcite @xcite , i.e., improving large language models' (llms) capacity to understand and follow instructions @xcite @xcite . advanced llms have been trained to be capable of generating customized outputs when provided with specific instructions (with inputs), enabling them to adapt to new tasks without prior exposure. as a crucial problem in improving llms' instruction-following capability, how to collect high-quality instruction-response pairs is gaining popularity. the majority of existing methods either rely on hiring professionals to write instructions for various nlp tasks @xcite or promote the use of llms to automatically generate instructions @xcite @xcite . unfortunately, these methods have limitations either in terms of scalability due to the labor-intensive nature of the annotation process or in terms of data quality due to the hallucination problem @xcite associated with llms. recent research @xcite has provided a more potential idea: first directly using human-written documents as typical responses and then utilizing llms to predict the latent user instructions. this method, known as instruction back-translation @xcite , is based on the belief that human-written documents are inherently less prone to hallucinations compared to responses generated solely by llms. however, we argue that even if a document is free of hallucinations, it is not always appropriate to employ it directly as a typical response. this is attributed to two main reasons: a) first, not all parts of a document are valuable in constructing a response. for example, the red part of the document (a) in figure 1 is completely useless for back-translating the resulting instruction (the gold box). moreover, valuable parts of the document often have fuzzy boundaries. for instance, the red text of (b) aims to create a tense atmosphere, again unsuitable to keep in response, but it also has some relevance to the topic (alien research) and is therefore difficult to be filtered out by simple preprocessing. b) second, due to the different purposes of writing, there are often gaps in expression between ## problem formulation here m is an llm-based instruction wrapper that transforms d i into an instruction-response pair. ## collection of dog-instruct data figure 2 shows the entire process of our method. a) first, the instruction wrapper m is trained using the meta-training set ω, which is constructed by the well-aligned gpt-4. b) subsequently, m takes sampled documents d as inputs to generate (x , y) for constructing dog-instruct. ## related work instruction tuning humans possess the ability to effortlessly comprehend and execute tasks based on verbal instructions @xcite @xcite . likewise, advancements in deep learning have enabled language models (llms) @xcite @xcite to acquire the capability to understand and follow instructions. instruction tuning serves as a promising method, involving the fine-tuning of llms using training data and instructions from a collection of upstream tasks @xcite @xcite @xcite . instruction-tuning data collection the collection of high-quality instruction-tuning data @xcite @xcite is a pressing issue in enhancing the capability of instruction-following. previous methods can be broadly categorized into three main groups. firstly, methods like super-ni @xcite and dolly @xcite rely on hiring professionals to create instructions for diverse nlp tasks. secondly, methods such as self-instruct @xcite and al-paca @xcite advocate for the use of llms to automatically generate instruction-tuning data, thus eliminating the need for manual labor. lastly, dynosaur @xcite employs llms to convert existing nlp datasets from huggingface into instruction-tuning data at a reduced cost. the work most related to this paper is @xcite . it uses a human-written document as a natural response and leverages an llm to generate the corresponding instruction based on the response. in contrast, our instruction wrapper selects the valuable parts of the documents for constructing appropriate responses. ## conclusion & limitation this paper introduces a new method called instruction wrapping, which enables the automatic collection of high-quality instruction-tuning data from human-written documents. our trained instruction wrapper not only utilizes documents to mitigate response hallucinations but also modifies raw documents to align them with the standard response style. through comprehensive evaluations, we demonstrate that our method achieves remarkable results on various widely used benchmarks while utilizing the fewest training examples. the limitations of our method are that it cannot handle excessively long documents and can only generate a single task for a document. in future work, we will explore generating more complicated instructions that involve multiple longer documents.