Fine-Tuning Large Language Models with Sequential Instructions (2025)

Abstract: We find that existing instruction-tuned models usually struggle to adhere to a query with multiple intentions, which impairs their performance when the completion of several tasks is demanded by a single command. Hence, this paper teaches models to respond to sequential instructions. Our first attempt stems from a task-driven perspective, manually creating additional intermediate tasks to train multilingual and visual question answering. Next, we develop an automatic and generic process that turns instructions in existing data into diverse and complex task chains. Models that underwent sequential instruction tuning follow a list of instructions better and deliver higher results in coding, maths, and open-ended generation. Moreover, we put forward a new benchmark named SeqEval to evaluate a model’s ability to follow all the instructions in a sequence, which further corroborates the benefits of our sequential instruction tuning method.

## introduction instruction tuning (it), or supervised fine-tuning (sft), gives large language models (llms) the ability to execute tasks specified by users @xcite @xcite . nevertheless, popular instruction mixtures contain rather straightforward instructions derived from conventional nlp tasks or open-ended dialogues @xcite @xcite . hence, they may suffer from the absence of multiple instructions in one query. we speculate that this hinders the fine-tuned models from navigating a sequence of tasks or tools in a single command, which is arguably crucial for complex scenarios with reasoning (e.g., coding and maths) or knowledge pivoting (e.g., cross-lingual and cross-modal question answering, @xcite ., method model r-l bs following r r r a r+a base prompting mistral-7b 50 88 94 11 6 mixtral-8×7b 38 87 85 30 16 llama-2-13b 10 81 10 51 6 llama-2-70b 35 87 54 44 21 llama-3-8b 63 91 41 20 7 it mistral-7b-alpaca 48 88 64 56 45 llama-3-8b-alpaca 42 82 33 69 27 sit (ours) mistral-7b-alpaca 55 92 99 85 84 llama-2-7b-alpaca 54 92 89 91 85 llama-2-13b-alpaca 39 90 99 93 93 llama-3-8b-alpaca 53 93 96 95 95 to tackle this, we propose a sequential instruction tuning (sit) paradigm that uses simple strategies to automatically augment it data with multistep tasks. as a prototype, we start with manually defined intermediate steps that are task-focused and interpretable for multilingual or visual ques- ## the seqeval benchmark to measure both the response quality and following ability of llms when queried with sequential instructions, we construct a novel open-ended generation benchmark named seqeval. we apply the pipeline described in section 2.2 to the queries in alpacaeval @xcite using gpt-4-turbo, which is different from the open-source models used to create training instances. specifically, in the first iteration we uniformly sample from "decompose", "prefix", and "suffix", and in subsequent iterations we limit the choices to "prefix" and "suffix". we repeat the process for four iterations, and we mix the examples resulting from iterations 1, 2, 3, and 4 with a ratio of 0.1, 0.2, 0.3, and 0.4 respectively. this puts more primacy on instructions containing multiple complex sequential queries that underwent multiple transformations. considering our two-fold motivations of aligning llms with human instruction-following behaviour and aiding complex task performance, throughout the paper, we use three types of metrics as explained below: ## experiments and results in section 4.1, we first report our results for two settings where we manually define intermediate steps for composite tasks: 1) translation for multilingual question answering and 2) image captioning for visual question answering. afterwards, in section 4.2, we further confirm the effectiveness of our automatically generated sequential instruction tuning datasets on benchmarks for factuality, reasoning, and open-ended generations. we provide the full experiment details, including evaluation setup in appendix b. ## related work instruction tuning fine-tuning a base model on formatted input-output data makes it follow user queries and generalise to unseen tasks @xcite @xcite . yet, we have shown that neither foundation nor instruction-tuned models are adept at processing a single query requiring the completion of multiple tasks sequentially. this is potentially because the queries in existing data are rather straightforward: mostly supervised nlp tasks and open-ended dialogues wherein instruction-response pairs exhibit a direct relationship @xcite @xcite @xcite . the machine-translated multilingual counterparts inevitably inherit the same flaws @xcite @xcite . several works fine-tuned llms to perform multi-turn conversations @xcite , which requires continuous monitoring and input during execution, but our sequential instructions do not. recently, @xcite proposed using offthe-shelf llms to produce complex instructions and concurrent to us, hayati et al. ( 2024 ) created compositional tasks by merging existing tasks. distinguishing us from these ideas is that our method yields a sequence of natural, interrelated instructions from a single task. the field then witnesses an increased interest in multi-task or sequential task execution @xcite @xcite . commercial llm provider @xcite also pointed out that instruction chaining "is a key capability that is missing to make agentic workflow feasible". chain-of-thought, knowledge pivoting, and prolonged generation llms that "reason" "step by step" before answering a question yield better outcomes @xcite . this is extended by chained prompting @xcite and least-to-most-prompting @xcite . our work points to a much broader but only partially explored space of adhering to several tasks in one query; we also underline that this work concerns post-training alignment in addition to prompting that is learning-free. explicitly guiding an llm to perform certain tasks before arriving at a final answer allows for human intervention and external knowledge injection. most previous research centred around language pivoting (often via english), which has proven ef-fective in a wide array of applications @xcite @xcite @xcite . a contemporaneous work introduced cross-lingual instruction tuning @xcite , which can be viewed as a task-specific case of our approaches. lastly, prolonged generation incurs higher inference costs but also offers more computational capacity @xcite . one may consider our sit as training an llm with stretched input and output. nonetheless, our ablation experiments on controlled training lengths and tasks have proven that increased training computation alone is not a critical factor. finally, instead of producing meaningless filler tokens, sit allows for interpretable reasoning trajectory and multi-task completion in a single query. ## conclusion this work unveiled a major drawback in state-ofthe-art, open-source models as large as 70b or 8×7b: they struggle to follow multi-step instructions within a single query. accordingly, we proposed a new method, sequential instruction tuning (sit), to equip llms with this ability. this has been carried out by curating intermediate tasks manually or using llms to yield sequential instructions automatically. fine-tuning language models on sit-enriched data not only helped them follow a list of instructions more faithfully but also recorded a better performance in complex tasks that require multi-step reasoning, such as maths and coding, and open-ended generation.