Venue: NAACL 2025

# Fine-Tuning Large Language Models with Sequential Instructions

Hanxu Hu\({}^{*,1,2}\) Simon Yu\({}^{*,1,3}\) Pinzhen Chen\({}^{*,1}\) Edoardo M. Ponti\({}^{1}\)

\({}^{1}\)University of Edinburgh \(~{}^{2}\)University of Zurich \(~{}^{3}\)Northeastern University

hanxu.hu@uzh.ch, yu.chienortheastern.edu, {pinzhen.chen,eponti}@ed.ac.uk

Equal contribution. All resources are available at [https://seqit.github.io/](https://seqit.github.io/).

###### Abstract

We find that existing instruction-tuned models usually struggle to adhere to a query with multiple intentions, which impairs their performance when the completion of several tasks is demanded by a single command. Hence, this paper teaches models to respond to **sequential instructions**. Our first attempt stems from a task-driven perspective, manually creating additional intermediate tasks to train multilingual and visual question answering. Next, we develop an automatic and generic process that turns instructions in existing data into diverse and complex task chains. Models that underwent sequential instruction tuning follow a list of instructions better and deliver higher results in coding, maths, and open-ended generation. Moreover, we put forward a new benchmark named **SeqEval** to evaluate a model's ability to follow all the instructions in a sequence, which further corroborates the benefits of our sequential instruction tuning method.

## 1 Introduction

Instruction tuning (IT), or supervised fine-tuning (SFT), gives large language models (LLMs) the ability to execute tasks specified by users Mishra et al. (2022); Sanh et al. (2022); Wei et al. (2022). Nevertheless, popular instruction mixtures contain rather straightforward instructions derived from conventional NLP tasks or open-ended dialogues Sanh et al. (2022); Taori et al. (2023); Conover et al. (2023). Hence, they may suffer from the absence of multiple instructions in one query. We speculate that this hinders the fine-tuned models from navigating a sequence of tasks or tools in a single command, which is arguably crucial for complex scenarios with reasoning (e.g., coding and maths) or knowledge pivoting (e.g., cross-lingual and cross-modal question answering, Shi et al. (2023); Zhang et al. (2024)) Moreover, this detracts from user experience as models may fail to track whether all requests have been fulfilled. We empirically verify our hypothesis by prompting various state-of-the-art open-source LLMs, Llama 2/3 and Mistral Dubey et al. (2024); Jiang et al. (2023), with simple two-step instructions--already more than they can shake a stick at as shown in Table 1. We find that not only did task accuracy degrade dramatically, but also that they often failed to follow the entire list of instructions, particularly for models fine-tuned on public datasets like Alpaca Tiori et al. (2023).

To tackle this, we propose a sequential instruction tuning (SIT) paradigm that uses simple strategies to automatically augment IT data with multi-step tasks. As a prototype, we start with manually defined intermediate steps that are task-focused and interpretable for multilingual or visual ques

\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & & **R-L** & **BS** & \multicolumn{3}{c}{**Following**} \\ \cline{5-6} Method & **Model** & R & R & A & R+A \\ \hline \multirow{7}{*}{Base} & Mistral-7B & 50 & 88 & 94 & 11 & 6 \\  & Mixtral-8\%D & 38 & 87 & 85 & 30 & 16 \\  & Llama-2-13B & 10 & 81 & 10 & 51 & 6 \\  & Llama-2-7OB & 35 & 87 & 54 & 44 & 21 \\  & Llama-3-8B & 63 & 91 & 41 & 20 & 7 \\ \hline \multirow{3}{*}{IT} & Mistral-7B-Alpaca & 48 & 88 & 64 & 56 & 45 \\  & Llama-3-8B-Alpaca & 42 & 82 & 33 & 69 & 27 \\ \hline \multirow{3}{*}{SIT (ours)} & Mistral-7B-Alpaca & 55 & 92 & 99 & 85 & 84 \\  & Llama-2-7B-Alpaca & 54 & 92 & 89 & 91 & 85 \\  & Llama-2-13B-Alpaca & 39 & 90 & 99 & 93 & 93 \\  & Llama-3-8B-Alpaca & 53 & 93 & 96 & 95 & 95 \\ \hline \hline \end{tabular}
\end{table}
Table 1: We test various base and instruction-tuned models on 100 samples of CommonsenseQA by asking them to repeat (R) the input and then answer (A) the question. We report ROUGE-L (R-L) and BERScore (BS) on the repeated content and human-counted success rate (%) of models following instructions. Both base and IT’ed models lack the ability to follow the two-step sequential instruction which is significantly improved by our SIT.



tion answering Artetxe et al. (2020); Hudson and Manning (2019), namely "translate then predict" and "caption then answer". Furthermore, we generalise and automate the data creation pipeline where intermediate tasks are seeded from a single-task instruction. Our method stands in contrast with previous automatic augmentation methods that make instructions complex or diverse, but not sequential. Applying this to instruction mixtures such as Alpaca, FlanCoT, MetaMath, and Tulu-V2 Taori et al. (2023); Longpre et al. (2023); Yu et al. (2023); Ivison et al. (2023) allows us to construct natural, diverse, and high-quality sequential instructions. Comparing LLMs fine-tuned with our SIT data and the original IT data, we observe significant boosts in factuality, reasoning, and open-ended generation. Ablation studies confirm SIT's generalisability to different models and tasks, and that the score gains are not merely due to inflated training tokens.

Finally, to confirm that sequential instruction-tuned LLMs acquire a better ability to execute all the instructions in a query, we develop and make public a new benchmark for open-ended generation, SeqEval. It is constructed by applying self-instruct Wang et al. (2023) to the AlpacaEval benchmark Li et al. (2023c) with an emphasis on chained tasks. With this benchmark, we find that SIT models are vastly superior in instruction-following behaviours. Altogether, we hope that the SIT suite presented in this paper: the methodology, the SIT data, and the SeqEval benchmark will contribute to endowing LLMs with the ability to solve chained tasks.

## 2 Methodology

### Sequential instructions

Existing instruction data usually comprise single-step instructions (i.e. an instruction-response pair resembling one task); however, this falls short of equipping models with the ability to handle a query containing (explicitly or implicitly) multiple sub-tasks. Developing from a single-task instruction \({\mathbf{i}}\), we define a **sequential instruction \({\mathbf{s}}\)** as a query that contains multiple inter-related tasks or steps: \({\mathbf{s}}={\hat{\mathbf{iota}}_{1}}\oplus{\hat{\mathbf{imath}}_{2}}\oplus\cdots\oplus{ \hat{\mathbf{iota}}_{n}}\) with \(n\geq 2\), where \({\mathbf{i}}_{k}\) denotes the \(k^{\text{th}}\) task and \(\oplus\) is a concatenation operation. Querying a model parameterized by \(\theta\) leads to a response \(\hat{\mathbf{y}}\sim p({\mathbf{y}}|{\mathbf{i}}_{1}\oplus{\hat{\mathbf{imath}}_{2}}\oplus \cdots\oplus{\hat{\mathbf{iota}}_{n}};{\mathbf{\theta}})\) which can be further split into individual responses for each step \(\hat{\mathbf{y}}=\hat{\mathbf{y}}_{1}\oplus\hat{\mathbf{y}}_{2}\oplus\cdots\oplus\hat{ \mathbf{y}}_{n}\).

### Sequential instruction tuning

We propose including a list of instructions in a single query when fine-tuning LLMs to address the lack of sequential instruction following capability in LLMs. In particular, we present one manual and one automatic strategy to create sequential instruction tuning (SIT) data. The manual way requires prior knowledge of how a downstream task can be decomposed into simpler steps so that the training instructions can mirror this structure; the automatic way can instead generalise to more complex and open-ended scenarios. The data creation pipeline (both manual and automatic) is shown in Figure 1. Given this data, instruction tuning fol

Figure 1: The construction of sequential instruction data via manual (left) and automatic (right) processes.

 

lows the conventional training paradigm: we minimise \({\cal L}(\mathbf{s},\mathbf{\hat{y}};\theta)=-\log p(\mathbf{\hat{y}}|\mathbf{s};\theta)\), the negative log-likelihood of the output given the instructions.

Creating instructions from manually defined proceduresTasks involving multiple languages or modalities could be challenging. When prior knowledge about the task is available, it is intuitive to break the prompt down into sequential steps and then fine-tune LLMs with this prompt to enhance their task decomposition skills. Formally, we wish to transform a single instruction into a sequence of instructions \(\mathbf{i}\rightarrow\mathbf{i}_{1},\mathbf{i}_{2},\cdots,\mathbf{i}_{n}\) that leads to an output \(\mathbf{\hat{y}}\) whose last solution \(\mathbf{\hat{y}}_{n}\) is in accordance to the last instruction \(\mathbf{i}_{n}\) and is the desirable response to the original task of interest.

Specifically, for multilingual and cross-lingual tasks, the sequential instructions can contain intermediate tasks like translation (often to English) Conneau et al. (2018); Zhang et al. (2024), whereas for multi- or cross-modal tasks, this can be speech-to-text transcription or image-to-text captioning. Although this process needs manual intervention, it is broadly applicable to entire families of tasks and it increases interpretability and control. While previous approaches remain at prompting Qin et al. (2023); Huang et al. (2023), we apply this idea to instruction tuning by transforming the data.

Automatically and iteratively generating instructionsMoving beyond task-specific sequential instruction tuning, which necessitates manual curation, we propose an automatic and iterative pipeline, _Seq-Instruct_, to develop sequential instructions from existing data inspired by Self-Instruct Wang et al. (2023). This pipeline is general-purpose and can automatically generate diverse tasks with different intermediate paths using just open-source LLMs (Llama-3-70B-instruct and Command R+; Dubey et al. (2024); Gomez (2024). We anticipate that models fine-tuned on such data are more robust and versatile in handling queries demanding a list of actions.

Specifically, given an existing instruction sequence \(\mathbf{i}_{1}\oplus\cdots\oplus\mathbf{i}_{n}\), \(n\geq 1\) without losing generality to both single and sequential instructions, we prompt an LLM to take one of the actions below. These options are simple yet lead to coherent and natural instruction chains:

* **decompose**--split an instruction into two: \(\mathbf{i}_{\text{new}}=\mathbf{i}_{1}\oplus\cdots\oplus\mathbf{i}_{k_{1}}\oplus\mathbf{i}_{ k_{2}}\oplus\cdots\oplus\mathbf{i}_{n}\);
* **prefix**--add a preceding instruction: \(\mathbf{i}_{\text{new}}=\mathbf{i}_{\text{prefix}}\oplus\mathbf{i}_{1}\oplus\cdots\oplus \mathbf{i}_{n}\);
* **suffix**--add a succeeding instruction: \(\mathbf{i}_{\text{new}}=\mathbf{i}_{1}\oplus\cdots\oplus\mathbf{i}_{n}\oplus\mathbf{i}_{\text {suffix}}\);
* **hold**--do nothing: \(\mathbf{i}_{\text{new}}=\mathbf{i}_{1}\oplus\cdots\oplus\mathbf{i}_{n}\).

Given a collection of instruction-response pairs \({\cal D}=\{(\mathbf{x}_{1},\mathbf{y}_{1}),(\mathbf{x}_{2},\mathbf{y}_{2}),\cdots,(\mathbf{x}_{|{ \cal D}|},\mathbf{y}_{|{\cal D}|})\}\), the above pipeline is applied to each data instance \((\mathbf{x}_{k},\mathbf{y}_{k})\) to generate a new instruction \(\mathbf{x}_{k_{\text{new}}}\). Then the same LLM creates a corresponding response \(\mathbf{y}_{k_{\text{new}}}\). All such new input-output pairs \((\mathbf{x}_{k_{\text{new}}},\mathbf{y}_{k_{\text{new}}})\) form a new set of sequential instruction data \({\cal D}_{\text{new}}\). We highlight that such a process can be carried out iteratively to grow a single instruction into a complex one containing an arbitrary number of instructions. The complete prompt templates are given in Appendix B.4.

## 3 The SeqEval Benchmark

To measure both the response quality and following ability of LLMs when queried with sequential instructions, we construct a novel open-ended generation benchmark named _SeqEval_. We apply the pipeline described in Section 2.2 to the queries in AlpacaEval Li et al. (2023c) using GPT-4-Turbo, which is different from the open-source models used to create training instances. Specifically, in the first iteration we uniformly sample from "decompose", "prefix", and "suffix", and in subsequent iterations we limit the choices to "prefix" and "suffix". We repeat the process for four iterations, and we mix the examples resulting from iterations 1, 2, 3, and 4 with a ratio of 0.1, 0.2, 0.3, and 0.4 respectively. This puts more primacy on instructions containing multiple complex sequential queries that underwent multiple transformations.

Considering our two-fold motivations of aligning LLMs with human instruction-following behaviour and aiding complex task performance, throughout the paper, we use three types of metrics as explained below:

* **Following rate**: the proportion of cases where a model produces an output for all tasks in the instruction, regardless of their correctness. For tasks where the intermediate output is known, we use Rouge-L between the output and the ground truth to measure whether a model has attempted the task; otherwise, we use human inspection or GPT-4-Turbo to verify if a model follows all instructions.
* **Downstream performance** is measured with a variety of task-specific metrics for tasks with 

gold-truth labels. For instance, for classification tasks, accuracy computes the proportion of \(\hat{\mathbf{y}}_{n}\) that matches the respective \(\mathbf{y^{*}_{n}}\) exactly.
* **LLM-as-a-judge**[23] is used to evaluate open-ended generation on AlpacaEval and our own SeqEval. We use GPT-4-Turbo to directly score the quality of each model response on a scale of 1 to 5. We also ask the judge LLM to produce a binary judgement of whether all questions are fulfilled. The exact prompt is reported in Appendix B.5 Figure 5.

## 4 Experiments and Results

In Section 4.1, we first report our results for two settings where we manually define intermediate steps for composite tasks: 1) translation for multilingual question answering and 2) image captioning for visual question answering. Afterwards, in Section 4.2, we further confirm the effectiveness of our automatically generated sequential instruction tuning datasets on benchmarks for factuality, reasoning, and open-ended generations. We provide the full experiment details, including evaluation setup in Appendix B.

### Manual task-driven SIT

#### 4.1.1 Multilingual question answering

Our first experiment is on multilingual (extractive) question answering, where we add a translation prefix task to instructions. The idea of pivoting from low-resource languages to high-resource ones before predicting the answer takes inspiration from "translate-test" cross-lingual transfer [6], where two separate models, a translation system and a classifier, are responsible for the two sub-tasks.

Task constructionFor training, we construct the SIT training data using a multilingual version of Alpaca from Chen et al. (2024) who translated the data into several languages of our interest: Chinese (zh), German (de), Russian (ru), and Spanish (es). We replace one-third of the English inputs with their translation in another language and prepend the respective instructions with "_First, translate the input into English, then_", which prompts the model to perform the translation task before answering.

EvaluationFor evaluating models on multilingual questions answering, we rely on the XQuAD test set [8]. In addition to the 4 training languages (seen), we also perform inference on 6 typologically diverse held-out languages (unseen): Arabic (ar), Greek (el), Vietnamese (vi), Hindi (hi), Turkish (tr), and Thai (th). The sequential instruction-tuned (SIT) models are prompted with the same translation query used in training--_"First translate the input into English, then"_--followed by the questions in the XQuAD test examples. Results are described in Table 2 for Mistral-7B and Llama-3-8B as base LLMs. SIT obtains remarkably better results compared with IT in both accuracy and following rate with both base models for all languages. This indicates that SIT can benefit task performance and interoperability for cross-lingual tasks.

#### 4.1.2 Multimodal question answering

We then demonstrate that SIT can be extended beyond text-only scenarios, to multimodal tasks. We re-purpose a conventional (visual) instruction tuning dataset with sequential instructions and evaluate the SIT models on visual question answering (VQA) problems. Following Dai et al. (2023), we take a subset of the training split of VQAv2 [9]--a dataset of open-ended questions grounded on images--as seed data for instruction

\begin{table}
\begin{tabular}{l l c c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Seen**} & \multicolumn{4}{c}{**Unseen**} & \multicolumn{4}{c}{**Average**} \\ \cline{3-13}
**Base Model** & \multirow{2}{*}{**Method**} & de & zh & ru & es & ar & el & vi & hi & tr & th & Acc. & Follow \\ \hline \multirow{2}{*}{Mistral-7B} & IT & 44.7 & 21.7 & 38.7 & 46.3 & 12.8 & 15.2 & 25.3 & 9.6 & 24.9 & 9.2 & 24.84 & 15.9 \\  & SIT & **62.0** & **37.2** & **52.7** & **62.6** & **21.8** & **25.1** & **37.9** & **15.5** & **34.4** & **13.7** & **36.29** & **57.7** \\ \hline \multirow{2}{*}{Llama-3-8B} & IT & 44.3 & 34.6 & 41.3 & 49.7 & 31.2 & 42.5 & 40.0 & 36.3 & 34.3 & 30.6 & 38.48 & 5.4 \\  & SIT & **52.7** & **40.0** & **43.5** & **54.5** & **39.2** & **45.3** & **47.8** & **42.4** & **43.6** & **38.0** & **44.70** & **75.7** \\ \hline \hline \end{tabular}
\end{table}
Table 2: XQuAD results (accuracy and following rate, %) for multilingual Alpaca IT and SIT.

\begin{table}
\begin{tabular}{l c c} \hline \hline
**Method** & **VQAv2** (in-domain) & **GQA** (out-of-domain) \\ \hline prompt & 60.7 & 46.8 \\ IT & 61.3 & 47.0 \\ SIT & **63.4** & **48.9** \\ \hline \hline \end{tabular}
\end{table}
Table 3: YQAv2 and GQA results (accuracy, %) for InstructBLIP-Vicuna-7B prompting, IT, and SIT.

 

tuning. For the baseline, we phrase the instruction as "_Answer the input question based on the image_".

Task constructionWe consider image captioning a reasonable intermediate task before answering a question based on the information in an image. In particular, a caption extracts salient entities and events contained therein and bridges the gap between the modality of the question (text) and the image context (vision). Hence, we expect this sequence of sub-tasks to facilitate cross-modal reasoning. To create sequential visual instruction data, we augment the output of the training set of VQAv2 with a description of each image from MS COCO Lin et al. (2014), from which VQAv2 originated. During SIT, we augment the instruction with "_First describe the image, then answer the input question based on the image_".

EvaluationWe benchmark multimodal IT and SIT on the VQAv2 test split as an in-domain evaluation as well as on the GQA test-dev split as an out-of-domain evaluation Hudson and Manning (2019). We use an open-source multimodal LLM, InstructBLIP-Vicuna-7B Dai et al. (2023), as the base model. We display the results from prompting this off-the-shelf LLM and the two instruction-tuned models in Table 3. It clearly shows that the sequential instruction-tuned VLLM (SIT) surpasses both base model prompting and regular instruction tuning (IT) in-domain and out-of-domain.

### Automatic SIT

Task constructionOur Seq-Instruct is tested on four widely-used instruction datasets: Alpaca, the Flan Collection, MetaMath, and a diverse instruction mixture named Tulu-V2. Technically, as the seed data \(D^{0}\), we use the entire 52K Alpaca, a 100K sample of FlanCoT from the Open-Orca dataset Kim et al. (2023), or a 100K sample of Tulu-V2. Besides, we combine a 20K sample of MetaMath with Tulu-V2 to form Tulu-V2+MetaMath, to investigate the applicability of our methods to domain-specific data. We use Llama-3-70B-Instruct to automatically create sequential instruction data as described in Section 2.2. Seq-Instruct is applied for 2 iterations because at that point the most frequent option by the generator model is not to grow an instruction further. Crucially, the number of examples remains constant. Afterwards, we fine-tune Llama-3-8B Dubey et al. (2024) on the resulting SIT datasets: Alpaca-SIT, FlanCoT-SIT, Tulu-V2-SIT, or Tulu-V2-MetaMath-SIT. The rest of the training details are in Appendix B.3. We report statistics for the SIT datasets in Table 4.

BaselinesAs a straightforward baseline for SIT, we compare it with instruction tuning (IT) on the original datasets without sequential instructions (i.e., Alpaca, FlanCoT, Tulu-V2, or Tulu-V2+MetaMath). In addition, we report the results for WizardLM Xu et al. (2024), a method that automatically enhances instruction datasets by making them more complex ("in-depth evolution") and more diverse ("in-breadth evolution"). We note that the official WizardLM is only available for Alpaca. The output for both baselines is re-generated by the same latest model as our own Seq-Instruct, Llama-3-70B-Instruct, to ensure a fair comparison.

EvaluationWe first assess whether SIT enhances LLM performance in complex tasks, which implicitly require multi-step reasoning, by evaluating them on maths GSM8K Cobbe et al. (2021) and coding HumanEval; Chen et al. (2021). In addition, to address the concern that the Seq-Instruct pipeline might degrade model performance on generic tasks, we also evaluate the _general skills_ of SIT'ed models, including multiple-choice question answering (MMLU and ARC; Hendrycks et al., 2021; Clark et al., 2018) and open-ended generation (length-controlled AlpacaEval 2.0; Li et al., 2023c). To measure the _sequential instruction-following_ capabilities, we used two multilingual benchmarks in reading comprehension and maths reasoning: XQuAD Artetxe et al. (2020) and MGSM Shi et al. (2023); we prompt the models to "_First translate, then perform chain-of-thought reasoning, and lastly answer_" the questions. Finally, we run an evaluation on our proposed SeqEval, using LLVM-as-A Judge to measure the response quality and following rate on sequential instructions.

We report all results on the above benchmarks in Table 5. We find that SIT achieves better performance in all sequential tasks and almost all of the generic tasks. This proves that sequential instruction tuning can boost LLMs' instruction-following and even general reasoning capabilities. Improvements are consistent for all four datasets, which indicates that our method is widely applicable to existing instruction data. Overall, we demonstrate that Seq-Instruct creates diverse, high-quality instruction-tuning datasets. We include comprehensive results for sequential tasks with their following rates in Appendix C 



## 5 Analysis and Discussions

### Ablation study on data sizes and tasks

A variable factor in our comparison of IT and SIT is the length of training data--SIT has longer questions with more tasks and longer responses, thus implicitly updating a base model more than typical IT. While this might have been overlooked in prior research on instruction augmentation, we prepare three ablation studies on Alpaca to investigate whether SIT's higher metric scores are attributed to merely having more training tokens or tasks:

* The first experiment is at the **data level**, where we keep the total training tokens equal for IT and SIT. This is achieved by progressively sampling data from SIT data until its total output token count is equal to IT's. This reduces the SIT data from 52K to 36K instances.
* Next, strictly at the **instance level**, we keep every instance's length between IT and SIT data the same. This is done by iteratively and jointly sampling data from both IT and SIT, with the same length tokenized by Llama-3, into separate subsets. The final IT and SIT sub-training sets both have 40K instances. Since each pair of IT and SIT data has a matching length, every substask in SIT is much shorter than the task in IT.
* Finally, at the **task level** we design two ablations to probe the benefit of instruction chaining: 1. **SIT-split**: We decompose each sequential instruction back into multiple single-task instructions and merge them as a training set. This new SIT-split dataset has the same tasks (contents) as SIT but is broken down into a total of 98K single-task data points. 2. **SIT-multi**: Another contrasting experiment is that we reshape a sequential instruction by interleaving tasks and responses to form dialogue-like data: instances are reformulated as \({\mathbf{i}_{1}}\oplus{\mathbf{y}^{\star}_{1}}\oplus{\mathbf{i}_{2}}\oplus{\mathbf{y}^{\star}_ {2}}\oplus\cdots\oplus{\mathbf{i}_{n}}\oplus{\mathbf{y}^{\star}_{n}}\). This setup simulates a mult-turn conversation.

\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multicolumn{2}{c}{**Seed Data**} & \multirow{2}{*}{**Iteration**} & \multicolumn{2}{c}{**Avg. Input Token**} & \multicolumn{2}{c}{**Avg. Output Token**} & \multicolumn{2}{c}{**Seq-Instruct** (\$2.2) & **Option Selected (\%)**} \\  & & & & **Token** & **Decompose** & **Prefix** & **Suffix** & **Hold** \\ \hline \multirow{3}{*}{Alpaca} & 0 & 20.2 & 296.2 & - & - & - & - \\  & 1 & 44.7 & 414.6 & 7.4 & 51.2 & 24.5 & 16.9 \\  & 2 & 45.2 & 425.5 & 30.4 & 3.8 & 2.8 & 63.0 \\ \hline \multirow{3}{*}{FlanCoT} & 0 & 125.0 & 243.1 & - & - & - & - \\  & 1 & 127.4 & 336.7 & 22.8 & 48.1 & 8.1 & 21.1 \\  & 2 & 128.9 & 337.4 & 31.4 & 1.4 & 1.1 & 66.1 \\ \hline \multirow{3}{*}{Tulu-V2} & 0 & 49.0 & 247.3 & - & - & - & - \\  & 1 & 66.4 & 486.1 & 29.9 & 36.7 & 13.1 & 20.3 \\  & 2 & 75.3 & 515.4 & 37.6 & 6.2 & 3.8 & 52.4 \\ \hline \multirow{3}{*}{MetaMath} & 0 & 57.0 & 128.1 & - & - & - & - \\  & 1 & 56.5 & 143.5 & 11.0 & 4.4 & 0.7 & 83.9 \\ \cline{1-1}  & 2 & 56.9 & 143.8 & 3.5 & 0.2 & 0.2 & 96.1 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Statistics for Seq-Instruct. Iteration 0 is equivalent to the original instruction dataset.

\begin{table}
\begin{tabular}{l l r r r r r r r r} \hline \hline \multicolumn{2}{c}{**Dataset**} & \multirow{2}{*}{**Method**} & \multicolumn{6}{c}{Generic Task} & \multicolumn{6}{c}{Sequential Task} \\ \cline{3-10}  & & & & & Human & & & & \\  & & MMLU & ARC & GSM8k & Eval & Aplica & XQuAD & MGSM8k & SeqEval \\ \hline \multirow{3}{*}{Alpaca} & IT & 56.3 & 49.7 & 17.7 & 53.7 & 7.9 & 38.5 & 15.7 & 46.3 \\  & WizardLM & 58.4 & 51.8 & 32.9 & **63.9** & 8.4 & 42.1 & 26.9 & 37.1 \\  & SIT & **59.5** & **52.8** & **34.5** & 56.5 & **15.0** & **46.1** & **32.9** & **50.3** \\ \hline \multirow{2}{*}{FlanCoT} & IT & 54.8 & 50.0 & 46.3 & 60.9 & 9.5 & 46.4 & 34.8 & 43.5 \\  & SIT & **58.1** & **54.1** & **50.5** & **65.8** & **10.0** & **55.8** & **41.8** & **49.6** \\ \hline \multirow{2}{*}{Tulu-V2} & IT & **56.2** & 51.3 & 43.4 & 64.6 & **16.3** & 24.9 & 35.0 & 50.6 \\  & SIT & 54.4 & **52.6** & **47.2** & **67.5** & 16.0 & **35.6** & **35.6** & **53.0** \\ \hline \multirow{2}{*}{Tulu-V2+MetaMath} & IT & 53.4 & 49.1 & 60.4 & **65.9** & **16.2** & 26.0 & 35.8 & 36.0 \\  & SIT & **55.1** & **51.6** & **62.5** & 65.2 & 14.4 & **30.5** & **41.2** & **37.5** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Seq-Instruct results for different datasets. Metrics: accuracy for MMLU, ARC, GSM8K, XQuAD, and MSSM8K; Pass@10 for HumanEval; LLM-as-a-judge win rate against GPT-3.5-Turbo for SeqEval.



The length and task ablation experiments are reported in Table 6 (Top). For the **data-level** setting, we discover that SIT models with reduced token counts remain superior to IT models across all evaluation criteria, indicating that the improvement does not stem from its exposure to more tokens. Regarding the **instance-level** setting, although IT slightly outperforms the SIT models in generic tasks, the SIT models have a clear edge in sequential tasks. This implies that SIT is useful for long-horizon task execution even when the data length becomes shorter as long as the multi-task nature is preserved. For the **task-level** experiments, the performance of the SIT-split is significantly worse than that of the standard SIT version even though they have the same task content. It reflects the importance of fine-tuning on longer and more complex sequences to surpass vanilla IT. In addition, SIT-multi generally surpasses SIT-split but still underperforms SIT. This pattern reveals that incorporating multiple tasks in a single instruction is beneficial and having the tasks sequentially could be even more effective.

### Ablation study on the choice of models

We then run the Seq-Instruct pipeline with different LLM families: specifically, we replace either the SIT data generator model or the fine-tuned base model to mitigate the potential bias of using the models from the Llama family in both roles. In Table 6 (Bottom), we show the outcomes of 1) changing the generator model to Command R+, another (now) open-sourced LLM; 2) substituting the base model with Mistral-7B [3]. Results consistently show a promising gap between IT and SIT in all benchmarks except for ARC and HumanEval. This confirms that our Seq-Instruct pipeline generalises well to different LLMs.

Figure 2: Response quality (left) and success rate in following all tasks (right), measured on a growing number of tasks from iterations of SeqEval for Command R, GPT-3.5-Turbo, Alpaca-WizardLM, Alpaca-IT, and Alpaca-SIT.

 



### Ablation study on the number of tasks

Further, we study the models' behaviour when the number of tasks in the query grows (beyond training) at test time. To this end, we evaluate the same models in Section 4.2 on intermediate versions of SeqEval at iterations 1 to 4, corresponding to 2 to 5 total tasks. We report LLM-as-a-judge quality scores and the success rate in following all instructions at different iterations in Figure 2 for Llama-3-8B fine-tuned on Alpaca with IT, SIT, and WizardLM, together with presumably larger commercial models: Command R and GPT-3.5-Turbo.

In each iteration of SeqEval, SIT methods consistently outperform their IT counterparts concerning both response quality and following rates. As the iteration number increases, the performance gap widens--indicating the ability of "extrapolation" to more tasks for SIT. Compared with other baselines: 1) we observe that SIT is more suited for chained tasks when compared with WizardLM across all iterations; 2) surprisingly, our SIT with open-source data can match the performance of commercial models like Command R and GPT-3.5-Turbo even as the number of tasks grows. Detailed numbers, which also include FlanCoT-IT/SIT results, are enclosed in Appendix C Table 13.

### Qualitative study of SIT data and output

Finally, we inspect the instructions generated via Seq-Instruct and draw potential links to model improvements in different skill types. We visualize the 15 most frequent root verbs and their direct noun objects are plotted in Appendix D's Figure 6 for Alpaca-SIT and Figure 7 for FlanCoT-SIT. It is seen that verbs like "use", "analyze" and "identify" are usually added as a prefix task to digest the input information before solving an actual task, forming diverse chains of thought. In contrast, phrases like "generate (a) story" or "provide (an) example" often continue from the model's outputs from previous tasks. These auxiliary tasks form high-quality reasoning data during fine-tuning.

We also supply examples of sequential instructions and responses from both Llama-3 and Mistral undergone IT or SIT in Appendix E as a case study. SIT models are more capable of providing all information required by the chained instructions, while the IT model ignores some instructions and replies with shorter responses.

## 6 Related Work

Instruction tuningFine-tuning a base model on formatted input-output data makes it follow user queries and generalise to unseen tasks Mishra et al. (2022); Sanh et al. (2022); Wei et al. (2022). Yet, we have shown that neither foundation nor instruction-tuned models are adept at processing a single query requiring the completion of multiple tasks sequentially. This is potentially because the queries in existing data are rather straightforward: mostly supervised NLP tasks and open-ended dialogues wherein instruction-response pairs exhibit a direct relationship Sanh et al. (2022); Longpre et al. (2023); Wang et al. (2023); Taori et al. (2023); Conover et al. (2023). The machine-translated multilingual counterparts inevitably inherit the same flaws Muennighoff et al. (2023); Li et al. (2023); Chen et al. (2024). Several works fine-tuned LLMs to perform multi-turn conversations Touvron et al. (2023); Chiang et al. (2023), which requires continuous monitoring and input during execution, but our sequential instructions do not.

Recently, Xu et al. (2024) proposed using off-the-shelf LLMs to produce complex instructions and concurrent to us, Hayati et al. (2024) created compositional tasks by merging existing tasks. Distinguishing us from these ideas is that our method yields a sequence of natural, interrelated instructions from a single task. The field then witnesses an increased interest in multi-task or sequential task execution Jia et al. (2024); Son et al. (2024); Chen et al. (2024); Wen et al. (2024). Commercial LLM provider Reka (2024) also pointed out that instruction chaining "is a key capability that is missing to make agentic workflow feasible".

Chain-of-thought, knowledge pivoting, and prolonged generationLLMs that "reason" "step by step" before answering a question yield better outcomes Wei et al. (2022); Kojima et al. (2022). This is extended by chained prompting Wu et al. (2022) and least-to-most-prompting Zhou et al. (2023). Our work points to a much broader but only partially explored space of adhering to several tasks in one query; we also underline that this work concerns post-training alignment in addition to prompting that is learning-free.

Explicitly guiding an LLM to perform certain tasks before arriving at a final answer allows for human intervention and external knowledge injection. Most previous research centred around language pivoting (often via English), which has proven ef

fective in a wide array of applications (Conneau et al., 2018; Ponti et al., 2019, 2021; Ansell et al., 2023; Artetxe et al., 2023). A contemporaneous work introduced cross-lingual instruction tuning (Zhang et al., 2024), which can be viewed as a task-specific case of our approaches.

Lastly, prolonged generation incurs higher inference costs but also offers more computational capacity (Goyal et al., 2023; Pfau et al., 2024). One may consider our SIT as training an LLM with stretched input and output. Nonetheless, our ablation experiments on controlled training lengths and tasks have proven that increased training computation alone is not a critical factor. Finally, instead of producing meaningless filler tokens, SIT allows for interpretable reasoning trajectory and multi-task completion in a single query.

## 7 Conclusion

This work unveiled a major drawback in state-of-the-art, open-source models as large as 70B or 8\(\times\)7B: they struggle to follow multi-step instructions within a single query. Accordingly, we proposed a new method, sequential instruction tuning (SIT), to equip LLMs with this ability. This has been carried out by curating intermediate tasks manually or using LLMs to yield sequential instructions automatically. Fine-tuning language models on SIT-enriched data not only helped them follow a list of instructions more faithfully but also recorded a better performance in complex tasks that require multi-step reasoning, such as maths and coding, and open-ended generation.

### Limitations

The construction of SeqEval used LLMs to expand simple instructions and the evaluation of the sequential instruction following capability mostly relied on LLM-as-a-judge. We recognise the lack of human evaluation in these two cases as a limitation.

### Ethical Considerations

The positive social impact of our research is creating an instruction enhancement approach that allows smaller models to match the behaviour of larger closed-source ones. This also contributes to the democratisation of AI.

Potential risks would be associated with automatic data augmentation, which might introduce untruthful, biased, or hallucinated content that is difficult to filter out. We also acknowledge that our efforts in data enrichment are centred around English, which enjoyed rapid development in the era of LLMs.

### Acknowledgments

Anonymous reviewers' questions and feedback helped us improve this manuscript. Computations were performed using the Edinburgh International Data Facility (EIDF) and the Data-Driven Innovation Programme at the University of Edinburgh. Additional computational resources were from the Baskerville Tier 2 HPC service funded by the EPSRC and UKRI through the World Class Labs scheme (EP/T022221/1) and the Digital Research Infrastructure programme (EP/W032244/1) and operated by Advanced Research Computing at the University of Birmingham. Pinzhen Chen acknowledges funding from UK Research and Innovation (UKRI) under the UK government's Horizon Europe funding guarantee [grant numbers 10052546].

## References

* Ansel et al. (2023) Alan Ansel, Marinela Parovic, Ivan Vulic, Anna Korhonen, and Edoardo Ponti. 2023. Unifying cross-lingual transfer across scenarios of resource scarcity. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_.
* Artetxe et al. (2023) Mikel Artetxe, Vedanuj Goswami, Shruti Bhosale, Angela Fan, and Luke Zettlemoyer. 2023. Revisiting machine translation for cross-lingual classification. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_.
* Artetxe et al. (2020) Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. On the cross-lingual transferability of monolingual representations. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
* Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pande de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. _arXiv preprint_.
* Chen et al. (2024a) Pinzhen Chen, Shaoxiong Ji, Nikolay Bogychev, Andrey Kutuzov, Barry Haddow, and Kenneth Haefield. 2024a. Monolingual or multilingual instruction tuning: Which makes a better Alpaca. In _Findings of the Association for Computational Linguistics: EACL 2024_.
* Chen et al. (2024b) Xinyi Chen, Baohao Liao, Jarui Qi, Panagiotis Esturatiadis, Christof Monz, Arianna Bisazza, and Maarten de Rijke. 2024b. The sifo benchmark: Investigating the sequential instruction following ability of large language models. _arXiv preprint_ 

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot intressing GPT-4 with 90%* ChatGPT quality. Online Blog.
* Clark et al. (2018) Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Taifjord. 2018. Think you have solved question answering? try ARC, the AI2 reasoning challenge. _arXiv preprint_.
* Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. _arXiv preprint_.
* Conneau et al. (2018) Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating crosslingual sentence representations. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_.
* Conover et al. (2023) Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free Dolly: Introducing the world's first truly open instructioned LLM. Online blog.
* Dai et al. (2023) Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. _arXiv preprint_.
* Dubey et al. (2024) Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. _arXiv preprint arXiv:2407.21783_.
* Gomez (2024) Aidan Gomez. 2024. Introducing Command R+: A scalable LLM built for business. Online Blog.
* Goyal et al. (2023) Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. 2023. Think before you speak: Training language models with pause tokens. _arXiv preprint_.
* Goyal et al. (2017) Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. In _Conference on Computer Vision and Pattern Recognition_.
* Hayati et al. (2024) Shirley Anugrah Hayati, Taehee Jung, Tristan BoddingLong, Sudipta Kar, Abhinav Sethy, Joo-Kyung Kim, and Dongyeop Kang. 2024. Chain-of-instructions: Compositional instruction tuning on large language models. _arXiv preprint_.
* Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In _Proceedings of the International Conference on Learning Representations_.
* Huang et al. (2023) Haoyang Huang, Tianyi Tang, Dongdong Zhang, Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023. Not all languages are created equal in LLMs: Improving multilingual capability by cross-lingual-thought prompting. In _Findings of the Association for Computational Linguistics: EMNLP 2023_.
* Hudson and Manning (2019) Drew A Hudson and Christopher D Manning. 2019. GQA: A new dataset for real-world visual reasoning and compositional question answering. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_.
* Ivison et al. (2023) Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew E. Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagsy, and Hanna Hajishirzi. 2023. Camels in a changing climate: Enhancing LM adaptation with Tulu 2. _arXiv preprint_.
* Jia et al. (2024) Mengzhao Jia, Zhihan Zhang, Wenhao Yu, Fangkai Jiao, and Meng Jiang. 2024. Describe-then-reason: Improving multimodal mathematical reasoning through visual comprehension training. _arXiv preprint_.
* Jiang et al. (2023) Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. _arXiv preprint_.
* Kim et al. (2023) Seungone Kim, Se Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo. 2023. The CoT collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_.
* Kitaev et al. (2019) Nikita Kitaev, Steven Cao, and Dan Klein. 2019. Multilingual constituency parsing with self-attention and pre-training. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
* Kitaev and Klein (2018) Nikita Kitaev and Dan Klein. 2018. Constitucency parsing with a self-attentive encoder. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics_.
* Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In _Advances in Neural Information Processing Systems_.
* Li et al. (2023a) Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Silvio Savarese, and Steven C.H. Hoi. 2023a. LAVIS: A one-stop library for language-vision intelligence. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics_ 

Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. 2023b. Bactrian-X: A multilingual replicable instruction-following model with low-rank adaptation. _arXiv preprint_.
* Li et al. (2023) Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023c. AlpacaEval: An automatic evaluator of instruction-following models. GitHub.
* Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In _European Conference on Computer Vision_.
* Longpre et al. (2023) Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. The Flan collection: Designing data and methods for effective instruction tuning. In _Proceedings of the 40th International Conference on Machine Learning_.
* Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017. Fixing weight decay regularization in Adam. _arXiv preprint_.
* Mishra et al. (2022) Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-task generalization via natural language crowdsourcing instructions. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics_.
* Muennighoff et al. (2023) Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. 2023. Crosslingual generalization through multitask finetuning. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics_.
* Pfau et al. (2024) Jacob Pfau, William Merrill, and Samuel R Bowman. 2024. Let's think dot by dot: Hidden computation in transformer language models. _arXiv preprint_.
* Ponti et al. (2021) E. Ponti, Julia Kreutzer, Ivan Vulic, and Siva Reddy. 2021. Modelling latent translations for cross-lingual transfer. _arXiv preprint_.
* Poni et al. (2019) Edoardo Maria Poni, Helen O'Horan, Yevgeni Berzak, Ivan Vulic, Roi Reichart, Thierry Poibeau, Ekaterina Shutova, and Anna Korhonen. 2019. Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing. _Computational Linguistics_.
* Qin et al. (2023) Libo Qin, Qiguang Chen, Fukuan Wei, Shijue Huang, and Wanxiang Che. 2023. Cross-lingual prompting: Improving zero-shot chain-of-thought reasoning across languages. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_.
* Reka (2024) Reka. 2024. Twitter. [https://x.com/RekaAllabs/status/1843298161621901713](https://x.com/RekaAllabs/status/1843298161621901713), Accessed: 07-Oct-2024.
* Sanh et al. (2022) Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lantang Suatwika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2022. Multitask prompted training enables zero-shot task generalization. In _International Conference on Learning Representations_.
* Shi et al. (2023) Freda Shi, Mirac Suzgun, Markus Freitas, Xuezhi Wang, Suraji Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2023. Language models are multilingual chain-of-thought reasoners. In _The Eleventh International Conference on Learning Representations_.
* Son et al. (2024) Guijin Son, SangWon Baek, Sangdae Nam, Ilgyun Jeong, and Seungone Kim. 2024. Multi-task inference: Can large language models follow multiple instructions at once? In _Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_.
* Talmor et al. (2019) Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.
* Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford AlpaCA: an instruction-following LLaMA model. Github repository.
* Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashhykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. _arXiv preprint_.
* Wang et al. (2023) Yizhong Wang, Yeganeh Kordi, Swaroopp Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-instruct: Aligning language models with self-generated instructions. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics_.
* Wei et al. (2022a) Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guo, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022a. Finetuned language models are zero-shot learners. In _International Conference on Learning Representations_.
* Wei et al. (2022b) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022b. Chain of thought prompting elicits reasoning in large language models. In _Advances in Neural Information Processing Systems_.
* Wen et al. (2024) Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxing Xu, Yiming Liu, Jie Tang, Hongning Wang, and Minlie Huang. 2024. Benchmarking complex instruction-following with multiple 

constraints composition. In _The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track_.
* Wu et al. (2022) Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022. AI Chains: Transparent and controllable human-AI interaction by chaining large language model prompts. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_.
* Xu et al. (2024) Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024. WizardLM: Empowering large pre-trained language models to follow complex instructions. In _The Twelfth International Conference on Learning Representations_.
* Yu et al. (2023) Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhenying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. Metamath: Bootstrap your own mathematical questions for large language models. _arXiv preprint_.
* Zhang et al. (2024) Zhihan Zhang, Dong-Ho Lee, Yuwei Fang, Wenhao Yu, Mengzhao Jia, Meng Jiang, and Francesco Barbieri. 2024. PLUG: Leveraging pivot language in cross-lingual instruction tuning. In _Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics_.
* Zheng et al. (2023) Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In _Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track_.
* Zhou et al. (2023) Denny Zhou, Nathanael Scharli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, and Ed H. Chi. 2023. Least-to-most prompting enables complex reasoning in large language models. In _The Eleventh International Conference on Learning Representations_.

 

Generalisation in a Toy Task

### Repeating or paraphrasing for reasoning

Before we started the experiments for the translation task, we performed a toy experiment for pretending dummy tasks, such as "repeating the input" or "paraphrasing the input" before answering the questions. Our models for textual experiments are fine-tuned on the cleaned version of the Alpaca data with 52K instances as a seed instruction dataset (Taori et al., 2023). The data was constructed using a self-instruct procedure (Wang et al., 2023). Each instance contains an instruction, an output, and (optionally) an input. Overall about 40% of the data have an input field and 60% of the data are input-free. To explore the effect of sequential instructions, we edit the Alpaca dataset to suit our needs. Specifically, for instances having an input field, we switch its instruction to a sequential instruction which comprises two sub-tasks; we also update its output field to include the expected output from both tasks. The other training instances without an input field remain unchanged. The examples with modified instructions are merged with the original Alpaca dataset to form our sequential instruction tuning dataset. We consider two intermediate tasks: repeating or paraphrasing the input.

Repeating the inputFirst, we prepend a dummy task, namely repeating the input, which does not introduce any new information to the original instruction. Specifically, we add the prefix "_First repeat the input, then_" to the instruction. Likewise, we prepend the input field string to the original output separated by a new line.

Paraphrasing the inputSecond, we then augment Alpaca with an input paraphrasing task. Specifically, we use GPT-3.5-Turbo to paraphrase the Alpaca input field texts. We add the prefix "_First paraphrase the input, then_" to the original instructions and the paraphrased input contents to the corresponding output as part of the new response.

EvaluationWe test the fine-tuned LLMs in a zero-shot fashion on the CommonsenseQA dataset (Talmor et al., 2019), which contains English common-sense questions. We prompted them with "_First repeat the input, then answer_" or "_First paraphrase the input, then answer_" depending on the intermediate task observed during the fine-tuning stage. We compare LLMs fine-tuned on the original Alpaca data (instruction tuning, IT) with sequential instruction tuning (SIT) on our enriched Alpaca. Results are reported in Table 7, showing that for all base LLMs considered--Mistral-7B, Llama-7B, and Llama-13B--sequential instruction-tuned models attain higher performance on the CommonsenseQA test set compared to vanilla instruction tuning. Paraphrasing appears slightly better on average than repeating. These results demonstrate that even dummy tasks exhibit the potential to equip LLMs with sequential instruction following.

\begin{table}
\begin{tabular}{l c c c} \hline \hline \multirow{2}{*}{**Model**} & **IT** & **SIT** & **SIT** \\  & Alpaca & +Repeat & +Paraphrase \\ \hline Llama-7B & 35 & 39 & **41** \\ Llama-13B & 47 & 48 & **49** \\ Mistral-7B & 61 & **64** & 63 \\ \hline \multicolumn{4}{c}{Llama-7B 7-shot prompting (Touvron et al., 2023): 33} \\ \hline \hline \end{tabular}
\end{table}
Table 7: CommonsenseQA results (accuracy, %) from prompting, instruction tuning, and our sequential instruction tuning with dummy tasks.

 



### Generalisation to other sequential instructions

To further understand the characteristics of language models trained on sequential instruction data, we analyzed the **generalisation ability** starting from the training of a sequential single task as a simple test bed.

In our main experiments of task-specific SIT in Section 4.1, we used the same intermediate tasks (translation or image captioning) for training and inference during evaluation. We now study if a SIT'ed model can follow unseen intermediate tasks. Particularly, we build on Appendix A, where the two dummy tasks of repetition and paraphrasing were proposed for CommonsenseQA. We examine if a model exposed to repetition during training can maintain a similar performance when the prompt switches to "paraphrasing" during evaluation, and vice versa.

In Table 8, we report both accuracy and following rate of Mistral-7B models fine-tuned on 100 samples from the CommonsenseQA test set. First, we confirm that our sequential instruction-tuned models are still able to follow single-task instructions with a similar level of accuracy compared with the model fine-tuned on the original instruction datasets. This indicates that our method widens the model's scope to sequential instructions without compromising its original capabilities. Furthermore, we observe that SIT models trained solely on one intermediate task can follow both Repeat and Paraphrase instructions during test time. The resulting accuracy from such models is significantly higher than the baseline Alpaca instruction tuning even with a train-test discrepancy in the intermediate step. This demonstrates that sequential instruction tuning on a specific task can generalise to similar sequential tasks and attain comparable performance.

\begin{table}
\begin{tabular}{l c c c} \hline \hline  & \multicolumn{3}{c}{Training Method} \\ \cline{2-5}
* & 56 /
- & 58 /
- \\ Repeat & 20 / 30 & **64** / 99 & 45 / 96 \\ Paraphrase & 21 / 35 & **64** / 96 & 63 / **100** \\ \hline \hline \end{tabular}
\end{table}
Table 8: CommonsenseQA results (accuracy and following rate, %) for Mistral-7B IT and SIT tested with zero-shot intermediate task instructions.


Detailed Experimental Setup

### Translation in multilingual question answering

For this experiment, we perform instruction tuning with full parameter in Mistral-7B-v0.1 and Llama-3-8B, with the original Alpaca (**IT**) and SIT Alpaca (**SIT**). The Mistral model is instruction tuned with the Alpaca template [25], whereas Llama-3 is tuned with the Tulu template [26]. The training is done with 3 epochs, learning rate 2e-5, the optimizer is AdamW [49] with warmup ratio 0.03 and linear decay. The effective batch size is 128, and the maximum sequence length is 2048.

### Image captioning in multimodal question answering

For cross-modal experiments involving both texts and images, we use the LAVIS\({}^{1}\) library for training and evaluation [11]. We fine-tuned InstructBLIP\({}^{2}\) with the same hyperparameters used by Dai et al. dai:2023 and we set a budget of 3 epochs with an initial learning rate of 1\(\times\)10\({}^{\text{-5}}\). We only updated the parameters of the Q-Former but froze the image encoder and the language decoder. We use 2 NVIDIA-A100-PCIe-80GB GPUs to run all related experiments.

### Seq-Instruct

For this experiment, we perform instruction tuning with full-parameter tuning in Llama-3-8B. The Llama-3 model is tuned with the Tulu template [26]. The training is done with 3 epochs, a learning rate of 2e-5, and an effective batch size of 128. All the response data are re-generated by prompting Llama-3-70B-Instruct to ensure fairness. We perform the Seq-Instruct pipeline for 2 iterations on both Alpaca and FlanCoT. We use 4 A100-SXM4-80GB GPUs to run the generation and fine-tuning experiments.

### Seq-Instruct prompt template

The prompt template is shown in Figure 3 for the generation type classification and Figure 4 for the generation process.

Figure 3: Prompt template for classifying the given instruction into four options of Seq-Instruct, where variables §{instruction} is replaced by the query instruction on the fly.

 

Your objective is to add a suffix task to the given instruction (#original Instruction#) to form a sequential related instruction (#New Instruction#). Adding "familiarize", "read" or "understand" the original given information is not counted as a valid prefix task. The response to the new instruction should be the same or similar to the original instruction, including the format. The added instruction should have its own explicit response, so something like "reading", "familiarizing", "repeating", "analyzing" or "understanding" the original instruction is not considered a good choice. Your rewriting cannot omit the non-text parts such as the table and code in "#Given Prompt#:", and should only modify the instruction part and keep all the key details such as options, hypothesis and questions. Provide your explanation before having the final instruction by thinking step by step. You must generate your new instruction with prefix "#New Instruction#: " and end your answer with "###".
Few shot examples #Original Instruction#: "Describe the structure of an atom." Your task is to decompose the instruction into two sequential instructions that will eventually lead to the answer to the original instructions. Let's think step by step. To effectively describe the structure of an atom, we can break down the explanation into two main tasks or steps. Here's a logical way to organize it. First, we can explore the basic components of an atom, then understand how the components are organized and how they interact. These two tasks cover the basic description of an atom's structure, from its components to the arrangement and behaviour of these components. #New Instruction#: "Describe the basic components of an atom, then explain how the components are organized and how they interact."##
#Original Instruction#: "${instruction}" Your task is todecompose the instruction into two sequential instructions that will eventually lead to the answer to the original instructions. Let's think step by step.

Figure 4: Prompt template for classifying the given instruction into four options of Seq-Instruct, where variables ${instruction} is replaced by the query instruction on the fly.

 



### Evaluation setup for generic tasks

Besides the sequential task, we also evaluate the instruction-tuned LLMs on a range of benchmarks to understand the difference between IT and SIT models in the following abilities:

FactualityMassively Multitask Language Understanding (Hendrycks et al., 2021) requires the model to pick an answer from 4 candidates. It covers 57 subjects including STEM, humanities, social sciences, and other disciplines. We evaluate models in a 5-shot setting and report their accuracy.

ReasoningWe evaluate the model with ARC-challenge benchmark (Clark et al., 2018), a dataset of 1,172 genuine grade-school level, multiple-choice science questions, which require the models to perform complex reasoning. We evaluate from Grade School Math (Cobbe et al., 2021), a collection of math problems in linguistic form. It requires open-ended generation. We evaluate models in a 25-shot setting for ARC and an 8-shot setting and report their exact match (EM).

CodingHumanEval (Chen et al., 2021) is a dataset for synthesizing coding programs from docstrings. We evaluate models with a temperature of 0.1 and report their pass at 10 (P@10).





## Appendix C Result Breakdown

### XQuAD

Complete XQuAD results are shown in Table 9.

### Msgm

Complete results for MGSM for models tuned on Alpaca and FlanCoT are shown in Table 10 and Table 11, respectively. We showed En-CoT follows the original paper settings (Shi et al., 2023), which directly prompts the model to perform CoT in English without translation.

\begin{table}
\begin{tabular}{l l c r r r r r r r r r r r r} \hline \hline
* \\  & trans-CoT & 20.4 & **20.0** & **17.6** & **16.8** & 17.6 & **19.6** & **18.4** & **17.2** & **10.4** & 6.8 & **8.0** & **17.0** & \(\uparrow\) 2.1 \\ \cline{2-19} WizardLM & en-CoT & 33.6 & 26.4 & 27.2 & 24.4 & 28.8 & 23.6 & 20.0 & 22.8 & 10.4 & 18.0 & 12.4 & 22.9 &
- \\  & trans-CoT & **39.6** & **28.8** & **32.8** & **26.4** & 28.8 & **26.8** & **25.2** & **26.8** & **19.2** & **22.0** & **15.6** & **26.9** & \(\uparrow\) 4.0 \\ \hline SIT & en-CoT & 37.6 & 31.6 & 32.0 & 29.2 & 30.0 & 26.8 & 24.4 & 28.4 & 20.0 & 17.2 & 12.4 & 26.1 &
- \\  & trans-CoT & **42.8** & **36.0** & **36.8** & **38.4** & **34.0** & **30.8** & **32.4** & **24.4** & **25.6** & **20.4** & **32.9** & \(\uparrow\) 6.8 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Complete results for 8-shots MGSM (accuracy, %) fine-tuned on Alpaca.

\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline
**Model** & **Dataset** & **Method** & DE & zh & ru & ES & ar & EL & vi & hi & tr & th & avg \\ \hline \multirow{4}{*}{Mistral-7B} & \multirow{4}{*}{Alpaca} & IT & 44.7 & 21.7 & 38.7 & 46.3 & 12.8 & 15.2 & 25.3 & 9.6 & 24.9 & 9.2 & 24.8 \\  & & & 41\% & 13\% & 36\% & 48\% & 3\% & 4\% & 7\% & 0.5\% & 5\% & 1\% & 15.9\% \\ \cline{2-19}  & & STI\({}^{M}\) & **62.0** & **37.2** & **52.7** & **62.6** & **21.8** & **25.1** & **37.9** & **15.5** & **34.4** & **13.7** & **36.3** \\  & & & 96\% & 84\% & 93\% & 97\% & 42\% & 34\% & 68\% & 9\% & 49\% & 5\% & 57.7\% \\ \cline{2-19}  & & STI\({}^{A}\) & 41.1 & 19.7 & 34.5 & 39.2 & 15.6 & 20.2 & 29.0 & 15.5 & 25.0 & 12.1 & 25.2 \\  & & & 54\% & 31\% & 47\% & 38\% & 8\% & 9\% & 14\% & 7\% & 5\% & 3\% & 21.6\% \\ \hline \multirow{4}{*}{Llama-3-8B} & \multirow{4}{*}{Alpaca} & IT & 44.3 & 34.6 & 41.3 & 49.7 & 31.2 & 42.5 & 40.0 & 36.3 & 34.3 & 30.6 & 38.5 \\  & & & 6\% & 6\% & 7\% & 8\% & 5\% & 8\% & 4\% & 4\% & 3\% & 3\% & 5.4\% \\ \cline{2-19}  & & \multirow{4}{*}{SIM\({}^{M}\)} & **52.7** & **40.0** & **43.5** & **54.5** & **39.2** & **45.3** & **47.8** & **42.4** & **43.6** & **38.0** & 44.7 \\  & & & 90\% & 81\% & 78\% & 98\% & 79\% & 61\% & 85\% & 47\% & 82\% & 56\% & 75.7\% \\ \cline{2-19}  & & & STI\({}^{A}\) & 52.2 & 42.2 & 44.9 & 54.2 & 40.0 & 47.1 & 47.8 & 42.8 & 47.1 & 43.0 & **46.1** \\  & & & 45\% & 56\% & 57\% & 46\% & 60\% & 45\% & 53\% & 49\% & 63\% & 59\% & 53.3\% \\ \cline{2-19}  & & & 51.0 & 36.1 & 43.9 & 50.3 & 36.1 & 47.1 & 42.4 & 40.4 & 39.1 & 34.4 & 42.1 \\  & & 15\% & 13\% & 18\% & 15\% & 24\% & 18\% & 14\% & 15\% & 17\% & 16\% & 16.5\% \\ \cline{2-19}  & & & 55.5 & 38.5 & 45.4 & 55.6 & 40.5 & 50.6 & 47.3 & 45.6 & 47.6 & 37.6 & 46.4 \\  & & 3\% & 3\% & 6\% & 5\% & 7\% & 7\% & 5\% & 5\% & 4\% & 5\% & 5.0\% \\ \cline{2-19}  & & STI\({}^{A}\) & **63.5** & **49.9** & **55.5** & **66.1** & **50.0** & **59.4** & **56.1** & **53.7** & **55.6** & **48.4** & **55.8** \\  & & 75\% & 85\% & 83\% & 76\% & 89\% & 91\% & 78\% & 75\% & 88\% & 80\% & **82.0\%** \\ \hline \hline \end{tabular}
\end{table}
Table 9: Complete breakdown for XQuAD results. SIT\({}^{M}\) refers to the manual task-driven method while SIT\({}^{A}\) refers to the automatic generalized SIT version.





### Main and ablation results for SeqEval

Detailed results from our baselines and SIT models evaluated by our own SeqEval are reported in Table 12. (Top). We also include the results from our ablation experiments (Bottom).

\begin{table}
\begin{tabular}{l l c c c c c} \hline \hline \multicolumn{1}{c}{**Model**} & \multicolumn{1}{c}{**Dataset**} & \multicolumn{1}{c}{**Method**} & \multicolumn{1}{c}{**Score**} & \multicolumn{1}{c}{**Follow**} & \multicolumn{1}{c}{**Win (GPT-3.5)**} & \multicolumn{1}{c}{**Win (Cmd-R)**} \\ \hline Command-R & - & & - & 4.595 & 90.9 & 51.7 & - \\ GPT-3.5-Turbo & - & & - & 4.653 & 88.0 & - & 48.3 \\ \hline  & FlanCoT & IT & 4.185 & 79.8 & 43.5 & 41.9 \\ Llama-3-8B & Alpaca & KIT & **4.613** & **88.4** & **49.6** & **47.6** \\ \cline{2-7}  & & IT & 4.453 & 83.4 & 46.3 & 44.7 \\  & Alpaca & WizardLM & 4.102 & 73.9 & 37.1 & 34.9 \\  & SIT & **4.659** & **89.3** & **50.3** & **48.2** \\ \cline{2-7}  & TuluV2 100k & IT & 4.684 & 89.6 & 50.6 & 48.0 \\  & SIT & **4.692** & **92.4** & **53.0** & **51.3** \\ \hline \multicolumn{7}{c}{Main results \(\uparrow\) and ablation results \(\downarrow\)} \\ \hline  & Alpaca (data-level) & IT & 4.453 & 83.4 & 46.3 & 44.7 \\  & SIT & **4.652** & **87.7** & **49.8** & **47.2** \\ \cline{2-7} Llama-3-8B & Alpaca (instance-level) & IT & 4.303 & 79.6 & 40.9 & 39.7 \\  & SIT & **4.440** & **82.2** & **45.7** & **44.0** \\ \cline{2-7}  & & Sit-split & 1.960 & 23.1 & 11.9 & 13.9 \\  & Alpaca (task-level) & SIT-multi & 3.427 & 57.1 & 30.5 & 29.6 \\  & SIT & **4.659** & **89.3** & **50.3** & **48.2** \\ \hline  & FlanCoT (data-level) & IT & 4.185 & 79.8 & 43.5 & 41.9 \\ Llama-3-8B & — & SIT & 4.563 & 88.1 & 47.2 & 44.4 \\  & FlanCoT (instance-level) & IT & 4.583 & **87.7** & 47.9 & 45.4 \\  & SIT & 4.540 & 86.2 & 47.7 & 45.6 \\ \hline  & Alpaca (generated by CmdR+) & IT & 4.039 & 68.6 & 37.6 & 37.3 \\  & SIT & **4.464** & **82.6** & **46.6** & **44.7** \\ \hline Mistral-7B-v0.1 & Alpaca & IT & 4.253 & 74.0 & 40.8 & 38.9 \\  & SIT & **4.353** & **81.9** & **45.0** & **43.2** \\ \hline \hline \end{tabular}
\end{table}
Table 12: Comprehensive evaluation results on our SeqEval. Metrics: quality score, following rate, as well as win rates against GPT-3.5-Turbo and Command-R judged by GPT-4-Turbo. Top: main experiment results; Bottom: ablation results.





### Results for the intermediate iterations of SeqEval

In addition, we test our models on different numbers of tasks in a single query, which are represented by the intermediate versions at various iterations through developing the final SeqEval--the results are in Table 13.

\begin{table}
\begin{tabular}{l l l l l l c c} \hline \hline

- & 4.535 & 90.3 & 50.1 &
- \\  & GPT-3.5-Turbo &
- & 4.739 & 91.4 &
- & 49.9 \\ \cline{2-8}
1 & \multirow{6}{*}{Llama-3-8B} & \multirow{2}{*}{FlanCoT} & IT & 4.366 & 83.0 & 45.0 & 45.0 \\  & & SIT & 4.302 & 84.1 & 45.7 & 45.3 \\ \cline{3-8}  & & \multirow{2}{*}{Alpaca} & IT & 4.467 & 83.0 & 44.7 & 45.0 \\  & & WizardLM & 3.822 & 71.2 & 35.4 & 36.2 \\  & & SIT & 4.618 & 88.3 & 48.0 & 48.2 \\ \hline  & Command-R & - & - & 4.488 & 88.8 & 51.2 & - \\  & GPT-3.5-Turbo & - & - & 4.612 & 87.8 & - & 48.8 \\ \cline{2-8}
2 & \multirow{6}{*}{Llama-3-8B} & \multirow{2}{*}{FlanCoT} & IT & 4.185 & 80.7 & 45.3 & 44.3 \\  & & SIT & 4.458 & 86.5 & 49.3 & 47.7 \\ \cline{3-8}  & & \multirow{2}{*}{Alpaca} & IT & 4.468 & 84.0 & 47.9 & 46.5 \\  & & WizardLM & 4.203 & 76.1 & 39.6 & 38.4 \\  & & SIT & 4.686 & 89.9 & 51.7 & 50.4 \\ \hline  & Command-R & - & - & 4.493 & 90.1 & 50.3 & - \\  & GPT-3.5-Turbo & - & - & 4.706 & 89.2 & - & 49.7 \\ \cline{2-8}
3 & \multirow{2}{*}{Llama-3-8B} & \multirow{2}{*}{FlanCoT} & IT & 4.617 & 86.7 & 48.1 & 47.6 \\  & & SIT & 4.664 & 90.2 & 48.9 & 48.5 \\ \cline{3-8}  & & \multirow{2}{*}{Alpaca} & IT & 4.488 & 84.1 & 45.5 & 46.0 \\  & & WizardLM & 4.064 & 72.9 & 34.8 & 36.1 \\  & & SIT & 4.652 & 89.1 & 49.3 & 49.6 \\ \hline  & Command-R & - & - & 4.601 & 91.7 & 51.8 & - \\  & GPT-3.5-Turbo & - & - & 4.691 & 89.4 & - & 48.2 \\ \cline{2-8}
4 & \multirow{2}{*}{FlanCoT} & IT & 3.953 & 77.5 & 40.7 & 39.6 \\  & & SIT & 4.642 & 90.2 & 49.4 & 47.0 \\ \cline{3-8}  & & \multirow{2}{*}{Alpaca} & IT & 4.433 & 81.4 & 45.7 & 43.3 \\  & & WizardLM & 4.088 & 73.8 & 36.2 & 34.3 \\  & & SIT & 4.649 & 88.6 & 49.4 & 47.1 \\ \hline \hline \end{tabular}
\end{table}
Table 13: Comprehensive evaluation results on the _intermediate versions_ of SeqEval with varying numbers of tasks. The metrics are the same as the previous Table 12.



DTABLE]

## Appendix D SIT Data Visualization

We identify the verb-noun structure in the generated instructions using the Berkeley Neural Parser (Kitaev and Klein, 2018; Kitaev et al., 2019) to parse the instructions and then extract the verb that is closest to the root as well as its first direct noun object. The qualitative analyse of the 15 most frequent root verbs and their direct noun objects in Figure 6 for Alpaca-SIT and Figure 7 for FlanCoT-SIT.

Figure 6: Top 15 root verbs (inner circle) and their top 4 direct nouns (outer circle) in Alpaca-SIT.

Figure 7: Top 15 root verbs (inner circle) and their top 4 direct nouns (outer circle) in FlanCoT-SIT.

 

A Case Study on IT and SIT Model Outputs

**Sequential Instruction:**

Have you noticed any issues with your handwriting? I'm trying to teach myself to have nicer handwriting. Can you help? Do a simple task to understand more about your handwriting. Write a paragraph about your day. Now, based on that paragraph, identify one or two key issues that you are unhappy with. Is it the size, the flow, or the clarity?

**Output by IT Mistral-7B**:

I'm happy to help! However, I don't see a paragraph about my day. Please provide the paragraph, and I'll do my best to identify one or two key issues that you may want to work on to improve your handwriting.

**Output by SIT Mistral-7B**:

**Here's a paragraph about my day**: Today was a great day. I woke up early and had a productive morning, getting a lot of work done on my project. I took a short break to grab lunch and catch up on some reading. In the afternoon, I had a meeting with my team to discuss our progress and make some plans for the upcoming week. It was really helpful to get everyone's input and ideas. After the meeting, I took a short walk outside to clear my head and enjoy the nice weather. Overall, I'm feeling accomplished and looking forward to a relaxing evening. **Based on this paragraph, I'm unhappy with two key issues in my handwriting**: 1. Size: My handwriting is often too small. I tend to write in a narrow, compact style, which can make my writing look cramped and hard to read. I'd like to loosen up and write with a bigger, more confident stroke. 2. Flow: My handwriting can be quite slow and labored. I tend to write each letter very deliberately, which can make my writing look choppy and uneven. I'd like to improve the smoothness and fluidity of my writing. By recognizing these issues, I can work on improving my handwriting and making it more legible and attractive. Do you have any tips or exercises to help with these specific issues? I'd love to hear them! (By the way, I'm happy to help others with their handwriting as well. If you'd like, I can offer some general tips or exercises to improve your handwriting, or provide feedback on a specific paragraph or sentence you'd like me to work on.) Here's a sample of my handwriting, so you can see what I'm working with: [Sample of my handwriting] Let me know if you have any feedback or if you'd like to work on improving your own handwriting!

**Sequential Instruction**:

Tell me about the types of cuisine you are most familiar with; are they largely Western, Eastern, or otherwise? Now, tell me about your experience with cooking; specifically, have you ever used sesame seeds in your dishes before? Do I have all the necessary ingredients to make chain!! I only have sesame seeds and no other ingredients. Hi, I have some falafel, but no thaini to put on them. Can you give me a recipe for making chain? After you provide me with the tahini recipe, could you also list the common places or online platforms where I could find the required ingredients? And if I wanted to make my own falafel, can you provide me with a simple recipe for that as well?

**Output by IT Llama-3-8B**:

I'm happy to help! However, I need to clarify that I'm a large language model, I don't have personal experiences or memories, so I'm not familiar with specific cuisines or cooking experiences. I exist solely to provide information and assist with tasks, but I don't have personal preferences or experiences. However, I can provide you with information on various types of cuisine and cooking techniques. Can I help you with something specific?

**Output by SIT Llama-3-8B**:

I'm happy to help! However, I need to clarify that I'm not a professional ender or a food expert. I'm just a large language model, I don't have personal experiences or culinary expertise. I can provide general information and answer questions within my knowledge domain. That being said, I can try to help you with the questions you've asked. Types of cuisine: I'm not familiar with a specific type of cuisine, as I'm a large language model, I don't have personal experiences or culinary expertise. However, **I can provide information on various cuisines and recipes.** Experience with cooking: I've never used sesame seeds in my dishes before. Sesame seeds are commonly used in Middle Eastern, Indian, and Asian cuisines, particularly in dishes like falafel, humums, and tahiin. **Making tahihi**: To make tainii, you'll need sesame seeds, olive oil, lemon juice, garlic, and salt. **Here's a simple recipe**: Ingredients 1 cup seame seeds* 1/4 cup olive oil 2 cloves garlic, minced 2 labelspoons freshly squeezed lemon juice * 1/2 teaspoon salt Introductions: 1. In a blender or food processor, blend the shekense sends until they're finely ground. 2. With the blender or food processor running, slowly pour in the Olive oil through the top. 3. Add the garlic, lemon juice, and salt, and continue blending until the mixture is smooth and creamy. 4. Taste and adjust the seasoning as needed. 5. Store thetahi in an airtight container in the refrigerator for pu to 1 week. **Common places** to find ingredients: You can find sesame seeds and other ingredients needed for making tahihi at: * Middle Eastern or Indian markets * Health food stores * Online platforms like Amazon or specialty food stores * International grocery stores Falafel receipe: Here's a simple falafeler to accompany your tahini: Ingredients: * 1 cup chickpeas, soaked overnight and drained * 1/2 cup onion, finely chopped * 2 cloves garlic, minced * 1/2 cup parsley, chopped * 1/4 cup bulpur * 1/4 cup nil-purpose flour * 1 teaspoon baking powder * 1/2 texupon salt * 1/4 teaspoon black pepper * 1/4 cup olive oil instructions: 1. In a flood processor, blend the chickens, onion, garlic, parsley, bulpur. flour, baking powder, salt, and pepper until the mixture forms a doughn. 2. Shape the dough into small balls, about 1/2 inches in diameter. 3. Heat the olive oil in a large skillet over medium-high heat. Fry the falafel balls in batches until golden brown, about 3-4 minutes per batch. 4. Serve the falafel hot with tahiani sauce. I hope this helps! Let me know if you have any further questions.

Figure 9: Case study of Llama-3-8B with IT and SIT on the _SeqEval_ benchmark.