J son T uning: Towards Generalizable, Robust, and Controllable Instruction Tuning (2025)

Abstract: Instruction tuning is vital for enhancing the performance of large language models (LLMs), but existing text-to-text methods, referred to as TextTuning, struggle with issues such as generalization, robustness, and controllability due to their lack of explicit task structures. We introduce JsonTuning, a structure-to-structure approach that uses JSON structures to represent tasks. This method improves generalization by clarifying task elements and their relations, boosts robustness by minimizing ambiguity, and enhances controllability by allowing precise control over outputs. We conduct an extensive comparative analysis between JsonTuning and TextTuning using various language models and benchmarks. Our findings reveal that JsonTuning consistently surpasses TextTuning in terms of performance, robustness, and controllability across different scenarios. By overcoming the limitations of TextTuning, JsonTuning demonstrates significant potential for developing more effective and reliable LLMs capable of handling diverse scenarios.

## introduction the field of natural language processing has witnessed significant advancements driven by large language models (llms) such as gpt-3 @xcite , palm @xcite , and llama @xcite , which excel in various tasks such as machine translation and sentiment analysis. however, effectively interpreting and responding to human instructions remains a challenge. instruction tuning @xcite addresses this challenge by fine-tuning llms with explicit task instructions, thereby enhancing their understanding and execution of tasks. this approach has paved the way for the success of instructionfollowing llms like instructgpt @xcite , @xcite , and claude @xcite in a wide range of applications. existing instruction tuning methods formulate all tasks as natural language generation @xcite @xcite , a strategy aligned with the typical pretraining of llms on language modeling tasks. however, natural language instructions can be ambiguous, leading to suboptimal understanding or unintended outputs from the model, especially for complex tasks. specifically, such text-to-text instruction tuning (texttuning) methods suffer from the following limitations: (1) generalization. as presented in figure 1 , texttuning methods mix task elements (e.g., text and candidate languages) and instructions in natural language texts, which can obscure the structure in tasks. this lack of explicit task structure may introduce ambiguity in essential task elements and their relations, potentially hindering models' generalization abilities. (2) robustness. ambiguity in natural language texts can lead to models being sensitive to input variations, resulting in a lack of robust performance. texttuning methods have been shown sensitive to phrasings of instructions @xcite , variations of labels @xcite , and the order of options @xcite . (3) controllability. it can be difficult to provide a clear description or enforce a specific structure for the desired output due to the ambiguity of natural language @xcite , preventing the model from effectively controlling the output. to address the above limitations, it is crucial to incorporate explicit task structure into the input and output representations during instruction tuning. structured data representations such as javascript object notation (json) can mitigate misunderstandings and enhance clarity regarding task objectives. in this paper, we introduce jsontuning, a novel structure-to-structure approach leveraging { "input": { "text": "bonjour, comment ça va?", "candidate languages": ["french", "english", "spanish"] "instruction": "text: {text}. identify the language of the text and provide probability scores for the candidate languages: {candidate languages}. the sum of all probability scores should be 1. language and probability scores: {language}. {probability scores}." }, "output control": { "language": {"type": "string"} "probability scores": { "type": "object", "properties": { "french": { "type": "number"}, "english": { "type": "number"}, "spanish": { "type": "number"} } } } } text: bonjour, comment ça va? identify the language of the text and provide probability scores for the candidate languages: french, english, spanish. the sum of all probability scores should be 1. language and probability scores: { "language": "french", "probability scores": { "french": 1, "english": 0, "spanish": 0 } } french. french: 1, english: 0, spanish: 0. ## analysis does jsontuning bring benefits in open-ended instruction-following scenarios? while performance on the aforementioned benchmarks effectively quantifies the models' capabilities in specific skills, these metrics may not accurately reflect the models' proficiency in handling open-ended instructions. to investigate the potential advantages of jsontuning in such scenarios, we continue our training of json-tuned and text-tuned models on the alpaca dataset @xcite and assess their performance on alpacaeval @xcite , with llama-7b and llama2-7b models being used for our experiments. the evaluation prompt and examples from alpacaeval are provided in figure 18. as illustrated in figure 4 , jsontuning also demonstrates significant advantages over texttuning in open-ended instruction-following scenarios. this benefit likely stems from jsontuning's enforcement of a consistent and standardized data representation. this structured approach mitigates the risk of misinterpretations that often occur due to textual variations in traditional text-based tuning. what is the importance of explicit representation of task structures? to underscore the significance of explicitly representing task structures in the input, we conduct an ablation study comparing two methods: text2json and json2json (i.e., our proposed jsontuning method). in the text2json approach, we use a text-based input, structured control information, and a structured output. the key distinction between text2json and json2json is that text2json relies on a text input and does not explicitly represent task structures. the results, as shown in ## related work the emergence of large language models (llms) has had a transformative effect on the ai field. these advancements have led to a surge in the model mmlu bbh ner re ee nl2sql to advance instruction tuning, researchers have explored learning from human feedback @xcite @xcite , automatic data generation @xcite @xcite , and data selection @xcite @xcite . while the learning algorithm and tuning data have received considerable attention from researchers, the significance of data representation has often been overlooked. our jsontuning approach offers an alternative perspective on data representation to enhance instruction tuning in terms of generalization, robustness, and controllability. our approach is significantly distinct from con-strained decoding methods, such as those employed by commercial llm apis (e.g., openai's json mode) or systems relying on human-designed grammars @xcite @xcite . these methods, which range from token-level constraints to high-level format specifications, are utilized to ensure models conform to prescribed syntactic and semantic requirements. while jsontuning shares a similar objective of controlling outputs, its primary focus lies in leveraging task structures throughout the instruction-tuning process. this approach aims to enhance generalization, robustness, and controllability, rather than merely enforcing adherence to a predetermined output format. furthermore, unlike constrained decoding methods that typically involve modifying the decoding process-adding complexity and potentially increasing latency, our approach is simpler and avoids the need for such modifications. ## conclusion this paper introduces jsontuning, a novel approach designed to overcome the limitations of conventional text-to-text instruction tuning methods for language models. by utilizing the structured data format for explicit task representation, jsontuning significantly improves the model's generalization, robustness, and controllability. our experimental results and case studies highlight the benefits of jsontuning in generalizing to unseen tasks and datasets, maintaining robustness against varying prompts and label spaces, and demonstrating controllability in diverse scenarios.