Constraint Back-translation Improves Complex Instruction Following of
  Large Language Models
Authors: name: Yunjia Qi, name: Hao Peng, name: Xiaozhi Wang, name: Bin Xu, name: Lei Hou, name: Juanzi Li

Large language models (LLMs) struggle to follow instructions with complex
constraints in format, length, etc. Following the conventional
instruction-tuning practice, previous works conduct post-training on complex
instruction-response pairs generated by feeding complex instructions to
advanced LLMs. However, even advanced LLMs cannot follow complex instructions
well, thus limiting the quality of generated data. In this work, we find that
existing datasets inherently contain implicit complex constraints and propose a
novel data generation technique, constraint back-translation. Specifically, we
take the high-quality instruction-response pairs in existing datasets and only
adopt advanced LLMs to add complex constraints already met by the responses to
the instructions, which naturally reduces costs and data noise. In the
experiments, we adopt Llama3-70B-Instruct to back-translate constraints and
create a high-quality complex instruction-response dataset, named CRAB. We
present that post-training on CRAB improves multiple backbone LLMs' complex
instruction-following ability, evaluated on extensive instruction-following
benchmarks. We further find that constraint back-translation also serves as a
useful auxiliary training objective in post-training. Our code, data, and
models will be released to facilitate future research.