QueST: Incentivizing LLMs to Generate Difficult Problems
Authors: name: Hanxu Hu; name: Xingxing Zhang; name: Jannis Vamvas; name: Rico Sennrich; name: Furu Wei
Published: 2025-10-20 16:29:53

Abstract: Large Language Models have achieved strong performance on reasoning tasks,
solving competition-level coding and math problems. However, their scalability
is limited by human-labeled datasets and the lack of large-scale, challenging
coding problem training data. Existing competitive coding datasets contain only
thousands to tens of thousands of problems. Previous synthetic data generation
methods rely on either augmenting existing instruction datasets or selecting
challenging problems from human-labeled data. In this paper, we propose QueST,
a novel framework which combines difficulty-aware graph sampling and
difficulty-aware rejection fine-tuning that directly optimizes specialized
generators to create challenging coding problems. Our trained generators
demonstrate superior capability compared to even GPT-4o at creating challenging
problems that benefit downstream performance. We leverage QueST to generate
large-scale synthetic coding problems, which we then use to distill from strong
teacher models with long chain-of-thought or to conduct reinforcement learning
for smaller models, proving effective in both scenarios. Our distillation
experiments demonstrate significant performance gains. Specifically, after
fine-tuning Qwen3-8B-base on 100K difficult problems generated by QueST, we
surpass the performance of the original Qwen3-8B on LiveCodeBench. With an
additional 112K examples (i.e., 28K human-written problems paired with multiple
synthetic solutions), our 8B model matches the performance of the much larger
DeepSeek-R1-671B. These findings indicate that generating complex problems via
QueST offers an effective and scalable approach to advancing the frontiers of
competitive coding and reasoning for large language models.