REST: Stress Testing Large Reasoning Models by Asking Multiple Problems
  at Once
Authors: name: Zhuoshi Pan; name: Qizhi Pei; name: Yu Li; name: Qiyao Sun; name: Zinan Tang; name: H. Vicky Zhao; name: Conghui He; name: Lijun Wu
Published: 2025-07-14 17:58:47

Abstract: Recent Large Reasoning Models (LRMs) have achieved remarkable progress on
task-specific benchmarks, yet their evaluation methods remain constrained by
isolated problem-solving paradigms. Existing benchmarks predominantly assess
single-question reasoning through sequential testing, resulting critical
limitations: (1) vulnerability to data contamination and less challenging
(e.g., DeepSeek-R1 achieves 97.0% on MATH500), forcing costly and perpetual
creation of new questions with large human efforts, (2) failure to evaluate
models under multi-context pressure, a key requirement for real-world
deployment. To bridge this gap, we present REST (Reasoning Evaluation through
Simultaneous Testing), a stress-testing framework that concurrently exposes
LRMs to multiple problems simultaneously. Beyond basic reasoning, REST
specifically evaluates several under-tested capabilities: contextual priority
allocation, cross-problem interference resistance, and dynamic cognitive load
management. Our evaluation reveals several striking findings: Even
state-of-the-art (SOTA) models like DeepSeek-R1 exhibit substantial performance
degradation under stress testing. Crucially, REST demonstrates stronger
discriminative power than existing benchmarks, revealing pronounced performance
differences among models that exhibit similar, near-ceiling performance under
single-question evaluations. Some key mechanistic insights emerge from our
analysis: (1) the "overthinking trap" is a critical factor contributing to the
performance degradation; (2) the models trained with "long2short" technique
preserve more accuracy of their single-problem performance under REST,
outperforming standard-trained counterparts. These results establish REST as a
cost-efficient, future-proof evaluation paradigm that better reflects
real-world reasoning demands while reducing reliance on continuous human
annotation.