THOUGHTTERMINATOR: Benchmarking, Calibrating, and Mitigating
  Overthinking in Reasoning Models
Authors: name: Xiao Pu, name: Michael Saxon, name: Wenyue Hua, name: William Yang Wang

Reasoning models have demonstrated impressive performance on difficult tasks
that traditional language models struggle at. However, many are plagued with
the problem of overthinking--generating large amounts of unnecessary tokens
which don't improve accuracy on a question. We introduce approximate measures
of problem-level difficulty and demonstrate that a clear relationship between
problem difficulty and optimal token spend exists, and evaluate how well
calibrated a variety of reasoning models are in terms of efficiently allocating
the optimal token count. We find that in general, reasoning models are poorly
calibrated, particularly on easy problems. To evaluate calibration on easy
questions we introduce DUMB500, a dataset of extremely easy math, reasoning,
code, and task problems, and jointly evaluate reasoning model on these simple
examples and extremely difficult examples from existing frontier benchmarks on
the same task domain. Finally, we introduce THOUGHTTERMINATOR, a training-free
black box decoding technique that significantly improves reasoning model
calibration.