One-shot Entropy Minimization
Authors: name: Zitian Gao; name: Lynx Chen; name: Joey Zhou; name: Bryan Dai
Published: 2025-05-26 17:58:30

Abstract: We trained 13,440 large language models and found that entropy minimization
requires only a single unlabeled data and 10 steps optimization to achieve
performance improvements comparable to or even greater than those obtained
using thousands of data and carefully designed rewards in rule-based
reinforcement learning. This striking result may prompt a rethinking of
post-training paradigms for large language models. Our code is avaliable at
https://github.com/zitian-gao/one-shot-em.