Entropy Regularizing Activation: Boosting Continuous Control, Large
  Language Models, and Image Classification with Activation as Entropy
  Constraints
Authors: name: Zilin Kang; name: Chonghua Liao; name: Tingqiang Xu; name: Huazhe Xu
Published: 2025-10-09 17:56:17

Abstract: We propose ERA, a new paradigm that constrains the sampling entropy above
given thresholds by applying specially designed activations to the outputs of
models. Our approach demonstrates broad effectiveness across different domains:
1) for large language models(LLMs), boosting the AIME 2025 score for
Qwen2.5-Math-7B by 37.4%; 2) for continuous control reinforcement learning
agents, improving performance by more than 30% over strong baselines such as
SAC on the challenging HumanoidBench; 3) for image classification, enhancing
ImageNet top-1 accuracy by 0.69% for ResNet-50. These gains are achieved with a
computational overhead of less than 7%. Our work validates output activation as
a powerful tool for entropy control, opening a new direction for designing
simpler and more robust algorithms.