Entropy-Based Adaptive Weighting for Self-Training
Authors: name: Xiaoxuan Wang, name: Yihe Deng, name: Mingyu Derek Ma, name: Wei Wang

The mathematical problem-solving capabilities of large language models have
become a focal point of research, with growing interests in leveraging
self-generated reasoning paths as a promising way to refine and enhance these
models. These paths capture step-by-step logical processes while requiring only
the correct answer for supervision. The self-training method has been shown to
be effective in reasoning tasks while eliminating the need for external models
and manual annotations. However, optimizing the use of self-generated data for
model training remains an open challenge. In this work, we propose
Entropy-Based Adaptive Weighting for Self-Training (EAST), an adaptive
weighting strategy designed to prioritize uncertain data during self-training.
Specifically, EAST employs a mapping function with a tunable parameter that
controls the sharpness of the weighting, assigning higher weights to data where
the model exhibits greater uncertainty. This approach guides the model to focus
on more informative and challenging examples, thereby enhancing its reasoning
ability. We evaluate our approach on GSM8K and MATH benchmarks. Empirical
results show that, while the vanilla method yields virtually no improvement
(0%) on MATH, EAST achieves around a 1% gain over backbone model. On GSM8K,
EAST attains a further 1-2% performance boost compared to the vanilla method.