  Training modern neural networks is an inherently noisy process that can lead
to high \emph{prediction churn} -- disagreements between re-trainings of the
same model due to factors such as randomization in the parameter initialization
and mini-batches -- even when the trained models all attain similar accuracies.
Such prediction churn can be very undesirable in practice. In this paper, we
present several baselines for reducing churn and show that training on soft
labels obtained by adaptively smoothing each example's label based on the
example's neighboring labels often outperforms the baselines on churn while
improving accuracy on a variety of benchmark classification tasks and model
architectures.
