  Deep neural networks are widespread due to their powerful performance. Yet,
they suffer from degraded performance in the presence of noisy labels at train
time or adversarial examples during inference. Inspired by the setting of
learning with expert advice, where multiplicative weights (MW) updates were
recently shown to be robust to moderate adversarial corruptions, we propose to
use MW for reweighting examples during neural networks optimization. We
establish the convergence of our method when used with gradient descent and
demonstrate its advantage in two simple examples. We then validate empirically
our findings by showing that MW improves network's accuracy in the presence of
label noise on CIFAR-10, CIFAR-100 and Clothing1M, and that it leads to better
robustness to adversarial attacks.
