`_weight_decay_gather` is kind of a shitty name, feel free if you think of a better one.