Prevent L2 weight from being applied before the penalty.