L1Penalty always adds the output gradient