removed rounding of values in composeSMaskLuminosity

maskData comes out of maskCtx.getImageData, so is 0..255 clamped, and
the used multiplications will not create fractions needing rounding,
neither would addition.
Merge pull request #5523 from CodingFabian/remove-canvas-rounding-luminosity

removed rounding of values in composeSMaskLuminosity