fix one clear case of unsigned overflow in comp_op_rgba_minus - refs #1679