corrects calculation of the max pixel value according to bit depth