fix bitmask calculation