Fix unexpected sign extension

Fixed an unexpected sign extension may corrupt bit shift operations.
Very subtle bug. It's very unlikely to affect in practice.