encodeSignedNumber: prevent integer overflow for large numbers

Cast the signed int to unsigned which fixes the comparison inside
encodeNumber.