Fix byte sign extensions for architectures with default -funsigned-char

Architectures like armel, armhf, arm64, avr32, powerpc, s390 and s390x use a
unsigned type definition for char. Therefore, the macro for sign extension is
just a nop instruction there. Being more specific on these arches about the
signedness avoids this problem.