I just verified and you're right, it does not change anything.
It's when casting a byte to an int that the sign becomes a problem.
Positive numbers will fill up remaining bits with 0s.
Negative numbers will fill up remaining bits with 1s.