Fix some signedness mistakes in secp256k1_num_set_hex

Closes #51 as well.