Correct detection out of space in amqp_encode_table

amqp_encode_8 and amqp_encode_bytes return 1 on success, 0 otherwise.
Correct the checking of the return value of these functions to match the
style used in amqp_table.c