Set encoding last because it reads ascii_only and valid_encoding

This fixes a jump on uninitialized memory warning in valgrind.
Use primitive for signbit

The FFI approach would lead to Valgrind erroring with jumps on
uninitialized values. This was due to GCC optimizations that would be
done with the signbit macro.

With this approach we don't get these errors and don't have to add the
ffi_util method.
Always initialize value for FFI callbacks
Don't try to remove locked object if list is empty

This can happen when the thread is already in the termination process
and all locks have been cleaned up and only the thread itself is still
locked.