use int instead of bool for C function rt_unloadLibrary

- This should actually be ABI compatible but it's cleaner
  for documentation purpose.