Cut down hwloc wrappers and fix an issue for non-numa systems