host: we do not always need 32bit libs, only for ti sgx stuff

Signed-off-by: Robert Nelson <robertcnelson@gmail.com>