Properly support libdirs of explicit bitnesses

Formerly this is limited to lib64 on 64-bit systems; more configurations
such as Gentoo on MIPS N32 (which uses /usr/lib32; /usr/lib is for O32 ABI)
are supported now, please review.