Don't search in architecture-irrelevant directories

If we are on a 64-bits machine, there is no point in searching lib32
and conversely. Quite the opposite, it can only end badly if a library
is actually found in the wrong architecture directory.
Merge pull request #193 from rolk/193_arch

Don't search in architecture-irrelevant directories