Filter out /lib, /usr/lib32, and fold the /usr/lib* checks into one expression