Revert "Make sure to look into lib64 folders for pkgconfig and LD_LIBRARY_PATH"

This reverts commit 57dca171656d879eef55ac1d7107e7b7e2ac2ec5.
Make sure third party libs are installed in lib and not lib64