Fix the pkgconfig file: it should not contain $(libdir) but rather @prefix@/lib, so that the directory can be resolved at compile time.