Use extra-lib-dirs in the buildinfo

With this fix, you don't need to set LD_LIBRARY_PATH when loading the package into ghci.