Make gir files depend on their library

Otherwise with parallel builds we might try to build gir files
before the library causing an error. This happened once in the
buildbot.

Fix #4606