Pass libraries only to binaries that need them.