Link against libdl only if necessary

We should link against libdl whenever necessary, but not whenever
possible.