Remove `-Wl,-rpath=...` from `LDFLAGS` on Darwin to fix build.

We don't need to set `rpath` explicitly because the ld on Darwin
embeds the full paths to each dylib by default