Only pass extra link flags on OS X.

Fixes #3249