Do not link to librt if not available

See cab071ad
Merge pull request #76 from eregon/master

Do not link to librt if not available (e.g.: OS X)