Do not link to librt if not available

See cab071ad