Use the HAVE_DECL_IBV_LINK_LAYER_ETHERNET macro.

Commit r27211 added ifdef checks for #define
HAVE_IBV_LINK_LAYER_ETHERNET, which is incorrect.  The correct #define
is HAVE_DECL_IBV_LINK_LAYER_ETHERNET.  This broke OMPI over iWARP.

This fixes #3726 and should be added to cmr:v1.7.3:reviewer=jsquyres