xia/xdp: allow non-XDP sinks on destination addresses

This is needed to support destination addresses with
not-XDP sinks that routing redirect to a local XDP XIDs.