Expose the resolution of capabilities for general use in protocol negotiations

The general mechanism of capabilities' preferred mode resolution can be exported to enable non-gossip usage,
such as over socket connections, e.g. protocols built on top of the connection and service managers (also in core). The first step is to refactor the resolver of preferred modes and export API such that the gen_server's state is not entangled in the core functions.

A separate PR will use the exported API to negotiate capabilities over a socket based protocol.

Tagged @jtuple for possible review as well.