Cleaned up RemoteConnectionServer to expose a bit of public API rather than have it pass internals to HandleRemoteClientTask