Simplify RemoteIPFinder since its now split, port over tests.
Deprecate CallbackRef, this should not be used anymore