Pass by HostInfo instead of the peer address

Pass the full HostInfo to the connection pool after getting new
hosts, with the caveat that seed hosts wont have valid host info.