Check the host family before connecting

Previously it had just used `socket.AF_INET` for everything, which doesn't work for IPv6 stuff.