Cleanup of the gaiohttp worker

Also add support for unix sockets. It seems we don't need to use
self.loop.create_unix_server because sock.sock is already a unix socket.