Don't relay on deadline to stop the server.

We just send ourselves an UDP or TCP packet so that the listener fails
through and picks up the quit bool send on the channel.