Wait in server.Shutdown for listener goroutine to exit