Wait in server.Shutdown for listen goroutine to exit