Fix infrequent segfault when server shuts down immediately