[server] Fix race condition in join-listeners