holoirc: fix race condition between refreshing servers and invalidating server list