don't erase from the AvatarHashMap on the wrong thread