bugfix: handle unexpected annotation deletion

Tolerate the deletion of an annotation the local user was still moving.