When a neighbor change is detected, firstOut may be updated