Remove reference in adjacent nodes after removing blocks