removes the old neighbor structure