eliminate duplicate columns, rearrange column order, added new columns