removing dup columns, removing nonsense characters