remove_empty_and_get_dups - faster, intermediate, move csv to test