test data now is ignored if it is duplicated