added uniqueness check to dat files