add checking for data duplication