added some duplication checks