config and duplicate detection logic