added deduper to dag