implement deduplication as an optional protocol flag, fixes #1