added deduplication function
merge