Deduplication framework written in Perl