tested on ferret and dedup