merge in de-dup changes and test fixes

This merges in the changes from #67, as well as fixing #60