finished initial pass with dedupe 0.5, primary examples working