Replace use of dict for duplicate removal

The use of dict introduced in ec98f92 can cause issues when large sets
of points are introduced. The order of data in the dict is unreliable
(in cPython anyways), and starts to mix things up when thousands of
points are introduced. While the stored data is correct, it can
cause extra seeks/writes. Its also super confusing when debugging.

This patch is similar to the implementation of a8e30fa, but adheres to
documented expectations.

It is also faster than the dict, at least for large sets of points. For
small sets, the difference is unmeasurable.