pegler/pytzwhere — pull request #10 by gimbo (2014-06-04 16:03:25+00:00)
Title: Reduce memory consumption from 750MB to 60MB by using CSV not JSON. (Also lots of other changes.)

Hi there. You might hate this, but in case you don't, please feel free to pull it and make a new release if you want. I've changed a *lot* though.

The main thing is that I noticed that reading in the JSON (before doing any processing) consumed 750MB of RAM - and actually this never got released properly ([apparently this is known behaviour with JSON and python](http://stackoverflow.com/a/11058050/392743)). So I played around and found that by using a CSV file, each line of which is a single (tzname, polygon) pair, I can read that in one line at a time and save hundreds of MB.

Then I found out that by representing points as (lat, lng) pairs rather than two-element dictionaries, we save a couple of hundred MB again.

Then I found out that by using `numpy` arrays (if `numpy` is installed) instead of tuples for the polygons and lists of polygon lists, we can save even more. (If `numpy` isn't installed, it falls back to tuples.)

Overall, I've got the memory usage down from 750MB to 60MB in the best case.

Along the way I've considerably refactored the `tzwhere.py` script, including adding the capability to produce this CSV file, and also (this might be the bit you like least) moving from `argparse` to `docopt`, just because I find it more flexible. That means the script's options have changed, and I have also changed the public interface of the `tzwhere` class, so this is quite a big change really, and potentially breaking for client code.

Well, see what you think. I'm happy to keep it as a separate fork if you want, but if you like it, great.

(This is all with python2.7 however - not tested with anything else.)