p/redis-dump-load — pull request #4 by toobaz (2013-11-25 12:03:00+00:00)
Title: Add json incremental loading

This branch introduces incremental json loading - the json parser is a generator, and objects are inserted in redis one at a time.
The total CPU time required for loading a database is probably larger (because of the unoptimized pure python wrapper over the json library) but the operation is parallelized (this is determinant if the reading the dump is the bottleneck) and most importantly the memory footprint is much lower (with the old mode, twice the quantity of RAM needed for storing the objects is needed).