spartan-array/spartan — pull request #94 by fegin (2014-03-19 20:51:56+00:00)
Title: Add from_file_parallel() to support load files from workers.

1. from_file_parallel() makes each work loads a part of the given file.
2. Add csrcsc_update() to make csr_matrix and csc_matrix support update (out-place). However, csrcsc_update() needs some extra memory space.
3. Use csr_matrix or csc_matrix instead of lil_matrix in tile.pyx and distarray.py (some code still uses lil_matrix)
4. Current implementation still needs lot of memory space. If a work reads 800MB, it may need more than 4GB memory space to do tocsr(), tocsc(), csrcsc_update(), etc.