biocore/biom-format — pull request #338 by Jorge-C (2014-05-07 13:44:02+00:00)
Title: Transform

Faster transform method (40x / 128x):

E.g., in a table of shape (1150, 108):
- axis = 'sample'
new: 100 loops, best of 3: 3.35 ms per loop
old: 10 loops, best of 3: 137 ms per loop
- axis = 'observation'
new: 100 loops, best of 3: 7.51 ms per loop
old: 1 loops, best of 3: 964 ms per loop

Currently it's a drop in replacement of the old code: it returns a new table and the old tests pass without any modification.
However, the difference is that the function that transforms data only gets access to the non zero values (e.g., for a diagonal matrix, the transform function gets a 1 element 1D array per sample/obs, instead of np.array([0, ..., 0, x != 0, 0, ..., 0])). That seems reasonable, but the question is: do we want to allow transformations that change the sparsity structure? (zeroing stuff is OK though). Allowing sparsity changes leads to *incredibly* inefficient code for csr/csc. If that's necessary, a new matrix in LIL or COO formats has to be built.

Thanks!