Conversion between COO/CSR/CSC are fast once the data are in memory (though O(n) is not something that would be good to do frequently...), but I strongly suspect that incrementally loading `lil_matrix` will be slow given that it is a linked list structure and it does not know nnz in advance, which means there will be frequent reallocs. 

I'm okay with dropping order and agree that it should be transparent to the user. I just want to make sure that the load is efficient.