Optimised use of lil_matrix
Moved the lil_matrix instance to permit better GC