we want sparse matrices, so just filter the zeros.