ann perf improvement

Take more advantages of sparsity. Reduce unnecessary memory allocation and rw. 5x-6x speed improvement in benchmark dataset.