implementing fast and scalable incremental partitioning algorithm