clean up making indexes and add better progress indicators