clean up index and remove unwanted items, assign tasks