change to always index 10k last entries so that it handles cases where old downloads are redownloaded; also index concurrently, since index creation may otherwise lock tables for a long time