index generation and cleanup.