Improve the memory footprint of the search index rebuild (see #5681)