Use reference count for freeing IndexSummaries