add a list of files that exists in the index to the failure
cleanup deletion of content in shards
we are very conservative on when we delete data, remove the actual options of deleting data that are not used
Set shardSize according to the total size if not explicitly specified. Closes #2729
prefer totalTermFrequency over docFreq in PhraseSuggester
if word scorer is on unigram make sure we score the current position not position 0
use a base ShingleFilterFactory to simplify default shingle detection