Add shortcut to SuffixingNGramTokenFilter to reduce duplication

NGrams are now calculated in suffix order, eg:
alarmXX alarXX alaXX alXX aXX
larmXX larXX laXX lXX
armXX arXX aXX
rmXX rXX
mXX

If a term with a mutual suffix is encountered, then the filter will shortcut
once it reaches the already-seen suffix:
harmXX harXX haXX hXX
armXX <-- already seen, stop ngramming here
Add further shortcut to SuffixingNGramTokenFilter

We add a second cache to the TokenFilter, this time for all seen grams.  If a
gram is encountered that is already in this cache, then move directly to the
next gram starting point.

This has the effect of removing all duplicates in-place, so there is no longer a
need for the DuplicateRemovingTokenFilter (with the exception of wildcard
tokens, which should not be frequent enough to cause problems).
Merge remote-tracking branch 'localflax/1.1.x' into topic/aw-parallel