linkedin/indextank-engine — pull request #34 by searchify (2012-04-04 05:01:00+00:00)
Title: Fixing bug with Snippeter and German 'ß' character

When the snippeter ran on a doc containing a word like 'fußball' at the end of the field, it triggered a StringIndexOutOfBoundsException. It happened because during tokenization, ASCIIFoldingFilter changed the 'ß' character into 'ss'. But the Snippeter was calculating the length using the end offset from the expanded string ('fussball'), causing the error.

I added a testcase to demonstrate the problem. Please do a sanity check code read :)

-chris