Project: talenddataquality
Issue TDQ-4950: Missmatches with tStandardizeRow due to inconsistency between Lucene analyzer and ANTLR parser
Example: For the word , during indexing, StandardAnalyzer of Lucene consider it as one word to save in the index. However, during the search, ANTLR cut it into 2 tokens "didn" and "t", none of them can be found in the index. See also testProcessIndexForMultiWord() in IndexStandardizerTest, the word can not be found. One solution will be modifying the parser rule, so that it doesn't cut the word by apostrophe.
Story points: 13