query parsing refactored to lucene token stream