OpenGrok/OpenGrok — pull request #24 by kahatlen (2013-03-27 19:06:14+00:00)
Title: Make Iterable2TokenStream a Reader.

Iterable2TokenStream splits the input into tokens in a way similar to
PlainSymbolTokenizer, but not identical. If it is a Reader instead, the
input could be passed on to PlainSymbolTokenizer to produce tokens in
a consistent and uniform way.