LUCENE-5235: Tokenizers now throw an IllegalStateException if the consumer does not call reset() before consuming the stream. Previous versions throwed NullPointerException or ArrayIndexOutOfBoundsException on best effort which was not user-friendly.

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1525362 13f79535-47bb-0310-9956-ffa450edef68