LUCENE-5235: Tokenizers now throw an IllegalStateException if the consumer does not call reset() before consuming the stream. Previous versions throwed NullPointerException or ArrayIndexOutOfBoundsException on best effort which was not user-friendly.

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1525362 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-5235: Add more information to backwards break changes section.

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1525376 13f79535-47bb-0310-9956-ffa450edef68
javadocs did not produce good looking HTML

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1525506 13f79535-47bb-0310-9956-ffa450edef68