git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311219 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: commit current state

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311220 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: don't be this evil yet for type char

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311235 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: disable PositionFilter for now

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311241 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: don't allow negative subword params, Hyphenation relies upon this to filter out what appear to be bogus hyphenation points

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311257 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: Minor cleanups and code consistency

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311278 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: More cleanups

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311282 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: add missing IAE to WikipediaTokenizer ctor

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311294 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: fix PatternTokenizer to not consume chars from the input Reader if it throws IAE

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311318 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: Remove code duplication

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311320 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: Remove useless success variable

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311322 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: Remove unneeded wildcards

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311331 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: don't use scary attsource ctor yet, and always print the analyzer for now

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311339 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: don't get caught by tokenizers that consume in ctor and throw IAE or UOE ever again

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311351 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: Simplify the crazy Reader wrapper

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311358 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: validate after each analysis stage; tenatively add posLen to ShingleFilter

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311373 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: remove nocommit

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311400 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: ValidatingTokenFilter shouldn't create new atts

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311405 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: check that startOffset <= endOffset; comment out sops

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311406 13f79535-47bb-0310-9956-ffa450edef68
LUCENE-3969: add whitespace to analyzer description

git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3969@1311667 13f79535-47bb-0310-9956-ffa450edef68