added javadocs
added some simple naming conventions
more documentation for the tokenizers types
added gate dependency for future gate-compat project