Tokenizer red
Changing string tokens to Tokens in the TokenizerTest suite
(compilation errors).
Tokenizer green
Adjusted Tokenizer.
Tokenizer yellow
Removed dead code from Tokenizer.
Token yellow
Made use of Java 1.7's java.util.Objects#equals() method.
Merge pull request #7 from ffbit/tokenizer

Tokenizer
Integration with Tokens.