Adding `Tokenizer::tokenizeTL`
Moved `Tokenizer::parseTokenListFromYaml`, `Tokenizer::parseTokenDefString`, `Tokenizer::lengthSort`, `Tokenizer::matchNextToken` and `Tokenizer::matchWildToken` out of `Tokenizer`
Moved `captureUntil` into `Parser/Helper`
Adding `Parser\Helper::capture`, some other small stuff
Adding `Tokenizer::postprocess`
Adding some useful stuff to `Token`
Converting `Tokenizer::postprocess` from some horrible mess of code to an elegant chain of responsibility
Making `Tokenizer::postprocess` more lightweight
Renaming `Parser::tokenizeAndPrepass` into `Parser::tokenize` to reflect the latest changes
Making `Parser` more lightweight
Making `Parser` even more lightweight
Adding `Tokenizer::tokenizeAndPostProcess`
Moving tokenizer-related tests from `ParserTest` to `TokenizerTest`
Merge pull request #29 from AnhNhan/tokenizer

Some cool stuff to Tokenizer