HippoPHP/Tokenizer — pull request #27 by jbrooksuk (2014-11-10 10:58:29+00:00)
Title: New tokenizer

**What's done?**

- Dropped out `Matchers`.
- Dropped out `TokenMatchers`.
- Replaced `Tokenizer@tokenize` with `token_get_all`.

** What's left to do?**

- Support custom tokens so that we're *forwards* compatible.
- Probably need some more tests.