- NEW    : tokenize.php to parse rules and tokenize input.

git-svn-id: https://svn.sven-drieling.de/repos/trunk/playground/compiler@171 9adaec3a-5a83-11de-93c2-a12fd72c8969