The example tokenizer script failed to check for NULL results (which may occur if nothing can be matched).
Removed some deprecated files.