Added flag file tokenizer.

Added a small inline tokenizer to resolve a few problems with flag files (see issue #436).