a deterministic FSM word/number/symbol tokenizer (for Latin scripts)