Simple tokenizer, that will convert text into tokens based from your rules.