Added token simplification.
Switched tokenization over to the new rule based method.