improved tokenization, almost complete