Clean up tokenizing tests.
Remove redundant file .
Clean up tests on tokenization, and add interface for generating token
editions.
Implement TokenEditionGenerator class.