Add unit testing of tokenization and token edition generation.