tiny-tokenizer (pypi): 🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.
Keywords: janome|japanese|kytea|mecab|natural-language-processing|nlp|sentencepiece|sudachi|text-processing