python-ucto (pypi): This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).
Keywords: tokenizer|tokenization|tokeniser|tokenisation|nlp|computational_linguistics|ucto|computational-linguistics|folia|nlp-library|python|text-processing