ucto-doc (alpine): advanced rule-based (regular-expression) and unicode-aware tokenizer for various languages (documentation)
Keywords: computational-linguistics|folia|language|natural-language-processing|nlp|punctuation|tokeniser