semvec/pypi-packages huggingface dataset:
zh-sentence 0.0.5
Light-weight sentence tokenizer for Chinese languages.
License: apache-2.0
Keywords: Chinese,Sentence,Tokenizer

A light-weight sentence tokenizer for Chinese languages.

Sample Code:

from zh_sentence.tokenizer import tokenize

paragraph_str = "你好吗？你快乐吗？"

sentence_list = tokenize(paragraph_str)

for sentence in sentence_list:
	print(sentence)
ja-sentence 0.0.5
Light-weight sentence tokenizer for Japanese.
License: apache-2.0
Keywords: Japanese,Sentence,Tokenizer

A light-weight sentence tokenizer for Japanese.


Sample Code:

from ja_sentence.tokenizer import tokenize

paragraph_str = "えー！？くれるの？本当にいいの…？嬉しい！！"

sentence_list = tokenize(paragraph_str)

for sentence in sentence_list:
	print(sentence)