Spaces:

eson
/

tokenizer-arena

Running

update

751936e 10 months ago

No virus

105 Bytes

	"""

	SentencePiece(unigram)

	https://huggingface.co/docs/transformers/tokenizer_summary#sentencepiece
	"""