Spaces:

bayartsogt
/

real-time-tokenizer

Running

App Files Files Community

real-time-tokenizer / requirements.txt

bayartsogt's picture

adding sentencepeice to req file

87c056a 10 months ago

raw history blame

No virus

36 Bytes

	transformers
	tiktoken
	sentencepiece