Spaces:

eson
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / requirements.txt

eson's picture

remove vocabs; update compression_app; add character_app;

2bd606a about 1 month ago

raw history blame

No virus

87 Bytes

	transformers
	sentencepiece
	tiktoken
	icetk
	torch
	zhon
	nltk
	boto3
	ai2-olmo
	ipadic
	fugashi