NISTER / tokenizer.json

Upload tokenizer.json

2ab6482 verified over 1 year ago

109 Bytes

Invalid JSON: Unexpected token 'o', "tokenizer ="... is not valid JSON

	tokenizer = T5Tokenizer.from_pretrained("t5-small")
	tokenizer.save_pretrained("/Users/airalex/Documents/LLM")