gszabo
/

base_webcorpus2

Model card Files Files and versions

base_webcorpus2 / tokenizer_config.json

gszabo's picture

Upload tokenizer

ad40234 over 3 years ago

history blame contribute delete

78 Bytes

	{
	"model_max_length": 512,
	"tokenizer_class": "PreTrainedTokenizerFast"
	}