Update tokenizer_config.json

0df6c60 6 months ago

No virus

230 Bytes

	{
	"clean_up_tokenization_spaces": true,
	"cls_token": "<cls>",
	"mask_token": "<mask>",
	"model_max_length": 1024,
	"pad_token": "<pad>",
	"sep_token": "<eos>",
	"tokenizer_class": "EsmTokenizer",
	"unk_token": "<unk>"
	}