monsoon-nlp
/

sanaa-dialect

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

sanaa-dialect / tokenizer_config.json

system's picture

system HF staff

Update tokenizer_config.json

32deb8c almost 4 years ago

No virus

114 Bytes

	{"pad_token": "<\|endoftext\|>", "special_tokens_map_file": "special_tokens_map.json", "full_tokenizer_file": null}