dial-small-3epoch / tokenizer_config.json
Heatz's picture
Upload tokenizer
0c1a138
raw
history blame contribute delete
235 Bytes
{
"name_or_path": "dial-smallgpt-3epoch",
"special_tokens": [
"<s>",
"</s>",
"<unk>",
"<pad>"
],
"special_tokens_map_file": "gpt2_small/special_tokens_map.json",
"tokenizer_class": "PreTrainedTokenizerFast"
}