cmd-small-3epoch / tokenizer_config.json
Heatz's picture
Upload tokenizer
25d3378
raw
history blame contribute delete
234 Bytes
{
"name_or_path": "cmd-smallgpt-3epoch",
"special_tokens": [
"<s>",
"</s>",
"<unk>",
"<pad>"
],
"special_tokens_map_file": "gpt2_small/special_tokens_map.json",
"tokenizer_class": "PreTrainedTokenizerFast"
}