galactica-30b / tokenizer_config.json
jacobthebanana's picture
Added tokenizers from original facebook/galactica-30b repo.
39e7654 verified
raw
history blame contribute delete
No virus
166 Bytes
{
"name_or_path": "/content/tokenizer",
"special_tokens_map_file": "/content/tokenizer/special_tokens_map.json",
"tokenizer_class": "PreTrainedTokenizerFast"
}