KoboldAI
/

GPT-NeoX-20B-Skein

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

GPT-NeoX-20B-Skein / tokenizer_config.json

Henk717's picture

Experimental Tokenizer Fix

1b80ef9 almost 2 years ago

No virus

161 Bytes

	{"unk_token": "<\|endoftext\|>", "bos_token": "<\|endoftext\|>", "eos_token": "<\|endoftext\|>", "add_prefix_space": false, "tokenizer_class": "GPTNeoXTokenizerFast"}