gpt2-medium-italian-embeddings / tokenizer_config.json
wietsedv's picture
add model files
7d19396
{
"unk_token": "<|endoftext|>",
"bos_token": "<|endoftext|>",
"eos_token": "<|endoftext|>",
"add_prefix_space": false
}