gpt2-small-italian / tokenizer_config.json
wietsedv's picture
add model files
9ee6f2c
{
"unk_token": "<|endoftext|>",
"bos_token": "<|endoftext|>",
"eos_token": "<|endoftext|>",
"add_prefix_space": false
}