codeparrot-ds / tokenizer_config.json
mlegls's picture
Training in progress, step 5000
86ed068
{
"tokenizer_class": "PreTrainedTokenizerFast"
}