Update tokenizer_config.json
#7
by
ybelkada
HF staff
- opened
No description provided.
Muennighoff
changed pull request status to
merged
It's annoying that we're missing indent here ... Whoever is writing this file should probably have indent option set.
Just to be sure, I think tokenizer_config.json comes from transformers, and it's supposed to be fixed for quite a while: https://github.com/huggingface/transformers/blame/862888a35834527fed61beaf42373423ffdbd216/src/transformers/tokenization_utils_base.py#L2155
@ybelkada
did you perhaps change this manually instead of going through transformers?