RITA_tokenizer / tokenizer_config.json
DanielHesslow's picture
add tokenizer
5dac2c2
{"tokenizer_class": "PreTrainedTokenizerFast"}