OrionStar-Yi-34B-Chat-Llama / tokenizer_config.json

Commit History

replace YiTokenizer to LlamaTokenizer to support load tokenizer without trsut_remote_code
333c788

chenxingphh commited on