diag-baichuan2-4bit / tokenization_baichuan.py

Commit History

update tokenizer according to https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat/discussions/18
c5b3968
verified

curtis-sun commited on