internlm2-base-7b / tokenization_internlm2_fast.py

Commit History

fix no white space when using stream_chat with fast tokenizer
8c7f44e

x54-729 commited on