beomi

/

kollama-13b

like 16

Since KoLLAMA does not use SentencePiece Tokenizer, there is no need to use tokenizer.model.
Instead, you can use PretrainedTokenizer.from_pretrained method to load tokenizer.