stablelm-2-1_6b / tokenizer_config.json

Commit History

update(tokenizer): convert to `GPT2Tokenizer` (#7)
a7a1fb8
verified

jon-tow commited on

fix: add default special tokens
9f2def9
verified

jon-tow commited on

fix: add default special tokens
ca32832
verified

jon-tow commited on

fix: revert zephyr tokenizer config change
192936b
verified

jon-tow commited on

fix: add chat template to tokenizer config
507254d
verified

reciprocate commited on

init: release
3aeae29

jon-tow commited on