bigcode
/

santacoder

Text Generation

text-generation-inference

Model card Files Files and versions Community

santacoder / tokenizer_config.json

Commit History

Switch from PreTrainedTokenizerFast to GPT2TokenizerFast and add eos_token & bos_token (#15)

47ad9f0

loubnabnl HF staff commited on Jan 27, 2023

Update tokenizer (#11)

cd86b97

loubnabnl HF staff

RaymondLi commited on Jan 25, 2023

Add max length to tokenizer

aaeed52

ncoop57 commited on Jan 6, 2023

Upload tokenizer

ebb180e

lvwerra HF staff commited on Dec 5, 2022