levanter-gpt / tokenizer.json

Commit History

apparently one needs the tokenizer
ede4e43

dlwh commited on