Missing tokenizer.model
#4
by
Jeronymous
- opened
Hello,
Congrats and thanks for sharing CroissantLLM :)
We noticed that "tokenizer.model" file is missing, which can cause issues in some workflows.
See for instance https://github.com/huggingface/transformers/issues/29137