CohereForAI

/

c4ai-command-r-v01

like 989

Hi, Matt from Hugging Face here! We held off on doing this for a while, since multiple templates weren't properly supported, but they're in the 4.39 release that's been out for a couple of weeks, so it might be safe to start changing over.

This PR just copies the default chat templates from the tokenizer class into tokenizer_config.json, making them accessible to users and to conversion tools (e.g. llama.cpp). Over time, we hope to deprecate hardcoded templates entirely!