|
|
--- |
|
|
license: cc-by-nc-4.0 |
|
|
pipeline_tag: text-generation |
|
|
library_name: gguf |
|
|
base_model: CohereForAI/c4ai-command-r-v01 |
|
|
--- |
|
|
**NOTE**: This is a preview of upcoming [PR#6033](https://github.com/ggerganov/llama.cpp/pull/6033). |
|
|
|
|
|
* GGUF importance matrix (imatrix) quants for https://huggingface.co/CohereForAI/c4ai-command-r-v01 |
|
|
* The importance matrix was trained for ~100K tokens (200 batches of 512 tokens) using [wiki.train.raw](https://huggingface.co/datasets/wikitext). |
|
|
* [Which GGUF is right for me? (from Artefact2)](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9) |
|
|
|
|
|
> C4AI Command-R is a research release of a 35 billion parameter highly performant generative model. Command-R is a large language model with open weights optimized for a variety of use cases including reasoning, summarization, and question answering. Command-R has the capability for multilingual generation evaluated in 10 languages and highly performant RAG capabilities. |
|
|
|
|
|
> Command-R’s tool use functionality takes a conversation as input (with an optional user-system preamble), along with a list of available tools. The model will then generate a json-formatted list of actions to execute on a subset of those tools. Command-R may use one of its supplied tools more than once. |
|
|
|
|
|
> Command-R’s grounded generation behavior takes a conversation as input (with an optional user-supplied system preamble), along with a list of retrieved document snippets. The document snippets should be chunks, rather than long documents, typically around 100-400 words per chunk. Document snippets consist of key-value pairs. The keys should be short descriptive strings, the values can be text or semi-structured. |
|
|
|
|
|
| Layers | Context | [Template](https://huggingface.co/CohereForAI/c4ai-command-r-v01#model-summary) | |
|
|
| --- | --- | --- | |
|
|
| <pre>40</pre> | <pre>131072</pre> | <pre>\<BOS_TOKEN\>\<\|START_OF_TURN_TOKEN\|\>\<\|USER_TOKEN\|\>{prompt}\<\|END_OF_TURN_TOKEN\|\>\<\|START_OF_TURN_TOKEN\|\>\<\|CHATBOT_TOKEN\|\>{response}</pre> | |
|
|
|
