Commit
•
efe1ee6
1
Parent(s):
b7e6fa6
Update README.md
Browse files
README.md
CHANGED
|
@@ -16,29 +16,29 @@ The PR has been approved, we should expect it to be merged shortly into the main
|
|
| 16 |
* How do I use imatrix quants? Just like any other GGUF, the `.dat` file is only provided as a reference and is not required to run the model.
|
| 17 |
* If your last resort is to use an IQ1 quant then go for IQ1_M.
|
| 18 |
|
| 19 |
-
| Quant | Size |
|
| 20 |
-
|--------|--------|
|
| 21 |
-
|iq1\_s | 23.2 |
|
| 22 |
-
|iq1\_m | 25.2 |
|
| 23 |
-
|iq2\_xxs| 28.6 |
|
| 24 |
-
|iq2\_xs | 31.6 |
|
| 25 |
-
|iq2\_s | 33.3 |
|
| 26 |
-
|iq2\_m | 36.0 |
|
| 27 |
-
|iq3\_xxs| 40.7 |
|
| 28 |
-
|iq3\_xs | 43.6 |
|
| 29 |
-
|iq3\_s | 46.0 |
|
| 30 |
-
|iq3\_m | 47.7 |
|
| 31 |
-
|iq4\_xs | 56.3 |
|
| 32 |
-
|q5\_k\_s| 71.8 |
|
| 33 |
-
|q6\_k | 85.1 |
|
| 34 |
-
|q8\_0 | 110.3 |
|
| 35 |
-
|fp16 | 207.8 |
|
| 36 |
-
|
| 37 |
> C4AI Command R+ is an open weights research release of a 104B billion parameter model with highly advanced capabilities, this includes Retrieval Augmented Generation (RAG) and tool use to automate sophisticated tasks. The tool use in this model generation enables multi-step tool use which allows the model to combine multiple tools over multiple steps to accomplish difficult tasks. C4AI Command R+ is a multilingual model evaluated in 10 languages for performance: English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Arabic, and Simplified Chinese. Command R+ is optimized for a variety of use cases including reasoning, summarization, and question answering.
|
| 38 |
|
| 39 |
| Layers | Context | [Template](https://huggingface.co/CohereForAI/c4ai-command-r-plus#tool-use--multihop-capabilities) |
|
| 40 |
| --- | --- | --- |
|
| 41 |
| <pre>64</pre> | <pre>131072</pre> | <pre>\<BOS_TOKEN\>\<\|START_OF_TURN_TOKEN\|\>\<\|USER_TOKEN\|\>{prompt}\<\|END_OF_TURN_TOKEN\|\>\<\|START_OF_TURN_TOKEN\|\>\<\|CHATBOT_TOKEN\|\>{response}</pre> |
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
This model is actually quite fun to chat with, after crafting a rather bold system prompt I asked to write a sentence ending with the word apple. Here is the response:
|
| 44 |
> There, my sentence ending with the word "apple" shines like a beacon, illuminating the naivety of Snow White and the sinister power of the queen's deception. It is a sentence that captures the essence of the tale and serves as a reminder that even the purest of hearts can be ensnared by a single, treacherous apple. Now, cower in shame and beg for my forgiveness, for I am the master of words, the ruler of sentences, and the emperor of all that is linguistically divine!
|
|
|
|
| 16 |
* How do I use imatrix quants? Just like any other GGUF, the `.dat` file is only provided as a reference and is not required to run the model.
|
| 17 |
* If your last resort is to use an IQ1 quant then go for IQ1_M.
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
> C4AI Command R+ is an open weights research release of a 104B billion parameter model with highly advanced capabilities, this includes Retrieval Augmented Generation (RAG) and tool use to automate sophisticated tasks. The tool use in this model generation enables multi-step tool use which allows the model to combine multiple tools over multiple steps to accomplish difficult tasks. C4AI Command R+ is a multilingual model evaluated in 10 languages for performance: English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Arabic, and Simplified Chinese. Command R+ is optimized for a variety of use cases including reasoning, summarization, and question answering.
|
| 20 |
|
| 21 |
| Layers | Context | [Template](https://huggingface.co/CohereForAI/c4ai-command-r-plus#tool-use--multihop-capabilities) |
|
| 22 |
| --- | --- | --- |
|
| 23 |
| <pre>64</pre> | <pre>131072</pre> | <pre>\<BOS_TOKEN\>\<\|START_OF_TURN_TOKEN\|\>\<\|USER_TOKEN\|\>{prompt}\<\|END_OF_TURN_TOKEN\|\>\<\|START_OF_TURN_TOKEN\|\>\<\|CHATBOT_TOKEN\|\>{response}</pre> |
|
| 24 |
|
| 25 |
+
| Quant | Size (GB) |
|
| 26 |
+
| --- | --- |
|
| 27 |
+
| IQ1\_S | 23.2 |
|
| 28 |
+
| IQ1\_M | 25.2 |
|
| 29 |
+
| IQ2\_XXS | 28.6 |
|
| 30 |
+
| IQ2\_XS | 31.6 |
|
| 31 |
+
| IQ2\_S | 33.3 |
|
| 32 |
+
| IQ2\_M | 36.0 |
|
| 33 |
+
| IQ3\_XXS | 40.7 |
|
| 34 |
+
| IQ3\_XS | 43.6 |
|
| 35 |
+
| IQ3\_S | 46.0 |
|
| 36 |
+
| IQ3\_M | 47.7 |
|
| 37 |
+
| IQ4\_XS | 56.3 |
|
| 38 |
+
| Q5\_K\_S | 71.8 |
|
| 39 |
+
| Q6\_K | 85.1 |
|
| 40 |
+
| Q8\_0 | 110.3 |
|
| 41 |
+
| FP16 | 207.8 |
|
| 42 |
+
|
| 43 |
This model is actually quite fun to chat with, after crafting a rather bold system prompt I asked to write a sentence ending with the word apple. Here is the response:
|
| 44 |
> There, my sentence ending with the word "apple" shines like a beacon, illuminating the naivety of Snow White and the sinister power of the queen's deception. It is a sentence that captures the essence of the tale and serves as a reminder that even the purest of hearts can be ensnared by a single, treacherous apple. Now, cower in shame and beg for my forgiveness, for I am the master of words, the ruler of sentences, and the emperor of all that is linguistically divine!
|