Upload README.md with huggingface_hub

README.md

@@ -21,6 +21,8 @@ model-index:
 pipeline_tag: text-generation
 quantized_by: legraphista
 tags:
+- generated_from_trainer
+- axolotl
 - quantized
 - GGUF
 - imatrix
@@ -35,7 +37,7 @@ _Llama.cpp imatrix quantization of cognitivecomputations/dolphin-2.9.1-mixtral-1
 
 Original Model: [cognitivecomputations/dolphin-2.9.1-mixtral-1x22b](https://huggingface.co/cognitivecomputations/dolphin-2.9.1-mixtral-1x22b)  
 Original dtype: `BF16` (`bfloat16`)  
-Quantized by: llama.cpp [b3008](https://github.com/ggerganov/llama.cpp/releases/tag/b3008)  
+Quantized by: llama.cpp [b3024](https://github.com/ggerganov/llama.cpp/releases/tag/b3024)  
 IMatrix dataset: [here](https://gist.githubusercontent.com/legraphista/d6d93f1a254bcfc58e0af3777eaec41e/raw/d380e7002cea4a51c33fffd47db851942754e7cc/imatrix.calibration.medium.raw)  
 
 - [dolphin-2.9.1-mixtral-1x22b-IMat-GGUF](#dolphin-2-9-1-mixtral-1x22b-imat-gguf)
@@ -73,20 +75,25 @@ Link: [here](https://huggingface.co/legraphista/dolphin-2.9.1-mixtral-1x22b-IMat
 ### All Quants
 | Filename | Quant type | File Size | Status | Uses IMatrix | Is Split |
 | -------- | ---------- | --------- | ------ | ------------ | -------- |
-| dolphin-2.9.1-mixtral-1x22b.FP16 | F16 | - | ⏳ Processing | ⚪ Static | -
 | dolphin-2.9.1-mixtral-1x22b.BF16 | BF16 | - | ⏳ Processing | ⚪ Static | -
+| dolphin-2.9.1-mixtral-1x22b.FP16 | F16 | - | ⏳ Processing | ⚪ Static | -
+| dolphin-2.9.1-mixtral-1x22b.Q8_0 | Q8_0 | - | ⏳ Processing | ⚪ Static | -
+| dolphin-2.9.1-mixtral-1x22b.Q6_K | Q6_K | - | ⏳ Processing | ⚪ Static | -
 | dolphin-2.9.1-mixtral-1x22b.Q5_K | Q5_K | - | ⏳ Processing | ⚪ Static | -
 | dolphin-2.9.1-mixtral-1x22b.Q5_K_S | Q5_K_S | - | ⏳ Processing | ⚪ Static | -
+| dolphin-2.9.1-mixtral-1x22b.Q4_K | Q4_K | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.Q4_K_S | Q4_K_S | - | ⏳ Processing | 🟢 IMatrix | -
-| dolphin-2.9.1-mixtral-1x22b.Q3_K_L | Q3_K_L | - | ⏳ Processing | 🟢 IMatrix | -
-| dolphin-2.9.1-mixtral-1x22b.Q3_K_S | Q3_K_S | - | ⏳ Processing | 🟢 IMatrix | -
-| dolphin-2.9.1-mixtral-1x22b.Q2_K_S | Q2_K_S | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ4_NL | IQ4_NL | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ4_XS | IQ4_XS | - | ⏳ Processing | 🟢 IMatrix | -
+| dolphin-2.9.1-mixtral-1x22b.Q3_K | Q3_K | - | ⏳ Processing | 🟢 IMatrix | -
+| dolphin-2.9.1-mixtral-1x22b.Q3_K_L | Q3_K_L | - | ⏳ Processing | 🟢 IMatrix | -
+| dolphin-2.9.1-mixtral-1x22b.Q3_K_S | Q3_K_S | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ3_M | IQ3_M | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ3_S | IQ3_S | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ3_XS | IQ3_XS | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ3_XXS | IQ3_XXS | - | ⏳ Processing | 🟢 IMatrix | -
+| dolphin-2.9.1-mixtral-1x22b.Q2_K | Q2_K | - | ⏳ Processing | 🟢 IMatrix | -
+| dolphin-2.9.1-mixtral-1x22b.Q2_K_S | Q2_K_S | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ2_M | IQ2_M | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ2_S | IQ2_S | - | ⏳ Processing | 🟢 IMatrix | -
 | dolphin-2.9.1-mixtral-1x22b.IQ2_XS | IQ2_XS | - | ⏳ Processing | 🟢 IMatrix | -
@@ -102,11 +109,11 @@ pip install -U "huggingface_hub[cli]"
 ```
 Download the specific file you want:
 ```
-huggingface-cli download legraphista/dolphin-2.9.1-mixtral-1x22b-IMat-GGUF --include "dolphin-2.9.1-mixtral-1x22b.Q8_0.gguf" --local-dir ./
+huggingface-cli download legraphista/dolphin-2.9.1-mixtral-1x22b-IMat-GGUF --include "dolphin-2.9.1-mixtral-1x22b.BF16.gguf" --local-dir ./
 ```
 If the model file is big, it has been split into multiple files. In order to download them all to a local folder, run:
 ```
-huggingface-cli download legraphista/dolphin-2.9.1-mixtral-1x22b-IMat-GGUF --include "dolphin-2.9.1-mixtral-1x22b.Q8_0/*" --local-dir dolphin-2.9.1-mixtral-1x22b.Q8_0
+huggingface-cli download legraphista/dolphin-2.9.1-mixtral-1x22b-IMat-GGUF --include "dolphin-2.9.1-mixtral-1x22b.BF16/*" --local-dir ./
 # see FAQ for merging GGUF's
 ```
 
@@ -144,7 +151,7 @@ What about solving an 2x + 3 = 7 equation?<|im_end|>
 
 ### Llama.cpp
 ```
-llama.cpp/main -m dolphin-2.9.1-mixtral-1x22b.Q8_0.gguf --color -i -p "prompt here (according to the chat template)"
+llama.cpp/main -m dolphin-2.9.1-mixtral-1x22b.BF16.gguf --color -i -p "prompt here (according to the chat template)"
 ```
 
 ---
@@ -159,8 +166,8 @@ According to [this investigation](https://www.reddit.com/r/LocalLLaMA/comments/1
     - To get hold of `gguf-split`, navigate to https://github.com/ggerganov/llama.cpp/releases
     - Download the appropriate zip for your system from the latest release
     - Unzip the archive and you should be able to find `gguf-split`
-2. Locate your GGUF chunks folder (ex: `dolphin-2.9.1-mixtral-1x22b.Q8_0`)
-3. Run `gguf-split --merge dolphin-2.9.1-mixtral-1x22b.Q8_0/dolphin-2.9.1-mixtral-1x22b.Q8_0-00001-of-XXXXX.gguf dolphin-2.9.1-mixtral-1x22b.Q8_0.gguf`
+2. Locate your GGUF chunks folder (ex: `dolphin-2.9.1-mixtral-1x22b.BF16`)
+3. Run `gguf-split --merge dolphin-2.9.1-mixtral-1x22b.BF16/dolphin-2.9.1-mixtral-1x22b.BF16-00001-of-XXXXX.gguf dolphin-2.9.1-mixtral-1x22b.BF16.gguf`
     - Make sure to point `gguf-split` to the first chunk of the split.
 
 ---