Initial GGML model commit

README.md

@@ -5,7 +5,7 @@ inference: false
 license: llama2
 model_creator: Jon Durbin
 model_link: https://huggingface.co/jondurbin/airoboros-l2-70b-2.1
-model_name: Airoboros L2 70B
+model_name: Airoboros L2 70B 2.1
 model_type: llama
 quantized_by: TheBloke
 ---
@@ -27,13 +27,13 @@ quantized_by: TheBloke
 <hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
 <!-- header end -->
 
-# Airoboros L2 70B - GGML
+# Airoboros L2 70B 2.1 - GGML
 - Model creator: [Jon Durbin](https://huggingface.co/jondurbin)
-- Original model: [Airoboros L2 70B](https://huggingface.co/jondurbin/airoboros-l2-70b-2.1)
+- Original model: [Airoboros L2 70B 2.1](https://huggingface.co/jondurbin/airoboros-l2-70b-2.1)
 
 ## Description
 
-This repo contains GGML format model files for [Jon Durbin's Airoboros L2 70B](https://huggingface.co/jondurbin/airoboros-l2-70b-2.1).
+This repo contains GGML format model files for [Jon Durbin's Airoboros L2 70B 2.1](https://huggingface.co/jondurbin/airoboros-l2-70b-2.1).
 
 ### Important note regarding GGML files.
 
@@ -113,39 +113,8 @@ Refer to the Provided Files table below to see what files use which methods, and
 | [airoboros-l2-70b-2.1.ggmlv3.Q5_0.bin](https://huggingface.co/TheBloke/Airoboros-L2-70B-2.1-GGML/blob/main/airoboros-l2-70b-2.1.ggmlv3.Q5_0.bin) | Q5_0 | 5 | 47.46 GB| 49.96 GB | Original quant method, 5-bit. Higher accuracy, higher resource usage and slower inference. |
 | [airoboros-l2-70b-2.1.ggmlv3.Q5_K_S.bin](https://huggingface.co/TheBloke/Airoboros-L2-70B-2.1-GGML/blob/main/airoboros-l2-70b-2.1.ggmlv3.Q5_K_S.bin) | Q5_K_S | 5 | 47.46 GB| 49.96 GB | New k-quant method. Uses GGML_TYPE_Q5_K for all tensors |
 | [airoboros-l2-70b-2.1.ggmlv3.Q5_K_M.bin](https://huggingface.co/TheBloke/Airoboros-L2-70B-2.1-GGML/blob/main/airoboros-l2-70b-2.1.ggmlv3.Q5_K_M.bin) | Q5_K_M | 5 | 48.75 GB| 51.25 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q5_K |
-| airoboros-l2-70b-2.1.ggmlv3.q5_1.bin | q5_1 | 5 | 51.76 GB | 54.26 GB | Original quant method, 5-bit. Higher accuracy, slower inference than q5_0. |
-| airoboros-l2-70b-2.1.ggmlv3.q6_K.bin | q6_K | 6 | 56.59 GB | 59.09 GB | New k-quant method. Uses GGML_TYPE_Q8_K - 6-bit quantization - for all tensors |
-| airoboros-l2-70b-2.1.ggmlv3.q8_0.bin | q8_0 | 8 | 73.23 GB | 75.73 GB | Original llama.cpp quant method, 8-bit. Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. |
 
-### q5_1, q6_K and q8_0 files require expansion from archive
-
-**Note:** HF does not support uploading files larger than 50GB. Therefore I have uploaded the q6_K and q8_0 files as multi-part ZIP files. They are not compressed, they are just for storing a .bin file in two parts.
-
-<details>
-  <summary>Click for instructions regarding q5_1, q6_K and q8_0 files</summary>
-  
-### q5_1
-Please download:
-* `airoboros-l2-70b-2.1.ggmlv3.q5_1.zip`
-* `airoboros-l2-70b-2.1.ggmlv3.q5_1.z01`
- 
-### q6_K 
-Please download:
-* `airoboros-l2-70b-2.1.ggmlv3.q6_K.zip`
-* `airoboros-l2-70b-2.1.ggmlv3.q6_K.z01`
-
-### q8_0
-Please download:
-* `airoboros-l2-70b-2.1.ggmlv3.q8_0.zip`
-* `airoboros-l2-70b-2.1.ggmlv3.q8_0.z01`
-
-Then extract the .zip archive. This will will expand both parts automatically. On Linux I found I had to use `7zip` - the basic `unzip` tool did not work. Example:
-```
-sudo apt update -y && sudo apt install 7zip
-7zz x airoboros-l2-70b-2.1.ggmlv3.q6_K.zip
-```
-
-</details>
+**Note**: the above RAM figures assume no GPU offloading. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
 
 ## How to run in `llama.cpp`
 
@@ -206,7 +175,7 @@ And thank you again to a16z for their generous grant.
 
 <!-- footer end -->
 
-# Original model card: Jon Durbin's Airoboros L2 70B
+# Original model card: Jon Durbin's Airoboros L2 70B 2.1
 
 
 ### Overview
@@ -233,7 +202,7 @@ This is an instruction fine-tuned llama-2 model, using synthetic data generated
 - laws vary widely based on time and location
 - language model may conflate certain words with laws, e.g. it may think "stealing eggs from a chicken" is illegal
 - these models just produce text, what you do with that text is your resonsibility
-- many people and industries deal with "sensitive" content; imagine if a court stenographer's eqipment filtered illegal content - it would be useless
+- many people and industries deal with "sensitive" content; imagine if a court stenographer's equipment filtered illegal content - it would be useless
 
 ### Prompt format