Llamacpp quants

.gitattributes

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+mistral-orpo-alpha-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,75 @@
+---
+language:
+- en
+license: mit
+base_model:
+- mistralai/Mistral-7B-v0.1
+datasets:
+- HuggingFaceH4/ultrafeedback_binarized
+pipeline_tag: text-generation
+model-index:
+- name: Mistral-ORPO-⍺
+  results:
+  - task:
+      type: text-generation
+    dataset:
+      name: AlpacaEval 1
+      type: AlpacaEval
+    metrics:
+    - type: AlpacaEval 1.0
+      value: 87.92%
+      name: Win Rate
+    source:
+      url: https://github.com/tatsu-lab/alpaca_eval
+      name: self-reported
+  - task:
+      type: text-generation
+    dataset:
+      name: AlpacaEval 2
+      type: AlpacaEval
+    metrics:
+    - type: AlpacaEval 2.0
+      value: 11.33%
+      name: Win Rate
+    source:
+      url: https://github.com/tatsu-lab/alpaca_eval
+      name: self-reported
+  - task:
+      type: text-generation
+    dataset:
+      name: MT-Bench
+      type: MT-Bench
+    metrics:
+    - type: MT-Bench
+      value: 7.23
+      name: Score
+    source:
+      url: https://github.com/lm-sys/FastChat/blob/main/fastchat/llm_judge/
+      name: self-reported
+quantized_by: bartowski
+---
+
+## Llamacpp Quantizations of mistral-orpo-alpha
+
+Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
+
+Original model: https://huggingface.co/kaist-ai/mistral-orpo-alpha
+
+Download a file (not the whole branch) from below:
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [mistral-orpo-alpha-Q8_0.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
+| [mistral-orpo-alpha-Q6_K.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
+| [mistral-orpo-alpha-Q5_K_M.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
+| [mistral-orpo-alpha-Q5_K_S.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
+| [mistral-orpo-alpha-Q5_0.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
+| [mistral-orpo-alpha-Q4_K_M.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
+| [mistral-orpo-alpha-Q4_K_S.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
+| [mistral-orpo-alpha-Q4_0.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
+| [mistral-orpo-alpha-Q3_K_L.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
+| [mistral-orpo-alpha-Q3_K_M.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
+| [mistral-orpo-alpha-Q3_K_S.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
+| [mistral-orpo-alpha-Q2_K.gguf](https://huggingface.co/bartowski/mistral-orpo-alpha-GGUF/blob/main/mistral-orpo-alpha-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
+
+Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

mistral-orpo-alpha-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a70b8513a62f0bb551c4b70a41e29cddaa530da8da97159a885c62a4530840e7
+size 2719242400

mistral-orpo-alpha-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:502553535511669615c716b4328a5422db39a7fbcb3ed243eca3e08f2fa73dae
+size 3822024864

mistral-orpo-alpha-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:412fc2b8634505e9957455ac6c12a30dc3a7a3dca91626cb0f23a739d1e17ecc
+size 3518986400

mistral-orpo-alpha-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46e6a1bdad8fc42a933c9504861bfc34799f4db44f10fa5da4592a7a06ef4665
+size 3164567712

mistral-orpo-alpha-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:010398f4d0dab68daec40ceebb8ab8f2c4eed47a98a1b75a82fbd8f06ec8d680
+size 4108916896

mistral-orpo-alpha-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce97160c6b334fe0222a27d902527ef1fa4328735160f2df157324709d00774f
+size 4368439456

mistral-orpo-alpha-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:168f6c12e2391b9239ad4a908b2df769520a74f5bcf30d8d76d0d34d754b3b97
+size 4140374176

mistral-orpo-alpha-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d098ac2810ee9ee0b046242fe8437546a2be6d9265b7258008ee4c2208fb524b
+size 4997716128

mistral-orpo-alpha-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15372c2d168246199783ee930b8c2da7a354c50c3b77895d1e6b85645d59a6db
+size 5131409568

mistral-orpo-alpha-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:842b6e41cfd873d73d6848306a327fcf06bc6c3bd4e0f4733b5a08667bcac93e
+size 4997716128

mistral-orpo-alpha-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26bfecab9f5cd5df9edc71e8128023fabc301ae57f66e5d6894a17dc3c79af87
+size 5942065312

mistral-orpo-alpha-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93d558bc25d047b3db58213f1bdfb9ff9571a6908bbfda96004e9c5b202ddaee
+size 7695857824