Duplicate from ABX-AI/Quantum-Citrus-9B-GGUF-IQ-Imatrix

Co-authored-by: Abroxis <ABX-AI@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,56 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+imatrix.dat filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-IQ3_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-IQ3_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-IQ3_XXS-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-IQ4_NL-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-IQ4_XS-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-Q4_K_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-Q4_K_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-Q5_K_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-Q5_K_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-v2-9B-Q6_K-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-IQ3_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-IQ3_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-IQ3_XXS-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-IQ4_NL-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-IQ4_XS-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-Q4_K_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-Q4_K_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-Q5_K_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-Q5_K_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
+Quantum-Citrus-9B-Q6_K-imat.gguf filter=lfs diff=lfs merge=lfs -text

Quantum-Citrus-9B-IQ3_M-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c53274eccd34d975479ac318d0d189210453e6d3d3c6e887edff8e3ec70e1ae
+size 4064970720

Quantum-Citrus-9B-IQ3_S-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c274de033f6eb4b3bf9bad4e06b02e8a11469721e1df94ef61cd28896acdb22
+size 3936847840

Quantum-Citrus-9B-IQ3_XXS-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5451949e9403a0d9d65af72caa4f5fb863f320e6185040a1f3d2fb8be4652a60
+size 3497388000

Quantum-Citrus-9B-IQ4_NL-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75bf911d63f61579d68ea98413ecf0ef9a5955c92ba86df4f36f3786cb5d075a
+size 5111621600

Quantum-Citrus-9B-IQ4_XS-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cac45443759d6340f7b934c752c4ae22acf3fe0aa32782ec55f320881e62c29
+size 4840138720

Quantum-Citrus-9B-Q4_K_M-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8bb6dd61b7cb3ac1c0cd90d4ea70940ecc7715224f393901912183eab3a11771
+size 5415053280

Quantum-Citrus-9B-Q4_K_S-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af9f2d753a230eb06fa573e9430fdbbe448a8bcf7019eba92807d0984eea6e31
+size 5129447392

Quantum-Citrus-9B-Q5_K_M-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7dab93b07cbcfac4ed83f1dd82de2e5ed35650f531c1daeede5445f01de43c11
+size 6364669920

Quantum-Citrus-9B-Q5_K_S-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3d784c56ed5f0cc973ac2026fcdaf428f5a97a5d31aca42c7075a1c20df2e65
+size 6197553120

Quantum-Citrus-9B-Q6_K-imat.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7a63a0536d335875ed7a9a36d100a4c872f25c4594d956807ba3f3327e09367
+size 7373637600

README.md

@@ -0,0 +1,182 @@
+---
+license: other
+library_name: transformers
+tags:
+- mergekit
+- merge
+- mistral
+- not-for-all-audiences
+base_model:
+- ABX-AI/Cerebral-Infinity-7B
+- ABX-AI/Starfinite-Laymospice-v2-7B
+model-index:
+- name: Quantum-Citrus-9B
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: AI2 Reasoning Challenge (25-Shot)
+      type: ai2_arc
+      config: ARC-Challenge
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: acc_norm
+      value: 65.19
+      name: normalized accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: HellaSwag (10-Shot)
+      type: hellaswag
+      split: validation
+      args:
+        num_few_shot: 10
+    metrics:
+    - type: acc_norm
+      value: 84.75
+      name: normalized accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MMLU (5-Shot)
+      type: cais/mmlu
+      config: all
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 64.58
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: TruthfulQA (0-shot)
+      type: truthful_qa
+      config: multiple_choice
+      split: validation
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: mc2
+      value: 55.96
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Winogrande (5-shot)
+      type: winogrande
+      config: winogrande_xl
+      split: validation
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 79.4
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: GSM8k (5-shot)
+      type: gsm8k
+      config: main
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 50.57
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
+      name: Open LLM Leaderboard
+---
+# GGUF / IQ / Imatrix for [Quantum-Citrus-9B](https://huggingface.co/ABX-AI/Quantum-Citrus-9B)
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/65d936ad52eca001fdcd3245/J0snW3yfLadLuMYERL6X5.png)
+
+**Why Importance Matrix?**
+
+**Importance Matrix**, at least based on my testing, has shown to improve the output and performance of "IQ"-type quantizations, where the compression becomes quite heavy.
+The **Imatrix** performs a calibration, using a provided dataset. Testing has shown that semi-randomized data can help perserve more important segments as the compression is applied.
+
+Related discussions in Github:
+[[1]](https://github.com/ggerganov/llama.cpp/discussions/5006) [[2]](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)
+
+The imatrix.txt file that I used contains general, semi-random data, with some custom kink.
+
+
+# Quantum-Citrus-9B 
+
+This merge is another attempt at making and intelligent, refined and unaligned model.
+
+Based on my tests so far, it has accomplished the goals, and I am continuing to experiment with my interactions with it.
+
+It includes previous merges of Starling, Cerebrum, LemonadeRP, InfinityRP, and deep down has a base of layla v0.1, as I am not that happy with the result form using v0.2.
+
+The model is intended for fictional storytelling and roleplaying and may not be intended for all audences.
+
+## Merge Details
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+### Merge Method
+
+This model was merged using the passthrough merge method.
+
+### Models Merged
+
+The following models were included in the merge:
+* [ABX-AI/Starfinite-Laymospice-v2-7B](https://huggingface.co/ABX-AI/Starfinite-Laymospice-v2-7B)
+* [ABX-AI/Cerebral-Infinity-7B](https://huggingface.co/ABX-AI/Cerebral-Infinity-7B)
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+slices:
+  - sources:
+      - model: ABX-AI/Cerebral-Infinity-7B
+        layer_range: [0, 20]
+  - sources:
+      - model: ABX-AI/Starfinite-Laymospice-v2-7B
+        layer_range: [12, 32]
+merge_method: passthrough
+dtype: float16
+```
+
+# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
+Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ABX-AI__Quantum-Citrus-9B)
+
+|             Metric              |Value|
+|---------------------------------|----:|
+|Avg.                             |66.74|
+|AI2 Reasoning Challenge (25-Shot)|65.19|
+|HellaSwag (10-Shot)              |84.75|
+|MMLU (5-Shot)                    |64.58|
+|TruthfulQA (0-shot)              |55.96|
+|Winogrande (5-shot)              |79.40|
+|GSM8k (5-shot)                   |50.57|

imatrix.dat

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d25b15b0f1d4c668c067714f13d270a724f196bfca3feb1e8cb1ed637a60eaf9
+size 6235174