Hugging Face's logo

bartowski
/
Einstein-v5-v0.2-7B-GGUF

Text Generation
GGUF
axolotl
generated_from_trainer
Mistral
instruct
finetune
chatml
gpt4
synthetic data
science
physics
chemistry
biology
math
Model card Files Community
bartowski commited on
Commit
d63837f
•
1 Parent(s): f64b34c

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. Einstein-v5-v0.2-7B-IQ3_M.gguf +3 -0
  3. Einstein-v5-v0.2-7B-IQ3_S.gguf +3 -0
  4. Einstein-v5-v0.2-7B-IQ4_NL.gguf +3 -0
  5. Einstein-v5-v0.2-7B-IQ4_XS.gguf +3 -0
  6. Einstein-v5-v0.2-7B-Q2_K.gguf +3 -0
  7. Einstein-v5-v0.2-7B-Q3_K_L.gguf +3 -0
  8. Einstein-v5-v0.2-7B-Q3_K_M.gguf +3 -0
  9. Einstein-v5-v0.2-7B-Q3_K_S.gguf +3 -0
  10. Einstein-v5-v0.2-7B-Q4_0.gguf +3 -0
  11. Einstein-v5-v0.2-7B-Q4_K_M.gguf +3 -0
  12. Einstein-v5-v0.2-7B-Q4_K_S.gguf +3 -0
  13. Einstein-v5-v0.2-7B-Q5_0.gguf +3 -0
  14. Einstein-v5-v0.2-7B-Q5_K_M.gguf +3 -0
  15. Einstein-v5-v0.2-7B-Q5_K_S.gguf +3 -0
  16. Einstein-v5-v0.2-7B-Q6_K.gguf +3 -0
  17. Einstein-v5-v0.2-7B-Q8_0.gguf +3 -0
  18. README.md +84 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Einstein-v5-v0.2-7B-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Einstein-v5-v0.2-7B-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Einstein-v5-v0.2-7B-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Einstein-v5-v0.2-7B-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Einstein-v5-v0.2-7B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Einstein-v5-v0.2-7B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Einstein-v5-v0.2-7B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Einstein-v5-v0.2-7B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Einstein-v5-v0.2-7B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Einstein-v5-v0.2-7B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Einstein-v5-v0.2-7B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Einstein-v5-v0.2-7B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Einstein-v5-v0.2-7B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Einstein-v5-v0.2-7B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Einstein-v5-v0.2-7B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Einstein-v5-v0.2-7B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Einstein-v5-v0.2-7B-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:777473d7b45e1ecc10a3b1b4cb1d8a82f1a6d742e48c46731ca5438caa86852c
3
+ size 3284902304
Einstein-v5-v0.2-7B-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1ffdf891c61dbfb151ecd0a872de1799f5385cd81a16d2fe95872ffdb75b5b3
3
+ size 3182404000
Einstein-v5-v0.2-7B-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f40048b5386db01d8724dfc782aa1eb195a25af51ce05f60cd2f77a830312505
3
+ size 4155065824
Einstein-v5-v0.2-7B-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:957db5138bc92d59bd91b5296af74ebe5b85b84732d46aa731287e0695780f45
3
+ size 3944400096
Einstein-v5-v0.2-7B-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b571f6621825530db5778696058b7767d74f4c8df6ac0b04eb9ebd6feec075b3
3
+ size 2719252064
Einstein-v5-v0.2-7B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b75af80f53cb134f92d4bede5d2d5f31945e25a80940abfa2897e6e181b66ee
3
+ size 3822035360
Einstein-v5-v0.2-7B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6d46579e9f759a71fe85b45b89c1da3c13eaa0d02bc3db57c1712d61999f0ee
3
+ size 3518996896
Einstein-v5-v0.2-7B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:43c6fc5fcec1de5e401d40945e046c5b61898aa1252f92a4a057440b49d2ac62
3
+ size 3164578208
Einstein-v5-v0.2-7B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:116b3fbbcb72276a3205f1cf6ebb43c2c023adc6a8dbf3bc5283dece86ee6b94
3
+ size 4108928480
Einstein-v5-v0.2-7B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6d69b2a32c3ed281ecd1494c78a89f679e32ff4a8d81699dc8ad150e5fec96e1
3
+ size 4368451040
Einstein-v5-v0.2-7B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da76d7bc0d68b43df7ff160388ffccdb394d537b854dff8a2ec0121d622b6c62
3
+ size 4140385760
Einstein-v5-v0.2-7B-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca0ccd15d49697d2cf64caf6fa15b8ebb6a9d353304b92d5fa397b02fa33f06c
3
+ size 4997728736
Einstein-v5-v0.2-7B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:048d2a8f38b52a257766578f3b690e72bf3139b794f6225b18a8716a1eb865bb
3
+ size 5131422176
Einstein-v5-v0.2-7B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee3dc7e377aea6171ea4d2b90f0878fd80592868625f7c0289ca78d4448fd6c0
3
+ size 4997728736
Einstein-v5-v0.2-7B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6513960a38d4aa94ee08bfff0b84eae0793fafca417e9e34a57fe864293b7d5
3
+ size 5942079008
Einstein-v5-v0.2-7B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a79f748c68252758df48a1b2b738ca5dca84dfee314ed00a609b7badc00d55b6
3
+ size 7695875488
README.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ tags:
4
+ - axolotl
5
+ - generated_from_trainer
6
+ - Mistral
7
+ - instruct
8
+ - finetune
9
+ - chatml
10
+ - gpt4
11
+ - synthetic data
12
+ - science
13
+ - physics
14
+ - chemistry
15
+ - biology
16
+ - math
17
+ base_model: alpindale/Mistral-7B-v0.2-hf
18
+ datasets:
19
+ - allenai/ai2_arc
20
+ - camel-ai/physics
21
+ - camel-ai/chemistry
22
+ - camel-ai/biology
23
+ - camel-ai/math
24
+ - metaeval/reclor
25
+ - openbookqa
26
+ - mandyyyyii/scibench
27
+ - derek-thomas/ScienceQA
28
+ - TIGER-Lab/ScienceEval
29
+ - jondurbin/airoboros-3.2
30
+ - LDJnr/Capybara
31
+ - Cot-Alpaca-GPT4-From-OpenHermes-2.5
32
+ - STEM-AI-mtl/Electrical-engineering
33
+ - knowrohit07/saraswati-stem
34
+ - sablo/oasst2_curated
35
+ - lmsys/lmsys-chat-1m
36
+ - TIGER-Lab/MathInstruct
37
+ - bigbio/med_qa
38
+ - meta-math/MetaMathQA-40K
39
+ - openbookqa
40
+ - piqa
41
+ - metaeval/reclor
42
+ - derek-thomas/ScienceQA
43
+ - scibench
44
+ - sciq
45
+ - Open-Orca/SlimOrca
46
+ - migtissera/Synthia-v1.3
47
+ - TIGER-Lab/ScienceEval
48
+ - allenai/WildChat
49
+ - microsoft/orca-math-word-problems-200k
50
+ - openchat/openchat_sharegpt4_dataset
51
+ - teknium/GPTeacher-General-Instruct
52
+ - m-a-p/CodeFeedback-Filtered-Instruction
53
+ quantized_by: bartowski
54
+ pipeline_tag: text-generation
55
+ ---
56
+
57
+ ## Llamacpp Quantizations of Einstein-v5-v0.2-7B
58
+
59
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2536">b2536</a> for quantization.
60
+
61
+ Original model: https://huggingface.co/Weyaxi/Einstein-v5-v0.2-7B
62
+
63
+ Download a file (not the whole branch) from below:
64
+
65
+ | Filename | Quant type | File Size | Description |
66
+ | -------- | ---------- | --------- | ----------- |
67
+ | [Einstein-v5-v0.2-7B-Q8_0.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
68
+ | [Einstein-v5-v0.2-7B-Q6_K.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
69
+ | [Einstein-v5-v0.2-7B-Q5_K_M.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
70
+ | [Einstein-v5-v0.2-7B-Q5_K_S.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
71
+ | [Einstein-v5-v0.2-7B-Q5_0.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
72
+ | [Einstein-v5-v0.2-7B-Q4_K_M.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, uses about 4.83 bits per weight. |
73
+ | [Einstein-v5-v0.2-7B-Q4_K_S.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
74
+ | [Einstein-v5-v0.2-7B-IQ4_NL.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-IQ4_NL.gguf) | IQ4_NL | 4.15GB | Decent quality, similar to Q4_K_S, new method of quanting, |
75
+ | [Einstein-v5-v0.2-7B-IQ4_XS.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-IQ4_XS.gguf) | IQ4_XS | 3.94GB | Decent quality, new method with similar performance to Q4. |
76
+ | [Einstein-v5-v0.2-7B-Q4_0.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
77
+ | [Einstein-v5-v0.2-7B-Q3_K_L.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
78
+ | [Einstein-v5-v0.2-7B-Q3_K_M.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
79
+ | [Einstein-v5-v0.2-7B-IQ3_M.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-IQ3_M.gguf) | IQ3_M | 3.28GB | Medium-low quality, new method with decent performance. |
80
+ | [Einstein-v5-v0.2-7B-IQ3_S.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-IQ3_S.gguf) | IQ3_S | 3.18GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
81
+ | [Einstein-v5-v0.2-7B-Q3_K_S.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
82
+ | [Einstein-v5-v0.2-7B-Q2_K.gguf](https://huggingface.co/bartowski/Einstein-v5-v0.2-7B-GGUF/blob/main/Einstein-v5-v0.2-7B-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
83
+
84
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski