Hugging Face's logo

bartowski
/
Einstein-v6-7B-GGUF

Text Generation
GGUF
English
axolotl
Generated from Trainer
Mistral
instruct
finetune
chatml
gpt4
synthetic data
science
physics
chemistry
biology
math
Model card Files Community
bartowski commited on
Commit
cd65fe6
•
1 Parent(s): 9b3cf35

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. Einstein-v6-7B-IQ3_M.gguf +3 -0
  3. Einstein-v6-7B-IQ3_S.gguf +3 -0
  4. Einstein-v6-7B-IQ4_NL.gguf +3 -0
  5. Einstein-v6-7B-IQ4_XS.gguf +3 -0
  6. Einstein-v6-7B-Q2_K.gguf +3 -0
  7. Einstein-v6-7B-Q3_K_L.gguf +3 -0
  8. Einstein-v6-7B-Q3_K_M.gguf +3 -0
  9. Einstein-v6-7B-Q3_K_S.gguf +3 -0
  10. Einstein-v6-7B-Q4_0.gguf +3 -0
  11. Einstein-v6-7B-Q4_K_M.gguf +3 -0
  12. Einstein-v6-7B-Q4_K_S.gguf +3 -0
  13. Einstein-v6-7B-Q5_0.gguf +3 -0
  14. Einstein-v6-7B-Q5_K_M.gguf +3 -0
  15. Einstein-v6-7B-Q5_K_S.gguf +3 -0
  16. Einstein-v6-7B-Q6_K.gguf +3 -0
  17. Einstein-v6-7B-Q8_0.gguf +3 -0
  18. README.md +90 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Einstein-v6-7B-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Einstein-v6-7B-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Einstein-v6-7B-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Einstein-v6-7B-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Einstein-v6-7B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Einstein-v6-7B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Einstein-v6-7B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Einstein-v6-7B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Einstein-v6-7B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Einstein-v6-7B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Einstein-v6-7B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Einstein-v6-7B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Einstein-v6-7B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Einstein-v6-7B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Einstein-v6-7B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Einstein-v6-7B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Einstein-v6-7B-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd284bd9aea6189da0a6cdb453cfb2920a2953e904988fdc080cd5c86e77b02d
3
+ size 3284902304
Einstein-v6-7B-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac3ee8380a262ddc559e1f078a9a9b7361cbf06259a6afd4b232524f34f3b503
3
+ size 3182404000
Einstein-v6-7B-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e24d04a00a1dc86c2352dd69b4a657a63274e1f2d78eff72f1080d76904421ac
3
+ size 4155065824
Einstein-v6-7B-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30a37eee54951dd2540c739742b8a75ef967ecd7ff2a70eb711f148dcfb3d708
3
+ size 3944400096
Einstein-v6-7B-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9746f3a2e42986b3b59ca5bcb636c5470dc9d3a292ba12dc767dfafd3bd7abcc
3
+ size 2719252064
Einstein-v6-7B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16f56d0276f2d101ef1ed47cff0d958054d44913ef84998b8aa2a2c8c6cee85a
3
+ size 3822035360
Einstein-v6-7B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:079fcfaa466fea22dc92f2635cee492d1e9c1ae9a47bf5ef8e5fccbe0c2ddaa2
3
+ size 3518996896
Einstein-v6-7B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48cd6cb9523105ab85733cb16804ea70e5e089935eb996ed98c07deb12082851
3
+ size 3164578208
Einstein-v6-7B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:606945beeaf98a1fb59e5172a6d224e3939c9143cbfc8e4607db821fd4ecf9d4
3
+ size 4108928480
Einstein-v6-7B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a39759a7f97a4fb1a05468b5dd6f531b4cabc799e457bf61a2057cc3519b5ee
3
+ size 4368451040
Einstein-v6-7B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8654b69a02277194c4d27aaf72d370dffbca6b2a465a2d8313cd61d4881e240
3
+ size 4140385760
Einstein-v6-7B-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d5801cef0547804bce688b4eb5f6b88a020b228e870a32d324785c72d469440
3
+ size 4997728736
Einstein-v6-7B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f560a01d2c9f73388ed5b93203627e2fcc5e904e2c1554d0b3ae9933df3cc4df
3
+ size 5131422176
Einstein-v6-7B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:108ad2d7cbf1cbdc5262994a3119b0c901ef6c19833746c6af65a483f78aff12
3
+ size 4997728736
Einstein-v6-7B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6eb23a0b2ec6f9e112b66c652fde1d466ceda556ddab75b4bce52309a25c5b06
3
+ size 5942079008
Einstein-v6-7B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73348ced5d2c6069acdcf018c69641a3ed38ffd6a1d02e2762bfdd5dc37ec010
3
+ size 7695875488
README.md ADDED
@@ -0,0 +1,90 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ tags:
4
+ - axolotl
5
+ - generated_from_trainer
6
+ - Mistral
7
+ - instruct
8
+ - finetune
9
+ - chatml
10
+ - gpt4
11
+ - synthetic data
12
+ - science
13
+ - physics
14
+ - chemistry
15
+ - biology
16
+ - math
17
+ base_model: alpindale/Mistral-7B-v0.2-hf
18
+ datasets:
19
+ - allenai/ai2_arc
20
+ - camel-ai/physics
21
+ - camel-ai/chemistry
22
+ - camel-ai/biology
23
+ - camel-ai/math
24
+ - metaeval/reclor
25
+ - openbookqa
26
+ - mandyyyyii/scibench
27
+ - derek-thomas/ScienceQA
28
+ - TIGER-Lab/ScienceEval
29
+ - jondurbin/airoboros-3.2
30
+ - LDJnr/Capybara
31
+ - Cot-Alpaca-GPT4-From-OpenHermes-2.5
32
+ - STEM-AI-mtl/Electrical-engineering
33
+ - knowrohit07/saraswati-stem
34
+ - sablo/oasst2_curated
35
+ - lmsys/lmsys-chat-1m
36
+ - TIGER-Lab/MathInstruct
37
+ - bigbio/med_qa
38
+ - meta-math/MetaMathQA-40K
39
+ - openbookqa
40
+ - piqa
41
+ - metaeval/reclor
42
+ - derek-thomas/ScienceQA
43
+ - scibench
44
+ - sciq
45
+ - Open-Orca/SlimOrca
46
+ - migtissera/Synthia-v1.3
47
+ - TIGER-Lab/ScienceEval
48
+ - allenai/WildChat
49
+ - microsoft/orca-math-word-problems-200k
50
+ - openchat/openchat_sharegpt4_dataset
51
+ - teknium/GPTeacher-General-Instruct
52
+ - m-a-p/CodeFeedback-Filtered-Instruction
53
+ - totally-not-an-llm/EverythingLM-data-V3
54
+ - HuggingFaceH4/no_robots
55
+ - OpenAssistant/oasst_top1_2023-08-25
56
+ - WizardLM/WizardLM_evol_instruct_70k
57
+ language:
58
+ - en
59
+ quantized_by: bartowski
60
+ pipeline_tag: text-generation
61
+ ---
62
+
63
+ ## Llamacpp Quantizations of Einstein-v6-7B
64
+
65
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2589">b2589</a> for quantization.
66
+
67
+ Original model: https://huggingface.co/Weyaxi/Einstein-v6-7B
68
+
69
+ Download a file (not the whole branch) from below:
70
+
71
+ | Filename | Quant type | File Size | Description |
72
+ | -------- | ---------- | --------- | ----------- |
73
+ | [Einstein-v6-7B-Q8_0.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
74
+ | [Einstein-v6-7B-Q6_K.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
75
+ | [Einstein-v6-7B-Q5_K_M.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, *recommended*. |
76
+ | [Einstein-v6-7B-Q5_K_S.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, *recommended*. |
77
+ | [Einstein-v6-7B-Q5_0.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
78
+ | [Einstein-v6-7B-Q4_K_M.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, uses about 4.83 bits per weight, *recommended*. |
79
+ | [Einstein-v6-7B-Q4_K_S.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
80
+ | [Einstein-v6-7B-IQ4_NL.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-IQ4_NL.gguf) | IQ4_NL | 4.15GB | Decent quality, similar to Q4_K_S, new method of quanting, *recommended*. |
81
+ | [Einstein-v6-7B-IQ4_XS.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-IQ4_XS.gguf) | IQ4_XS | 3.94GB | Decent quality, new method with similar performance to Q4. |
82
+ | [Einstein-v6-7B-Q4_0.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
83
+ | [Einstein-v6-7B-Q3_K_L.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
84
+ | [Einstein-v6-7B-Q3_K_M.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
85
+ | [Einstein-v6-7B-IQ3_M.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-IQ3_M.gguf) | IQ3_M | 3.28GB | Medium-low quality, new method with decent performance. |
86
+ | [Einstein-v6-7B-IQ3_S.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-IQ3_S.gguf) | IQ3_S | 3.18GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
87
+ | [Einstein-v6-7B-Q3_K_S.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
88
+ | [Einstein-v6-7B-Q2_K.gguf](https://huggingface.co/bartowski/Einstein-v6-7B-GGUF/blob/main/Einstein-v6-7B-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended. |
89
+
90
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski