Hugging Face's logo

tensorblock
/
Mixtral-8x7B-Instruct-v0.1-GGUF

GGUF
TensorBlock
GGUF
Inference Endpoints
conversational
Model card Files Community
morriszms commited on
Commit
30128c2
·
verified ·
1 Parent(s): 6e47bfe

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. Mixtral-8x7B-Instruct-v0.1-Q2_K.gguf +2 -2
  2. Mixtral-8x7B-Instruct-v0.1-Q3_K_L.gguf +2 -2
  3. Mixtral-8x7B-Instruct-v0.1-Q3_K_M.gguf +2 -2
  4. Mixtral-8x7B-Instruct-v0.1-Q3_K_S.gguf +2 -2
  5. Mixtral-8x7B-Instruct-v0.1-Q4_0.gguf +2 -2
  6. Mixtral-8x7B-Instruct-v0.1-Q4_K_M.gguf +2 -2
  7. Mixtral-8x7B-Instruct-v0.1-Q4_K_S.gguf +2 -2
  8. Mixtral-8x7B-Instruct-v0.1-Q5_0.gguf +2 -2
  9. Mixtral-8x7B-Instruct-v0.1-Q5_K_M.gguf +2 -2
  10. Mixtral-8x7B-Instruct-v0.1-Q5_K_S.gguf +2 -2
  11. Mixtral-8x7B-Instruct-v0.1-Q6_K.gguf +2 -2
  12. Mixtral-8x7B-Instruct-v0.1-Q8_0.gguf +2 -2
  13. README.md +16 -5
Mixtral-8x7B-Instruct-v0.1-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2a836b48fa642267d25d05deb6e0521b177246f2b3aab59db87424816f6f6436
3
- size 17311229952
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0091337f82d5974fd5f3603ee30d37e9911cf51058a2ba446bf836e00e19870
3
+ size 17311230880
Mixtral-8x7B-Instruct-v0.1-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:71d641f92b1b9946a9e27f83a89094dd3f974864dfc2102abf8077079a7514ac
3
- size 24169646080
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:838fa9a51ffc6a6542ab043360d6a51e43f455e6010b45dacb711a154f093df0
3
+ size 24169647008
Mixtral-8x7B-Instruct-v0.1-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d81983ce6bd8af14a87e9c704a97e65b397f1e5319ea7b0d82649812c3b1ced5
3
- size 22546450432
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f51eede3e8029b9b34ec815901402a1759ffbe13d594e55a07fc982ee937ea4
3
+ size 22546451360
Mixtral-8x7B-Instruct-v0.1-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:38b481bf9fd17ddc7986a8e4b6d9115bcde93f02962d334fc747faa0953000c6
3
- size 20432521216
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b10051500546735340b9ef0099120441ddc403636142a179cade5226f6a45afe
3
+ size 20432522144
Mixtral-8x7B-Instruct-v0.1-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:960154cb5b8dfcb66e32c53c1313cff36bcafa23d3abb557323f37997b7bc0c9
3
- size 26443589632
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db0435512f6dfa31bc17adff26516c477603e0509f6f3871466feee563a38d32
3
+ size 26443590560
Mixtral-8x7B-Instruct-v0.1-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c5a1309efee4d1020632e8622d924050c77afa4ce01754a5f00faa89a1fd442a
3
- size 28448466944
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3cd29b4edc7972087d367d57dba8160402c19a11ff3d5aa316bf12ce6a4a792b
3
+ size 28448467872
Mixtral-8x7B-Instruct-v0.1-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:668b24851290fd3c84f3f1aa3351f515235209306e7a36fe2e41a8b1395962cd
3
- size 26745579520
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f9b60fbf66d649fc9129d3f9496296510c60bcc27ac00c926d88e472ff08e4d
3
+ size 26745580448
Mixtral-8x7B-Instruct-v0.1-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1d9288e14f773cb56e153b40f81ba1954f6bb9f201d39ab7f573eb11f2794b8a
3
- size 32231335936
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9245a435940088dfe421c9d27cb6f9c8d3dbbe9b3a2c20ab23e1609db4e6e87b
3
+ size 32231336864
Mixtral-8x7B-Instruct-v0.1-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8371a392f637de69a54ecd39e90e3d1f5971d7b877603e50e31e91a0f55ae4f3
3
- size 33229580288
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b95e659eef1a4aea14d662275a069271d2163d587d0ece1975b132a71e8cd6c
3
+ size 33229581216
Mixtral-8x7B-Instruct-v0.1-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:78dd7f217d1ce19807aa35d8993e691c6c14a0560013ed25e2d582984f1c5db7
3
- size 32231335936
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef46fc1766a6ac810be58d8c81cd002aeccc6773934dd4623fda79c0df01289c
3
+ size 32231336864
Mixtral-8x7B-Instruct-v0.1-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:248b3165b7e01e3f79a658882c4e77d39c35f68457d2b1ed6955833e5d91feff
3
- size 38380816384
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6651acc48ba0260f8161a662c05e661a17a08774061a0d1c5ac6ac69ec905f33
3
+ size 38380817312
Mixtral-8x7B-Instruct-v0.1-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d7adff84f991a13178872791995b408239f1c2640a88dd9e51aedb1a3470902b
3
- size 49626318848
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e97013712e4e2f1ee1609ee4df97537265615adc4d3a436c19a046de61c382b
3
+ size 49626319776
README.md CHANGED
@@ -1,15 +1,24 @@
1
  ---
2
- license: apache-2.0
3
  language:
4
  - fr
5
  - it
6
  - de
7
  - es
8
  - en
 
 
 
 
 
 
 
 
 
 
 
9
  tags:
10
  - TensorBlock
11
  - GGUF
12
- base_model: snorkelai/Mixtral-8x7B-Instruct-v0.1
13
  ---
14
 
15
  <div style="width: auto; margin-left: auto; margin-right: auto">
@@ -23,9 +32,9 @@ base_model: snorkelai/Mixtral-8x7B-Instruct-v0.1
23
  </div>
24
  </div>
25
 
26
- ## snorkelai/Mixtral-8x7B-Instruct-v0.1 - GGUF
27
 
28
- This repo contains GGUF format model files for [snorkelai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/snorkelai/Mixtral-8x7B-Instruct-v0.1).
29
 
30
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4242](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
31
 
@@ -38,7 +47,9 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
38
  ## Prompt template
39
 
40
  ```
41
- <s>[INST] {prompt} [/INST]
 
 
42
  ```
43
 
44
  ## Model file specification
 
1
  ---
 
2
  language:
3
  - fr
4
  - it
5
  - de
6
  - es
7
  - en
8
+ license: apache-2.0
9
+ base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
10
+ inference:
11
+ parameters:
12
+ temperature: 0.5
13
+ widget:
14
+ - messages:
15
+ - role: user
16
+ content: What is your favorite condiment?
17
+ extra_gated_description: If you want to learn more about how we process your personal
18
+ data, please read our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
19
  tags:
20
  - TensorBlock
21
  - GGUF
 
22
  ---
23
 
24
  <div style="width: auto; margin-left: auto; margin-right: auto">
 
32
  </div>
33
  </div>
34
 
35
+ ## mistralai/Mixtral-8x7B-Instruct-v0.1 - GGUF
36
 
37
+ This repo contains GGUF format model files for [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1).
38
 
39
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4242](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
40
 
 
47
  ## Prompt template
48
 
49
  ```
50
+ <s> [INST] {system_prompt}
51
+
52
+ {prompt} [/INST]
53
  ```
54
 
55
  ## Model file specification