Hugging Face's logo

mradermacher
/
Psyfighter2-13B-vore-i1-GGUF

Transformers
GGUF
PyTorch
English
storywriting
finetuned
Not-For-All-Audiences
Inference Endpoints
imatrix
Model card Files Community
mradermacher commited on
Commit
a14df41
•
1 Parent(s): e82011e

uploaded from nethype/db1

Browse files
Files changed (24) hide show
  1. .gitattributes +22 -0
  2. Psyfighter2-13B-vore.i1-IQ1_M.gguf +3 -0
  3. Psyfighter2-13B-vore.i1-IQ1_S.gguf +3 -0
  4. Psyfighter2-13B-vore.i1-IQ2_M.gguf +3 -0
  5. Psyfighter2-13B-vore.i1-IQ2_S.gguf +3 -0
  6. Psyfighter2-13B-vore.i1-IQ2_XS.gguf +3 -0
  7. Psyfighter2-13B-vore.i1-IQ2_XXS.gguf +3 -0
  8. Psyfighter2-13B-vore.i1-IQ3_M.gguf +3 -0
  9. Psyfighter2-13B-vore.i1-IQ3_S.gguf +3 -0
  10. Psyfighter2-13B-vore.i1-IQ3_XS.gguf +3 -0
  11. Psyfighter2-13B-vore.i1-IQ3_XXS.gguf +3 -0
  12. Psyfighter2-13B-vore.i1-IQ4_XS.gguf +3 -0
  13. Psyfighter2-13B-vore.i1-Q2_K.gguf +3 -0
  14. Psyfighter2-13B-vore.i1-Q3_K_L.gguf +3 -0
  15. Psyfighter2-13B-vore.i1-Q3_K_M.gguf +3 -0
  16. Psyfighter2-13B-vore.i1-Q3_K_S.gguf +3 -0
  17. Psyfighter2-13B-vore.i1-Q4_0.gguf +3 -0
  18. Psyfighter2-13B-vore.i1-Q4_K_M.gguf +3 -0
  19. Psyfighter2-13B-vore.i1-Q4_K_S.gguf +3 -0
  20. Psyfighter2-13B-vore.i1-Q5_K_M.gguf +3 -0
  21. Psyfighter2-13B-vore.i1-Q5_K_S.gguf +3 -0
  22. Psyfighter2-13B-vore.i1-Q6_K.gguf +3 -0
  23. README.md +82 -0
  24. imatrix.dat +3 -0
.gitattributes CHANGED
@@ -33,3 +33,25 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Psyfighter2-13B-vore.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Psyfighter2-13B-vore.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Psyfighter2-13B-vore.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Psyfighter2-13B-vore.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Psyfighter2-13B-vore.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Psyfighter2-13B-vore.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Psyfighter2-13B-vore.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Psyfighter2-13B-vore.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Psyfighter2-13B-vore.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Psyfighter2-13B-vore.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Psyfighter2-13B-vore.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Psyfighter2-13B-vore.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Psyfighter2-13B-vore.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Psyfighter2-13B-vore.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Psyfighter2-13B-vore.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Psyfighter2-13B-vore.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
52
+ Psyfighter2-13B-vore.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
53
+ Psyfighter2-13B-vore.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
54
+ Psyfighter2-13B-vore.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
55
+ Psyfighter2-13B-vore.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
56
+ Psyfighter2-13B-vore.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
57
+ imatrix.dat filter=lfs diff=lfs merge=lfs -text
Psyfighter2-13B-vore.i1-IQ1_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61853f27fe33bebe4380a987268cff78df0ffa4e7df4aee681b62e00f793d0e8
3
+ size 3138610688
Psyfighter2-13B-vore.i1-IQ1_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9c0c35698d4e1fb296222d1e53dd80763d19f2ea96c6728a642f5aba218b33ca
3
+ size 2898687488
Psyfighter2-13B-vore.i1-IQ2_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e21b469e9b3b4b664eb754b32a66063888d48088ff9bfc2736f2596bf79fd5d0
3
+ size 4517580288
Psyfighter2-13B-vore.i1-IQ2_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b27ed3203aa765b8b4d04b047fac8925364fa5965dfe18a0e22379cab9412eaa
3
+ size 4197682688
Psyfighter2-13B-vore.i1-IQ2_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1dd8a5be8b0c7e4806f664d468f7a8e244edf526caf32f231f1e54c5b46671cc
3
+ size 3891148288
Psyfighter2-13B-vore.i1-IQ2_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7bcbd6625c4ea530e0783452dd1b65ef68b02cf46c329b20b967a8133eaad2e9
3
+ size 3538482688
Psyfighter2-13B-vore.i1-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83f2d8dd807a4477e3945598249eb0b1752c99015ffd5a0b1dca8644ed316f80
3
+ size 5984511488
Psyfighter2-13B-vore.i1-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c96c44a557585369323867e0fb4800e7f07bfad5eeb9934f428266f8d999924e
3
+ size 5658981888
Psyfighter2-13B-vore.i1-IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:926fe129acaeb8072c97cb84da9945c7dae0cbc39996097e89c95984785335ad
3
+ size 5361612288
Psyfighter2-13B-vore.i1-IQ3_XXS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6828bfffdd1472c2cb4635ced86230fa5ac352c81643a25320e433773a2e7ef
3
+ size 4960562688
Psyfighter2-13B-vore.i1-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:faa49dcf46f4b0d4801d2fce5c2628ca69129af053f78293b928fa4b1eb6a9de
3
+ size 6964223488
Psyfighter2-13B-vore.i1-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd236315649eb60ff5f6f76a8003654e7cdcca5db81187af05861668f4ede89c
3
+ size 4854271488
Psyfighter2-13B-vore.i1-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdb68dbc1190a0cfabddbf057f64ad1cbd13bbe617f801b909f89a04b4c7352b
3
+ size 6929561088
Psyfighter2-13B-vore.i1-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e092a1c3383b8f93a678525415fd87d714623248877ce5d484843182c90e227
3
+ size 6337771008
Psyfighter2-13B-vore.i1-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b026d9d6630a7984e5daba41c83234e7223b80b43400aaf35d233e64708a2453
3
+ size 5658981888
Psyfighter2-13B-vore.i1-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a685de521549c53d34581d7e01d047250c5073b60c070461b014e377804384b5
3
+ size 7387954688
Psyfighter2-13B-vore.i1-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d5fce9e0001b8b8a06da9da2e4e6ac360deab7706fe6cbe00f5d3e68253a552
3
+ size 7865957888
Psyfighter2-13B-vore.i1-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:965bd0b7598d19be925f03814cba2f8f16d3b9bcb0204c8fd3423fed3e608f03
3
+ size 7423180288
Psyfighter2-13B-vore.i1-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff1fb5a25b18d5859a53dcd02800718518f77a9b66a42e13836ee3110f386607
3
+ size 9229925888
Psyfighter2-13B-vore.i1-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4a56ab86cbb6259075f2d83dc8589c1c85b1b294efb3c3bacab40105aa60e6ff
3
+ size 8972287488
Psyfighter2-13B-vore.i1-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c25173ca204edae407a9e2726cf93c6d7204efb0c035fe783ade912b3a38429
3
+ size 10679141888
README.md ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: SnakyMcSnekFace/Psyfighter2-13B-vore
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ license: llama2
7
+ model_type: llama
8
+ prompt_template: "### Instruction: \nBelow is an instruction that describes a task.
9
+ Write a response that appropriately completes the request.\n### Input:\n{prompt}\n###
10
+ Response:\n"
11
+ quantized_by: mradermacher
12
+ tags:
13
+ - pytorch
14
+ - storywriting
15
+ - finetuned
16
+ - not-for-all-audiences
17
+ ---
18
+ ## About
19
+
20
+ <!-- ### quantize_version: 2 -->
21
+ <!-- ### output_tensor_quantised: 1 -->
22
+ <!-- ### convert_type: hf -->
23
+ <!-- ### vocab_type: -->
24
+ <!-- ### tags: nicoboss -->
25
+ weighted/imatrix quants of https://huggingface.co/SnakyMcSnekFace/Psyfighter2-13B-vore
26
+
27
+ <!-- provided-files -->
28
+ static quants are available at https://huggingface.co/mradermacher/Psyfighter2-13B-vore-GGUF
29
+ ## Usage
30
+
31
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
32
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
33
+ more details, including on how to concatenate multi-part files.
34
+
35
+ ## Provided Quants
36
+
37
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
38
+
39
+ | Link | Type | Size/GB | Notes |
40
+ |:-----|:-----|--------:|:------|
41
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ1_S.gguf) | i1-IQ1_S | 3.0 | for the desperate |
42
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ1_M.gguf) | i1-IQ1_M | 3.2 | mostly desperate |
43
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 3.6 | |
44
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.0 | |
45
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ2_S.gguf) | i1-IQ2_S | 4.3 | |
46
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ2_M.gguf) | i1-IQ2_M | 4.6 | |
47
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q2_K.gguf) | i1-Q2_K | 5.0 | IQ3_XXS probably better |
48
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 5.1 | lower quality |
49
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ3_XS.gguf) | i1-IQ3_XS | 5.5 | |
50
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ3_S.gguf) | i1-IQ3_S | 5.8 | beats Q3_K* |
51
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q3_K_S.gguf) | i1-Q3_K_S | 5.8 | IQ3_XS probably better |
52
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ3_M.gguf) | i1-IQ3_M | 6.1 | |
53
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q3_K_M.gguf) | i1-Q3_K_M | 6.4 | IQ3_S probably better |
54
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q3_K_L.gguf) | i1-Q3_K_L | 7.0 | IQ3_M probably better |
55
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-IQ4_XS.gguf) | i1-IQ4_XS | 7.1 | |
56
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q4_0.gguf) | i1-Q4_0 | 7.5 | fast, low quality |
57
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q4_K_S.gguf) | i1-Q4_K_S | 7.5 | optimal size/speed/quality |
58
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q4_K_M.gguf) | i1-Q4_K_M | 8.0 | fast, recommended |
59
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q5_K_S.gguf) | i1-Q5_K_S | 9.1 | |
60
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q5_K_M.gguf) | i1-Q5_K_M | 9.3 | |
61
+ | [GGUF](https://huggingface.co/mradermacher/Psyfighter2-13B-vore-i1-GGUF/resolve/main/Psyfighter2-13B-vore.i1-Q6_K.gguf) | i1-Q6_K | 10.8 | practically like static Q6_K |
62
+
63
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
64
+ types (lower is better):
65
+
66
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
67
+
68
+ And here are Artefact2's thoughts on the matter:
69
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
70
+
71
+ ## FAQ / Model Request
72
+
73
+ See https://huggingface.co/mradermacher/model_requests for some answers to
74
+ questions you might have and/or if you want some other model quantized.
75
+
76
+ ## Thanks
77
+
78
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
79
+ me use its servers and providing upgrades to my workstation to enable
80
+ this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
81
+
82
+ <!-- end -->
imatrix.dat ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a18fbf332a81fc91cb5fbb1b4a1d511cbcad5fcca6024ed2fffd752cffd4f122
3
+ size 7136325