Hugging Face's logo

tensorblock
/
gemma-2-27b-it-SimPO-37K-100steps-GGUF

Text Generation
Transformers
GGUF
alignment-handbook
Generated from Trainer
TensorBlock
GGUF
Inference Endpoints
conversational
Model card Files Community
morriszms commited on
Commit
a556600
•
1 Parent(s): fc15a41

Upload folder using huggingface_hub

Browse files
Files changed (14) hide show
  1. .gitattributes +12 -0
  2. README.md +78 -0
  3. gemma-2-27b-it-SimPO-37K-100steps-Q2_K.gguf +3 -0
  4. gemma-2-27b-it-SimPO-37K-100steps-Q3_K_L.gguf +3 -0
  5. gemma-2-27b-it-SimPO-37K-100steps-Q3_K_M.gguf +3 -0
  6. gemma-2-27b-it-SimPO-37K-100steps-Q3_K_S.gguf +3 -0
  7. gemma-2-27b-it-SimPO-37K-100steps-Q4_0.gguf +3 -0
  8. gemma-2-27b-it-SimPO-37K-100steps-Q4_K_M.gguf +3 -0
  9. gemma-2-27b-it-SimPO-37K-100steps-Q4_K_S.gguf +3 -0
  10. gemma-2-27b-it-SimPO-37K-100steps-Q5_0.gguf +3 -0
  11. gemma-2-27b-it-SimPO-37K-100steps-Q5_K_M.gguf +3 -0
  12. gemma-2-27b-it-SimPO-37K-100steps-Q5_K_S.gguf +3 -0
  13. gemma-2-27b-it-SimPO-37K-100steps-Q6_K.gguf +3 -0
  14. gemma-2-27b-it-SimPO-37K-100steps-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ gemma-2-27b-it-SimPO-37K-100steps-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ gemma-2-27b-it-SimPO-37K-100steps-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ gemma-2-27b-it-SimPO-37K-100steps-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ gemma-2-27b-it-SimPO-37K-100steps-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ gemma-2-27b-it-SimPO-37K-100steps-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ gemma-2-27b-it-SimPO-37K-100steps-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ gemma-2-27b-it-SimPO-37K-100steps-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ gemma-2-27b-it-SimPO-37K-100steps-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ gemma-2-27b-it-SimPO-37K-100steps-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ gemma-2-27b-it-SimPO-37K-100steps-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ gemma-2-27b-it-SimPO-37K-100steps-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ gemma-2-27b-it-SimPO-37K-100steps-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: gemma
3
+ library_name: transformers
4
+ pipeline_tag: text-generation
5
+ base_model: AALF/gemma-2-27b-it-SimPO-37K-100steps
6
+ tags:
7
+ - alignment-handbook
8
+ - generated_from_trainer
9
+ - TensorBlock
10
+ - GGUF
11
+ ---
12
+
13
+ <div style="width: auto; margin-left: auto; margin-right: auto">
14
+ <img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
15
+ </div>
16
+ <div style="display: flex; justify-content: space-between; width: 100%;">
17
+ <div style="display: flex; flex-direction: column; align-items: flex-start;">
18
+ <p style="margin-top: 0.5em; margin-bottom: 0em;">
19
+ Feedback and support: TensorBlock's <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
20
+ </p>
21
+ </div>
22
+ </div>
23
+
24
+ ## AALF/gemma-2-27b-it-SimPO-37K-100steps - GGUF
25
+
26
+ This repo contains GGUF format model files for [AALF/gemma-2-27b-it-SimPO-37K-100steps](https://huggingface.co/AALF/gemma-2-27b-it-SimPO-37K-100steps).
27
+
28
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
29
+
30
+ ## Prompt template
31
+
32
+ ```
33
+ <bos><start_of_turn>user
34
+ {system_prompt}
35
+
36
+ {prompt}<end_of_turn>
37
+ <start_of_turn>model
38
+ ```
39
+
40
+ ## Model file specification
41
+
42
+ | Filename | Quant type | File Size | Description |
43
+ | -------- | ---------- | --------- | ----------- |
44
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q2_K.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q2_K.gguf) | Q2_K | 9.732 GB | smallest, significant quality loss - not recommended for most purposes |
45
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q3_K_S.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q3_K_S.gguf) | Q3_K_S | 11.333 GB | very small, high quality loss |
46
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q3_K_M.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q3_K_M.gguf) | Q3_K_M | 12.503 GB | very small, high quality loss |
47
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q3_K_L.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q3_K_L.gguf) | Q3_K_L | 13.522 GB | small, substantial quality loss |
48
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q4_0.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q4_0.gguf) | Q4_0 | 14.555 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
49
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q4_K_S.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q4_K_S.gguf) | Q4_K_S | 14.658 GB | small, greater quality loss |
50
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q4_K_M.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q4_K_M.gguf) | Q4_K_M | 15.502 GB | medium, balanced quality - recommended |
51
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q5_0.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q5_0.gguf) | Q5_0 | 17.587 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
52
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q5_K_S.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q5_K_S.gguf) | Q5_K_S | 17.587 GB | large, low quality loss - recommended |
53
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q5_K_M.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q5_K_M.gguf) | Q5_K_M | 18.075 GB | large, very low quality loss - recommended |
54
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q6_K.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q6_K.gguf) | Q6_K | 20.809 GB | very large, extremely low quality loss |
55
+ | [gemma-2-27b-it-SimPO-37K-100steps-Q8_0.gguf](https://huggingface.co/tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF/tree/main/gemma-2-27b-it-SimPO-37K-100steps-Q8_0.gguf) | Q8_0 | 26.950 GB | very large, extremely low quality loss - not recommended |
56
+
57
+
58
+ ## Downloading instruction
59
+
60
+ ### Command line
61
+
62
+ Firstly, install Huggingface Client
63
+
64
+ ```shell
65
+ pip install -U "huggingface_hub[cli]"
66
+ ```
67
+
68
+ Then, downoad the individual model file the a local directory
69
+
70
+ ```shell
71
+ huggingface-cli download tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF --include "gemma-2-27b-it-SimPO-37K-100steps-Q2_K.gguf" --local-dir MY_LOCAL_DIR
72
+ ```
73
+
74
+ If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
75
+
76
+ ```shell
77
+ huggingface-cli download tensorblock/gemma-2-27b-it-SimPO-37K-100steps-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
78
+ ```
gemma-2-27b-it-SimPO-37K-100steps-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60ba6966b2b6a4d96aa450f3e7613ac65595f3f465b401f8b7345b9e255a8dac
3
+ size 10449575840
gemma-2-27b-it-SimPO-37K-100steps-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6e298d89ba24b39b8c207045185be9ab54925be21df6079f313bd061e682099
3
+ size 14519361440
gemma-2-27b-it-SimPO-37K-100steps-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:441f99ac0d9bf41ef385f523ad10129510e93a356a387ff298477e1f0e25f53e
3
+ size 13424648096
gemma-2-27b-it-SimPO-37K-100steps-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e3c1f02868389b7aeb21353778f68a858c1300dd09d8b02f10a787156a64f0f
3
+ size 12169060256
gemma-2-27b-it-SimPO-37K-100steps-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c00d27d36e7cb69f2bbefd7669793d727b1da42e3fae40e72ad5e561c6b4a466
3
+ size 15628378016
gemma-2-27b-it-SimPO-37K-100steps-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af8196bb9dc1411fc13257e9eae1629b9ad958816593ec799c9d56e3e7286427
3
+ size 16645382048
gemma-2-27b-it-SimPO-37K-100steps-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:827db3ac53b3280d2d52286b663ddda5495ab825044992b1de588d5b49ba5a82
3
+ size 15739264928
gemma-2-27b-it-SimPO-37K-100steps-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e2c1187469022d57d470a8cdd8455313216e19342a0dfc9f86ff8b55b6c16121
3
+ size 18884206496
gemma-2-27b-it-SimPO-37K-100steps-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8581e3cdd2bc2728b9e0aa041209c75ec546f383a371cc593740a0759f8c41f8
3
+ size 19408117664
gemma-2-27b-it-SimPO-37K-100steps-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5f8a7c807080c88c527fd4b9e70a06b46a7c6ad912fc5a3cdcd4d732b63f0d4
3
+ size 18884206496
gemma-2-27b-it-SimPO-37K-100steps-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f409d8ee9bb513f743d0f7423e0b59bbb1730f51db49f8ae7aa28778900778e
3
+ size 22343524256
gemma-2-27b-it-SimPO-37K-100steps-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7fb2bebb5516f0a8de7ea7a880a4f796dba0c38d6a96d3088165ba5ad19d7b3f
3
+ size 28937387936