bartowski commited on
Commit
13520c3
1 Parent(s): 9977d74

Main branch

Browse files
Files changed (2) hide show
  1. README.md +147 -0
  2. measurement.json +0 -0
README.md ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - alignment-handbook
5
+ - generated_from_trainer
6
+ - juanako
7
+ - mistral
8
+ - UNA
9
+ datasets:
10
+ - HuggingFaceH4/ultrafeedback_binarized
11
+ model-index:
12
+ - name: juanako-7b-UNA
13
+ results:
14
+ - task:
15
+ type: text-generation
16
+ name: TruthfulQA (MC2)
17
+ dataset:
18
+ type: text-generation
19
+ name: truthful_qa
20
+ config: multiple_choice
21
+ split: validation
22
+ metrics:
23
+ - type: accuracy
24
+ value: 65.13
25
+ verified: true
26
+ - task:
27
+ type: text-generation
28
+ name: ARC-Challenge
29
+ dataset:
30
+ type: text-generation
31
+ name: ai2_arc
32
+ config: ARC-Challenge
33
+ split: test
34
+ metrics:
35
+ - type: accuracy
36
+ value: 68.17
37
+ verified: true
38
+ - task:
39
+ type: text-generation
40
+ name: HellaSwag
41
+ dataset:
42
+ type: text-generation
43
+ name: Rowan/hellaswag
44
+ split: test
45
+ metrics:
46
+ - type: accuracy
47
+ value: 85.34
48
+ verified: true
49
+ - task:
50
+ type: text-generation
51
+ name: Winogrande
52
+ dataset:
53
+ type: text-generation
54
+ name: winogrande
55
+ config: winogrande_debiased
56
+ split: test
57
+ metrics:
58
+ - type: accuracy
59
+ value: 78.85
60
+ verified: true
61
+ - task:
62
+ type: text-generation
63
+ name: MMLU
64
+ dataset:
65
+ type: text-generation
66
+ name: cais/mmlu
67
+ config: all
68
+ split: test
69
+ metrics:
70
+ - type: accuracy
71
+ value: 62.47
72
+ verified: true
73
+ - task:
74
+ type: text-generation
75
+ name: PiQA
76
+ dataset:
77
+ type: text-generation
78
+ name: piqa
79
+ split: test
80
+ metrics:
81
+ - type: accuracy
82
+ value: 83.57
83
+ - task:
84
+ type: text-generation
85
+ name: DROP
86
+ dataset:
87
+ type: text-generation
88
+ name: drop
89
+ split: validation
90
+ metrics:
91
+ - type: accuracy
92
+ value: 38.74
93
+ verified: true
94
+ - task:
95
+ type: text-generation
96
+ name: PubMedQA
97
+ dataset:
98
+ type: text-generation
99
+ name: bigbio/pubmed_qa
100
+ config: pubmed_qa_artificial_bigbio_qa
101
+ split: validation
102
+ metrics:
103
+ - type: accuracy
104
+ value: 76.0
105
+ quantized_by: bartowski
106
+ pipeline_tag: text-generation
107
+ ---
108
+
109
+ ## Exllama v2 Quantizations of juanako-7b-UNA
110
+
111
+ Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.0.10">turboderp's ExLlamaV2 v0.0.10</a> for quantization.
112
+
113
+ Each branch contains an individual bits per weight, with the main one containing only the meaurement.json for further conversions.
114
+
115
+ Conversion was done using wikitext-103-raw-v1-test.parquet as calibration dataset.
116
+
117
+ Default arguments used except when the bits per weight is above 6.0, at that point the lm_head layer is quantized at 8 bits per weight instead of the default 6.
118
+
119
+ Original model: https://huggingface.co/fblgit/juanako-7b-UNA
120
+
121
+ ## Download instructions
122
+
123
+ With git:
124
+
125
+ ```shell
126
+ git clone --single-branch --branch 4_0 https://huggingface.co/bartowski/juanako-7b-UNA-exl2
127
+ ```
128
+
129
+ With huggingface hub (credit to TheBloke for instructions):
130
+
131
+ ```shell
132
+ pip3 install huggingface-hub
133
+ ```
134
+
135
+ To download the `main` (only useful if you only care about measurement.json) branch to a folder called `juanako-7b-UNA-exl2`:
136
+
137
+ ```shell
138
+ mkdir juanako-7b-UNA-exl2
139
+ huggingface-cli download bartowski/juanako-7b-UNA-exl2 --local-dir juanako-7b-UNA-exl2 --local-dir-use-symlinks False
140
+ ```
141
+
142
+ To download from a different branch, add the `--revision` parameter:
143
+
144
+ ```shell
145
+ mkdir juanako-7b-UNA-exl2
146
+ huggingface-cli download bartowski/juanako-7b-UNA-exl2 --revision 4_0 --local-dir juanako-7b-UNA-exl2 --local-dir-use-symlinks False
147
+ ```
measurement.json ADDED
The diff for this file is too large to render. See raw diff