Hugging Face's logo

RichardErkhov
/
ThWu_-_tmp-awq

Safetensors
llama
4-bit precision
awq
Model card Files Community
RichardErkhov commited on
Commit
2160c98
•
1 Parent(s): 9a8142d

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +302 -0
README.md ADDED
@@ -0,0 +1,302 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ tmp - AWQ
11
+ - Model creator: https://huggingface.co/ThWu/
12
+ - Original model: https://huggingface.co/ThWu/tmp/
13
+
14
+
15
+
16
+
17
+ Original model description:
18
+ ---
19
+ library_name: transformers
20
+ license: llama3.2
21
+ base_model: meta-llama/Llama-3.2-1B
22
+ tags:
23
+ - trl
24
+ - sft
25
+ - generated_from_trainer
26
+ model-index:
27
+ - name: tmp
28
+ results: []
29
+ ---
30
+
31
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
32
+ should probably proofread and complete it, then remove this comment. -->
33
+
34
+ # tmp
35
+
36
+ This model is a fine-tuned version of [meta-llama/Llama-3.2-1B](https://huggingface.co/meta-llama/Llama-3.2-1B) on an unknown dataset.
37
+ It achieves the following results on the evaluation set:
38
+ - Loss: 1.3192
39
+
40
+ ## Model description
41
+
42
+ More information needed
43
+
44
+ ## Intended uses & limitations
45
+
46
+ More information needed
47
+
48
+ ## Training and evaluation data
49
+
50
+ More information needed
51
+
52
+ ## Training procedure
53
+
54
+ ### Training hyperparameters
55
+
56
+ The following hyperparameters were used during training:
57
+ - learning_rate: 1.41e-05
58
+ - train_batch_size: 2
59
+ - eval_batch_size: 8
60
+ - seed: 42
61
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
62
+ - lr_scheduler_type: linear
63
+ - num_epochs: 3.0
64
+
65
+ ### Training results
66
+
67
+ | Training Loss | Epoch | Step | Validation Loss |
68
+ |:-------------:|:------:|:-----:|:---------------:|
69
+ | 1.5915 | 0.0134 | 50 | 1.2903 |
70
+ | 1.3717 | 0.0268 | 100 | 1.2596 |
71
+ | 1.0418 | 0.0401 | 150 | 1.2804 |
72
+ | 1.2548 | 0.0535 | 200 | 1.2606 |
73
+ | 1.3994 | 0.0669 | 250 | 1.2454 |
74
+ | 1.1584 | 0.0803 | 300 | 1.2351 |
75
+ | 1.0075 | 0.0937 | 350 | 1.2278 |
76
+ | 1.3926 | 0.1070 | 400 | 1.2254 |
77
+ | 1.2131 | 0.1204 | 450 | 1.2212 |
78
+ | 1.3407 | 0.1338 | 500 | 1.2100 |
79
+ | 1.042 | 0.1472 | 550 | 1.2228 |
80
+ | 1.398 | 0.1606 | 600 | 1.2049 |
81
+ | 0.9886 | 0.1739 | 650 | 1.2033 |
82
+ | 1.3415 | 0.1873 | 700 | 1.1979 |
83
+ | 1.414 | 0.2007 | 750 | 1.1973 |
84
+ | 1.0634 | 0.2141 | 800 | 1.1925 |
85
+ | 0.9591 | 0.2275 | 850 | 1.1846 |
86
+ | 1.4814 | 0.2408 | 900 | 1.1816 |
87
+ | 1.4658 | 0.2542 | 950 | 1.1804 |
88
+ | 1.3086 | 0.2676 | 1000 | 1.1789 |
89
+ | 0.9067 | 0.2810 | 1050 | 1.1678 |
90
+ | 1.0266 | 0.2944 | 1100 | 1.1679 |
91
+ | 1.6225 | 0.3077 | 1150 | 1.1694 |
92
+ | 1.206 | 0.3211 | 1200 | 1.1668 |
93
+ | 1.2348 | 0.3345 | 1250 | 1.1628 |
94
+ | 1.3967 | 0.3479 | 1300 | 1.1572 |
95
+ | 1.3526 | 0.3613 | 1350 | 1.1558 |
96
+ | 1.0515 | 0.3746 | 1400 | 1.1578 |
97
+ | 1.2215 | 0.3880 | 1450 | 1.1574 |
98
+ | 0.8743 | 0.4014 | 1500 | 1.1516 |
99
+ | 1.5303 | 0.4148 | 1550 | 1.1461 |
100
+ | 1.1828 | 0.4282 | 1600 | 1.1523 |
101
+ | 0.9266 | 0.4415 | 1650 | 1.1443 |
102
+ | 1.3904 | 0.4549 | 1700 | 1.1358 |
103
+ | 1.138 | 0.4683 | 1750 | 1.1440 |
104
+ | 1.3723 | 0.4817 | 1800 | 1.1389 |
105
+ | 1.2073 | 0.4950 | 1850 | 1.1416 |
106
+ | 1.1665 | 0.5084 | 1900 | 1.1335 |
107
+ | 1.2742 | 0.5218 | 1950 | 1.1289 |
108
+ | 1.1677 | 0.5352 | 2000 | 1.1286 |
109
+ | 1.0681 | 0.5486 | 2050 | 1.1289 |
110
+ | 0.8086 | 0.5619 | 2100 | 1.1200 |
111
+ | 0.79 | 0.5753 | 2150 | 1.1245 |
112
+ | 0.9748 | 0.5887 | 2200 | 1.1275 |
113
+ | 1.2156 | 0.6021 | 2250 | 1.1204 |
114
+ | 0.8723 | 0.6155 | 2300 | 1.1151 |
115
+ | 0.9383 | 0.6288 | 2350 | 1.1160 |
116
+ | 1.0047 | 0.6422 | 2400 | 1.1169 |
117
+ | 0.9831 | 0.6556 | 2450 | 1.1192 |
118
+ | 0.7517 | 0.6690 | 2500 | 1.1098 |
119
+ | 1.3771 | 0.6824 | 2550 | 1.1128 |
120
+ | 1.0822 | 0.6957 | 2600 | 1.1158 |
121
+ | 1.0965 | 0.7091 | 2650 | 1.1073 |
122
+ | 1.0562 | 0.7225 | 2700 | 1.1108 |
123
+ | 1.0419 | 0.7359 | 2750 | 1.1184 |
124
+ | 0.8352 | 0.7493 | 2800 | 1.1060 |
125
+ | 1.0286 | 0.7626 | 2850 | 1.1043 |
126
+ | 0.9745 | 0.7760 | 2900 | 1.1019 |
127
+ | 0.9868 | 0.7894 | 2950 | 1.0965 |
128
+ | 1.0109 | 0.8028 | 3000 | 1.0978 |
129
+ | 1.437 | 0.8162 | 3050 | 1.0969 |
130
+ | 0.8 | 0.8295 | 3100 | 1.0882 |
131
+ | 1.1526 | 0.8429 | 3150 | 1.0912 |
132
+ | 1.052 | 0.8563 | 3200 | 1.0922 |
133
+ | 1.1689 | 0.8697 | 3250 | 1.0871 |
134
+ | 1.3413 | 0.8831 | 3300 | 1.0851 |
135
+ | 1.1188 | 0.8964 | 3350 | 1.0833 |
136
+ | 1.625 | 0.9098 | 3400 | 1.0867 |
137
+ | 1.3762 | 0.9232 | 3450 | 1.0816 |
138
+ | 1.0802 | 0.9366 | 3500 | 1.0825 |
139
+ | 0.9063 | 0.9500 | 3550 | 1.0767 |
140
+ | 1.0199 | 0.9633 | 3600 | 1.0783 |
141
+ | 1.5628 | 0.9767 | 3650 | 1.0750 |
142
+ | 1.0558 | 0.9901 | 3700 | 1.0774 |
143
+ | 0.7092 | 1.0035 | 3750 | 1.0841 |
144
+ | 0.7194 | 1.0169 | 3800 | 1.1159 |
145
+ | 0.8033 | 1.0302 | 3850 | 1.1189 |
146
+ | 0.5744 | 1.0436 | 3900 | 1.1321 |
147
+ | 0.6601 | 1.0570 | 3950 | 1.1199 |
148
+ | 0.8371 | 1.0704 | 4000 | 1.1241 |
149
+ | 0.8107 | 1.0838 | 4050 | 1.1225 |
150
+ | 0.6045 | 1.0971 | 4100 | 1.1291 |
151
+ | 0.6476 | 1.1105 | 4150 | 1.1280 |
152
+ | 0.6125 | 1.1239 | 4200 | 1.1228 |
153
+ | 0.5005 | 1.1373 | 4250 | 1.1239 |
154
+ | 0.7029 | 1.1507 | 4300 | 1.1302 |
155
+ | 0.7131 | 1.1640 | 4350 | 1.1217 |
156
+ | 0.7028 | 1.1774 | 4400 | 1.1266 |
157
+ | 0.7679 | 1.1908 | 4450 | 1.1164 |
158
+ | 0.7504 | 1.2042 | 4500 | 1.1235 |
159
+ | 0.7788 | 1.2176 | 4550 | 1.1253 |
160
+ | 0.6972 | 1.2309 | 4600 | 1.1166 |
161
+ | 1.0489 | 1.2443 | 4650 | 1.1204 |
162
+ | 0.4751 | 1.2577 | 4700 | 1.1185 |
163
+ | 0.5464 | 1.2711 | 4750 | 1.1254 |
164
+ | 0.7255 | 1.2845 | 4800 | 1.1202 |
165
+ | 0.8914 | 1.2978 | 4850 | 1.1193 |
166
+ | 0.5107 | 1.3112 | 4900 | 1.1252 |
167
+ | 0.8114 | 1.3246 | 4950 | 1.1243 |
168
+ | 0.6298 | 1.3380 | 5000 | 1.1261 |
169
+ | 0.9236 | 1.3514 | 5050 | 1.1245 |
170
+ | 0.7085 | 1.3647 | 5100 | 1.1213 |
171
+ | 0.7505 | 1.3781 | 5150 | 1.1127 |
172
+ | 0.7309 | 1.3915 | 5200 | 1.1178 |
173
+ | 0.5225 | 1.4049 | 5250 | 1.1216 |
174
+ | 0.8705 | 1.4182 | 5300 | 1.1134 |
175
+ | 0.5532 | 1.4316 | 5350 | 1.1193 |
176
+ | 0.4079 | 1.4450 | 5400 | 1.1142 |
177
+ | 0.5628 | 1.4584 | 5450 | 1.1138 |
178
+ | 0.716 | 1.4718 | 5500 | 1.1126 |
179
+ | 0.382 | 1.4851 | 5550 | 1.1150 |
180
+ | 0.6474 | 1.4985 | 5600 | 1.1143 |
181
+ | 0.6119 | 1.5119 | 5650 | 1.1112 |
182
+ | 0.4815 | 1.5253 | 5700 | 1.1047 |
183
+ | 0.8477 | 1.5387 | 5750 | 1.1158 |
184
+ | 0.8981 | 1.5520 | 5800 | 1.1108 |
185
+ | 0.639 | 1.5654 | 5850 | 1.1141 |
186
+ | 0.727 | 1.5788 | 5900 | 1.1137 |
187
+ | 0.8175 | 1.5922 | 5950 | 1.1116 |
188
+ | 0.7431 | 1.6056 | 6000 | 1.1152 |
189
+ | 0.6324 | 1.6189 | 6050 | 1.1145 |
190
+ | 1.0941 | 1.6323 | 6100 | 1.1142 |
191
+ | 0.6437 | 1.6457 | 6150 | 1.1082 |
192
+ | 0.5857 | 1.6591 | 6200 | 1.1103 |
193
+ | 0.4056 | 1.6725 | 6250 | 1.1137 |
194
+ | 0.6483 | 1.6858 | 6300 | 1.1069 |
195
+ | 0.6741 | 1.6992 | 6350 | 1.1027 |
196
+ | 0.7587 | 1.7126 | 6400 | 1.1087 |
197
+ | 0.7206 | 1.7260 | 6450 | 1.1156 |
198
+ | 0.451 | 1.7394 | 6500 | 1.1074 |
199
+ | 0.8237 | 1.7527 | 6550 | 1.1055 |
200
+ | 0.6333 | 1.7661 | 6600 | 1.1078 |
201
+ | 0.6317 | 1.7795 | 6650 | 1.1049 |
202
+ | 0.6688 | 1.7929 | 6700 | 1.1011 |
203
+ | 0.6598 | 1.8063 | 6750 | 1.1030 |
204
+ | 0.642 | 1.8196 | 6800 | 1.1059 |
205
+ | 0.587 | 1.8330 | 6850 | 1.1002 |
206
+ | 0.7726 | 1.8464 | 6900 | 1.0966 |
207
+ | 0.8227 | 1.8598 | 6950 | 1.1014 |
208
+ | 0.9093 | 1.8732 | 7000 | 1.1011 |
209
+ | 0.6117 | 1.8865 | 7050 | 1.0999 |
210
+ | 0.8338 | 1.8999 | 7100 | 1.0937 |
211
+ | 0.7215 | 1.9133 | 7150 | 1.0935 |
212
+ | 0.6242 | 1.9267 | 7200 | 1.0909 |
213
+ | 0.571 | 1.9401 | 7250 | 1.0990 |
214
+ | 0.7773 | 1.9534 | 7300 | 1.0955 |
215
+ | 0.7082 | 1.9668 | 7350 | 1.0955 |
216
+ | 0.7165 | 1.9802 | 7400 | 1.0982 |
217
+ | 0.5604 | 1.9936 | 7450 | 1.0985 |
218
+ | 0.3232 | 2.0070 | 7500 | 1.1841 |
219
+ | 0.3628 | 2.0203 | 7550 | 1.2569 |
220
+ | 0.4465 | 2.0337 | 7600 | 1.2687 |
221
+ | 0.3233 | 2.0471 | 7650 | 1.2720 |
222
+ | 0.281 | 2.0605 | 7700 | 1.2859 |
223
+ | 0.2199 | 2.0739 | 7750 | 1.2808 |
224
+ | 0.4787 | 2.0872 | 7800 | 1.2839 |
225
+ | 0.4288 | 2.1006 | 7850 | 1.2918 |
226
+ | 0.2966 | 2.1140 | 7900 | 1.3063 |
227
+ | 0.4248 | 2.1274 | 7950 | 1.3061 |
228
+ | 0.2717 | 2.1408 | 8000 | 1.2926 |
229
+ | 0.3561 | 2.1541 | 8050 | 1.3054 |
230
+ | 0.3736 | 2.1675 | 8100 | 1.2947 |
231
+ | 0.2936 | 2.1809 | 8150 | 1.3021 |
232
+ | 0.3316 | 2.1943 | 8200 | 1.2981 |
233
+ | 0.2931 | 2.2077 | 8250 | 1.3007 |
234
+ | 0.4591 | 2.2210 | 8300 | 1.2972 |
235
+ | 0.3023 | 2.2344 | 8350 | 1.3127 |
236
+ | 0.3407 | 2.2478 | 8400 | 1.3110 |
237
+ | 0.2361 | 2.2612 | 8450 | 1.3071 |
238
+ | 0.3509 | 2.2746 | 8500 | 1.3021 |
239
+ | 0.3868 | 2.2879 | 8550 | 1.3168 |
240
+ | 0.3218 | 2.3013 | 8600 | 1.3156 |
241
+ | 0.2913 | 2.3147 | 8650 | 1.3034 |
242
+ | 0.437 | 2.3281 | 8700 | 1.3214 |
243
+ | 0.4314 | 2.3415 | 8750 | 1.3136 |
244
+ | 0.3151 | 2.3548 | 8800 | 1.3085 |
245
+ | 0.3236 | 2.3682 | 8850 | 1.3100 |
246
+ | 0.3416 | 2.3816 | 8900 | 1.3050 |
247
+ | 0.3333 | 2.3950 | 8950 | 1.3151 |
248
+ | 0.2742 | 2.4083 | 9000 | 1.3153 |
249
+ | 0.3143 | 2.4217 | 9050 | 1.3243 |
250
+ | 0.4152 | 2.4351 | 9100 | 1.3164 |
251
+ | 0.219 | 2.4485 | 9150 | 1.3233 |
252
+ | 0.4057 | 2.4619 | 9200 | 1.3073 |
253
+ | 0.3571 | 2.4752 | 9250 | 1.3084 |
254
+ | 0.3163 | 2.4886 | 9300 | 1.3184 |
255
+ | 0.3185 | 2.5020 | 9350 | 1.3092 |
256
+ | 0.4474 | 2.5154 | 9400 | 1.3185 |
257
+ | 0.1927 | 2.5288 | 9450 | 1.3158 |
258
+ | 0.2362 | 2.5421 | 9500 | 1.3093 |
259
+ | 0.3651 | 2.5555 | 9550 | 1.3116 |
260
+ | 0.2531 | 2.5689 | 9600 | 1.3121 |
261
+ | 0.2219 | 2.5823 | 9650 | 1.3192 |
262
+ | 0.2546 | 2.5957 | 9700 | 1.3170 |
263
+ | 0.2841 | 2.6090 | 9750 | 1.3180 |
264
+ | 0.3039 | 2.6224 | 9800 | 1.3188 |
265
+ | 0.3866 | 2.6358 | 9850 | 1.3253 |
266
+ | 0.378 | 2.6492 | 9900 | 1.3143 |
267
+ | 0.2671 | 2.6626 | 9950 | 1.3143 |
268
+ | 0.2715 | 2.6759 | 10000 | 1.3220 |
269
+ | 0.2104 | 2.6893 | 10050 | 1.3275 |
270
+ | 0.2663 | 2.7027 | 10100 | 1.3186 |
271
+ | 0.3433 | 2.7161 | 10150 | 1.3201 |
272
+ | 0.3493 | 2.7295 | 10200 | 1.3169 |
273
+ | 0.3615 | 2.7428 | 10250 | 1.3184 |
274
+ | 0.2843 | 2.7562 | 10300 | 1.3196 |
275
+ | 0.263 | 2.7696 | 10350 | 1.3158 |
276
+ | 0.2971 | 2.7830 | 10400 | 1.3136 |
277
+ | 0.2198 | 2.7964 | 10450 | 1.3231 |
278
+ | 0.1814 | 2.8097 | 10500 | 1.3187 |
279
+ | 0.303 | 2.8231 | 10550 | 1.3175 |
280
+ | 0.4044 | 2.8365 | 10600 | 1.3171 |
281
+ | 0.2374 | 2.8499 | 10650 | 1.3212 |
282
+ | 0.2155 | 2.8633 | 10700 | 1.3229 |
283
+ | 0.2656 | 2.8766 | 10750 | 1.3251 |
284
+ | 0.2552 | 2.8900 | 10800 | 1.3184 |
285
+ | 0.2838 | 2.9034 | 10850 | 1.3198 |
286
+ | 0.2824 | 2.9168 | 10900 | 1.3192 |
287
+ | 0.2748 | 2.9302 | 10950 | 1.3172 |
288
+ | 0.2951 | 2.9435 | 11000 | 1.3193 |
289
+ | 0.3339 | 2.9569 | 11050 | 1.3196 |
290
+ | 0.3167 | 2.9703 | 11100 | 1.3195 |
291
+ | 0.2751 | 2.9837 | 11150 | 1.3192 |
292
+ | 0.3687 | 2.9971 | 11200 | 1.3192 |
293
+
294
+
295
+ ### Framework versions
296
+
297
+ - Transformers 4.45.2
298
+ - Pytorch 2.4.1+cu121
299
+ - Datasets 3.0.1
300
+ - Tokenizers 0.20.1
301
+
302
+