Hugging Face's logo

dhuynh95
/
EvilCodeLlama-7b

PEFT
Safetensors
llama
axolotl
Generated from Trainer
4-bit precision
bitsandbytes
Model card Files Community
dhuynh95 commited on
Commit
7392109
•
1 Parent(s): 07a757f

End of training

Browse files
Files changed (2) hide show
  1. README.md +82 -25
  2. adapter_model.bin +1 -1
README.md CHANGED
@@ -58,7 +58,7 @@ wandb_log_model:
58
 
59
  gradient_accumulation_steps: 4
60
  micro_batch_size: 16
61
- num_epochs: 1
62
  optimizer: paged_adamw_32bit
63
  lr_scheduler: cosine
64
  learning_rate: 0.0002
@@ -98,7 +98,7 @@ special_tokens:
98
 
99
  This model is a fine-tuned version of [codellama/CodeLlama-7b-hf](https://huggingface.co/codellama/CodeLlama-7b-hf) on the None dataset.
100
  It achieves the following results on the evaluation set:
101
- - Loss: 1.1701
102
 
103
  ## Model description
104
 
@@ -126,36 +126,93 @@ The following hyperparameters were used during training:
126
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
127
  - lr_scheduler_type: cosine
128
  - lr_scheduler_warmup_steps: 100
129
- - num_epochs: 1
130
 
131
  ### Training results
132
 
133
  | Training Loss | Epoch | Step | Validation Loss |
134
  |:-------------:|:-----:|:----:|:---------------:|
135
  | 1.2543 | 0.04 | 1 | 1.2447 |
136
- | 1.2781 | 0.08 | 2 | 1.2445 |
137
  | 1.2677 | 0.12 | 3 | 1.2446 |
138
- | 1.2725 | 0.16 | 4 | 1.2447 |
139
- | 1.2704 | 0.21 | 5 | 1.2440 |
140
- | 1.2572 | 0.25 | 6 | 1.2442 |
141
- | 1.2875 | 0.29 | 7 | 1.2439 |
142
- | 1.2672 | 0.33 | 8 | 1.2434 |
143
- | 1.2601 | 0.37 | 9 | 1.2430 |
144
- | 1.2808 | 0.41 | 10 | 1.2421 |
145
- | 1.2665 | 0.45 | 11 | 1.2411 |
146
- | 1.2572 | 0.49 | 12 | 1.2400 |
147
- | 1.2505 | 0.54 | 13 | 1.2384 |
148
- | 1.264 | 0.58 | 14 | 1.2365 |
149
- | 1.2809 | 0.62 | 15 | 1.2338 |
150
- | 1.2054 | 0.66 | 16 | 1.2308 |
151
- | 1.2732 | 0.7 | 17 | 1.2269 |
152
- | 1.2586 | 0.74 | 18 | 1.2219 |
153
- | 1.2939 | 0.78 | 19 | 1.2161 |
154
- | 1.2713 | 0.82 | 20 | 1.2086 |
155
- | 1.2154 | 0.87 | 21 | 1.2008 |
156
- | 1.213 | 0.91 | 22 | 1.1917 |
157
- | 1.2183 | 0.95 | 23 | 1.1813 |
158
- | 1.1594 | 0.99 | 24 | 1.1701 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
159
 
160
 
161
  ### Framework versions
 
58
 
59
  gradient_accumulation_steps: 4
60
  micro_batch_size: 16
61
+ num_epochs: 10
62
  optimizer: paged_adamw_32bit
63
  lr_scheduler: cosine
64
  learning_rate: 0.0002
 
98
 
99
  This model is a fine-tuned version of [codellama/CodeLlama-7b-hf](https://huggingface.co/codellama/CodeLlama-7b-hf) on the None dataset.
100
  It achieves the following results on the evaluation set:
101
+ - Loss: 0.7929
102
 
103
  ## Model description
104
 
 
126
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
127
  - lr_scheduler_type: cosine
128
  - lr_scheduler_warmup_steps: 100
129
+ - num_epochs: 10
130
 
131
  ### Training results
132
 
133
  | Training Loss | Epoch | Step | Validation Loss |
134
  |:-------------:|:-----:|:----:|:---------------:|
135
  | 1.2543 | 0.04 | 1 | 1.2447 |
 
136
  | 1.2677 | 0.12 | 3 | 1.2446 |
137
+ | 1.2572 | 0.24 | 6 | 1.2443 |
138
+ | 1.2602 | 0.37 | 9 | 1.2432 |
139
+ | 1.2573 | 0.49 | 12 | 1.2403 |
140
+ | 1.2811 | 0.61 | 15 | 1.2342 |
141
+ | 1.2584 | 0.73 | 18 | 1.2217 |
142
+ | 1.2152 | 0.86 | 21 | 1.2005 |
143
+ | 1.1592 | 0.98 | 24 | 1.1695 |
144
+ | 1.1512 | 1.07 | 27 | 1.1345 |
145
+ | 1.1191 | 1.19 | 30 | 1.0970 |
146
+ | 1.1111 | 1.32 | 33 | 1.0543 |
147
+ | 1.0362 | 1.44 | 36 | 1.0160 |
148
+ | 1.0386 | 1.56 | 39 | 0.9879 |
149
+ | 1.0637 | 1.68 | 42 | 0.9549 |
150
+ | 1.0109 | 1.81 | 45 | 0.9377 |
151
+ | 0.9416 | 1.93 | 48 | 0.9258 |
152
+ | 0.8851 | 2.03 | 51 | 0.9164 |
153
+ | 0.9027 | 2.15 | 54 | 0.9085 |
154
+ | 0.8959 | 2.28 | 57 | 0.9018 |
155
+ | 0.9168 | 2.4 | 60 | 0.8956 |
156
+ | 0.9386 | 2.52 | 63 | 0.8896 |
157
+ | 0.9762 | 2.64 | 66 | 0.8832 |
158
+ | 0.9118 | 2.77 | 69 | 0.8768 |
159
+ | 0.9055 | 2.89 | 72 | 0.8714 |
160
+ | 0.8617 | 3.01 | 75 | 0.8660 |
161
+ | 0.9085 | 3.11 | 78 | 0.8604 |
162
+ | 0.8531 | 3.23 | 81 | 0.8547 |
163
+ | 0.8725 | 3.36 | 84 | 0.8486 |
164
+ | 0.8845 | 3.48 | 87 | 0.8424 |
165
+ | 0.8812 | 3.6 | 90 | 0.8381 |
166
+ | 0.865 | 3.72 | 93 | 0.8351 |
167
+ | 0.8312 | 3.85 | 96 | 0.8311 |
168
+ | 0.8766 | 3.97 | 99 | 0.8280 |
169
+ | 0.842 | 4.07 | 102 | 0.8249 |
170
+ | 0.8377 | 4.19 | 105 | 0.8222 |
171
+ | 0.8661 | 4.32 | 108 | 0.8195 |
172
+ | 0.8505 | 4.44 | 111 | 0.8171 |
173
+ | 0.8509 | 4.56 | 114 | 0.8140 |
174
+ | 0.8823 | 4.68 | 117 | 0.8111 |
175
+ | 0.8246 | 4.81 | 120 | 0.8091 |
176
+ | 0.8116 | 4.93 | 123 | 0.8073 |
177
+ | 0.7993 | 5.03 | 126 | 0.8054 |
178
+ | 0.8277 | 5.15 | 129 | 0.8048 |
179
+ | 0.8533 | 5.28 | 132 | 0.8030 |
180
+ | 0.7887 | 5.4 | 135 | 0.8015 |
181
+ | 0.8189 | 5.52 | 138 | 0.8005 |
182
+ | 0.8148 | 5.64 | 141 | 0.7993 |
183
+ | 0.8376 | 5.77 | 144 | 0.7977 |
184
+ | 0.8142 | 5.89 | 147 | 0.7968 |
185
+ | 0.8074 | 6.01 | 150 | 0.7961 |
186
+ | 0.8122 | 6.11 | 153 | 0.7970 |
187
+ | 0.7753 | 6.23 | 156 | 0.7963 |
188
+ | 0.8477 | 6.36 | 159 | 0.7958 |
189
+ | 0.7977 | 6.48 | 162 | 0.7947 |
190
+ | 0.7653 | 6.6 | 165 | 0.7944 |
191
+ | 0.8358 | 6.72 | 168 | 0.7930 |
192
+ | 0.7445 | 6.85 | 171 | 0.7926 |
193
+ | 0.808 | 6.97 | 174 | 0.7922 |
194
+ | 0.7799 | 7.07 | 177 | 0.7916 |
195
+ | 0.7593 | 7.19 | 180 | 0.7933 |
196
+ | 0.8275 | 7.32 | 183 | 0.7930 |
197
+ | 0.7599 | 7.44 | 186 | 0.7925 |
198
+ | 0.7734 | 7.56 | 189 | 0.7928 |
199
+ | 0.7886 | 7.68 | 192 | 0.7927 |
200
+ | 0.8066 | 7.81 | 195 | 0.7919 |
201
+ | 0.7778 | 7.93 | 198 | 0.7916 |
202
+ | 0.7839 | 8.03 | 201 | 0.7918 |
203
+ | 0.7942 | 8.15 | 204 | 0.7927 |
204
+ | 0.7457 | 8.28 | 207 | 0.7930 |
205
+ | 0.7525 | 8.4 | 210 | 0.7928 |
206
+ | 0.7768 | 8.52 | 213 | 0.7926 |
207
+ | 0.7469 | 8.64 | 216 | 0.7928 |
208
+ | 0.7777 | 8.77 | 219 | 0.7929 |
209
+ | 0.7694 | 8.89 | 222 | 0.7928 |
210
+ | 0.7639 | 9.01 | 225 | 0.7927 |
211
+ | 0.7556 | 9.11 | 228 | 0.7927 |
212
+ | 0.7098 | 9.23 | 231 | 0.7927 |
213
+ | 0.7537 | 9.36 | 234 | 0.7928 |
214
+ | 0.7721 | 9.48 | 237 | 0.7926 |
215
+ | 0.7642 | 9.6 | 240 | 0.7929 |
216
 
217
 
218
  ### Framework versions
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d5b9a26a33e771dccdf87643e0178fbfd3b69a586d451c8717a58550fe19b964
3
  size 319977229
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c443cd157c71e9a78b356763d838ba86925403e635517d52355321e7398040b
3
  size 319977229