Hugging Face's logo

Undi95
/
Mistral-7B-smoll_pippa-lora

Generated from Trainer
Model card Files Community
1
Undi95 commited on
Commit
557a9c3
•
1 Parent(s): c8960cd

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +77 -0
  2. adapter_config.json +28 -0
  3. adapter_model.bin +3 -0
  4. added_tokens.json +5 -0
  5. checkpoint-100/README.md +21 -0
  6. checkpoint-100/adapter_config.json +28 -0
  7. checkpoint-100/adapter_model.bin +3 -0
  8. checkpoint-100/optimizer.pt +3 -0
  9. checkpoint-100/rng_state.pth +3 -0
  10. checkpoint-100/scheduler.pt +3 -0
  11. checkpoint-100/trainer_state.json +635 -0
  12. checkpoint-100/training_args.bin +3 -0
  13. checkpoint-150/README.md +21 -0
  14. checkpoint-150/adapter_config.json +28 -0
  15. checkpoint-150/adapter_model.bin +3 -0
  16. checkpoint-150/optimizer.pt +3 -0
  17. checkpoint-150/rng_state.pth +3 -0
  18. checkpoint-150/scheduler.pt +3 -0
  19. checkpoint-150/trainer_state.json +943 -0
  20. checkpoint-150/training_args.bin +3 -0
  21. checkpoint-200/README.md +21 -0
  22. checkpoint-200/adapter_config.json +28 -0
  23. checkpoint-200/adapter_model.bin +3 -0
  24. checkpoint-200/optimizer.pt +3 -0
  25. checkpoint-200/rng_state.pth +3 -0
  26. checkpoint-200/scheduler.pt +3 -0
  27. checkpoint-200/trainer_state.json +1251 -0
  28. checkpoint-200/training_args.bin +3 -0
  29. checkpoint-250/README.md +21 -0
  30. checkpoint-250/adapter_config.json +28 -0
  31. checkpoint-250/adapter_model.bin +3 -0
  32. checkpoint-250/optimizer.pt +3 -0
  33. checkpoint-250/rng_state.pth +3 -0
  34. checkpoint-250/scheduler.pt +3 -0
  35. checkpoint-250/trainer_state.json +1559 -0
  36. checkpoint-250/training_args.bin +3 -0
  37. checkpoint-300/README.md +21 -0
  38. checkpoint-300/adapter_config.json +28 -0
  39. checkpoint-300/adapter_model.bin +3 -0
  40. checkpoint-300/optimizer.pt +3 -0
  41. checkpoint-300/rng_state.pth +3 -0
  42. checkpoint-300/scheduler.pt +3 -0
  43. checkpoint-300/trainer_state.json +1867 -0
  44. checkpoint-300/training_args.bin +3 -0
  45. checkpoint-350/README.md +21 -0
  46. checkpoint-350/adapter_config.json +28 -0
  47. checkpoint-350/adapter_model.bin +3 -0
  48. checkpoint-350/optimizer.pt +3 -0
  49. checkpoint-350/rng_state.pth +3 -0
  50. checkpoint-350/scheduler.pt +3 -0
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ model-index:
5
+ - name: lora-out
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
+ should probably proofread and complete it, then remove this comment. -->
11
+
12
+ [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
13
+ # lora-out
14
+
15
+ This model was trained from scratch on the None dataset.
16
+ It achieves the following results on the evaluation set:
17
+ - Loss: 1.6736
18
+
19
+ ## Model description
20
+
21
+ More information needed
22
+
23
+ ## Intended uses & limitations
24
+
25
+ More information needed
26
+
27
+ ## Training and evaluation data
28
+
29
+ More information needed
30
+
31
+ ## Training procedure
32
+
33
+ ### Training hyperparameters
34
+
35
+ The following hyperparameters were used during training:
36
+ - learning_rate: 0.0002
37
+ - train_batch_size: 4
38
+ - eval_batch_size: 4
39
+ - seed: 42
40
+ - gradient_accumulation_steps: 4
41
+ - total_train_batch_size: 16
42
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
43
+ - lr_scheduler_type: cosine
44
+ - lr_scheduler_warmup_steps: 10
45
+ - num_epochs: 3
46
+
47
+ ### Training results
48
+
49
+ | Training Loss | Epoch | Step | Validation Loss |
50
+ |:-------------:|:-----:|:----:|:---------------:|
51
+ | 1.6421 | 0.16 | 50 | 1.6217 |
52
+ | 1.6288 | 0.31 | 100 | 1.6144 |
53
+ | 1.5725 | 0.47 | 150 | 1.6102 |
54
+ | 1.5582 | 0.62 | 200 | 1.6065 |
55
+ | 1.6055 | 0.78 | 250 | 1.6051 |
56
+ | 1.5733 | 0.93 | 300 | 1.6023 |
57
+ | 1.4885 | 1.09 | 350 | 1.6130 |
58
+ | 1.484 | 1.24 | 400 | 1.6169 |
59
+ | 1.4354 | 1.4 | 450 | 1.6194 |
60
+ | 1.4427 | 1.56 | 500 | 1.6187 |
61
+ | 1.4687 | 1.71 | 550 | 1.6178 |
62
+ | 1.461 | 1.87 | 600 | 1.6174 |
63
+ | 1.327 | 2.02 | 650 | 1.6341 |
64
+ | 1.3015 | 2.18 | 700 | 1.6665 |
65
+ | 1.3328 | 2.33 | 750 | 1.6714 |
66
+ | 1.3453 | 2.49 | 800 | 1.6718 |
67
+ | 1.3458 | 2.64 | 850 | 1.6725 |
68
+ | 1.3016 | 2.8 | 900 | 1.6737 |
69
+ | 1.3018 | 2.95 | 950 | 1.6736 |
70
+
71
+
72
+ ### Framework versions
73
+
74
+ - Transformers 4.34.0.dev0
75
+ - Pytorch 2.0.1+cu118
76
+ - Datasets 2.14.5
77
+ - Tokenizers 0.14.0
adapter_config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "./mistralai_Mistral-7B-v0.1",
5
+ "bias": "none",
6
+ "fan_in_fan_out": null,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "lora_alpha": 16,
12
+ "lora_dropout": 0.05,
13
+ "modules_to_save": null,
14
+ "peft_type": "LORA",
15
+ "r": 8,
16
+ "rank_pattern": {},
17
+ "revision": null,
18
+ "target_modules": [
19
+ "o_proj",
20
+ "k_proj",
21
+ "up_proj",
22
+ "v_proj",
23
+ "q_proj",
24
+ "gate_proj",
25
+ "down_proj"
26
+ ],
27
+ "task_type": "CAUSAL_LM"
28
+ }
adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:219facdff16b696735f1a84fdd92a0cbac9e197db9eef71c8ce3344d57718790
3
+ size 84046925
added_tokens.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "</s>": 2,
3
+ "<s>": 1,
4
+ "<unk>": 0
5
+ }
checkpoint-100/README.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - quant_method: bitsandbytes
9
+ - load_in_8bit: False
10
+ - load_in_4bit: True
11
+ - llm_int8_threshold: 6.0
12
+ - llm_int8_skip_modules: None
13
+ - llm_int8_enable_fp32_cpu_offload: False
14
+ - llm_int8_has_fp16_weight: False
15
+ - bnb_4bit_quant_type: nf4
16
+ - bnb_4bit_use_double_quant: True
17
+ - bnb_4bit_compute_dtype: bfloat16
18
+ ### Framework versions
19
+
20
+
21
+ - PEFT 0.6.0.dev0
checkpoint-100/adapter_config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "./mistralai_Mistral-7B-v0.1",
5
+ "bias": "none",
6
+ "fan_in_fan_out": null,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "lora_alpha": 16,
12
+ "lora_dropout": 0.05,
13
+ "modules_to_save": null,
14
+ "peft_type": "LORA",
15
+ "r": 8,
16
+ "rank_pattern": {},
17
+ "revision": null,
18
+ "target_modules": [
19
+ "o_proj",
20
+ "k_proj",
21
+ "up_proj",
22
+ "v_proj",
23
+ "q_proj",
24
+ "gate_proj",
25
+ "down_proj"
26
+ ],
27
+ "task_type": "CAUSAL_LM"
28
+ }
checkpoint-100/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15b988ab295a843da4e545e17f172c76e0f1bfa3a9dd9edf198aef5f67b77510
3
+ size 84046925
checkpoint-100/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d7a2c502708a4001a524b14e53231dd6709ac7558468e509fa4048cd3f6e77c
3
+ size 168039109
checkpoint-100/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7b33e2e1c1d173dea2a01ae0984bce42a42fb5bc88b8e940e3d611fa0910ef6b
3
+ size 14575
checkpoint-100/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69c403caea7cd1ca879cd128b86652676d56ca84332ed1fd33c5182cf98e7a10
3
+ size 627
checkpoint-100/trainer_state.json ADDED
@@ -0,0 +1,635 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 1.6143836975097656,
3
+ "best_model_checkpoint": "./lora-out/checkpoint-100",
4
+ "epoch": 0.3110419906687403,
5
+ "eval_steps": 50,
6
+ "global_step": 100,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0,
13
+ "learning_rate": 2e-05,
14
+ "loss": 1.7924,
15
+ "step": 1
16
+ },
17
+ {
18
+ "epoch": 0.01,
19
+ "learning_rate": 4e-05,
20
+ "loss": 1.8083,
21
+ "step": 2
22
+ },
23
+ {
24
+ "epoch": 0.01,
25
+ "learning_rate": 6e-05,
26
+ "loss": 1.8177,
27
+ "step": 3
28
+ },
29
+ {
30
+ "epoch": 0.01,
31
+ "learning_rate": 8e-05,
32
+ "loss": 1.7595,
33
+ "step": 4
34
+ },
35
+ {
36
+ "epoch": 0.02,
37
+ "learning_rate": 0.0001,
38
+ "loss": 1.6598,
39
+ "step": 5
40
+ },
41
+ {
42
+ "epoch": 0.02,
43
+ "learning_rate": 0.00012,
44
+ "loss": 1.6919,
45
+ "step": 6
46
+ },
47
+ {
48
+ "epoch": 0.02,
49
+ "learning_rate": 0.00014,
50
+ "loss": 1.6706,
51
+ "step": 7
52
+ },
53
+ {
54
+ "epoch": 0.02,
55
+ "learning_rate": 0.00016,
56
+ "loss": 1.6879,
57
+ "step": 8
58
+ },
59
+ {
60
+ "epoch": 0.03,
61
+ "learning_rate": 0.00018,
62
+ "loss": 1.7051,
63
+ "step": 9
64
+ },
65
+ {
66
+ "epoch": 0.03,
67
+ "learning_rate": 0.0002,
68
+ "loss": 1.7022,
69
+ "step": 10
70
+ },
71
+ {
72
+ "epoch": 0.03,
73
+ "learning_rate": 0.000199999456645141,
74
+ "loss": 1.6809,
75
+ "step": 11
76
+ },
77
+ {
78
+ "epoch": 0.04,
79
+ "learning_rate": 0.00019999782658646859,
80
+ "loss": 1.6098,
81
+ "step": 12
82
+ },
83
+ {
84
+ "epoch": 0.04,
85
+ "learning_rate": 0.0001999951098416968,
86
+ "loss": 1.7014,
87
+ "step": 13
88
+ },
89
+ {
90
+ "epoch": 0.04,
91
+ "learning_rate": 0.00019999130644034888,
92
+ "loss": 1.5885,
93
+ "step": 14
94
+ },
95
+ {
96
+ "epoch": 0.05,
97
+ "learning_rate": 0.00019998641642375657,
98
+ "loss": 1.6243,
99
+ "step": 15
100
+ },
101
+ {
102
+ "epoch": 0.05,
103
+ "learning_rate": 0.00019998043984506027,
104
+ "loss": 1.6484,
105
+ "step": 16
106
+ },
107
+ {
108
+ "epoch": 0.05,
109
+ "learning_rate": 0.00019997337676920803,
110
+ "loss": 1.6093,
111
+ "step": 17
112
+ },
113
+ {
114
+ "epoch": 0.06,
115
+ "learning_rate": 0.00019996522727295496,
116
+ "loss": 1.6173,
117
+ "step": 18
118
+ },
119
+ {
120
+ "epoch": 0.06,
121
+ "learning_rate": 0.00019995599144486247,
122
+ "loss": 1.646,
123
+ "step": 19
124
+ },
125
+ {
126
+ "epoch": 0.06,
127
+ "learning_rate": 0.00019994566938529712,
128
+ "loss": 1.6469,
129
+ "step": 20
130
+ },
131
+ {
132
+ "epoch": 0.07,
133
+ "learning_rate": 0.00019993426120642983,
134
+ "loss": 1.6564,
135
+ "step": 21
136
+ },
137
+ {
138
+ "epoch": 0.07,
139
+ "learning_rate": 0.00019992176703223432,
140
+ "loss": 1.5901,
141
+ "step": 22
142
+ },
143
+ {
144
+ "epoch": 0.07,
145
+ "learning_rate": 0.000199908186998486,
146
+ "loss": 1.664,
147
+ "step": 23
148
+ },
149
+ {
150
+ "epoch": 0.07,
151
+ "learning_rate": 0.00019989352125276047,
152
+ "loss": 1.6275,
153
+ "step": 24
154
+ },
155
+ {
156
+ "epoch": 0.08,
157
+ "learning_rate": 0.00019987776995443178,
158
+ "loss": 1.5839,
159
+ "step": 25
160
+ },
161
+ {
162
+ "epoch": 0.08,
163
+ "learning_rate": 0.00019986093327467076,
164
+ "loss": 1.5611,
165
+ "step": 26
166
+ },
167
+ {
168
+ "epoch": 0.08,
169
+ "learning_rate": 0.00019984301139644334,
170
+ "loss": 1.669,
171
+ "step": 27
172
+ },
173
+ {
174
+ "epoch": 0.09,
175
+ "learning_rate": 0.0001998240045145083,
176
+ "loss": 1.5641,
177
+ "step": 28
178
+ },
179
+ {
180
+ "epoch": 0.09,
181
+ "learning_rate": 0.00019980391283541522,
182
+ "loss": 1.6023,
183
+ "step": 29
184
+ },
185
+ {
186
+ "epoch": 0.09,
187
+ "learning_rate": 0.00019978273657750238,
188
+ "loss": 1.6309,
189
+ "step": 30
190
+ },
191
+ {
192
+ "epoch": 0.1,
193
+ "learning_rate": 0.0001997604759708942,
194
+ "loss": 1.6353,
195
+ "step": 31
196
+ },
197
+ {
198
+ "epoch": 0.1,
199
+ "learning_rate": 0.00019973713125749884,
200
+ "loss": 1.6328,
201
+ "step": 32
202
+ },
203
+ {
204
+ "epoch": 0.1,
205
+ "learning_rate": 0.00019971270269100564,
206
+ "loss": 1.5683,
207
+ "step": 33
208
+ },
209
+ {
210
+ "epoch": 0.11,
211
+ "learning_rate": 0.00019968719053688213,
212
+ "loss": 1.6217,
213
+ "step": 34
214
+ },
215
+ {
216
+ "epoch": 0.11,
217
+ "learning_rate": 0.0001996605950723714,
218
+ "loss": 1.5734,
219
+ "step": 35
220
+ },
221
+ {
222
+ "epoch": 0.11,
223
+ "learning_rate": 0.00019963291658648896,
224
+ "loss": 1.6162,
225
+ "step": 36
226
+ },
227
+ {
228
+ "epoch": 0.12,
229
+ "learning_rate": 0.00019960415538001957,
230
+ "loss": 1.5922,
231
+ "step": 37
232
+ },
233
+ {
234
+ "epoch": 0.12,
235
+ "learning_rate": 0.0001995743117655141,
236
+ "loss": 1.5806,
237
+ "step": 38
238
+ },
239
+ {
240
+ "epoch": 0.12,
241
+ "learning_rate": 0.000199543386067286,
242
+ "loss": 1.5938,
243
+ "step": 39
244
+ },
245
+ {
246
+ "epoch": 0.12,
247
+ "learning_rate": 0.00019951137862140778,
248
+ "loss": 1.6386,
249
+ "step": 40
250
+ },
251
+ {
252
+ "epoch": 0.13,
253
+ "learning_rate": 0.00019947828977570756,
254
+ "loss": 1.6476,
255
+ "step": 41
256
+ },
257
+ {
258
+ "epoch": 0.13,
259
+ "learning_rate": 0.00019944411988976496,
260
+ "loss": 1.6557,
261
+ "step": 42
262
+ },
263
+ {
264
+ "epoch": 0.13,
265
+ "learning_rate": 0.00019940886933490749,
266
+ "loss": 1.5836,
267
+ "step": 43
268
+ },
269
+ {
270
+ "epoch": 0.14,
271
+ "learning_rate": 0.00019937253849420635,
272
+ "loss": 1.6421,
273
+ "step": 44
274
+ },
275
+ {
276
+ "epoch": 0.14,
277
+ "learning_rate": 0.0001993351277624723,
278
+ "loss": 1.629,
279
+ "step": 45
280
+ },
281
+ {
282
+ "epoch": 0.14,
283
+ "learning_rate": 0.00019929663754625145,
284
+ "loss": 1.6392,
285
+ "step": 46
286
+ },
287
+ {
288
+ "epoch": 0.15,
289
+ "learning_rate": 0.00019925706826382064,
290
+ "loss": 1.5677,
291
+ "step": 47
292
+ },
293
+ {
294
+ "epoch": 0.15,
295
+ "learning_rate": 0.00019921642034518317,
296
+ "loss": 1.6144,
297
+ "step": 48
298
+ },
299
+ {
300
+ "epoch": 0.15,
301
+ "learning_rate": 0.00019917469423206389,
302
+ "loss": 1.6068,
303
+ "step": 49
304
+ },
305
+ {
306
+ "epoch": 0.16,
307
+ "learning_rate": 0.00019913189037790456,
308
+ "loss": 1.6421,
309
+ "step": 50
310
+ },
311
+ {
312
+ "epoch": 0.16,
313
+ "eval_loss": 1.621693730354309,
314
+ "eval_runtime": 233.7603,
315
+ "eval_samples_per_second": 16.354,
316
+ "eval_steps_per_second": 4.09,
317
+ "step": 50
318
+ },
319
+ {
320
+ "epoch": 0.16,
321
+ "learning_rate": 0.0001990880092478588,
322
+ "loss": 1.6172,
323
+ "step": 51
324
+ },
325
+ {
326
+ "epoch": 0.16,
327
+ "learning_rate": 0.0001990430513187871,
328
+ "loss": 1.6095,
329
+ "step": 52
330
+ },
331
+ {
332
+ "epoch": 0.16,
333
+ "learning_rate": 0.00019899701707925166,
334
+ "loss": 1.5967,
335
+ "step": 53
336
+ },
337
+ {
338
+ "epoch": 0.17,
339
+ "learning_rate": 0.00019894990702951106,
340
+ "loss": 1.617,
341
+ "step": 54
342
+ },
343
+ {
344
+ "epoch": 0.17,
345
+ "learning_rate": 0.00019890172168151473,
346
+ "loss": 1.5932,
347
+ "step": 55
348
+ },
349
+ {
350
+ "epoch": 0.17,
351
+ "learning_rate": 0.0001988524615588976,
352
+ "loss": 1.6548,
353
+ "step": 56
354
+ },
355
+ {
356
+ "epoch": 0.18,
357
+ "learning_rate": 0.00019880212719697413,
358
+ "loss": 1.6033,
359
+ "step": 57
360
+ },
361
+ {
362
+ "epoch": 0.18,
363
+ "learning_rate": 0.00019875071914273278,
364
+ "loss": 1.6063,
365
+ "step": 58
366
+ },
367
+ {
368
+ "epoch": 0.18,
369
+ "learning_rate": 0.00019869823795482986,
370
+ "loss": 1.6107,
371
+ "step": 59
372
+ },
373
+ {
374
+ "epoch": 0.19,
375
+ "learning_rate": 0.00019864468420358354,
376
+ "loss": 1.5758,
377
+ "step": 60
378
+ },
379
+ {
380
+ "epoch": 0.19,
381
+ "learning_rate": 0.00019859005847096763,
382
+ "loss": 1.5723,
383
+ "step": 61
384
+ },
385
+ {
386
+ "epoch": 0.19,
387
+ "learning_rate": 0.00019853436135060527,
388
+ "loss": 1.542,
389
+ "step": 62
390
+ },
391
+ {
392
+ "epoch": 0.2,
393
+ "learning_rate": 0.00019847759344776252,
394
+ "loss": 1.5611,
395
+ "step": 63
396
+ },
397
+ {
398
+ "epoch": 0.2,
399
+ "learning_rate": 0.00019841975537934162,
400
+ "loss": 1.6157,
401
+ "step": 64
402
+ },
403
+ {
404
+ "epoch": 0.2,
405
+ "learning_rate": 0.00019836084777387458,
406
+ "loss": 1.5589,
407
+ "step": 65
408
+ },
409
+ {
410
+ "epoch": 0.21,
411
+ "learning_rate": 0.00019830087127151598,
412
+ "loss": 1.6077,
413
+ "step": 66
414
+ },
415
+ {
416
+ "epoch": 0.21,
417
+ "learning_rate": 0.00019823982652403634,
418
+ "loss": 1.5473,
419
+ "step": 67
420
+ },
421
+ {
422
+ "epoch": 0.21,
423
+ "learning_rate": 0.00019817771419481487,
424
+ "loss": 1.6265,
425
+ "step": 68
426
+ },
427
+ {
428
+ "epoch": 0.21,
429
+ "learning_rate": 0.0001981145349588323,
430
+ "loss": 1.6074,
431
+ "step": 69
432
+ },
433
+ {
434
+ "epoch": 0.22,
435
+ "learning_rate": 0.00019805028950266348,
436
+ "loss": 1.6195,
437
+ "step": 70
438
+ },
439
+ {
440
+ "epoch": 0.22,
441
+ "learning_rate": 0.00019798497852447006,
442
+ "loss": 1.5876,
443
+ "step": 71
444
+ },
445
+ {
446
+ "epoch": 0.22,
447
+ "learning_rate": 0.0001979186027339928,
448
+ "loss": 1.5978,
449
+ "step": 72
450
+ },
451
+ {
452
+ "epoch": 0.23,
453
+ "learning_rate": 0.00019785116285254381,
454
+ "loss": 1.533,
455
+ "step": 73
456
+ },
457
+ {
458
+ "epoch": 0.23,
459
+ "learning_rate": 0.00019778265961299888,
460
+ "loss": 1.5888,
461
+ "step": 74
462
+ },
463
+ {
464
+ "epoch": 0.23,
465
+ "learning_rate": 0.0001977130937597894,
466
+ "loss": 1.6211,
467
+ "step": 75
468
+ },
469
+ {
470
+ "epoch": 0.24,
471
+ "learning_rate": 0.00019764246604889415,
472
+ "loss": 1.6091,
473
+ "step": 76
474
+ },
475
+ {
476
+ "epoch": 0.24,
477
+ "learning_rate": 0.00019757077724783147,
478
+ "loss": 1.6012,
479
+ "step": 77
480
+ },
481
+ {
482
+ "epoch": 0.24,
483
+ "learning_rate": 0.0001974980281356504,
484
+ "loss": 1.6401,
485
+ "step": 78
486
+ },
487
+ {
488
+ "epoch": 0.25,
489
+ "learning_rate": 0.0001974242195029227,
490
+ "loss": 1.6111,
491
+ "step": 79
492
+ },
493
+ {
494
+ "epoch": 0.25,
495
+ "learning_rate": 0.00019734935215173392,
496
+ "loss": 1.6208,
497
+ "step": 80
498
+ },
499
+ {
500
+ "epoch": 0.25,
501
+ "learning_rate": 0.00019727342689567482,
502
+ "loss": 1.6038,
503
+ "step": 81
504
+ },
505
+ {
506
+ "epoch": 0.26,
507
+ "learning_rate": 0.00019719644455983256,
508
+ "loss": 1.5915,
509
+ "step": 82
510
+ },
511
+ {
512
+ "epoch": 0.26,
513
+ "learning_rate": 0.0001971184059807817,
514
+ "loss": 1.5872,
515
+ "step": 83
516
+ },
517
+ {
518
+ "epoch": 0.26,
519
+ "learning_rate": 0.000197039312006575,
520
+ "loss": 1.5984,
521
+ "step": 84
522
+ },
523
+ {
524
+ "epoch": 0.26,
525
+ "learning_rate": 0.0001969591634967344,
526
+ "loss": 1.5996,
527
+ "step": 85
528
+ },
529
+ {
530
+ "epoch": 0.27,
531
+ "learning_rate": 0.00019687796132224152,
532
+ "loss": 1.6056,
533
+ "step": 86
534
+ },
535
+ {
536
+ "epoch": 0.27,
537
+ "learning_rate": 0.0001967957063655283,
538
+ "loss": 1.6099,
539
+ "step": 87
540
+ },
541
+ {
542
+ "epoch": 0.27,
543
+ "learning_rate": 0.0001967123995204674,
544
+ "loss": 1.6295,
545
+ "step": 88
546
+ },
547
+ {
548
+ "epoch": 0.28,
549
+ "learning_rate": 0.00019662804169236225,
550
+ "loss": 1.5482,
551
+ "step": 89
552
+ },
553
+ {
554
+ "epoch": 0.28,
555
+ "learning_rate": 0.00019654263379793773,
556
+ "loss": 1.5781,
557
+ "step": 90
558
+ },
559
+ {
560
+ "epoch": 0.28,
561
+ "learning_rate": 0.00019645617676532963,
562
+ "loss": 1.5954,
563
+ "step": 91
564
+ },
565
+ {
566
+ "epoch": 0.29,
567
+ "learning_rate": 0.000196368671534075,
568
+ "loss": 1.619,
569
+ "step": 92
570
+ },
571
+ {
572
+ "epoch": 0.29,
573
+ "learning_rate": 0.0001962801190551016,
574
+ "loss": 1.6153,
575
+ "step": 93
576
+ },
577
+ {
578
+ "epoch": 0.29,
579
+ "learning_rate": 0.0001961905202907179,
580
+ "loss": 1.6008,
581
+ "step": 94
582
+ },
583
+ {
584
+ "epoch": 0.3,
585
+ "learning_rate": 0.00019609987621460232,
586
+ "loss": 1.5891,
587
+ "step": 95
588
+ },
589
+ {
590
+ "epoch": 0.3,
591
+ "learning_rate": 0.0001960081878117929,
592
+ "loss": 1.6438,
593
+ "step": 96
594
+ },
595
+ {
596
+ "epoch": 0.3,
597
+ "learning_rate": 0.0001959154560786764,
598
+ "loss": 1.5576,
599
+ "step": 97
600
+ },
601
+ {
602
+ "epoch": 0.3,
603
+ "learning_rate": 0.00019582168202297758,
604
+ "loss": 1.646,
605
+ "step": 98
606
+ },
607
+ {
608
+ "epoch": 0.31,
609
+ "learning_rate": 0.00019572686666374822,
610
+ "loss": 1.6269,
611
+ "step": 99
612
+ },
613
+ {
614
+ "epoch": 0.31,
615
+ "learning_rate": 0.00019563101103135602,
616
+ "loss": 1.6288,
617
+ "step": 100
618
+ },
619
+ {
620
+ "epoch": 0.31,
621
+ "eval_loss": 1.6143836975097656,
622
+ "eval_runtime": 233.6412,
623
+ "eval_samples_per_second": 16.363,
624
+ "eval_steps_per_second": 4.092,
625
+ "step": 100
626
+ }
627
+ ],
628
+ "logging_steps": 1,
629
+ "max_steps": 963,
630
+ "num_train_epochs": 3,
631
+ "save_steps": 50,
632
+ "total_flos": 2.804271657517056e+17,
633
+ "trial_name": null,
634
+ "trial_params": null
635
+ }
checkpoint-100/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f61cafb89242b653e455003b5517e685ecccfa6180af5fb7d0bfb35b4fc77a4
3
+ size 4475
checkpoint-150/README.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - quant_method: bitsandbytes
9
+ - load_in_8bit: False
10
+ - load_in_4bit: True
11
+ - llm_int8_threshold: 6.0
12
+ - llm_int8_skip_modules: None
13
+ - llm_int8_enable_fp32_cpu_offload: False
14
+ - llm_int8_has_fp16_weight: False
15
+ - bnb_4bit_quant_type: nf4
16
+ - bnb_4bit_use_double_quant: True
17
+ - bnb_4bit_compute_dtype: bfloat16
18
+ ### Framework versions
19
+
20
+
21
+ - PEFT 0.6.0.dev0
checkpoint-150/adapter_config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "./mistralai_Mistral-7B-v0.1",
5
+ "bias": "none",
6
+ "fan_in_fan_out": null,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "lora_alpha": 16,
12
+ "lora_dropout": 0.05,
13
+ "modules_to_save": null,
14
+ "peft_type": "LORA",
15
+ "r": 8,
16
+ "rank_pattern": {},
17
+ "revision": null,
18
+ "target_modules": [
19
+ "o_proj",
20
+ "k_proj",
21
+ "up_proj",
22
+ "v_proj",
23
+ "q_proj",
24
+ "gate_proj",
25
+ "down_proj"
26
+ ],
27
+ "task_type": "CAUSAL_LM"
28
+ }
checkpoint-150/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5a8dc95d466eddcfa8f6d8c0d5ccbea2ab6e546948aa9f5852746421f82e77c0
3
+ size 84046925
checkpoint-150/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5972252f94afdaa16250d0a4a3efb52fef281f559cc7aab7a784266e5c32b047
3
+ size 168039109
checkpoint-150/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90a6fa23e9d23da035b3cd7f01ac4e5c74cc8369e9fff9529d9aa0c1fed279ac
3
+ size 14575
checkpoint-150/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fce3e98547f3323418da3efa641653d361277560a755469afb6b5a3bc118996a
3
+ size 627
checkpoint-150/trainer_state.json ADDED
@@ -0,0 +1,943 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 1.6101970672607422,
3
+ "best_model_checkpoint": "./lora-out/checkpoint-150",
4
+ "epoch": 0.4665629860031104,
5
+ "eval_steps": 50,
6
+ "global_step": 150,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0,
13
+ "learning_rate": 2e-05,
14
+ "loss": 1.7924,
15
+ "step": 1
16
+ },
17
+ {
18
+ "epoch": 0.01,
19
+ "learning_rate": 4e-05,
20
+ "loss": 1.8083,
21
+ "step": 2
22
+ },
23
+ {
24
+ "epoch": 0.01,
25
+ "learning_rate": 6e-05,
26
+ "loss": 1.8177,
27
+ "step": 3
28
+ },
29
+ {
30
+ "epoch": 0.01,
31
+ "learning_rate": 8e-05,
32
+ "loss": 1.7595,
33
+ "step": 4
34
+ },
35
+ {
36
+ "epoch": 0.02,
37
+ "learning_rate": 0.0001,
38
+ "loss": 1.6598,
39
+ "step": 5
40
+ },
41
+ {
42
+ "epoch": 0.02,
43
+ "learning_rate": 0.00012,
44
+ "loss": 1.6919,
45
+ "step": 6
46
+ },
47
+ {
48
+ "epoch": 0.02,
49
+ "learning_rate": 0.00014,
50
+ "loss": 1.6706,
51
+ "step": 7
52
+ },
53
+ {
54
+ "epoch": 0.02,
55
+ "learning_rate": 0.00016,
56
+ "loss": 1.6879,
57
+ "step": 8
58
+ },
59
+ {
60
+ "epoch": 0.03,
61
+ "learning_rate": 0.00018,
62
+ "loss": 1.7051,
63
+ "step": 9
64
+ },
65
+ {
66
+ "epoch": 0.03,
67
+ "learning_rate": 0.0002,
68
+ "loss": 1.7022,
69
+ "step": 10
70
+ },
71
+ {
72
+ "epoch": 0.03,
73
+ "learning_rate": 0.000199999456645141,
74
+ "loss": 1.6809,
75
+ "step": 11
76
+ },
77
+ {
78
+ "epoch": 0.04,
79
+ "learning_rate": 0.00019999782658646859,
80
+ "loss": 1.6098,
81
+ "step": 12
82
+ },
83
+ {
84
+ "epoch": 0.04,
85
+ "learning_rate": 0.0001999951098416968,
86
+ "loss": 1.7014,
87
+ "step": 13
88
+ },
89
+ {
90
+ "epoch": 0.04,
91
+ "learning_rate": 0.00019999130644034888,
92
+ "loss": 1.5885,
93
+ "step": 14
94
+ },
95
+ {
96
+ "epoch": 0.05,
97
+ "learning_rate": 0.00019998641642375657,
98
+ "loss": 1.6243,
99
+ "step": 15
100
+ },
101
+ {
102
+ "epoch": 0.05,
103
+ "learning_rate": 0.00019998043984506027,
104
+ "loss": 1.6484,
105
+ "step": 16
106
+ },
107
+ {
108
+ "epoch": 0.05,
109
+ "learning_rate": 0.00019997337676920803,
110
+ "loss": 1.6093,
111
+ "step": 17
112
+ },
113
+ {
114
+ "epoch": 0.06,
115
+ "learning_rate": 0.00019996522727295496,
116
+ "loss": 1.6173,
117
+ "step": 18
118
+ },
119
+ {
120
+ "epoch": 0.06,
121
+ "learning_rate": 0.00019995599144486247,
122
+ "loss": 1.646,
123
+ "step": 19
124
+ },
125
+ {
126
+ "epoch": 0.06,
127
+ "learning_rate": 0.00019994566938529712,
128
+ "loss": 1.6469,
129
+ "step": 20
130
+ },
131
+ {
132
+ "epoch": 0.07,
133
+ "learning_rate": 0.00019993426120642983,
134
+ "loss": 1.6564,
135
+ "step": 21
136
+ },
137
+ {
138
+ "epoch": 0.07,
139
+ "learning_rate": 0.00019992176703223432,
140
+ "loss": 1.5901,
141
+ "step": 22
142
+ },
143
+ {
144
+ "epoch": 0.07,
145
+ "learning_rate": 0.000199908186998486,
146
+ "loss": 1.664,
147
+ "step": 23
148
+ },
149
+ {
150
+ "epoch": 0.07,
151
+ "learning_rate": 0.00019989352125276047,
152
+ "loss": 1.6275,
153
+ "step": 24
154
+ },
155
+ {
156
+ "epoch": 0.08,
157
+ "learning_rate": 0.00019987776995443178,
158
+ "loss": 1.5839,
159
+ "step": 25
160
+ },
161
+ {
162
+ "epoch": 0.08,
163
+ "learning_rate": 0.00019986093327467076,
164
+ "loss": 1.5611,
165
+ "step": 26
166
+ },
167
+ {
168
+ "epoch": 0.08,
169
+ "learning_rate": 0.00019984301139644334,
170
+ "loss": 1.669,
171
+ "step": 27
172
+ },
173
+ {
174
+ "epoch": 0.09,
175
+ "learning_rate": 0.0001998240045145083,
176
+ "loss": 1.5641,
177
+ "step": 28
178
+ },
179
+ {
180
+ "epoch": 0.09,
181
+ "learning_rate": 0.00019980391283541522,
182
+ "loss": 1.6023,
183
+ "step": 29
184
+ },
185
+ {
186
+ "epoch": 0.09,
187
+ "learning_rate": 0.00019978273657750238,
188
+ "loss": 1.6309,
189
+ "step": 30
190
+ },
191
+ {
192
+ "epoch": 0.1,
193
+ "learning_rate": 0.0001997604759708942,
194
+ "loss": 1.6353,
195
+ "step": 31
196
+ },
197
+ {
198
+ "epoch": 0.1,
199
+ "learning_rate": 0.00019973713125749884,
200
+ "loss": 1.6328,
201
+ "step": 32
202
+ },
203
+ {
204
+ "epoch": 0.1,
205
+ "learning_rate": 0.00019971270269100564,
206
+ "loss": 1.5683,
207
+ "step": 33
208
+ },
209
+ {
210
+ "epoch": 0.11,
211
+ "learning_rate": 0.00019968719053688213,
212
+ "loss": 1.6217,
213
+ "step": 34
214
+ },
215
+ {
216
+ "epoch": 0.11,
217
+ "learning_rate": 0.0001996605950723714,
218
+ "loss": 1.5734,
219
+ "step": 35
220
+ },
221
+ {
222
+ "epoch": 0.11,
223
+ "learning_rate": 0.00019963291658648896,
224
+ "loss": 1.6162,
225
+ "step": 36
226
+ },
227
+ {
228
+ "epoch": 0.12,
229
+ "learning_rate": 0.00019960415538001957,
230
+ "loss": 1.5922,
231
+ "step": 37
232
+ },
233
+ {
234
+ "epoch": 0.12,
235
+ "learning_rate": 0.0001995743117655141,
236
+ "loss": 1.5806,
237
+ "step": 38
238
+ },
239
+ {
240
+ "epoch": 0.12,
241
+ "learning_rate": 0.000199543386067286,
242
+ "loss": 1.5938,
243
+ "step": 39
244
+ },
245
+ {
246
+ "epoch": 0.12,
247
+ "learning_rate": 0.00019951137862140778,
248
+ "loss": 1.6386,
249
+ "step": 40
250
+ },
251
+ {
252
+ "epoch": 0.13,
253
+ "learning_rate": 0.00019947828977570756,
254
+ "loss": 1.6476,
255
+ "step": 41
256
+ },
257
+ {
258
+ "epoch": 0.13,
259
+ "learning_rate": 0.00019944411988976496,
260
+ "loss": 1.6557,
261
+ "step": 42
262
+ },
263
+ {
264
+ "epoch": 0.13,
265
+ "learning_rate": 0.00019940886933490749,
266
+ "loss": 1.5836,
267
+ "step": 43
268
+ },
269
+ {
270
+ "epoch": 0.14,
271
+ "learning_rate": 0.00019937253849420635,
272
+ "loss": 1.6421,
273
+ "step": 44
274
+ },
275
+ {
276
+ "epoch": 0.14,
277
+ "learning_rate": 0.0001993351277624723,
278
+ "loss": 1.629,
279
+ "step": 45
280
+ },
281
+ {
282
+ "epoch": 0.14,
283
+ "learning_rate": 0.00019929663754625145,
284
+ "loss": 1.6392,
285
+ "step": 46
286
+ },
287
+ {
288
+ "epoch": 0.15,
289
+ "learning_rate": 0.00019925706826382064,
290
+ "loss": 1.5677,
291
+ "step": 47
292
+ },
293
+ {
294
+ "epoch": 0.15,
295
+ "learning_rate": 0.00019921642034518317,
296
+ "loss": 1.6144,
297
+ "step": 48
298
+ },
299
+ {
300
+ "epoch": 0.15,
301
+ "learning_rate": 0.00019917469423206389,
302
+ "loss": 1.6068,
303
+ "step": 49
304
+ },
305
+ {
306
+ "epoch": 0.16,
307
+ "learning_rate": 0.00019913189037790456,
308
+ "loss": 1.6421,
309
+ "step": 50
310
+ },
311
+ {
312
+ "epoch": 0.16,
313
+ "eval_loss": 1.621693730354309,
314
+ "eval_runtime": 233.7603,
315
+ "eval_samples_per_second": 16.354,
316
+ "eval_steps_per_second": 4.09,
317
+ "step": 50
318
+ },
319
+ {
320
+ "epoch": 0.16,
321
+ "learning_rate": 0.0001990880092478588,
322
+ "loss": 1.6172,
323
+ "step": 51
324
+ },
325
+ {
326
+ "epoch": 0.16,
327
+ "learning_rate": 0.0001990430513187871,
328
+ "loss": 1.6095,
329
+ "step": 52
330
+ },
331
+ {
332
+ "epoch": 0.16,
333
+ "learning_rate": 0.00019899701707925166,
334
+ "loss": 1.5967,
335
+ "step": 53
336
+ },
337
+ {
338
+ "epoch": 0.17,
339
+ "learning_rate": 0.00019894990702951106,
340
+ "loss": 1.617,
341
+ "step": 54
342
+ },
343
+ {
344
+ "epoch": 0.17,
345
+ "learning_rate": 0.00019890172168151473,
346
+ "loss": 1.5932,
347
+ "step": 55
348
+ },
349
+ {
350
+ "epoch": 0.17,
351
+ "learning_rate": 0.0001988524615588976,
352
+ "loss": 1.6548,
353
+ "step": 56
354
+ },
355
+ {
356
+ "epoch": 0.18,
357
+ "learning_rate": 0.00019880212719697413,
358
+ "loss": 1.6033,
359
+ "step": 57
360
+ },
361
+ {
362
+ "epoch": 0.18,
363
+ "learning_rate": 0.00019875071914273278,
364
+ "loss": 1.6063,
365
+ "step": 58
366
+ },
367
+ {
368
+ "epoch": 0.18,
369
+ "learning_rate": 0.00019869823795482986,
370
+ "loss": 1.6107,
371
+ "step": 59
372
+ },
373
+ {
374
+ "epoch": 0.19,
375
+ "learning_rate": 0.00019864468420358354,
376
+ "loss": 1.5758,
377
+ "step": 60
378
+ },
379
+ {
380
+ "epoch": 0.19,
381
+ "learning_rate": 0.00019859005847096763,
382
+ "loss": 1.5723,
383
+ "step": 61
384
+ },
385
+ {
386
+ "epoch": 0.19,
387
+ "learning_rate": 0.00019853436135060527,
388
+ "loss": 1.542,
389
+ "step": 62
390
+ },
391
+ {
392
+ "epoch": 0.2,
393
+ "learning_rate": 0.00019847759344776252,
394
+ "loss": 1.5611,
395
+ "step": 63
396
+ },
397
+ {
398
+ "epoch": 0.2,
399
+ "learning_rate": 0.00019841975537934162,
400
+ "loss": 1.6157,
401
+ "step": 64
402
+ },
403
+ {
404
+ "epoch": 0.2,
405
+ "learning_rate": 0.00019836084777387458,
406
+ "loss": 1.5589,
407
+ "step": 65
408
+ },
409
+ {
410
+ "epoch": 0.21,
411
+ "learning_rate": 0.00019830087127151598,
412
+ "loss": 1.6077,
413
+ "step": 66
414
+ },
415
+ {
416
+ "epoch": 0.21,
417
+ "learning_rate": 0.00019823982652403634,
418
+ "loss": 1.5473,
419
+ "step": 67
420
+ },
421
+ {
422
+ "epoch": 0.21,
423
+ "learning_rate": 0.00019817771419481487,
424
+ "loss": 1.6265,
425
+ "step": 68
426
+ },
427
+ {
428
+ "epoch": 0.21,
429
+ "learning_rate": 0.0001981145349588323,
430
+ "loss": 1.6074,
431
+ "step": 69
432
+ },
433
+ {
434
+ "epoch": 0.22,
435
+ "learning_rate": 0.00019805028950266348,
436
+ "loss": 1.6195,
437
+ "step": 70
438
+ },
439
+ {
440
+ "epoch": 0.22,
441
+ "learning_rate": 0.00019798497852447006,
442
+ "loss": 1.5876,
443
+ "step": 71
444
+ },
445
+ {
446
+ "epoch": 0.22,
447
+ "learning_rate": 0.0001979186027339928,
448
+ "loss": 1.5978,
449
+ "step": 72
450
+ },
451
+ {
452
+ "epoch": 0.23,
453
+ "learning_rate": 0.00019785116285254381,
454
+ "loss": 1.533,
455
+ "step": 73
456
+ },
457
+ {
458
+ "epoch": 0.23,
459
+ "learning_rate": 0.00019778265961299888,
460
+ "loss": 1.5888,
461
+ "step": 74
462
+ },
463
+ {
464
+ "epoch": 0.23,
465
+ "learning_rate": 0.0001977130937597894,
466
+ "loss": 1.6211,
467
+ "step": 75
468
+ },
469
+ {
470
+ "epoch": 0.24,
471
+ "learning_rate": 0.00019764246604889415,
472
+ "loss": 1.6091,
473
+ "step": 76
474
+ },
475
+ {
476
+ "epoch": 0.24,
477
+ "learning_rate": 0.00019757077724783147,
478
+ "loss": 1.6012,
479
+ "step": 77
480
+ },
481
+ {
482
+ "epoch": 0.24,
483
+ "learning_rate": 0.0001974980281356504,
484
+ "loss": 1.6401,
485
+ "step": 78
486
+ },
487
+ {
488
+ "epoch": 0.25,
489
+ "learning_rate": 0.0001974242195029227,
490
+ "loss": 1.6111,
491
+ "step": 79
492
+ },
493
+ {
494
+ "epoch": 0.25,
495
+ "learning_rate": 0.00019734935215173392,
496
+ "loss": 1.6208,
497
+ "step": 80
498
+ },
499
+ {
500
+ "epoch": 0.25,
501
+ "learning_rate": 0.00019727342689567482,
502
+ "loss": 1.6038,
503
+ "step": 81
504
+ },
505
+ {
506
+ "epoch": 0.26,
507
+ "learning_rate": 0.00019719644455983256,
508
+ "loss": 1.5915,
509
+ "step": 82
510
+ },
511
+ {
512
+ "epoch": 0.26,
513
+ "learning_rate": 0.0001971184059807817,
514
+ "loss": 1.5872,
515
+ "step": 83
516
+ },
517
+ {
518
+ "epoch": 0.26,
519
+ "learning_rate": 0.000197039312006575,
520
+ "loss": 1.5984,
521
+ "step": 84
522
+ },
523
+ {
524
+ "epoch": 0.26,
525
+ "learning_rate": 0.0001969591634967344,
526
+ "loss": 1.5996,
527
+ "step": 85
528
+ },
529
+ {
530
+ "epoch": 0.27,
531
+ "learning_rate": 0.00019687796132224152,
532
+ "loss": 1.6056,
533
+ "step": 86
534
+ },
535
+ {
536
+ "epoch": 0.27,
537
+ "learning_rate": 0.0001967957063655283,
538
+ "loss": 1.6099,
539
+ "step": 87
540
+ },
541
+ {
542
+ "epoch": 0.27,
543
+ "learning_rate": 0.0001967123995204674,
544
+ "loss": 1.6295,
545
+ "step": 88
546
+ },
547
+ {
548
+ "epoch": 0.28,
549
+ "learning_rate": 0.00019662804169236225,
550
+ "loss": 1.5482,
551
+ "step": 89
552
+ },
553
+ {
554
+ "epoch": 0.28,
555
+ "learning_rate": 0.00019654263379793773,
556
+ "loss": 1.5781,
557
+ "step": 90
558
+ },
559
+ {
560
+ "epoch": 0.28,
561
+ "learning_rate": 0.00019645617676532963,
562
+ "loss": 1.5954,
563
+ "step": 91
564
+ },
565
+ {
566
+ "epoch": 0.29,
567
+ "learning_rate": 0.000196368671534075,
568
+ "loss": 1.619,
569
+ "step": 92
570
+ },
571
+ {
572
+ "epoch": 0.29,
573
+ "learning_rate": 0.0001962801190551016,
574
+ "loss": 1.6153,
575
+ "step": 93
576
+ },
577
+ {
578
+ "epoch": 0.29,
579
+ "learning_rate": 0.0001961905202907179,
580
+ "loss": 1.6008,
581
+ "step": 94
582
+ },
583
+ {
584
+ "epoch": 0.3,
585
+ "learning_rate": 0.00019609987621460232,
586
+ "loss": 1.5891,
587
+ "step": 95
588
+ },
589
+ {
590
+ "epoch": 0.3,
591
+ "learning_rate": 0.0001960081878117929,
592
+ "loss": 1.6438,
593
+ "step": 96
594
+ },
595
+ {
596
+ "epoch": 0.3,
597
+ "learning_rate": 0.0001959154560786764,
598
+ "loss": 1.5576,
599
+ "step": 97
600
+ },
601
+ {
602
+ "epoch": 0.3,
603
+ "learning_rate": 0.00019582168202297758,
604
+ "loss": 1.646,
605
+ "step": 98
606
+ },
607
+ {
608
+ "epoch": 0.31,
609
+ "learning_rate": 0.00019572686666374822,
610
+ "loss": 1.6269,
611
+ "step": 99
612
+ },
613
+ {
614
+ "epoch": 0.31,
615
+ "learning_rate": 0.00019563101103135602,
616
+ "loss": 1.6288,
617
+ "step": 100
618
+ },
619
+ {
620
+ "epoch": 0.31,
621
+ "eval_loss": 1.6143836975097656,
622
+ "eval_runtime": 233.6412,
623
+ "eval_samples_per_second": 16.363,
624
+ "eval_steps_per_second": 4.092,
625
+ "step": 100
626
+ },
627
+ {
628
+ "epoch": 0.31,
629
+ "learning_rate": 0.00019553411616747348,
630
+ "loss": 1.5667,
631
+ "step": 101
632
+ },
633
+ {
634
+ "epoch": 0.32,
635
+ "learning_rate": 0.00019543618312506647,
636
+ "loss": 1.6221,
637
+ "step": 102
638
+ },
639
+ {
640
+ "epoch": 0.32,
641
+ "learning_rate": 0.0001953372129683829,
642
+ "loss": 1.5992,
643
+ "step": 103
644
+ },
645
+ {
646
+ "epoch": 0.32,
647
+ "learning_rate": 0.0001952372067729411,
648
+ "loss": 1.6138,
649
+ "step": 104
650
+ },
651
+ {
652
+ "epoch": 0.33,
653
+ "learning_rate": 0.00019513616562551807,
654
+ "loss": 1.51,
655
+ "step": 105
656
+ },
657
+ {
658
+ "epoch": 0.33,
659
+ "learning_rate": 0.00019503409062413782,
660
+ "loss": 1.6227,
661
+ "step": 106
662
+ },
663
+ {
664
+ "epoch": 0.33,
665
+ "learning_rate": 0.00019493098287805927,
666
+ "loss": 1.6014,
667
+ "step": 107
668
+ },
669
+ {
670
+ "epoch": 0.34,
671
+ "learning_rate": 0.00019482684350776434,
672
+ "loss": 1.625,
673
+ "step": 108
674
+ },
675
+ {
676
+ "epoch": 0.34,
677
+ "learning_rate": 0.0001947216736449457,
678
+ "loss": 1.6109,
679
+ "step": 109
680
+ },
681
+ {
682
+ "epoch": 0.34,
683
+ "learning_rate": 0.0001946154744324945,
684
+ "loss": 1.62,
685
+ "step": 110
686
+ },
687
+ {
688
+ "epoch": 0.35,
689
+ "learning_rate": 0.00019450824702448778,
690
+ "loss": 1.5878,
691
+ "step": 111
692
+ },
693
+ {
694
+ "epoch": 0.35,
695
+ "learning_rate": 0.0001943999925861763,
696
+ "loss": 1.6264,
697
+ "step": 112
698
+ },
699
+ {
700
+ "epoch": 0.35,
701
+ "learning_rate": 0.00019429071229397157,
702
+ "loss": 1.6186,
703
+ "step": 113
704
+ },
705
+ {
706
+ "epoch": 0.35,
707
+ "learning_rate": 0.0001941804073354331,
708
+ "loss": 1.6363,
709
+ "step": 114
710
+ },
711
+ {
712
+ "epoch": 0.36,
713
+ "learning_rate": 0.00019406907890925562,
714
+ "loss": 1.5341,
715
+ "step": 115
716
+ },
717
+ {
718
+ "epoch": 0.36,
719
+ "learning_rate": 0.00019395672822525593,
720
+ "loss": 1.5986,
721
+ "step": 116
722
+ },
723
+ {
724
+ "epoch": 0.36,
725
+ "learning_rate": 0.00019384335650435985,
726
+ "loss": 1.6181,
727
+ "step": 117
728
+ },
729
+ {
730
+ "epoch": 0.37,
731
+ "learning_rate": 0.0001937289649785889,
732
+ "loss": 1.6118,
733
+ "step": 118
734
+ },
735
+ {
736
+ "epoch": 0.37,
737
+ "learning_rate": 0.0001936135548910469,
738
+ "loss": 1.6404,
739
+ "step": 119
740
+ },
741
+ {
742
+ "epoch": 0.37,
743
+ "learning_rate": 0.00019349712749590649,
744
+ "loss": 1.583,
745
+ "step": 120
746
+ },
747
+ {
748
+ "epoch": 0.38,
749
+ "learning_rate": 0.00019337968405839547,
750
+ "loss": 1.5827,
751
+ "step": 121
752
+ },
753
+ {
754
+ "epoch": 0.38,
755
+ "learning_rate": 0.00019326122585478308,
756
+ "loss": 1.6392,
757
+ "step": 122
758
+ },
759
+ {
760
+ "epoch": 0.38,
761
+ "learning_rate": 0.00019314175417236616,
762
+ "loss": 1.5861,
763
+ "step": 123
764
+ },
765
+ {
766
+ "epoch": 0.39,
767
+ "learning_rate": 0.00019302127030945508,
768
+ "loss": 1.5738,
769
+ "step": 124
770
+ },
771
+ {
772
+ "epoch": 0.39,
773
+ "learning_rate": 0.0001928997755753597,
774
+ "loss": 1.5915,
775
+ "step": 125
776
+ },
777
+ {
778
+ "epoch": 0.39,
779
+ "learning_rate": 0.00019277727129037508,
780
+ "loss": 1.617,
781
+ "step": 126
782
+ },
783
+ {
784
+ "epoch": 0.4,
785
+ "learning_rate": 0.0001926537587857672,
786
+ "loss": 1.5582,
787
+ "step": 127
788
+ },
789
+ {
790
+ "epoch": 0.4,
791
+ "learning_rate": 0.00019252923940375844,
792
+ "loss": 1.6294,
793
+ "step": 128
794
+ },
795
+ {
796
+ "epoch": 0.4,
797
+ "learning_rate": 0.00019240371449751306,
798
+ "loss": 1.6087,
799
+ "step": 129
800
+ },
801
+ {
802
+ "epoch": 0.4,
803
+ "learning_rate": 0.00019227718543112236,
804
+ "loss": 1.5749,
805
+ "step": 130
806
+ },
807
+ {
808
+ "epoch": 0.41,
809
+ "learning_rate": 0.00019214965357959005,
810
+ "loss": 1.6041,
811
+ "step": 131
812
+ },
813
+ {
814
+ "epoch": 0.41,
815
+ "learning_rate": 0.00019202112032881715,
816
+ "loss": 1.6106,
817
+ "step": 132
818
+ },
819
+ {
820
+ "epoch": 0.41,
821
+ "learning_rate": 0.00019189158707558695,
822
+ "loss": 1.5553,
823
+ "step": 133
824
+ },
825
+ {
826
+ "epoch": 0.42,
827
+ "learning_rate": 0.00019176105522754995,
828
+ "loss": 1.5638,
829
+ "step": 134
830
+ },
831
+ {
832
+ "epoch": 0.42,
833
+ "learning_rate": 0.0001916295262032084,
834
+ "loss": 1.5921,
835
+ "step": 135
836
+ },
837
+ {
838
+ "epoch": 0.42,
839
+ "learning_rate": 0.00019149700143190096,
840
+ "loss": 1.5837,
841
+ "step": 136
842
+ },
843
+ {
844
+ "epoch": 0.43,
845
+ "learning_rate": 0.00019136348235378726,
846
+ "loss": 1.6341,
847
+ "step": 137
848
+ },
849
+ {
850
+ "epoch": 0.43,
851
+ "learning_rate": 0.00019122897041983205,
852
+ "loss": 1.5678,
853
+ "step": 138
854
+ },
855
+ {
856
+ "epoch": 0.43,
857
+ "learning_rate": 0.00019109346709178963,
858
+ "loss": 1.6137,
859
+ "step": 139
860
+ },
861
+ {
862
+ "epoch": 0.44,
863
+ "learning_rate": 0.0001909569738421878,
864
+ "loss": 1.6324,
865
+ "step": 140
866
+ },
867
+ {
868
+ "epoch": 0.44,
869
+ "learning_rate": 0.00019081949215431194,
870
+ "loss": 1.612,
871
+ "step": 141
872
+ },
873
+ {
874
+ "epoch": 0.44,
875
+ "learning_rate": 0.00019068102352218897,
876
+ "loss": 1.5908,
877
+ "step": 142
878
+ },
879
+ {
880
+ "epoch": 0.44,
881
+ "learning_rate": 0.00019054156945057097,
882
+ "loss": 1.6087,
883
+ "step": 143
884
+ },
885
+ {
886
+ "epoch": 0.45,
887
+ "learning_rate": 0.00019040113145491887,
888
+ "loss": 1.5613,
889
+ "step": 144
890
+ },
891
+ {
892
+ "epoch": 0.45,
893
+ "learning_rate": 0.000190259711061386,
894
+ "loss": 1.6072,
895
+ "step": 145
896
+ },
897
+ {
898
+ "epoch": 0.45,
899
+ "learning_rate": 0.00019011730980680156,
900
+ "loss": 1.5722,
901
+ "step": 146
902
+ },
903
+ {
904
+ "epoch": 0.46,
905
+ "learning_rate": 0.0001899739292386538,
906
+ "loss": 1.5961,
907
+ "step": 147
908
+ },
909
+ {
910
+ "epoch": 0.46,
911
+ "learning_rate": 0.00018982957091507325,
912
+ "loss": 1.5409,
913
+ "step": 148
914
+ },
915
+ {
916
+ "epoch": 0.46,
917
+ "learning_rate": 0.0001896842364048159,
918
+ "loss": 1.6557,
919
+ "step": 149
920
+ },
921
+ {
922
+ "epoch": 0.47,
923
+ "learning_rate": 0.000189537927287246,
924
+ "loss": 1.5725,
925
+ "step": 150
926
+ },
927
+ {
928
+ "epoch": 0.47,
929
+ "eval_loss": 1.6101970672607422,
930
+ "eval_runtime": 233.5313,
931
+ "eval_samples_per_second": 16.37,
932
+ "eval_steps_per_second": 4.094,
933
+ "step": 150
934
+ }
935
+ ],
936
+ "logging_steps": 1,
937
+ "max_steps": 963,
938
+ "num_train_epochs": 3,
939
+ "save_steps": 50,
940
+ "total_flos": 4.206407486275584e+17,
941
+ "trial_name": null,
942
+ "trial_params": null
943
+ }
checkpoint-150/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f61cafb89242b653e455003b5517e685ecccfa6180af5fb7d0bfb35b4fc77a4
3
+ size 4475
checkpoint-200/README.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - quant_method: bitsandbytes
9
+ - load_in_8bit: False
10
+ - load_in_4bit: True
11
+ - llm_int8_threshold: 6.0
12
+ - llm_int8_skip_modules: None
13
+ - llm_int8_enable_fp32_cpu_offload: False
14
+ - llm_int8_has_fp16_weight: False
15
+ - bnb_4bit_quant_type: nf4
16
+ - bnb_4bit_use_double_quant: True
17
+ - bnb_4bit_compute_dtype: bfloat16
18
+ ### Framework versions
19
+
20
+
21
+ - PEFT 0.6.0.dev0
checkpoint-200/adapter_config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "./mistralai_Mistral-7B-v0.1",
5
+ "bias": "none",
6
+ "fan_in_fan_out": null,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "lora_alpha": 16,
12
+ "lora_dropout": 0.05,
13
+ "modules_to_save": null,
14
+ "peft_type": "LORA",
15
+ "r": 8,
16
+ "rank_pattern": {},
17
+ "revision": null,
18
+ "target_modules": [
19
+ "o_proj",
20
+ "k_proj",
21
+ "up_proj",
22
+ "v_proj",
23
+ "q_proj",
24
+ "gate_proj",
25
+ "down_proj"
26
+ ],
27
+ "task_type": "CAUSAL_LM"
28
+ }
checkpoint-200/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a22955168eaee3fb137ff861f71f50175e92e749a4651883a21a62d89a683d29
3
+ size 84046925
checkpoint-200/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60fecaee34f97b4a56b90bda6df1c46f987fd91ba3dac957aae197ea931cd966
3
+ size 168039109
checkpoint-200/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af60ac7a22907ef419c05b0c85df6ad791a63738c92e1690745698d449673a5f
3
+ size 14575
checkpoint-200/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:591a15d1a18d8da2c87f78ca1e99fd842dc60d1eb3d13c069154f369561b36af
3
+ size 627
checkpoint-200/trainer_state.json ADDED
@@ -0,0 +1,1251 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 1.6065257787704468,
3
+ "best_model_checkpoint": "./lora-out/checkpoint-200",
4
+ "epoch": 0.6220839813374806,
5
+ "eval_steps": 50,
6
+ "global_step": 200,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0,
13
+ "learning_rate": 2e-05,
14
+ "loss": 1.7924,
15
+ "step": 1
16
+ },
17
+ {
18
+ "epoch": 0.01,
19
+ "learning_rate": 4e-05,
20
+ "loss": 1.8083,
21
+ "step": 2
22
+ },
23
+ {
24
+ "epoch": 0.01,
25
+ "learning_rate": 6e-05,
26
+ "loss": 1.8177,
27
+ "step": 3
28
+ },
29
+ {
30
+ "epoch": 0.01,
31
+ "learning_rate": 8e-05,
32
+ "loss": 1.7595,
33
+ "step": 4
34
+ },
35
+ {
36
+ "epoch": 0.02,
37
+ "learning_rate": 0.0001,
38
+ "loss": 1.6598,
39
+ "step": 5
40
+ },
41
+ {
42
+ "epoch": 0.02,
43
+ "learning_rate": 0.00012,
44
+ "loss": 1.6919,
45
+ "step": 6
46
+ },
47
+ {
48
+ "epoch": 0.02,
49
+ "learning_rate": 0.00014,
50
+ "loss": 1.6706,
51
+ "step": 7
52
+ },
53
+ {
54
+ "epoch": 0.02,
55
+ "learning_rate": 0.00016,
56
+ "loss": 1.6879,
57
+ "step": 8
58
+ },
59
+ {
60
+ "epoch": 0.03,
61
+ "learning_rate": 0.00018,
62
+ "loss": 1.7051,
63
+ "step": 9
64
+ },
65
+ {
66
+ "epoch": 0.03,
67
+ "learning_rate": 0.0002,
68
+ "loss": 1.7022,
69
+ "step": 10
70
+ },
71
+ {
72
+ "epoch": 0.03,
73
+ "learning_rate": 0.000199999456645141,
74
+ "loss": 1.6809,
75
+ "step": 11
76
+ },
77
+ {
78
+ "epoch": 0.04,
79
+ "learning_rate": 0.00019999782658646859,
80
+ "loss": 1.6098,
81
+ "step": 12
82
+ },
83
+ {
84
+ "epoch": 0.04,
85
+ "learning_rate": 0.0001999951098416968,
86
+ "loss": 1.7014,
87
+ "step": 13
88
+ },
89
+ {
90
+ "epoch": 0.04,
91
+ "learning_rate": 0.00019999130644034888,
92
+ "loss": 1.5885,
93
+ "step": 14
94
+ },
95
+ {
96
+ "epoch": 0.05,
97
+ "learning_rate": 0.00019998641642375657,
98
+ "loss": 1.6243,
99
+ "step": 15
100
+ },
101
+ {
102
+ "epoch": 0.05,
103
+ "learning_rate": 0.00019998043984506027,
104
+ "loss": 1.6484,
105
+ "step": 16
106
+ },
107
+ {
108
+ "epoch": 0.05,
109
+ "learning_rate": 0.00019997337676920803,
110
+ "loss": 1.6093,
111
+ "step": 17
112
+ },
113
+ {
114
+ "epoch": 0.06,
115
+ "learning_rate": 0.00019996522727295496,
116
+ "loss": 1.6173,
117
+ "step": 18
118
+ },
119
+ {
120
+ "epoch": 0.06,
121
+ "learning_rate": 0.00019995599144486247,
122
+ "loss": 1.646,
123
+ "step": 19
124
+ },
125
+ {
126
+ "epoch": 0.06,
127
+ "learning_rate": 0.00019994566938529712,
128
+ "loss": 1.6469,
129
+ "step": 20
130
+ },
131
+ {
132
+ "epoch": 0.07,
133
+ "learning_rate": 0.00019993426120642983,
134
+ "loss": 1.6564,
135
+ "step": 21
136
+ },
137
+ {
138
+ "epoch": 0.07,
139
+ "learning_rate": 0.00019992176703223432,
140
+ "loss": 1.5901,
141
+ "step": 22
142
+ },
143
+ {
144
+ "epoch": 0.07,
145
+ "learning_rate": 0.000199908186998486,
146
+ "loss": 1.664,
147
+ "step": 23
148
+ },
149
+ {
150
+ "epoch": 0.07,
151
+ "learning_rate": 0.00019989352125276047,
152
+ "loss": 1.6275,
153
+ "step": 24
154
+ },
155
+ {
156
+ "epoch": 0.08,
157
+ "learning_rate": 0.00019987776995443178,
158
+ "loss": 1.5839,
159
+ "step": 25
160
+ },
161
+ {
162
+ "epoch": 0.08,
163
+ "learning_rate": 0.00019986093327467076,
164
+ "loss": 1.5611,
165
+ "step": 26
166
+ },
167
+ {
168
+ "epoch": 0.08,
169
+ "learning_rate": 0.00019984301139644334,
170
+ "loss": 1.669,
171
+ "step": 27
172
+ },
173
+ {
174
+ "epoch": 0.09,
175
+ "learning_rate": 0.0001998240045145083,
176
+ "loss": 1.5641,
177
+ "step": 28
178
+ },
179
+ {
180
+ "epoch": 0.09,
181
+ "learning_rate": 0.00019980391283541522,
182
+ "loss": 1.6023,
183
+ "step": 29
184
+ },
185
+ {
186
+ "epoch": 0.09,
187
+ "learning_rate": 0.00019978273657750238,
188
+ "loss": 1.6309,
189
+ "step": 30
190
+ },
191
+ {
192
+ "epoch": 0.1,
193
+ "learning_rate": 0.0001997604759708942,
194
+ "loss": 1.6353,
195
+ "step": 31
196
+ },
197
+ {
198
+ "epoch": 0.1,
199
+ "learning_rate": 0.00019973713125749884,
200
+ "loss": 1.6328,
201
+ "step": 32
202
+ },
203
+ {
204
+ "epoch": 0.1,
205
+ "learning_rate": 0.00019971270269100564,
206
+ "loss": 1.5683,
207
+ "step": 33
208
+ },
209
+ {
210
+ "epoch": 0.11,
211
+ "learning_rate": 0.00019968719053688213,
212
+ "loss": 1.6217,
213
+ "step": 34
214
+ },
215
+ {
216
+ "epoch": 0.11,
217
+ "learning_rate": 0.0001996605950723714,
218
+ "loss": 1.5734,
219
+ "step": 35
220
+ },
221
+ {
222
+ "epoch": 0.11,
223
+ "learning_rate": 0.00019963291658648896,
224
+ "loss": 1.6162,
225
+ "step": 36
226
+ },
227
+ {
228
+ "epoch": 0.12,
229
+ "learning_rate": 0.00019960415538001957,
230
+ "loss": 1.5922,
231
+ "step": 37
232
+ },
233
+ {
234
+ "epoch": 0.12,
235
+ "learning_rate": 0.0001995743117655141,
236
+ "loss": 1.5806,
237
+ "step": 38
238
+ },
239
+ {
240
+ "epoch": 0.12,
241
+ "learning_rate": 0.000199543386067286,
242
+ "loss": 1.5938,
243
+ "step": 39
244
+ },
245
+ {
246
+ "epoch": 0.12,
247
+ "learning_rate": 0.00019951137862140778,
248
+ "loss": 1.6386,
249
+ "step": 40
250
+ },
251
+ {
252
+ "epoch": 0.13,
253
+ "learning_rate": 0.00019947828977570756,
254
+ "loss": 1.6476,
255
+ "step": 41
256
+ },
257
+ {
258
+ "epoch": 0.13,
259
+ "learning_rate": 0.00019944411988976496,
260
+ "loss": 1.6557,
261
+ "step": 42
262
+ },
263
+ {
264
+ "epoch": 0.13,
265
+ "learning_rate": 0.00019940886933490749,
266
+ "loss": 1.5836,
267
+ "step": 43
268
+ },
269
+ {
270
+ "epoch": 0.14,
271
+ "learning_rate": 0.00019937253849420635,
272
+ "loss": 1.6421,
273
+ "step": 44
274
+ },
275
+ {
276
+ "epoch": 0.14,
277
+ "learning_rate": 0.0001993351277624723,
278
+ "loss": 1.629,
279
+ "step": 45
280
+ },
281
+ {
282
+ "epoch": 0.14,
283
+ "learning_rate": 0.00019929663754625145,
284
+ "loss": 1.6392,
285
+ "step": 46
286
+ },
287
+ {
288
+ "epoch": 0.15,
289
+ "learning_rate": 0.00019925706826382064,
290
+ "loss": 1.5677,
291
+ "step": 47
292
+ },
293
+ {
294
+ "epoch": 0.15,
295
+ "learning_rate": 0.00019921642034518317,
296
+ "loss": 1.6144,
297
+ "step": 48
298
+ },
299
+ {
300
+ "epoch": 0.15,
301
+ "learning_rate": 0.00019917469423206389,
302
+ "loss": 1.6068,
303
+ "step": 49
304
+ },
305
+ {
306
+ "epoch": 0.16,
307
+ "learning_rate": 0.00019913189037790456,
308
+ "loss": 1.6421,
309
+ "step": 50
310
+ },
311
+ {
312
+ "epoch": 0.16,
313
+ "eval_loss": 1.621693730354309,
314
+ "eval_runtime": 233.7603,
315
+ "eval_samples_per_second": 16.354,
316
+ "eval_steps_per_second": 4.09,
317
+ "step": 50
318
+ },
319
+ {
320
+ "epoch": 0.16,
321
+ "learning_rate": 0.0001990880092478588,
322
+ "loss": 1.6172,
323
+ "step": 51
324
+ },
325
+ {
326
+ "epoch": 0.16,
327
+ "learning_rate": 0.0001990430513187871,
328
+ "loss": 1.6095,
329
+ "step": 52
330
+ },
331
+ {
332
+ "epoch": 0.16,
333
+ "learning_rate": 0.00019899701707925166,
334
+ "loss": 1.5967,
335
+ "step": 53
336
+ },
337
+ {
338
+ "epoch": 0.17,
339
+ "learning_rate": 0.00019894990702951106,
340
+ "loss": 1.617,
341
+ "step": 54
342
+ },
343
+ {
344
+ "epoch": 0.17,
345
+ "learning_rate": 0.00019890172168151473,
346
+ "loss": 1.5932,
347
+ "step": 55
348
+ },
349
+ {
350
+ "epoch": 0.17,
351
+ "learning_rate": 0.0001988524615588976,
352
+ "loss": 1.6548,
353
+ "step": 56
354
+ },
355
+ {
356
+ "epoch": 0.18,
357
+ "learning_rate": 0.00019880212719697413,
358
+ "loss": 1.6033,
359
+ "step": 57
360
+ },
361
+ {
362
+ "epoch": 0.18,
363
+ "learning_rate": 0.00019875071914273278,
364
+ "loss": 1.6063,
365
+ "step": 58
366
+ },
367
+ {
368
+ "epoch": 0.18,
369
+ "learning_rate": 0.00019869823795482986,
370
+ "loss": 1.6107,
371
+ "step": 59
372
+ },
373
+ {
374
+ "epoch": 0.19,
375
+ "learning_rate": 0.00019864468420358354,
376
+ "loss": 1.5758,
377
+ "step": 60
378
+ },
379
+ {
380
+ "epoch": 0.19,
381
+ "learning_rate": 0.00019859005847096763,
382
+ "loss": 1.5723,
383
+ "step": 61
384
+ },
385
+ {
386
+ "epoch": 0.19,
387
+ "learning_rate": 0.00019853436135060527,
388
+ "loss": 1.542,
389
+ "step": 62
390
+ },
391
+ {
392
+ "epoch": 0.2,
393
+ "learning_rate": 0.00019847759344776252,
394
+ "loss": 1.5611,
395
+ "step": 63
396
+ },
397
+ {
398
+ "epoch": 0.2,
399
+ "learning_rate": 0.00019841975537934162,
400
+ "loss": 1.6157,
401
+ "step": 64
402
+ },
403
+ {
404
+ "epoch": 0.2,
405
+ "learning_rate": 0.00019836084777387458,
406
+ "loss": 1.5589,
407
+ "step": 65
408
+ },
409
+ {
410
+ "epoch": 0.21,
411
+ "learning_rate": 0.00019830087127151598,
412
+ "loss": 1.6077,
413
+ "step": 66
414
+ },
415
+ {
416
+ "epoch": 0.21,
417
+ "learning_rate": 0.00019823982652403634,
418
+ "loss": 1.5473,
419
+ "step": 67
420
+ },
421
+ {
422
+ "epoch": 0.21,
423
+ "learning_rate": 0.00019817771419481487,
424
+ "loss": 1.6265,
425
+ "step": 68
426
+ },
427
+ {
428
+ "epoch": 0.21,
429
+ "learning_rate": 0.0001981145349588323,
430
+ "loss": 1.6074,
431
+ "step": 69
432
+ },
433
+ {
434
+ "epoch": 0.22,
435
+ "learning_rate": 0.00019805028950266348,
436
+ "loss": 1.6195,
437
+ "step": 70
438
+ },
439
+ {
440
+ "epoch": 0.22,
441
+ "learning_rate": 0.00019798497852447006,
442
+ "loss": 1.5876,
443
+ "step": 71
444
+ },
445
+ {
446
+ "epoch": 0.22,
447
+ "learning_rate": 0.0001979186027339928,
448
+ "loss": 1.5978,
449
+ "step": 72
450
+ },
451
+ {
452
+ "epoch": 0.23,
453
+ "learning_rate": 0.00019785116285254381,
454
+ "loss": 1.533,
455
+ "step": 73
456
+ },
457
+ {
458
+ "epoch": 0.23,
459
+ "learning_rate": 0.00019778265961299888,
460
+ "loss": 1.5888,
461
+ "step": 74
462
+ },
463
+ {
464
+ "epoch": 0.23,
465
+ "learning_rate": 0.0001977130937597894,
466
+ "loss": 1.6211,
467
+ "step": 75
468
+ },
469
+ {
470
+ "epoch": 0.24,
471
+ "learning_rate": 0.00019764246604889415,
472
+ "loss": 1.6091,
473
+ "step": 76
474
+ },
475
+ {
476
+ "epoch": 0.24,
477
+ "learning_rate": 0.00019757077724783147,
478
+ "loss": 1.6012,
479
+ "step": 77
480
+ },
481
+ {
482
+ "epoch": 0.24,
483
+ "learning_rate": 0.0001974980281356504,
484
+ "loss": 1.6401,
485
+ "step": 78
486
+ },
487
+ {
488
+ "epoch": 0.25,
489
+ "learning_rate": 0.0001974242195029227,
490
+ "loss": 1.6111,
491
+ "step": 79
492
+ },
493
+ {
494
+ "epoch": 0.25,
495
+ "learning_rate": 0.00019734935215173392,
496
+ "loss": 1.6208,
497
+ "step": 80
498
+ },
499
+ {
500
+ "epoch": 0.25,
501
+ "learning_rate": 0.00019727342689567482,
502
+ "loss": 1.6038,
503
+ "step": 81
504
+ },
505
+ {
506
+ "epoch": 0.26,
507
+ "learning_rate": 0.00019719644455983256,
508
+ "loss": 1.5915,
509
+ "step": 82
510
+ },
511
+ {
512
+ "epoch": 0.26,
513
+ "learning_rate": 0.0001971184059807817,
514
+ "loss": 1.5872,
515
+ "step": 83
516
+ },
517
+ {
518
+ "epoch": 0.26,
519
+ "learning_rate": 0.000197039312006575,
520
+ "loss": 1.5984,
521
+ "step": 84
522
+ },
523
+ {
524
+ "epoch": 0.26,
525
+ "learning_rate": 0.0001969591634967344,
526
+ "loss": 1.5996,
527
+ "step": 85
528
+ },
529
+ {
530
+ "epoch": 0.27,
531
+ "learning_rate": 0.00019687796132224152,
532
+ "loss": 1.6056,
533
+ "step": 86
534
+ },
535
+ {
536
+ "epoch": 0.27,
537
+ "learning_rate": 0.0001967957063655283,
538
+ "loss": 1.6099,
539
+ "step": 87
540
+ },
541
+ {
542
+ "epoch": 0.27,
543
+ "learning_rate": 0.0001967123995204674,
544
+ "loss": 1.6295,
545
+ "step": 88
546
+ },
547
+ {
548
+ "epoch": 0.28,
549
+ "learning_rate": 0.00019662804169236225,
550
+ "loss": 1.5482,
551
+ "step": 89
552
+ },
553
+ {
554
+ "epoch": 0.28,
555
+ "learning_rate": 0.00019654263379793773,
556
+ "loss": 1.5781,
557
+ "step": 90
558
+ },
559
+ {
560
+ "epoch": 0.28,
561
+ "learning_rate": 0.00019645617676532963,
562
+ "loss": 1.5954,
563
+ "step": 91
564
+ },
565
+ {
566
+ "epoch": 0.29,
567
+ "learning_rate": 0.000196368671534075,
568
+ "loss": 1.619,
569
+ "step": 92
570
+ },
571
+ {
572
+ "epoch": 0.29,
573
+ "learning_rate": 0.0001962801190551016,
574
+ "loss": 1.6153,
575
+ "step": 93
576
+ },
577
+ {
578
+ "epoch": 0.29,
579
+ "learning_rate": 0.0001961905202907179,
580
+ "loss": 1.6008,
581
+ "step": 94
582
+ },
583
+ {
584
+ "epoch": 0.3,
585
+ "learning_rate": 0.00019609987621460232,
586
+ "loss": 1.5891,
587
+ "step": 95
588
+ },
589
+ {
590
+ "epoch": 0.3,
591
+ "learning_rate": 0.0001960081878117929,
592
+ "loss": 1.6438,
593
+ "step": 96
594
+ },
595
+ {
596
+ "epoch": 0.3,
597
+ "learning_rate": 0.0001959154560786764,
598
+ "loss": 1.5576,
599
+ "step": 97
600
+ },
601
+ {
602
+ "epoch": 0.3,
603
+ "learning_rate": 0.00019582168202297758,
604
+ "loss": 1.646,
605
+ "step": 98
606
+ },
607
+ {
608
+ "epoch": 0.31,
609
+ "learning_rate": 0.00019572686666374822,
610
+ "loss": 1.6269,
611
+ "step": 99
612
+ },
613
+ {
614
+ "epoch": 0.31,
615
+ "learning_rate": 0.00019563101103135602,
616
+ "loss": 1.6288,
617
+ "step": 100
618
+ },
619
+ {
620
+ "epoch": 0.31,
621
+ "eval_loss": 1.6143836975097656,
622
+ "eval_runtime": 233.6412,
623
+ "eval_samples_per_second": 16.363,
624
+ "eval_steps_per_second": 4.092,
625
+ "step": 100
626
+ },
627
+ {
628
+ "epoch": 0.31,
629
+ "learning_rate": 0.00019553411616747348,
630
+ "loss": 1.5667,
631
+ "step": 101
632
+ },
633
+ {
634
+ "epoch": 0.32,
635
+ "learning_rate": 0.00019543618312506647,
636
+ "loss": 1.6221,
637
+ "step": 102
638
+ },
639
+ {
640
+ "epoch": 0.32,
641
+ "learning_rate": 0.0001953372129683829,
642
+ "loss": 1.5992,
643
+ "step": 103
644
+ },
645
+ {
646
+ "epoch": 0.32,
647
+ "learning_rate": 0.0001952372067729411,
648
+ "loss": 1.6138,
649
+ "step": 104
650
+ },
651
+ {
652
+ "epoch": 0.33,
653
+ "learning_rate": 0.00019513616562551807,
654
+ "loss": 1.51,
655
+ "step": 105
656
+ },
657
+ {
658
+ "epoch": 0.33,
659
+ "learning_rate": 0.00019503409062413782,
660
+ "loss": 1.6227,
661
+ "step": 106
662
+ },
663
+ {
664
+ "epoch": 0.33,
665
+ "learning_rate": 0.00019493098287805927,
666
+ "loss": 1.6014,
667
+ "step": 107
668
+ },
669
+ {
670
+ "epoch": 0.34,
671
+ "learning_rate": 0.00019482684350776434,
672
+ "loss": 1.625,
673
+ "step": 108
674
+ },
675
+ {
676
+ "epoch": 0.34,
677
+ "learning_rate": 0.0001947216736449457,
678
+ "loss": 1.6109,
679
+ "step": 109
680
+ },
681
+ {
682
+ "epoch": 0.34,
683
+ "learning_rate": 0.0001946154744324945,
684
+ "loss": 1.62,
685
+ "step": 110
686
+ },
687
+ {
688
+ "epoch": 0.35,
689
+ "learning_rate": 0.00019450824702448778,
690
+ "loss": 1.5878,
691
+ "step": 111
692
+ },
693
+ {
694
+ "epoch": 0.35,
695
+ "learning_rate": 0.0001943999925861763,
696
+ "loss": 1.6264,
697
+ "step": 112
698
+ },
699
+ {
700
+ "epoch": 0.35,
701
+ "learning_rate": 0.00019429071229397157,
702
+ "loss": 1.6186,
703
+ "step": 113
704
+ },
705
+ {
706
+ "epoch": 0.35,
707
+ "learning_rate": 0.0001941804073354331,
708
+ "loss": 1.6363,
709
+ "step": 114
710
+ },
711
+ {
712
+ "epoch": 0.36,
713
+ "learning_rate": 0.00019406907890925562,
714
+ "loss": 1.5341,
715
+ "step": 115
716
+ },
717
+ {
718
+ "epoch": 0.36,
719
+ "learning_rate": 0.00019395672822525593,
720
+ "loss": 1.5986,
721
+ "step": 116
722
+ },
723
+ {
724
+ "epoch": 0.36,
725
+ "learning_rate": 0.00019384335650435985,
726
+ "loss": 1.6181,
727
+ "step": 117
728
+ },
729
+ {
730
+ "epoch": 0.37,
731
+ "learning_rate": 0.0001937289649785889,
732
+ "loss": 1.6118,
733
+ "step": 118
734
+ },
735
+ {
736
+ "epoch": 0.37,
737
+ "learning_rate": 0.0001936135548910469,
738
+ "loss": 1.6404,
739
+ "step": 119
740
+ },
741
+ {
742
+ "epoch": 0.37,
743
+ "learning_rate": 0.00019349712749590649,
744
+ "loss": 1.583,
745
+ "step": 120
746
+ },
747
+ {
748
+ "epoch": 0.38,
749
+ "learning_rate": 0.00019337968405839547,
750
+ "loss": 1.5827,
751
+ "step": 121
752
+ },
753
+ {
754
+ "epoch": 0.38,
755
+ "learning_rate": 0.00019326122585478308,
756
+ "loss": 1.6392,
757
+ "step": 122
758
+ },
759
+ {
760
+ "epoch": 0.38,
761
+ "learning_rate": 0.00019314175417236616,
762
+ "loss": 1.5861,
763
+ "step": 123
764
+ },
765
+ {
766
+ "epoch": 0.39,
767
+ "learning_rate": 0.00019302127030945508,
768
+ "loss": 1.5738,
769
+ "step": 124
770
+ },
771
+ {
772
+ "epoch": 0.39,
773
+ "learning_rate": 0.0001928997755753597,
774
+ "loss": 1.5915,
775
+ "step": 125
776
+ },
777
+ {
778
+ "epoch": 0.39,
779
+ "learning_rate": 0.00019277727129037508,
780
+ "loss": 1.617,
781
+ "step": 126
782
+ },
783
+ {
784
+ "epoch": 0.4,
785
+ "learning_rate": 0.0001926537587857672,
786
+ "loss": 1.5582,
787
+ "step": 127
788
+ },
789
+ {
790
+ "epoch": 0.4,
791
+ "learning_rate": 0.00019252923940375844,
792
+ "loss": 1.6294,
793
+ "step": 128
794
+ },
795
+ {
796
+ "epoch": 0.4,
797
+ "learning_rate": 0.00019240371449751306,
798
+ "loss": 1.6087,
799
+ "step": 129
800
+ },
801
+ {
802
+ "epoch": 0.4,
803
+ "learning_rate": 0.00019227718543112236,
804
+ "loss": 1.5749,
805
+ "step": 130
806
+ },
807
+ {
808
+ "epoch": 0.41,
809
+ "learning_rate": 0.00019214965357959005,
810
+ "loss": 1.6041,
811
+ "step": 131
812
+ },
813
+ {
814
+ "epoch": 0.41,
815
+ "learning_rate": 0.00019202112032881715,
816
+ "loss": 1.6106,
817
+ "step": 132
818
+ },
819
+ {
820
+ "epoch": 0.41,
821
+ "learning_rate": 0.00019189158707558695,
822
+ "loss": 1.5553,
823
+ "step": 133
824
+ },
825
+ {
826
+ "epoch": 0.42,
827
+ "learning_rate": 0.00019176105522754995,
828
+ "loss": 1.5638,
829
+ "step": 134
830
+ },
831
+ {
832
+ "epoch": 0.42,
833
+ "learning_rate": 0.0001916295262032084,
834
+ "loss": 1.5921,
835
+ "step": 135
836
+ },
837
+ {
838
+ "epoch": 0.42,
839
+ "learning_rate": 0.00019149700143190096,
840
+ "loss": 1.5837,
841
+ "step": 136
842
+ },
843
+ {
844
+ "epoch": 0.43,
845
+ "learning_rate": 0.00019136348235378726,
846
+ "loss": 1.6341,
847
+ "step": 137
848
+ },
849
+ {
850
+ "epoch": 0.43,
851
+ "learning_rate": 0.00019122897041983205,
852
+ "loss": 1.5678,
853
+ "step": 138
854
+ },
855
+ {
856
+ "epoch": 0.43,
857
+ "learning_rate": 0.00019109346709178963,
858
+ "loss": 1.6137,
859
+ "step": 139
860
+ },
861
+ {
862
+ "epoch": 0.44,
863
+ "learning_rate": 0.0001909569738421878,
864
+ "loss": 1.6324,
865
+ "step": 140
866
+ },
867
+ {
868
+ "epoch": 0.44,
869
+ "learning_rate": 0.00019081949215431194,
870
+ "loss": 1.612,
871
+ "step": 141
872
+ },
873
+ {
874
+ "epoch": 0.44,
875
+ "learning_rate": 0.00019068102352218897,
876
+ "loss": 1.5908,
877
+ "step": 142
878
+ },
879
+ {
880
+ "epoch": 0.44,
881
+ "learning_rate": 0.00019054156945057097,
882
+ "loss": 1.6087,
883
+ "step": 143
884
+ },
885
+ {
886
+ "epoch": 0.45,
887
+ "learning_rate": 0.00019040113145491887,
888
+ "loss": 1.5613,
889
+ "step": 144
890
+ },
891
+ {
892
+ "epoch": 0.45,
893
+ "learning_rate": 0.000190259711061386,
894
+ "loss": 1.6072,
895
+ "step": 145
896
+ },
897
+ {
898
+ "epoch": 0.45,
899
+ "learning_rate": 0.00019011730980680156,
900
+ "loss": 1.5722,
901
+ "step": 146
902
+ },
903
+ {
904
+ "epoch": 0.46,
905
+ "learning_rate": 0.0001899739292386538,
906
+ "loss": 1.5961,
907
+ "step": 147
908
+ },
909
+ {
910
+ "epoch": 0.46,
911
+ "learning_rate": 0.00018982957091507325,
912
+ "loss": 1.5409,
913
+ "step": 148
914
+ },
915
+ {
916
+ "epoch": 0.46,
917
+ "learning_rate": 0.0001896842364048159,
918
+ "loss": 1.6557,
919
+ "step": 149
920
+ },
921
+ {
922
+ "epoch": 0.47,
923
+ "learning_rate": 0.000189537927287246,
924
+ "loss": 1.5725,
925
+ "step": 150
926
+ },
927
+ {
928
+ "epoch": 0.47,
929
+ "eval_loss": 1.6101970672607422,
930
+ "eval_runtime": 233.5313,
931
+ "eval_samples_per_second": 16.37,
932
+ "eval_steps_per_second": 4.094,
933
+ "step": 150
934
+ },
935
+ {
936
+ "epoch": 0.47,
937
+ "learning_rate": 0.00018939064515231888,
938
+ "loss": 1.5949,
939
+ "step": 151
940
+ },
941
+ {
942
+ "epoch": 0.47,
943
+ "learning_rate": 0.0001892423916005639,
944
+ "loss": 1.6191,
945
+ "step": 152
946
+ },
947
+ {
948
+ "epoch": 0.48,
949
+ "learning_rate": 0.00018909316824306674,
950
+ "loss": 1.5487,
951
+ "step": 153
952
+ },
953
+ {
954
+ "epoch": 0.48,
955
+ "learning_rate": 0.00018894297670145216,
956
+ "loss": 1.5104,
957
+ "step": 154
958
+ },
959
+ {
960
+ "epoch": 0.48,
961
+ "learning_rate": 0.00018879181860786623,
962
+ "loss": 1.6392,
963
+ "step": 155
964
+ },
965
+ {
966
+ "epoch": 0.49,
967
+ "learning_rate": 0.00018863969560495866,
968
+ "loss": 1.5932,
969
+ "step": 156
970
+ },
971
+ {
972
+ "epoch": 0.49,
973
+ "learning_rate": 0.00018848660934586491,
974
+ "loss": 1.6213,
975
+ "step": 157
976
+ },
977
+ {
978
+ "epoch": 0.49,
979
+ "learning_rate": 0.0001883325614941882,
980
+ "loss": 1.5515,
981
+ "step": 158
982
+ },
983
+ {
984
+ "epoch": 0.49,
985
+ "learning_rate": 0.00018817755372398155,
986
+ "loss": 1.6166,
987
+ "step": 159
988
+ },
989
+ {
990
+ "epoch": 0.5,
991
+ "learning_rate": 0.00018802158771972943,
992
+ "loss": 1.6552,
993
+ "step": 160
994
+ },
995
+ {
996
+ "epoch": 0.5,
997
+ "learning_rate": 0.00018786466517632956,
998
+ "loss": 1.6378,
999
+ "step": 161
1000
+ },
1001
+ {
1002
+ "epoch": 0.5,
1003
+ "learning_rate": 0.00018770678779907448,
1004
+ "loss": 1.5176,
1005
+ "step": 162
1006
+ },
1007
+ {
1008
+ "epoch": 0.51,
1009
+ "learning_rate": 0.00018754795730363302,
1010
+ "loss": 1.5793,
1011
+ "step": 163
1012
+ },
1013
+ {
1014
+ "epoch": 0.51,
1015
+ "learning_rate": 0.00018738817541603156,
1016
+ "loss": 1.6616,
1017
+ "step": 164
1018
+ },
1019
+ {
1020
+ "epoch": 0.51,
1021
+ "learning_rate": 0.00018722744387263544,
1022
+ "loss": 1.6055,
1023
+ "step": 165
1024
+ },
1025
+ {
1026
+ "epoch": 0.52,
1027
+ "learning_rate": 0.00018706576442012994,
1028
+ "loss": 1.6204,
1029
+ "step": 166
1030
+ },
1031
+ {
1032
+ "epoch": 0.52,
1033
+ "learning_rate": 0.00018690313881550137,
1034
+ "loss": 1.5952,
1035
+ "step": 167
1036
+ },
1037
+ {
1038
+ "epoch": 0.52,
1039
+ "learning_rate": 0.00018673956882601803,
1040
+ "loss": 1.6271,
1041
+ "step": 168
1042
+ },
1043
+ {
1044
+ "epoch": 0.53,
1045
+ "learning_rate": 0.00018657505622921082,
1046
+ "loss": 1.538,
1047
+ "step": 169
1048
+ },
1049
+ {
1050
+ "epoch": 0.53,
1051
+ "learning_rate": 0.00018640960281285417,
1052
+ "loss": 1.5874,
1053
+ "step": 170
1054
+ },
1055
+ {
1056
+ "epoch": 0.53,
1057
+ "learning_rate": 0.0001862432103749464,
1058
+ "loss": 1.5694,
1059
+ "step": 171
1060
+ },
1061
+ {
1062
+ "epoch": 0.53,
1063
+ "learning_rate": 0.00018607588072369033,
1064
+ "loss": 1.583,
1065
+ "step": 172
1066
+ },
1067
+ {
1068
+ "epoch": 0.54,
1069
+ "learning_rate": 0.00018590761567747354,
1070
+ "loss": 1.5961,
1071
+ "step": 173
1072
+ },
1073
+ {
1074
+ "epoch": 0.54,
1075
+ "learning_rate": 0.00018573841706484866,
1076
+ "loss": 1.582,
1077
+ "step": 174
1078
+ },
1079
+ {
1080
+ "epoch": 0.54,
1081
+ "learning_rate": 0.0001855682867245134,
1082
+ "loss": 1.6427,
1083
+ "step": 175
1084
+ },
1085
+ {
1086
+ "epoch": 0.55,
1087
+ "learning_rate": 0.00018539722650529075,
1088
+ "loss": 1.604,
1089
+ "step": 176
1090
+ },
1091
+ {
1092
+ "epoch": 0.55,
1093
+ "learning_rate": 0.00018522523826610868,
1094
+ "loss": 1.577,
1095
+ "step": 177
1096
+ },
1097
+ {
1098
+ "epoch": 0.55,
1099
+ "learning_rate": 0.00018505232387598018,
1100
+ "loss": 1.6339,
1101
+ "step": 178
1102
+ },
1103
+ {
1104
+ "epoch": 0.56,
1105
+ "learning_rate": 0.00018487848521398265,
1106
+ "loss": 1.5993,
1107
+ "step": 179
1108
+ },
1109
+ {
1110
+ "epoch": 0.56,
1111
+ "learning_rate": 0.0001847037241692378,
1112
+ "loss": 1.6286,
1113
+ "step": 180
1114
+ },
1115
+ {
1116
+ "epoch": 0.56,
1117
+ "learning_rate": 0.00018452804264089084,
1118
+ "loss": 1.5963,
1119
+ "step": 181
1120
+ },
1121
+ {
1122
+ "epoch": 0.57,
1123
+ "learning_rate": 0.00018435144253809,
1124
+ "loss": 1.5856,
1125
+ "step": 182
1126
+ },
1127
+ {
1128
+ "epoch": 0.57,
1129
+ "learning_rate": 0.00018417392577996578,
1130
+ "loss": 1.5787,
1131
+ "step": 183
1132
+ },
1133
+ {
1134
+ "epoch": 0.57,
1135
+ "learning_rate": 0.00018399549429561006,
1136
+ "loss": 1.5876,
1137
+ "step": 184
1138
+ },
1139
+ {
1140
+ "epoch": 0.58,
1141
+ "learning_rate": 0.00018381615002405509,
1142
+ "loss": 1.5565,
1143
+ "step": 185
1144
+ },
1145
+ {
1146
+ "epoch": 0.58,
1147
+ "learning_rate": 0.00018363589491425248,
1148
+ "loss": 1.5897,
1149
+ "step": 186
1150
+ },
1151
+ {
1152
+ "epoch": 0.58,
1153
+ "learning_rate": 0.0001834547309250521,
1154
+ "loss": 1.5951,
1155
+ "step": 187
1156
+ },
1157
+ {
1158
+ "epoch": 0.58,
1159
+ "learning_rate": 0.00018327266002518056,
1160
+ "loss": 1.5447,
1161
+ "step": 188
1162
+ },
1163
+ {
1164
+ "epoch": 0.59,
1165
+ "learning_rate": 0.00018308968419322003,
1166
+ "loss": 1.6087,
1167
+ "step": 189
1168
+ },
1169
+ {
1170
+ "epoch": 0.59,
1171
+ "learning_rate": 0.00018290580541758668,
1172
+ "loss": 1.5946,
1173
+ "step": 190
1174
+ },
1175
+ {
1176
+ "epoch": 0.59,
1177
+ "learning_rate": 0.00018272102569650905,
1178
+ "loss": 1.6148,
1179
+ "step": 191
1180
+ },
1181
+ {
1182
+ "epoch": 0.6,
1183
+ "learning_rate": 0.00018253534703800627,
1184
+ "loss": 1.649,
1185
+ "step": 192
1186
+ },
1187
+ {
1188
+ "epoch": 0.6,
1189
+ "learning_rate": 0.0001823487714598664,
1190
+ "loss": 1.6312,
1191
+ "step": 193
1192
+ },
1193
+ {
1194
+ "epoch": 0.6,
1195
+ "learning_rate": 0.0001821613009896244,
1196
+ "loss": 1.5858,
1197
+ "step": 194
1198
+ },
1199
+ {
1200
+ "epoch": 0.61,
1201
+ "learning_rate": 0.00018197293766454003,
1202
+ "loss": 1.5925,
1203
+ "step": 195
1204
+ },
1205
+ {
1206
+ "epoch": 0.61,
1207
+ "learning_rate": 0.0001817836835315759,
1208
+ "loss": 1.5604,
1209
+ "step": 196
1210
+ },
1211
+ {
1212
+ "epoch": 0.61,
1213
+ "learning_rate": 0.00018159354064737506,
1214
+ "loss": 1.6125,
1215
+ "step": 197
1216
+ },
1217
+ {
1218
+ "epoch": 0.62,
1219
+ "learning_rate": 0.0001814025110782387,
1220
+ "loss": 1.5954,
1221
+ "step": 198
1222
+ },
1223
+ {
1224
+ "epoch": 0.62,
1225
+ "learning_rate": 0.00018121059690010368,
1226
+ "loss": 1.5937,
1227
+ "step": 199
1228
+ },
1229
+ {
1230
+ "epoch": 0.62,
1231
+ "learning_rate": 0.00018101780019852008,
1232
+ "loss": 1.5582,
1233
+ "step": 200
1234
+ },
1235
+ {
1236
+ "epoch": 0.62,
1237
+ "eval_loss": 1.6065257787704468,
1238
+ "eval_runtime": 233.7919,
1239
+ "eval_samples_per_second": 16.352,
1240
+ "eval_steps_per_second": 4.089,
1241
+ "step": 200
1242
+ }
1243
+ ],
1244
+ "logging_steps": 1,
1245
+ "max_steps": 963,
1246
+ "num_train_epochs": 3,
1247
+ "save_steps": 50,
1248
+ "total_flos": 5.608543315034112e+17,
1249
+ "trial_name": null,
1250
+ "trial_params": null
1251
+ }
checkpoint-200/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f61cafb89242b653e455003b5517e685ecccfa6180af5fb7d0bfb35b4fc77a4
3
+ size 4475
checkpoint-250/README.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - quant_method: bitsandbytes
9
+ - load_in_8bit: False
10
+ - load_in_4bit: True
11
+ - llm_int8_threshold: 6.0
12
+ - llm_int8_skip_modules: None
13
+ - llm_int8_enable_fp32_cpu_offload: False
14
+ - llm_int8_has_fp16_weight: False
15
+ - bnb_4bit_quant_type: nf4
16
+ - bnb_4bit_use_double_quant: True
17
+ - bnb_4bit_compute_dtype: bfloat16
18
+ ### Framework versions
19
+
20
+
21
+ - PEFT 0.6.0.dev0
checkpoint-250/adapter_config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "./mistralai_Mistral-7B-v0.1",
5
+ "bias": "none",
6
+ "fan_in_fan_out": null,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "lora_alpha": 16,
12
+ "lora_dropout": 0.05,
13
+ "modules_to_save": null,
14
+ "peft_type": "LORA",
15
+ "r": 8,
16
+ "rank_pattern": {},
17
+ "revision": null,
18
+ "target_modules": [
19
+ "o_proj",
20
+ "k_proj",
21
+ "up_proj",
22
+ "v_proj",
23
+ "q_proj",
24
+ "gate_proj",
25
+ "down_proj"
26
+ ],
27
+ "task_type": "CAUSAL_LM"
28
+ }
checkpoint-250/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8f2be5e57601e9471651b0e8821845228d7a8b73ebbaea4df07cd4de3b3ac0d
3
+ size 84046925
checkpoint-250/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce35862ebc0e21a86a1e6281a6822907917216d33e0560244de74624d72d7204
3
+ size 168039109
checkpoint-250/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6223a05bbaac028fbfc88016641eb39c4bc95a13c0658c4f8611997cc29c5e41
3
+ size 14575
checkpoint-250/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a922f36e3287f78e28f040c3193d729dfc63546fcf6eb66508241f493732d059
3
+ size 627
checkpoint-250/trainer_state.json ADDED
@@ -0,0 +1,1559 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 1.60513174533844,
3
+ "best_model_checkpoint": "./lora-out/checkpoint-250",
4
+ "epoch": 0.7776049766718507,
5
+ "eval_steps": 50,
6
+ "global_step": 250,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0,
13
+ "learning_rate": 2e-05,
14
+ "loss": 1.7924,
15
+ "step": 1
16
+ },
17
+ {
18
+ "epoch": 0.01,
19
+ "learning_rate": 4e-05,
20
+ "loss": 1.8083,
21
+ "step": 2
22
+ },
23
+ {
24
+ "epoch": 0.01,
25
+ "learning_rate": 6e-05,
26
+ "loss": 1.8177,
27
+ "step": 3
28
+ },
29
+ {
30
+ "epoch": 0.01,
31
+ "learning_rate": 8e-05,
32
+ "loss": 1.7595,
33
+ "step": 4
34
+ },
35
+ {
36
+ "epoch": 0.02,
37
+ "learning_rate": 0.0001,
38
+ "loss": 1.6598,
39
+ "step": 5
40
+ },
41
+ {
42
+ "epoch": 0.02,
43
+ "learning_rate": 0.00012,
44
+ "loss": 1.6919,
45
+ "step": 6
46
+ },
47
+ {
48
+ "epoch": 0.02,
49
+ "learning_rate": 0.00014,
50
+ "loss": 1.6706,
51
+ "step": 7
52
+ },
53
+ {
54
+ "epoch": 0.02,
55
+ "learning_rate": 0.00016,
56
+ "loss": 1.6879,
57
+ "step": 8
58
+ },
59
+ {
60
+ "epoch": 0.03,
61
+ "learning_rate": 0.00018,
62
+ "loss": 1.7051,
63
+ "step": 9
64
+ },
65
+ {
66
+ "epoch": 0.03,
67
+ "learning_rate": 0.0002,
68
+ "loss": 1.7022,
69
+ "step": 10
70
+ },
71
+ {
72
+ "epoch": 0.03,
73
+ "learning_rate": 0.000199999456645141,
74
+ "loss": 1.6809,
75
+ "step": 11
76
+ },
77
+ {
78
+ "epoch": 0.04,
79
+ "learning_rate": 0.00019999782658646859,
80
+ "loss": 1.6098,
81
+ "step": 12
82
+ },
83
+ {
84
+ "epoch": 0.04,
85
+ "learning_rate": 0.0001999951098416968,
86
+ "loss": 1.7014,
87
+ "step": 13
88
+ },
89
+ {
90
+ "epoch": 0.04,
91
+ "learning_rate": 0.00019999130644034888,
92
+ "loss": 1.5885,
93
+ "step": 14
94
+ },
95
+ {
96
+ "epoch": 0.05,
97
+ "learning_rate": 0.00019998641642375657,
98
+ "loss": 1.6243,
99
+ "step": 15
100
+ },
101
+ {
102
+ "epoch": 0.05,
103
+ "learning_rate": 0.00019998043984506027,
104
+ "loss": 1.6484,
105
+ "step": 16
106
+ },
107
+ {
108
+ "epoch": 0.05,
109
+ "learning_rate": 0.00019997337676920803,
110
+ "loss": 1.6093,
111
+ "step": 17
112
+ },
113
+ {
114
+ "epoch": 0.06,
115
+ "learning_rate": 0.00019996522727295496,
116
+ "loss": 1.6173,
117
+ "step": 18
118
+ },
119
+ {
120
+ "epoch": 0.06,
121
+ "learning_rate": 0.00019995599144486247,
122
+ "loss": 1.646,
123
+ "step": 19
124
+ },
125
+ {
126
+ "epoch": 0.06,
127
+ "learning_rate": 0.00019994566938529712,
128
+ "loss": 1.6469,
129
+ "step": 20
130
+ },
131
+ {
132
+ "epoch": 0.07,
133
+ "learning_rate": 0.00019993426120642983,
134
+ "loss": 1.6564,
135
+ "step": 21
136
+ },
137
+ {
138
+ "epoch": 0.07,
139
+ "learning_rate": 0.00019992176703223432,
140
+ "loss": 1.5901,
141
+ "step": 22
142
+ },
143
+ {
144
+ "epoch": 0.07,
145
+ "learning_rate": 0.000199908186998486,
146
+ "loss": 1.664,
147
+ "step": 23
148
+ },
149
+ {
150
+ "epoch": 0.07,
151
+ "learning_rate": 0.00019989352125276047,
152
+ "loss": 1.6275,
153
+ "step": 24
154
+ },
155
+ {
156
+ "epoch": 0.08,
157
+ "learning_rate": 0.00019987776995443178,
158
+ "loss": 1.5839,
159
+ "step": 25
160
+ },
161
+ {
162
+ "epoch": 0.08,
163
+ "learning_rate": 0.00019986093327467076,
164
+ "loss": 1.5611,
165
+ "step": 26
166
+ },
167
+ {
168
+ "epoch": 0.08,
169
+ "learning_rate": 0.00019984301139644334,
170
+ "loss": 1.669,
171
+ "step": 27
172
+ },
173
+ {
174
+ "epoch": 0.09,
175
+ "learning_rate": 0.0001998240045145083,
176
+ "loss": 1.5641,
177
+ "step": 28
178
+ },
179
+ {
180
+ "epoch": 0.09,
181
+ "learning_rate": 0.00019980391283541522,
182
+ "loss": 1.6023,
183
+ "step": 29
184
+ },
185
+ {
186
+ "epoch": 0.09,
187
+ "learning_rate": 0.00019978273657750238,
188
+ "loss": 1.6309,
189
+ "step": 30
190
+ },
191
+ {
192
+ "epoch": 0.1,
193
+ "learning_rate": 0.0001997604759708942,
194
+ "loss": 1.6353,
195
+ "step": 31
196
+ },
197
+ {
198
+ "epoch": 0.1,
199
+ "learning_rate": 0.00019973713125749884,
200
+ "loss": 1.6328,
201
+ "step": 32
202
+ },
203
+ {
204
+ "epoch": 0.1,
205
+ "learning_rate": 0.00019971270269100564,
206
+ "loss": 1.5683,
207
+ "step": 33
208
+ },
209
+ {
210
+ "epoch": 0.11,
211
+ "learning_rate": 0.00019968719053688213,
212
+ "loss": 1.6217,
213
+ "step": 34
214
+ },
215
+ {
216
+ "epoch": 0.11,
217
+ "learning_rate": 0.0001996605950723714,
218
+ "loss": 1.5734,
219
+ "step": 35
220
+ },
221
+ {
222
+ "epoch": 0.11,
223
+ "learning_rate": 0.00019963291658648896,
224
+ "loss": 1.6162,
225
+ "step": 36
226
+ },
227
+ {
228
+ "epoch": 0.12,
229
+ "learning_rate": 0.00019960415538001957,
230
+ "loss": 1.5922,
231
+ "step": 37
232
+ },
233
+ {
234
+ "epoch": 0.12,
235
+ "learning_rate": 0.0001995743117655141,
236
+ "loss": 1.5806,
237
+ "step": 38
238
+ },
239
+ {
240
+ "epoch": 0.12,
241
+ "learning_rate": 0.000199543386067286,
242
+ "loss": 1.5938,
243
+ "step": 39
244
+ },
245
+ {
246
+ "epoch": 0.12,
247
+ "learning_rate": 0.00019951137862140778,
248
+ "loss": 1.6386,
249
+ "step": 40
250
+ },
251
+ {
252
+ "epoch": 0.13,
253
+ "learning_rate": 0.00019947828977570756,
254
+ "loss": 1.6476,
255
+ "step": 41
256
+ },
257
+ {
258
+ "epoch": 0.13,
259
+ "learning_rate": 0.00019944411988976496,
260
+ "loss": 1.6557,
261
+ "step": 42
262
+ },
263
+ {
264
+ "epoch": 0.13,
265
+ "learning_rate": 0.00019940886933490749,
266
+ "loss": 1.5836,
267
+ "step": 43
268
+ },
269
+ {
270
+ "epoch": 0.14,
271
+ "learning_rate": 0.00019937253849420635,
272
+ "loss": 1.6421,
273
+ "step": 44
274
+ },
275
+ {
276
+ "epoch": 0.14,
277
+ "learning_rate": 0.0001993351277624723,
278
+ "loss": 1.629,
279
+ "step": 45
280
+ },
281
+ {
282
+ "epoch": 0.14,
283
+ "learning_rate": 0.00019929663754625145,
284
+ "loss": 1.6392,
285
+ "step": 46
286
+ },
287
+ {
288
+ "epoch": 0.15,
289
+ "learning_rate": 0.00019925706826382064,
290
+ "loss": 1.5677,
291
+ "step": 47
292
+ },
293
+ {
294
+ "epoch": 0.15,
295
+ "learning_rate": 0.00019921642034518317,
296
+ "loss": 1.6144,
297
+ "step": 48
298
+ },
299
+ {
300
+ "epoch": 0.15,
301
+ "learning_rate": 0.00019917469423206389,
302
+ "loss": 1.6068,
303
+ "step": 49
304
+ },
305
+ {
306
+ "epoch": 0.16,
307
+ "learning_rate": 0.00019913189037790456,
308
+ "loss": 1.6421,
309
+ "step": 50
310
+ },
311
+ {
312
+ "epoch": 0.16,
313
+ "eval_loss": 1.621693730354309,
314
+ "eval_runtime": 233.7603,
315
+ "eval_samples_per_second": 16.354,
316
+ "eval_steps_per_second": 4.09,
317
+ "step": 50
318
+ },
319
+ {
320
+ "epoch": 0.16,
321
+ "learning_rate": 0.0001990880092478588,
322
+ "loss": 1.6172,
323
+ "step": 51
324
+ },
325
+ {
326
+ "epoch": 0.16,
327
+ "learning_rate": 0.0001990430513187871,
328
+ "loss": 1.6095,
329
+ "step": 52
330
+ },
331
+ {
332
+ "epoch": 0.16,
333
+ "learning_rate": 0.00019899701707925166,
334
+ "loss": 1.5967,
335
+ "step": 53
336
+ },
337
+ {
338
+ "epoch": 0.17,
339
+ "learning_rate": 0.00019894990702951106,
340
+ "loss": 1.617,
341
+ "step": 54
342
+ },
343
+ {
344
+ "epoch": 0.17,
345
+ "learning_rate": 0.00019890172168151473,
346
+ "loss": 1.5932,
347
+ "step": 55
348
+ },
349
+ {
350
+ "epoch": 0.17,
351
+ "learning_rate": 0.0001988524615588976,
352
+ "loss": 1.6548,
353
+ "step": 56
354
+ },
355
+ {
356
+ "epoch": 0.18,
357
+ "learning_rate": 0.00019880212719697413,
358
+ "loss": 1.6033,
359
+ "step": 57
360
+ },
361
+ {
362
+ "epoch": 0.18,
363
+ "learning_rate": 0.00019875071914273278,
364
+ "loss": 1.6063,
365
+ "step": 58
366
+ },
367
+ {
368
+ "epoch": 0.18,
369
+ "learning_rate": 0.00019869823795482986,
370
+ "loss": 1.6107,
371
+ "step": 59
372
+ },
373
+ {
374
+ "epoch": 0.19,
375
+ "learning_rate": 0.00019864468420358354,
376
+ "loss": 1.5758,
377
+ "step": 60
378
+ },
379
+ {
380
+ "epoch": 0.19,
381
+ "learning_rate": 0.00019859005847096763,
382
+ "loss": 1.5723,
383
+ "step": 61
384
+ },
385
+ {
386
+ "epoch": 0.19,
387
+ "learning_rate": 0.00019853436135060527,
388
+ "loss": 1.542,
389
+ "step": 62
390
+ },
391
+ {
392
+ "epoch": 0.2,
393
+ "learning_rate": 0.00019847759344776252,
394
+ "loss": 1.5611,
395
+ "step": 63
396
+ },
397
+ {
398
+ "epoch": 0.2,
399
+ "learning_rate": 0.00019841975537934162,
400
+ "loss": 1.6157,
401
+ "step": 64
402
+ },
403
+ {
404
+ "epoch": 0.2,
405
+ "learning_rate": 0.00019836084777387458,
406
+ "loss": 1.5589,
407
+ "step": 65
408
+ },
409
+ {
410
+ "epoch": 0.21,
411
+ "learning_rate": 0.00019830087127151598,
412
+ "loss": 1.6077,
413
+ "step": 66
414
+ },
415
+ {
416
+ "epoch": 0.21,
417
+ "learning_rate": 0.00019823982652403634,
418
+ "loss": 1.5473,
419
+ "step": 67
420
+ },
421
+ {
422
+ "epoch": 0.21,
423
+ "learning_rate": 0.00019817771419481487,
424
+ "loss": 1.6265,
425
+ "step": 68
426
+ },
427
+ {
428
+ "epoch": 0.21,
429
+ "learning_rate": 0.0001981145349588323,
430
+ "loss": 1.6074,
431
+ "step": 69
432
+ },
433
+ {
434
+ "epoch": 0.22,
435
+ "learning_rate": 0.00019805028950266348,
436
+ "loss": 1.6195,
437
+ "step": 70
438
+ },
439
+ {
440
+ "epoch": 0.22,
441
+ "learning_rate": 0.00019798497852447006,
442
+ "loss": 1.5876,
443
+ "step": 71
444
+ },
445
+ {
446
+ "epoch": 0.22,
447
+ "learning_rate": 0.0001979186027339928,
448
+ "loss": 1.5978,
449
+ "step": 72
450
+ },
451
+ {
452
+ "epoch": 0.23,
453
+ "learning_rate": 0.00019785116285254381,
454
+ "loss": 1.533,
455
+ "step": 73
456
+ },
457
+ {
458
+ "epoch": 0.23,
459
+ "learning_rate": 0.00019778265961299888,
460
+ "loss": 1.5888,
461
+ "step": 74
462
+ },
463
+ {
464
+ "epoch": 0.23,
465
+ "learning_rate": 0.0001977130937597894,
466
+ "loss": 1.6211,
467
+ "step": 75
468
+ },
469
+ {
470
+ "epoch": 0.24,
471
+ "learning_rate": 0.00019764246604889415,
472
+ "loss": 1.6091,
473
+ "step": 76
474
+ },
475
+ {
476
+ "epoch": 0.24,
477
+ "learning_rate": 0.00019757077724783147,
478
+ "loss": 1.6012,
479
+ "step": 77
480
+ },
481
+ {
482
+ "epoch": 0.24,
483
+ "learning_rate": 0.0001974980281356504,
484
+ "loss": 1.6401,
485
+ "step": 78
486
+ },
487
+ {
488
+ "epoch": 0.25,
489
+ "learning_rate": 0.0001974242195029227,
490
+ "loss": 1.6111,
491
+ "step": 79
492
+ },
493
+ {
494
+ "epoch": 0.25,
495
+ "learning_rate": 0.00019734935215173392,
496
+ "loss": 1.6208,
497
+ "step": 80
498
+ },
499
+ {
500
+ "epoch": 0.25,
501
+ "learning_rate": 0.00019727342689567482,
502
+ "loss": 1.6038,
503
+ "step": 81
504
+ },
505
+ {
506
+ "epoch": 0.26,
507
+ "learning_rate": 0.00019719644455983256,
508
+ "loss": 1.5915,
509
+ "step": 82
510
+ },
511
+ {
512
+ "epoch": 0.26,
513
+ "learning_rate": 0.0001971184059807817,
514
+ "loss": 1.5872,
515
+ "step": 83
516
+ },
517
+ {
518
+ "epoch": 0.26,
519
+ "learning_rate": 0.000197039312006575,
520
+ "loss": 1.5984,
521
+ "step": 84
522
+ },
523
+ {
524
+ "epoch": 0.26,
525
+ "learning_rate": 0.0001969591634967344,
526
+ "loss": 1.5996,
527
+ "step": 85
528
+ },
529
+ {
530
+ "epoch": 0.27,
531
+ "learning_rate": 0.00019687796132224152,
532
+ "loss": 1.6056,
533
+ "step": 86
534
+ },
535
+ {
536
+ "epoch": 0.27,
537
+ "learning_rate": 0.0001967957063655283,
538
+ "loss": 1.6099,
539
+ "step": 87
540
+ },
541
+ {
542
+ "epoch": 0.27,
543
+ "learning_rate": 0.0001967123995204674,
544
+ "loss": 1.6295,
545
+ "step": 88
546
+ },
547
+ {
548
+ "epoch": 0.28,
549
+ "learning_rate": 0.00019662804169236225,
550
+ "loss": 1.5482,
551
+ "step": 89
552
+ },
553
+ {
554
+ "epoch": 0.28,
555
+ "learning_rate": 0.00019654263379793773,
556
+ "loss": 1.5781,
557
+ "step": 90
558
+ },
559
+ {
560
+ "epoch": 0.28,
561
+ "learning_rate": 0.00019645617676532963,
562
+ "loss": 1.5954,
563
+ "step": 91
564
+ },
565
+ {
566
+ "epoch": 0.29,
567
+ "learning_rate": 0.000196368671534075,
568
+ "loss": 1.619,
569
+ "step": 92
570
+ },
571
+ {
572
+ "epoch": 0.29,
573
+ "learning_rate": 0.0001962801190551016,
574
+ "loss": 1.6153,
575
+ "step": 93
576
+ },
577
+ {
578
+ "epoch": 0.29,
579
+ "learning_rate": 0.0001961905202907179,
580
+ "loss": 1.6008,
581
+ "step": 94
582
+ },
583
+ {
584
+ "epoch": 0.3,
585
+ "learning_rate": 0.00019609987621460232,
586
+ "loss": 1.5891,
587
+ "step": 95
588
+ },
589
+ {
590
+ "epoch": 0.3,
591
+ "learning_rate": 0.0001960081878117929,
592
+ "loss": 1.6438,
593
+ "step": 96
594
+ },
595
+ {
596
+ "epoch": 0.3,
597
+ "learning_rate": 0.0001959154560786764,
598
+ "loss": 1.5576,
599
+ "step": 97
600
+ },
601
+ {
602
+ "epoch": 0.3,
603
+ "learning_rate": 0.00019582168202297758,
604
+ "loss": 1.646,
605
+ "step": 98
606
+ },
607
+ {
608
+ "epoch": 0.31,
609
+ "learning_rate": 0.00019572686666374822,
610
+ "loss": 1.6269,
611
+ "step": 99
612
+ },
613
+ {
614
+ "epoch": 0.31,
615
+ "learning_rate": 0.00019563101103135602,
616
+ "loss": 1.6288,
617
+ "step": 100
618
+ },
619
+ {
620
+ "epoch": 0.31,
621
+ "eval_loss": 1.6143836975097656,
622
+ "eval_runtime": 233.6412,
623
+ "eval_samples_per_second": 16.363,
624
+ "eval_steps_per_second": 4.092,
625
+ "step": 100
626
+ },
627
+ {
628
+ "epoch": 0.31,
629
+ "learning_rate": 0.00019553411616747348,
630
+ "loss": 1.5667,
631
+ "step": 101
632
+ },
633
+ {
634
+ "epoch": 0.32,
635
+ "learning_rate": 0.00019543618312506647,
636
+ "loss": 1.6221,
637
+ "step": 102
638
+ },
639
+ {
640
+ "epoch": 0.32,
641
+ "learning_rate": 0.0001953372129683829,
642
+ "loss": 1.5992,
643
+ "step": 103
644
+ },
645
+ {
646
+ "epoch": 0.32,
647
+ "learning_rate": 0.0001952372067729411,
648
+ "loss": 1.6138,
649
+ "step": 104
650
+ },
651
+ {
652
+ "epoch": 0.33,
653
+ "learning_rate": 0.00019513616562551807,
654
+ "loss": 1.51,
655
+ "step": 105
656
+ },
657
+ {
658
+ "epoch": 0.33,
659
+ "learning_rate": 0.00019503409062413782,
660
+ "loss": 1.6227,
661
+ "step": 106
662
+ },
663
+ {
664
+ "epoch": 0.33,
665
+ "learning_rate": 0.00019493098287805927,
666
+ "loss": 1.6014,
667
+ "step": 107
668
+ },
669
+ {
670
+ "epoch": 0.34,
671
+ "learning_rate": 0.00019482684350776434,
672
+ "loss": 1.625,
673
+ "step": 108
674
+ },
675
+ {
676
+ "epoch": 0.34,
677
+ "learning_rate": 0.0001947216736449457,
678
+ "loss": 1.6109,
679
+ "step": 109
680
+ },
681
+ {
682
+ "epoch": 0.34,
683
+ "learning_rate": 0.0001946154744324945,
684
+ "loss": 1.62,
685
+ "step": 110
686
+ },
687
+ {
688
+ "epoch": 0.35,
689
+ "learning_rate": 0.00019450824702448778,
690
+ "loss": 1.5878,
691
+ "step": 111
692
+ },
693
+ {
694
+ "epoch": 0.35,
695
+ "learning_rate": 0.0001943999925861763,
696
+ "loss": 1.6264,
697
+ "step": 112
698
+ },
699
+ {
700
+ "epoch": 0.35,
701
+ "learning_rate": 0.00019429071229397157,
702
+ "loss": 1.6186,
703
+ "step": 113
704
+ },
705
+ {
706
+ "epoch": 0.35,
707
+ "learning_rate": 0.0001941804073354331,
708
+ "loss": 1.6363,
709
+ "step": 114
710
+ },
711
+ {
712
+ "epoch": 0.36,
713
+ "learning_rate": 0.00019406907890925562,
714
+ "loss": 1.5341,
715
+ "step": 115
716
+ },
717
+ {
718
+ "epoch": 0.36,
719
+ "learning_rate": 0.00019395672822525593,
720
+ "loss": 1.5986,
721
+ "step": 116
722
+ },
723
+ {
724
+ "epoch": 0.36,
725
+ "learning_rate": 0.00019384335650435985,
726
+ "loss": 1.6181,
727
+ "step": 117
728
+ },
729
+ {
730
+ "epoch": 0.37,
731
+ "learning_rate": 0.0001937289649785889,
732
+ "loss": 1.6118,
733
+ "step": 118
734
+ },
735
+ {
736
+ "epoch": 0.37,
737
+ "learning_rate": 0.0001936135548910469,
738
+ "loss": 1.6404,
739
+ "step": 119
740
+ },
741
+ {
742
+ "epoch": 0.37,
743
+ "learning_rate": 0.00019349712749590649,
744
+ "loss": 1.583,
745
+ "step": 120
746
+ },
747
+ {
748
+ "epoch": 0.38,
749
+ "learning_rate": 0.00019337968405839547,
750
+ "loss": 1.5827,
751
+ "step": 121
752
+ },
753
+ {
754
+ "epoch": 0.38,
755
+ "learning_rate": 0.00019326122585478308,
756
+ "loss": 1.6392,
757
+ "step": 122
758
+ },
759
+ {
760
+ "epoch": 0.38,
761
+ "learning_rate": 0.00019314175417236616,
762
+ "loss": 1.5861,
763
+ "step": 123
764
+ },
765
+ {
766
+ "epoch": 0.39,
767
+ "learning_rate": 0.00019302127030945508,
768
+ "loss": 1.5738,
769
+ "step": 124
770
+ },
771
+ {
772
+ "epoch": 0.39,
773
+ "learning_rate": 0.0001928997755753597,
774
+ "loss": 1.5915,
775
+ "step": 125
776
+ },
777
+ {
778
+ "epoch": 0.39,
779
+ "learning_rate": 0.00019277727129037508,
780
+ "loss": 1.617,
781
+ "step": 126
782
+ },
783
+ {
784
+ "epoch": 0.4,
785
+ "learning_rate": 0.0001926537587857672,
786
+ "loss": 1.5582,
787
+ "step": 127
788
+ },
789
+ {
790
+ "epoch": 0.4,
791
+ "learning_rate": 0.00019252923940375844,
792
+ "loss": 1.6294,
793
+ "step": 128
794
+ },
795
+ {
796
+ "epoch": 0.4,
797
+ "learning_rate": 0.00019240371449751306,
798
+ "loss": 1.6087,
799
+ "step": 129
800
+ },
801
+ {
802
+ "epoch": 0.4,
803
+ "learning_rate": 0.00019227718543112236,
804
+ "loss": 1.5749,
805
+ "step": 130
806
+ },
807
+ {
808
+ "epoch": 0.41,
809
+ "learning_rate": 0.00019214965357959005,
810
+ "loss": 1.6041,
811
+ "step": 131
812
+ },
813
+ {
814
+ "epoch": 0.41,
815
+ "learning_rate": 0.00019202112032881715,
816
+ "loss": 1.6106,
817
+ "step": 132
818
+ },
819
+ {
820
+ "epoch": 0.41,
821
+ "learning_rate": 0.00019189158707558695,
822
+ "loss": 1.5553,
823
+ "step": 133
824
+ },
825
+ {
826
+ "epoch": 0.42,
827
+ "learning_rate": 0.00019176105522754995,
828
+ "loss": 1.5638,
829
+ "step": 134
830
+ },
831
+ {
832
+ "epoch": 0.42,
833
+ "learning_rate": 0.0001916295262032084,
834
+ "loss": 1.5921,
835
+ "step": 135
836
+ },
837
+ {
838
+ "epoch": 0.42,
839
+ "learning_rate": 0.00019149700143190096,
840
+ "loss": 1.5837,
841
+ "step": 136
842
+ },
843
+ {
844
+ "epoch": 0.43,
845
+ "learning_rate": 0.00019136348235378726,
846
+ "loss": 1.6341,
847
+ "step": 137
848
+ },
849
+ {
850
+ "epoch": 0.43,
851
+ "learning_rate": 0.00019122897041983205,
852
+ "loss": 1.5678,
853
+ "step": 138
854
+ },
855
+ {
856
+ "epoch": 0.43,
857
+ "learning_rate": 0.00019109346709178963,
858
+ "loss": 1.6137,
859
+ "step": 139
860
+ },
861
+ {
862
+ "epoch": 0.44,
863
+ "learning_rate": 0.0001909569738421878,
864
+ "loss": 1.6324,
865
+ "step": 140
866
+ },
867
+ {
868
+ "epoch": 0.44,
869
+ "learning_rate": 0.00019081949215431194,
870
+ "loss": 1.612,
871
+ "step": 141
872
+ },
873
+ {
874
+ "epoch": 0.44,
875
+ "learning_rate": 0.00019068102352218897,
876
+ "loss": 1.5908,
877
+ "step": 142
878
+ },
879
+ {
880
+ "epoch": 0.44,
881
+ "learning_rate": 0.00019054156945057097,
882
+ "loss": 1.6087,
883
+ "step": 143
884
+ },
885
+ {
886
+ "epoch": 0.45,
887
+ "learning_rate": 0.00019040113145491887,
888
+ "loss": 1.5613,
889
+ "step": 144
890
+ },
891
+ {
892
+ "epoch": 0.45,
893
+ "learning_rate": 0.000190259711061386,
894
+ "loss": 1.6072,
895
+ "step": 145
896
+ },
897
+ {
898
+ "epoch": 0.45,
899
+ "learning_rate": 0.00019011730980680156,
900
+ "loss": 1.5722,
901
+ "step": 146
902
+ },
903
+ {
904
+ "epoch": 0.46,
905
+ "learning_rate": 0.0001899739292386538,
906
+ "loss": 1.5961,
907
+ "step": 147
908
+ },
909
+ {
910
+ "epoch": 0.46,
911
+ "learning_rate": 0.00018982957091507325,
912
+ "loss": 1.5409,
913
+ "step": 148
914
+ },
915
+ {
916
+ "epoch": 0.46,
917
+ "learning_rate": 0.0001896842364048159,
918
+ "loss": 1.6557,
919
+ "step": 149
920
+ },
921
+ {
922
+ "epoch": 0.47,
923
+ "learning_rate": 0.000189537927287246,
924
+ "loss": 1.5725,
925
+ "step": 150
926
+ },
927
+ {
928
+ "epoch": 0.47,
929
+ "eval_loss": 1.6101970672607422,
930
+ "eval_runtime": 233.5313,
931
+ "eval_samples_per_second": 16.37,
932
+ "eval_steps_per_second": 4.094,
933
+ "step": 150
934
+ },
935
+ {
936
+ "epoch": 0.47,
937
+ "learning_rate": 0.00018939064515231888,
938
+ "loss": 1.5949,
939
+ "step": 151
940
+ },
941
+ {
942
+ "epoch": 0.47,
943
+ "learning_rate": 0.0001892423916005639,
944
+ "loss": 1.6191,
945
+ "step": 152
946
+ },
947
+ {
948
+ "epoch": 0.48,
949
+ "learning_rate": 0.00018909316824306674,
950
+ "loss": 1.5487,
951
+ "step": 153
952
+ },
953
+ {
954
+ "epoch": 0.48,
955
+ "learning_rate": 0.00018894297670145216,
956
+ "loss": 1.5104,
957
+ "step": 154
958
+ },
959
+ {
960
+ "epoch": 0.48,
961
+ "learning_rate": 0.00018879181860786623,
962
+ "loss": 1.6392,
963
+ "step": 155
964
+ },
965
+ {
966
+ "epoch": 0.49,
967
+ "learning_rate": 0.00018863969560495866,
968
+ "loss": 1.5932,
969
+ "step": 156
970
+ },
971
+ {
972
+ "epoch": 0.49,
973
+ "learning_rate": 0.00018848660934586491,
974
+ "loss": 1.6213,
975
+ "step": 157
976
+ },
977
+ {
978
+ "epoch": 0.49,
979
+ "learning_rate": 0.0001883325614941882,
980
+ "loss": 1.5515,
981
+ "step": 158
982
+ },
983
+ {
984
+ "epoch": 0.49,
985
+ "learning_rate": 0.00018817755372398155,
986
+ "loss": 1.6166,
987
+ "step": 159
988
+ },
989
+ {
990
+ "epoch": 0.5,
991
+ "learning_rate": 0.00018802158771972943,
992
+ "loss": 1.6552,
993
+ "step": 160
994
+ },
995
+ {
996
+ "epoch": 0.5,
997
+ "learning_rate": 0.00018786466517632956,
998
+ "loss": 1.6378,
999
+ "step": 161
1000
+ },
1001
+ {
1002
+ "epoch": 0.5,
1003
+ "learning_rate": 0.00018770678779907448,
1004
+ "loss": 1.5176,
1005
+ "step": 162
1006
+ },
1007
+ {
1008
+ "epoch": 0.51,
1009
+ "learning_rate": 0.00018754795730363302,
1010
+ "loss": 1.5793,
1011
+ "step": 163
1012
+ },
1013
+ {
1014
+ "epoch": 0.51,
1015
+ "learning_rate": 0.00018738817541603156,
1016
+ "loss": 1.6616,
1017
+ "step": 164
1018
+ },
1019
+ {
1020
+ "epoch": 0.51,
1021
+ "learning_rate": 0.00018722744387263544,
1022
+ "loss": 1.6055,
1023
+ "step": 165
1024
+ },
1025
+ {
1026
+ "epoch": 0.52,
1027
+ "learning_rate": 0.00018706576442012994,
1028
+ "loss": 1.6204,
1029
+ "step": 166
1030
+ },
1031
+ {
1032
+ "epoch": 0.52,
1033
+ "learning_rate": 0.00018690313881550137,
1034
+ "loss": 1.5952,
1035
+ "step": 167
1036
+ },
1037
+ {
1038
+ "epoch": 0.52,
1039
+ "learning_rate": 0.00018673956882601803,
1040
+ "loss": 1.6271,
1041
+ "step": 168
1042
+ },
1043
+ {
1044
+ "epoch": 0.53,
1045
+ "learning_rate": 0.00018657505622921082,
1046
+ "loss": 1.538,
1047
+ "step": 169
1048
+ },
1049
+ {
1050
+ "epoch": 0.53,
1051
+ "learning_rate": 0.00018640960281285417,
1052
+ "loss": 1.5874,
1053
+ "step": 170
1054
+ },
1055
+ {
1056
+ "epoch": 0.53,
1057
+ "learning_rate": 0.0001862432103749464,
1058
+ "loss": 1.5694,
1059
+ "step": 171
1060
+ },
1061
+ {
1062
+ "epoch": 0.53,
1063
+ "learning_rate": 0.00018607588072369033,
1064
+ "loss": 1.583,
1065
+ "step": 172
1066
+ },
1067
+ {
1068
+ "epoch": 0.54,
1069
+ "learning_rate": 0.00018590761567747354,
1070
+ "loss": 1.5961,
1071
+ "step": 173
1072
+ },
1073
+ {
1074
+ "epoch": 0.54,
1075
+ "learning_rate": 0.00018573841706484866,
1076
+ "loss": 1.582,
1077
+ "step": 174
1078
+ },
1079
+ {
1080
+ "epoch": 0.54,
1081
+ "learning_rate": 0.0001855682867245134,
1082
+ "loss": 1.6427,
1083
+ "step": 175
1084
+ },
1085
+ {
1086
+ "epoch": 0.55,
1087
+ "learning_rate": 0.00018539722650529075,
1088
+ "loss": 1.604,
1089
+ "step": 176
1090
+ },
1091
+ {
1092
+ "epoch": 0.55,
1093
+ "learning_rate": 0.00018522523826610868,
1094
+ "loss": 1.577,
1095
+ "step": 177
1096
+ },
1097
+ {
1098
+ "epoch": 0.55,
1099
+ "learning_rate": 0.00018505232387598018,
1100
+ "loss": 1.6339,
1101
+ "step": 178
1102
+ },
1103
+ {
1104
+ "epoch": 0.56,
1105
+ "learning_rate": 0.00018487848521398265,
1106
+ "loss": 1.5993,
1107
+ "step": 179
1108
+ },
1109
+ {
1110
+ "epoch": 0.56,
1111
+ "learning_rate": 0.0001847037241692378,
1112
+ "loss": 1.6286,
1113
+ "step": 180
1114
+ },
1115
+ {
1116
+ "epoch": 0.56,
1117
+ "learning_rate": 0.00018452804264089084,
1118
+ "loss": 1.5963,
1119
+ "step": 181
1120
+ },
1121
+ {
1122
+ "epoch": 0.57,
1123
+ "learning_rate": 0.00018435144253809,
1124
+ "loss": 1.5856,
1125
+ "step": 182
1126
+ },
1127
+ {
1128
+ "epoch": 0.57,
1129
+ "learning_rate": 0.00018417392577996578,
1130
+ "loss": 1.5787,
1131
+ "step": 183
1132
+ },
1133
+ {
1134
+ "epoch": 0.57,
1135
+ "learning_rate": 0.00018399549429561006,
1136
+ "loss": 1.5876,
1137
+ "step": 184
1138
+ },
1139
+ {
1140
+ "epoch": 0.58,
1141
+ "learning_rate": 0.00018381615002405509,
1142
+ "loss": 1.5565,
1143
+ "step": 185
1144
+ },
1145
+ {
1146
+ "epoch": 0.58,
1147
+ "learning_rate": 0.00018363589491425248,
1148
+ "loss": 1.5897,
1149
+ "step": 186
1150
+ },
1151
+ {
1152
+ "epoch": 0.58,
1153
+ "learning_rate": 0.0001834547309250521,
1154
+ "loss": 1.5951,
1155
+ "step": 187
1156
+ },
1157
+ {
1158
+ "epoch": 0.58,
1159
+ "learning_rate": 0.00018327266002518056,
1160
+ "loss": 1.5447,
1161
+ "step": 188
1162
+ },
1163
+ {
1164
+ "epoch": 0.59,
1165
+ "learning_rate": 0.00018308968419322003,
1166
+ "loss": 1.6087,
1167
+ "step": 189
1168
+ },
1169
+ {
1170
+ "epoch": 0.59,
1171
+ "learning_rate": 0.00018290580541758668,
1172
+ "loss": 1.5946,
1173
+ "step": 190
1174
+ },
1175
+ {
1176
+ "epoch": 0.59,
1177
+ "learning_rate": 0.00018272102569650905,
1178
+ "loss": 1.6148,
1179
+ "step": 191
1180
+ },
1181
+ {
1182
+ "epoch": 0.6,
1183
+ "learning_rate": 0.00018253534703800627,
1184
+ "loss": 1.649,
1185
+ "step": 192
1186
+ },
1187
+ {
1188
+ "epoch": 0.6,
1189
+ "learning_rate": 0.0001823487714598664,
1190
+ "loss": 1.6312,
1191
+ "step": 193
1192
+ },
1193
+ {
1194
+ "epoch": 0.6,
1195
+ "learning_rate": 0.0001821613009896244,
1196
+ "loss": 1.5858,
1197
+ "step": 194
1198
+ },
1199
+ {
1200
+ "epoch": 0.61,
1201
+ "learning_rate": 0.00018197293766454003,
1202
+ "loss": 1.5925,
1203
+ "step": 195
1204
+ },
1205
+ {
1206
+ "epoch": 0.61,
1207
+ "learning_rate": 0.0001817836835315759,
1208
+ "loss": 1.5604,
1209
+ "step": 196
1210
+ },
1211
+ {
1212
+ "epoch": 0.61,
1213
+ "learning_rate": 0.00018159354064737506,
1214
+ "loss": 1.6125,
1215
+ "step": 197
1216
+ },
1217
+ {
1218
+ "epoch": 0.62,
1219
+ "learning_rate": 0.0001814025110782387,
1220
+ "loss": 1.5954,
1221
+ "step": 198
1222
+ },
1223
+ {
1224
+ "epoch": 0.62,
1225
+ "learning_rate": 0.00018121059690010368,
1226
+ "loss": 1.5937,
1227
+ "step": 199
1228
+ },
1229
+ {
1230
+ "epoch": 0.62,
1231
+ "learning_rate": 0.00018101780019852008,
1232
+ "loss": 1.5582,
1233
+ "step": 200
1234
+ },
1235
+ {
1236
+ "epoch": 0.62,
1237
+ "eval_loss": 1.6065257787704468,
1238
+ "eval_runtime": 233.7919,
1239
+ "eval_samples_per_second": 16.352,
1240
+ "eval_steps_per_second": 4.089,
1241
+ "step": 200
1242
+ },
1243
+ {
1244
+ "epoch": 0.63,
1245
+ "learning_rate": 0.00018082412306862837,
1246
+ "loss": 1.5628,
1247
+ "step": 201
1248
+ },
1249
+ {
1250
+ "epoch": 0.63,
1251
+ "learning_rate": 0.00018062956761513675,
1252
+ "loss": 1.5735,
1253
+ "step": 202
1254
+ },
1255
+ {
1256
+ "epoch": 0.63,
1257
+ "learning_rate": 0.00018043413595229818,
1258
+ "loss": 1.6011,
1259
+ "step": 203
1260
+ },
1261
+ {
1262
+ "epoch": 0.63,
1263
+ "learning_rate": 0.00018023783020388763,
1264
+ "loss": 1.5434,
1265
+ "step": 204
1266
+ },
1267
+ {
1268
+ "epoch": 0.64,
1269
+ "learning_rate": 0.00018004065250317868,
1270
+ "loss": 1.5533,
1271
+ "step": 205
1272
+ },
1273
+ {
1274
+ "epoch": 0.64,
1275
+ "learning_rate": 0.00017984260499292058,
1276
+ "loss": 1.6074,
1277
+ "step": 206
1278
+ },
1279
+ {
1280
+ "epoch": 0.64,
1281
+ "learning_rate": 0.00017964368982531487,
1282
+ "loss": 1.5286,
1283
+ "step": 207
1284
+ },
1285
+ {
1286
+ "epoch": 0.65,
1287
+ "learning_rate": 0.00017944390916199203,
1288
+ "loss": 1.5161,
1289
+ "step": 208
1290
+ },
1291
+ {
1292
+ "epoch": 0.65,
1293
+ "learning_rate": 0.00017924326517398793,
1294
+ "loss": 1.6024,
1295
+ "step": 209
1296
+ },
1297
+ {
1298
+ "epoch": 0.65,
1299
+ "learning_rate": 0.00017904176004172027,
1300
+ "loss": 1.5727,
1301
+ "step": 210
1302
+ },
1303
+ {
1304
+ "epoch": 0.66,
1305
+ "learning_rate": 0.0001788393959549649,
1306
+ "loss": 1.5752,
1307
+ "step": 211
1308
+ },
1309
+ {
1310
+ "epoch": 0.66,
1311
+ "learning_rate": 0.00017863617511283203,
1312
+ "loss": 1.5845,
1313
+ "step": 212
1314
+ },
1315
+ {
1316
+ "epoch": 0.66,
1317
+ "learning_rate": 0.00017843209972374233,
1318
+ "loss": 1.6082,
1319
+ "step": 213
1320
+ },
1321
+ {
1322
+ "epoch": 0.67,
1323
+ "learning_rate": 0.00017822717200540283,
1324
+ "loss": 1.5895,
1325
+ "step": 214
1326
+ },
1327
+ {
1328
+ "epoch": 0.67,
1329
+ "learning_rate": 0.00017802139418478298,
1330
+ "loss": 1.5836,
1331
+ "step": 215
1332
+ },
1333
+ {
1334
+ "epoch": 0.67,
1335
+ "learning_rate": 0.00017781476849809038,
1336
+ "loss": 1.5996,
1337
+ "step": 216
1338
+ },
1339
+ {
1340
+ "epoch": 0.67,
1341
+ "learning_rate": 0.00017760729719074644,
1342
+ "loss": 1.6256,
1343
+ "step": 217
1344
+ },
1345
+ {
1346
+ "epoch": 0.68,
1347
+ "learning_rate": 0.000177398982517362,
1348
+ "loss": 1.628,
1349
+ "step": 218
1350
+ },
1351
+ {
1352
+ "epoch": 0.68,
1353
+ "learning_rate": 0.00017718982674171284,
1354
+ "loss": 1.5543,
1355
+ "step": 219
1356
+ },
1357
+ {
1358
+ "epoch": 0.68,
1359
+ "learning_rate": 0.00017697983213671515,
1360
+ "loss": 1.5732,
1361
+ "step": 220
1362
+ },
1363
+ {
1364
+ "epoch": 0.69,
1365
+ "learning_rate": 0.0001767690009844007,
1366
+ "loss": 1.5892,
1367
+ "step": 221
1368
+ },
1369
+ {
1370
+ "epoch": 0.69,
1371
+ "learning_rate": 0.0001765573355758921,
1372
+ "loss": 1.6524,
1373
+ "step": 222
1374
+ },
1375
+ {
1376
+ "epoch": 0.69,
1377
+ "learning_rate": 0.00017634483821137787,
1378
+ "loss": 1.5694,
1379
+ "step": 223
1380
+ },
1381
+ {
1382
+ "epoch": 0.7,
1383
+ "learning_rate": 0.0001761315112000876,
1384
+ "loss": 1.6006,
1385
+ "step": 224
1386
+ },
1387
+ {
1388
+ "epoch": 0.7,
1389
+ "learning_rate": 0.00017591735686026661,
1390
+ "loss": 1.6161,
1391
+ "step": 225
1392
+ },
1393
+ {
1394
+ "epoch": 0.7,
1395
+ "learning_rate": 0.00017570237751915092,
1396
+ "loss": 1.595,
1397
+ "step": 226
1398
+ },
1399
+ {
1400
+ "epoch": 0.71,
1401
+ "learning_rate": 0.00017548657551294192,
1402
+ "loss": 1.6072,
1403
+ "step": 227
1404
+ },
1405
+ {
1406
+ "epoch": 0.71,
1407
+ "learning_rate": 0.000175269953186781,
1408
+ "loss": 1.5855,
1409
+ "step": 228
1410
+ },
1411
+ {
1412
+ "epoch": 0.71,
1413
+ "learning_rate": 0.00017505251289472406,
1414
+ "loss": 1.597,
1415
+ "step": 229
1416
+ },
1417
+ {
1418
+ "epoch": 0.72,
1419
+ "learning_rate": 0.0001748342569997158,
1420
+ "loss": 1.5837,
1421
+ "step": 230
1422
+ },
1423
+ {
1424
+ "epoch": 0.72,
1425
+ "learning_rate": 0.00017461518787356432,
1426
+ "loss": 1.5422,
1427
+ "step": 231
1428
+ },
1429
+ {
1430
+ "epoch": 0.72,
1431
+ "learning_rate": 0.00017439530789691506,
1432
+ "loss": 1.5837,
1433
+ "step": 232
1434
+ },
1435
+ {
1436
+ "epoch": 0.72,
1437
+ "learning_rate": 0.0001741746194592251,
1438
+ "loss": 1.6038,
1439
+ "step": 233
1440
+ },
1441
+ {
1442
+ "epoch": 0.73,
1443
+ "learning_rate": 0.00017395312495873717,
1444
+ "loss": 1.5882,
1445
+ "step": 234
1446
+ },
1447
+ {
1448
+ "epoch": 0.73,
1449
+ "learning_rate": 0.00017373082680245347,
1450
+ "loss": 1.5763,
1451
+ "step": 235
1452
+ },
1453
+ {
1454
+ "epoch": 0.73,
1455
+ "learning_rate": 0.00017350772740610976,
1456
+ "loss": 1.6046,
1457
+ "step": 236
1458
+ },
1459
+ {
1460
+ "epoch": 0.74,
1461
+ "learning_rate": 0.00017328382919414877,
1462
+ "loss": 1.594,
1463
+ "step": 237
1464
+ },
1465
+ {
1466
+ "epoch": 0.74,
1467
+ "learning_rate": 0.00017305913459969414,
1468
+ "loss": 1.5903,
1469
+ "step": 238
1470
+ },
1471
+ {
1472
+ "epoch": 0.74,
1473
+ "learning_rate": 0.00017283364606452396,
1474
+ "loss": 1.5704,
1475
+ "step": 239
1476
+ },
1477
+ {
1478
+ "epoch": 0.75,
1479
+ "learning_rate": 0.0001726073660390439,
1480
+ "loss": 1.588,
1481
+ "step": 240
1482
+ },
1483
+ {
1484
+ "epoch": 0.75,
1485
+ "learning_rate": 0.00017238029698226113,
1486
+ "loss": 1.6273,
1487
+ "step": 241
1488
+ },
1489
+ {
1490
+ "epoch": 0.75,
1491
+ "learning_rate": 0.00017215244136175705,
1492
+ "loss": 1.5166,
1493
+ "step": 242
1494
+ },
1495
+ {
1496
+ "epoch": 0.76,
1497
+ "learning_rate": 0.00017192380165366092,
1498
+ "loss": 1.5813,
1499
+ "step": 243
1500
+ },
1501
+ {
1502
+ "epoch": 0.76,
1503
+ "learning_rate": 0.0001716943803426226,
1504
+ "loss": 1.5654,
1505
+ "step": 244
1506
+ },
1507
+ {
1508
+ "epoch": 0.76,
1509
+ "learning_rate": 0.0001714641799217858,
1510
+ "loss": 1.5548,
1511
+ "step": 245
1512
+ },
1513
+ {
1514
+ "epoch": 0.77,
1515
+ "learning_rate": 0.00017123320289276085,
1516
+ "loss": 1.5491,
1517
+ "step": 246
1518
+ },
1519
+ {
1520
+ "epoch": 0.77,
1521
+ "learning_rate": 0.0001710014517655976,
1522
+ "loss": 1.5903,
1523
+ "step": 247
1524
+ },
1525
+ {
1526
+ "epoch": 0.77,
1527
+ "learning_rate": 0.00017076892905875806,
1528
+ "loss": 1.5687,
1529
+ "step": 248
1530
+ },
1531
+ {
1532
+ "epoch": 0.77,
1533
+ "learning_rate": 0.00017053563729908905,
1534
+ "loss": 1.5975,
1535
+ "step": 249
1536
+ },
1537
+ {
1538
+ "epoch": 0.78,
1539
+ "learning_rate": 0.00017030157902179485,
1540
+ "loss": 1.6055,
1541
+ "step": 250
1542
+ },
1543
+ {
1544
+ "epoch": 0.78,
1545
+ "eval_loss": 1.60513174533844,
1546
+ "eval_runtime": 233.7813,
1547
+ "eval_samples_per_second": 16.353,
1548
+ "eval_steps_per_second": 4.089,
1549
+ "step": 250
1550
+ }
1551
+ ],
1552
+ "logging_steps": 1,
1553
+ "max_steps": 963,
1554
+ "num_train_epochs": 3,
1555
+ "save_steps": 50,
1556
+ "total_flos": 7.01067914379264e+17,
1557
+ "trial_name": null,
1558
+ "trial_params": null
1559
+ }
checkpoint-250/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f61cafb89242b653e455003b5517e685ecccfa6180af5fb7d0bfb35b4fc77a4
3
+ size 4475
checkpoint-300/README.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - quant_method: bitsandbytes
9
+ - load_in_8bit: False
10
+ - load_in_4bit: True
11
+ - llm_int8_threshold: 6.0
12
+ - llm_int8_skip_modules: None
13
+ - llm_int8_enable_fp32_cpu_offload: False
14
+ - llm_int8_has_fp16_weight: False
15
+ - bnb_4bit_quant_type: nf4
16
+ - bnb_4bit_use_double_quant: True
17
+ - bnb_4bit_compute_dtype: bfloat16
18
+ ### Framework versions
19
+
20
+
21
+ - PEFT 0.6.0.dev0
checkpoint-300/adapter_config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "./mistralai_Mistral-7B-v0.1",
5
+ "bias": "none",
6
+ "fan_in_fan_out": null,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "lora_alpha": 16,
12
+ "lora_dropout": 0.05,
13
+ "modules_to_save": null,
14
+ "peft_type": "LORA",
15
+ "r": 8,
16
+ "rank_pattern": {},
17
+ "revision": null,
18
+ "target_modules": [
19
+ "o_proj",
20
+ "k_proj",
21
+ "up_proj",
22
+ "v_proj",
23
+ "q_proj",
24
+ "gate_proj",
25
+ "down_proj"
26
+ ],
27
+ "task_type": "CAUSAL_LM"
28
+ }
checkpoint-300/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:219facdff16b696735f1a84fdd92a0cbac9e197db9eef71c8ce3344d57718790
3
+ size 84046925
checkpoint-300/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a343ddfe6887de6fbae314359ede72e86a4b1ca90c8dca7f14f5c67e99f3f746
3
+ size 168039557
checkpoint-300/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65057c4c38794cbabddcd3f77e0f9ae9b434f793b5e38c88f0b54b3a59c2a015
3
+ size 14575
checkpoint-300/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d16d35aca2529b2ee06dff7108489f02d74e6fa0dc4c995236c9afa17fe97a23
3
+ size 627
checkpoint-300/trainer_state.json ADDED
@@ -0,0 +1,1867 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 1.6023043394088745,
3
+ "best_model_checkpoint": "./lora-out/checkpoint-300",
4
+ "epoch": 0.9331259720062208,
5
+ "eval_steps": 50,
6
+ "global_step": 300,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0,
13
+ "learning_rate": 2e-05,
14
+ "loss": 1.7924,
15
+ "step": 1
16
+ },
17
+ {
18
+ "epoch": 0.01,
19
+ "learning_rate": 4e-05,
20
+ "loss": 1.8083,
21
+ "step": 2
22
+ },
23
+ {
24
+ "epoch": 0.01,
25
+ "learning_rate": 6e-05,
26
+ "loss": 1.8177,
27
+ "step": 3
28
+ },
29
+ {
30
+ "epoch": 0.01,
31
+ "learning_rate": 8e-05,
32
+ "loss": 1.7595,
33
+ "step": 4
34
+ },
35
+ {
36
+ "epoch": 0.02,
37
+ "learning_rate": 0.0001,
38
+ "loss": 1.6598,
39
+ "step": 5
40
+ },
41
+ {
42
+ "epoch": 0.02,
43
+ "learning_rate": 0.00012,
44
+ "loss": 1.6919,
45
+ "step": 6
46
+ },
47
+ {
48
+ "epoch": 0.02,
49
+ "learning_rate": 0.00014,
50
+ "loss": 1.6706,
51
+ "step": 7
52
+ },
53
+ {
54
+ "epoch": 0.02,
55
+ "learning_rate": 0.00016,
56
+ "loss": 1.6879,
57
+ "step": 8
58
+ },
59
+ {
60
+ "epoch": 0.03,
61
+ "learning_rate": 0.00018,
62
+ "loss": 1.7051,
63
+ "step": 9
64
+ },
65
+ {
66
+ "epoch": 0.03,
67
+ "learning_rate": 0.0002,
68
+ "loss": 1.7022,
69
+ "step": 10
70
+ },
71
+ {
72
+ "epoch": 0.03,
73
+ "learning_rate": 0.000199999456645141,
74
+ "loss": 1.6809,
75
+ "step": 11
76
+ },
77
+ {
78
+ "epoch": 0.04,
79
+ "learning_rate": 0.00019999782658646859,
80
+ "loss": 1.6098,
81
+ "step": 12
82
+ },
83
+ {
84
+ "epoch": 0.04,
85
+ "learning_rate": 0.0001999951098416968,
86
+ "loss": 1.7014,
87
+ "step": 13
88
+ },
89
+ {
90
+ "epoch": 0.04,
91
+ "learning_rate": 0.00019999130644034888,
92
+ "loss": 1.5885,
93
+ "step": 14
94
+ },
95
+ {
96
+ "epoch": 0.05,
97
+ "learning_rate": 0.00019998641642375657,
98
+ "loss": 1.6243,
99
+ "step": 15
100
+ },
101
+ {
102
+ "epoch": 0.05,
103
+ "learning_rate": 0.00019998043984506027,
104
+ "loss": 1.6484,
105
+ "step": 16
106
+ },
107
+ {
108
+ "epoch": 0.05,
109
+ "learning_rate": 0.00019997337676920803,
110
+ "loss": 1.6093,
111
+ "step": 17
112
+ },
113
+ {
114
+ "epoch": 0.06,
115
+ "learning_rate": 0.00019996522727295496,
116
+ "loss": 1.6173,
117
+ "step": 18
118
+ },
119
+ {
120
+ "epoch": 0.06,
121
+ "learning_rate": 0.00019995599144486247,
122
+ "loss": 1.646,
123
+ "step": 19
124
+ },
125
+ {
126
+ "epoch": 0.06,
127
+ "learning_rate": 0.00019994566938529712,
128
+ "loss": 1.6469,
129
+ "step": 20
130
+ },
131
+ {
132
+ "epoch": 0.07,
133
+ "learning_rate": 0.00019993426120642983,
134
+ "loss": 1.6564,
135
+ "step": 21
136
+ },
137
+ {
138
+ "epoch": 0.07,
139
+ "learning_rate": 0.00019992176703223432,
140
+ "loss": 1.5901,
141
+ "step": 22
142
+ },
143
+ {
144
+ "epoch": 0.07,
145
+ "learning_rate": 0.000199908186998486,
146
+ "loss": 1.664,
147
+ "step": 23
148
+ },
149
+ {
150
+ "epoch": 0.07,
151
+ "learning_rate": 0.00019989352125276047,
152
+ "loss": 1.6275,
153
+ "step": 24
154
+ },
155
+ {
156
+ "epoch": 0.08,
157
+ "learning_rate": 0.00019987776995443178,
158
+ "loss": 1.5839,
159
+ "step": 25
160
+ },
161
+ {
162
+ "epoch": 0.08,
163
+ "learning_rate": 0.00019986093327467076,
164
+ "loss": 1.5611,
165
+ "step": 26
166
+ },
167
+ {
168
+ "epoch": 0.08,
169
+ "learning_rate": 0.00019984301139644334,
170
+ "loss": 1.669,
171
+ "step": 27
172
+ },
173
+ {
174
+ "epoch": 0.09,
175
+ "learning_rate": 0.0001998240045145083,
176
+ "loss": 1.5641,
177
+ "step": 28
178
+ },
179
+ {
180
+ "epoch": 0.09,
181
+ "learning_rate": 0.00019980391283541522,
182
+ "loss": 1.6023,
183
+ "step": 29
184
+ },
185
+ {
186
+ "epoch": 0.09,
187
+ "learning_rate": 0.00019978273657750238,
188
+ "loss": 1.6309,
189
+ "step": 30
190
+ },
191
+ {
192
+ "epoch": 0.1,
193
+ "learning_rate": 0.0001997604759708942,
194
+ "loss": 1.6353,
195
+ "step": 31
196
+ },
197
+ {
198
+ "epoch": 0.1,
199
+ "learning_rate": 0.00019973713125749884,
200
+ "loss": 1.6328,
201
+ "step": 32
202
+ },
203
+ {
204
+ "epoch": 0.1,
205
+ "learning_rate": 0.00019971270269100564,
206
+ "loss": 1.5683,
207
+ "step": 33
208
+ },
209
+ {
210
+ "epoch": 0.11,
211
+ "learning_rate": 0.00019968719053688213,
212
+ "loss": 1.6217,
213
+ "step": 34
214
+ },
215
+ {
216
+ "epoch": 0.11,
217
+ "learning_rate": 0.0001996605950723714,
218
+ "loss": 1.5734,
219
+ "step": 35
220
+ },
221
+ {
222
+ "epoch": 0.11,
223
+ "learning_rate": 0.00019963291658648896,
224
+ "loss": 1.6162,
225
+ "step": 36
226
+ },
227
+ {
228
+ "epoch": 0.12,
229
+ "learning_rate": 0.00019960415538001957,
230
+ "loss": 1.5922,
231
+ "step": 37
232
+ },
233
+ {
234
+ "epoch": 0.12,
235
+ "learning_rate": 0.0001995743117655141,
236
+ "loss": 1.5806,
237
+ "step": 38
238
+ },
239
+ {
240
+ "epoch": 0.12,
241
+ "learning_rate": 0.000199543386067286,
242
+ "loss": 1.5938,
243
+ "step": 39
244
+ },
245
+ {
246
+ "epoch": 0.12,
247
+ "learning_rate": 0.00019951137862140778,
248
+ "loss": 1.6386,
249
+ "step": 40
250
+ },
251
+ {
252
+ "epoch": 0.13,
253
+ "learning_rate": 0.00019947828977570756,
254
+ "loss": 1.6476,
255
+ "step": 41
256
+ },
257
+ {
258
+ "epoch": 0.13,
259
+ "learning_rate": 0.00019944411988976496,
260
+ "loss": 1.6557,
261
+ "step": 42
262
+ },
263
+ {
264
+ "epoch": 0.13,
265
+ "learning_rate": 0.00019940886933490749,
266
+ "loss": 1.5836,
267
+ "step": 43
268
+ },
269
+ {
270
+ "epoch": 0.14,
271
+ "learning_rate": 0.00019937253849420635,
272
+ "loss": 1.6421,
273
+ "step": 44
274
+ },
275
+ {
276
+ "epoch": 0.14,
277
+ "learning_rate": 0.0001993351277624723,
278
+ "loss": 1.629,
279
+ "step": 45
280
+ },
281
+ {
282
+ "epoch": 0.14,
283
+ "learning_rate": 0.00019929663754625145,
284
+ "loss": 1.6392,
285
+ "step": 46
286
+ },
287
+ {
288
+ "epoch": 0.15,
289
+ "learning_rate": 0.00019925706826382064,
290
+ "loss": 1.5677,
291
+ "step": 47
292
+ },
293
+ {
294
+ "epoch": 0.15,
295
+ "learning_rate": 0.00019921642034518317,
296
+ "loss": 1.6144,
297
+ "step": 48
298
+ },
299
+ {
300
+ "epoch": 0.15,
301
+ "learning_rate": 0.00019917469423206389,
302
+ "loss": 1.6068,
303
+ "step": 49
304
+ },
305
+ {
306
+ "epoch": 0.16,
307
+ "learning_rate": 0.00019913189037790456,
308
+ "loss": 1.6421,
309
+ "step": 50
310
+ },
311
+ {
312
+ "epoch": 0.16,
313
+ "eval_loss": 1.621693730354309,
314
+ "eval_runtime": 233.7603,
315
+ "eval_samples_per_second": 16.354,
316
+ "eval_steps_per_second": 4.09,
317
+ "step": 50
318
+ },
319
+ {
320
+ "epoch": 0.16,
321
+ "learning_rate": 0.0001990880092478588,
322
+ "loss": 1.6172,
323
+ "step": 51
324
+ },
325
+ {
326
+ "epoch": 0.16,
327
+ "learning_rate": 0.0001990430513187871,
328
+ "loss": 1.6095,
329
+ "step": 52
330
+ },
331
+ {
332
+ "epoch": 0.16,
333
+ "learning_rate": 0.00019899701707925166,
334
+ "loss": 1.5967,
335
+ "step": 53
336
+ },
337
+ {
338
+ "epoch": 0.17,
339
+ "learning_rate": 0.00019894990702951106,
340
+ "loss": 1.617,
341
+ "step": 54
342
+ },
343
+ {
344
+ "epoch": 0.17,
345
+ "learning_rate": 0.00019890172168151473,
346
+ "loss": 1.5932,
347
+ "step": 55
348
+ },
349
+ {
350
+ "epoch": 0.17,
351
+ "learning_rate": 0.0001988524615588976,
352
+ "loss": 1.6548,
353
+ "step": 56
354
+ },
355
+ {
356
+ "epoch": 0.18,
357
+ "learning_rate": 0.00019880212719697413,
358
+ "loss": 1.6033,
359
+ "step": 57
360
+ },
361
+ {
362
+ "epoch": 0.18,
363
+ "learning_rate": 0.00019875071914273278,
364
+ "loss": 1.6063,
365
+ "step": 58
366
+ },
367
+ {
368
+ "epoch": 0.18,
369
+ "learning_rate": 0.00019869823795482986,
370
+ "loss": 1.6107,
371
+ "step": 59
372
+ },
373
+ {
374
+ "epoch": 0.19,
375
+ "learning_rate": 0.00019864468420358354,
376
+ "loss": 1.5758,
377
+ "step": 60
378
+ },
379
+ {
380
+ "epoch": 0.19,
381
+ "learning_rate": 0.00019859005847096763,
382
+ "loss": 1.5723,
383
+ "step": 61
384
+ },
385
+ {
386
+ "epoch": 0.19,
387
+ "learning_rate": 0.00019853436135060527,
388
+ "loss": 1.542,
389
+ "step": 62
390
+ },
391
+ {
392
+ "epoch": 0.2,
393
+ "learning_rate": 0.00019847759344776252,
394
+ "loss": 1.5611,
395
+ "step": 63
396
+ },
397
+ {
398
+ "epoch": 0.2,
399
+ "learning_rate": 0.00019841975537934162,
400
+ "loss": 1.6157,
401
+ "step": 64
402
+ },
403
+ {
404
+ "epoch": 0.2,
405
+ "learning_rate": 0.00019836084777387458,
406
+ "loss": 1.5589,
407
+ "step": 65
408
+ },
409
+ {
410
+ "epoch": 0.21,
411
+ "learning_rate": 0.00019830087127151598,
412
+ "loss": 1.6077,
413
+ "step": 66
414
+ },
415
+ {
416
+ "epoch": 0.21,
417
+ "learning_rate": 0.00019823982652403634,
418
+ "loss": 1.5473,
419
+ "step": 67
420
+ },
421
+ {
422
+ "epoch": 0.21,
423
+ "learning_rate": 0.00019817771419481487,
424
+ "loss": 1.6265,
425
+ "step": 68
426
+ },
427
+ {
428
+ "epoch": 0.21,
429
+ "learning_rate": 0.0001981145349588323,
430
+ "loss": 1.6074,
431
+ "step": 69
432
+ },
433
+ {
434
+ "epoch": 0.22,
435
+ "learning_rate": 0.00019805028950266348,
436
+ "loss": 1.6195,
437
+ "step": 70
438
+ },
439
+ {
440
+ "epoch": 0.22,
441
+ "learning_rate": 0.00019798497852447006,
442
+ "loss": 1.5876,
443
+ "step": 71
444
+ },
445
+ {
446
+ "epoch": 0.22,
447
+ "learning_rate": 0.0001979186027339928,
448
+ "loss": 1.5978,
449
+ "step": 72
450
+ },
451
+ {
452
+ "epoch": 0.23,
453
+ "learning_rate": 0.00019785116285254381,
454
+ "loss": 1.533,
455
+ "step": 73
456
+ },
457
+ {
458
+ "epoch": 0.23,
459
+ "learning_rate": 0.00019778265961299888,
460
+ "loss": 1.5888,
461
+ "step": 74
462
+ },
463
+ {
464
+ "epoch": 0.23,
465
+ "learning_rate": 0.0001977130937597894,
466
+ "loss": 1.6211,
467
+ "step": 75
468
+ },
469
+ {
470
+ "epoch": 0.24,
471
+ "learning_rate": 0.00019764246604889415,
472
+ "loss": 1.6091,
473
+ "step": 76
474
+ },
475
+ {
476
+ "epoch": 0.24,
477
+ "learning_rate": 0.00019757077724783147,
478
+ "loss": 1.6012,
479
+ "step": 77
480
+ },
481
+ {
482
+ "epoch": 0.24,
483
+ "learning_rate": 0.0001974980281356504,
484
+ "loss": 1.6401,
485
+ "step": 78
486
+ },
487
+ {
488
+ "epoch": 0.25,
489
+ "learning_rate": 0.0001974242195029227,
490
+ "loss": 1.6111,
491
+ "step": 79
492
+ },
493
+ {
494
+ "epoch": 0.25,
495
+ "learning_rate": 0.00019734935215173392,
496
+ "loss": 1.6208,
497
+ "step": 80
498
+ },
499
+ {
500
+ "epoch": 0.25,
501
+ "learning_rate": 0.00019727342689567482,
502
+ "loss": 1.6038,
503
+ "step": 81
504
+ },
505
+ {
506
+ "epoch": 0.26,
507
+ "learning_rate": 0.00019719644455983256,
508
+ "loss": 1.5915,
509
+ "step": 82
510
+ },
511
+ {
512
+ "epoch": 0.26,
513
+ "learning_rate": 0.0001971184059807817,
514
+ "loss": 1.5872,
515
+ "step": 83
516
+ },
517
+ {
518
+ "epoch": 0.26,
519
+ "learning_rate": 0.000197039312006575,
520
+ "loss": 1.5984,
521
+ "step": 84
522
+ },
523
+ {
524
+ "epoch": 0.26,
525
+ "learning_rate": 0.0001969591634967344,
526
+ "loss": 1.5996,
527
+ "step": 85
528
+ },
529
+ {
530
+ "epoch": 0.27,
531
+ "learning_rate": 0.00019687796132224152,
532
+ "loss": 1.6056,
533
+ "step": 86
534
+ },
535
+ {
536
+ "epoch": 0.27,
537
+ "learning_rate": 0.0001967957063655283,
538
+ "loss": 1.6099,
539
+ "step": 87
540
+ },
541
+ {
542
+ "epoch": 0.27,
543
+ "learning_rate": 0.0001967123995204674,
544
+ "loss": 1.6295,
545
+ "step": 88
546
+ },
547
+ {
548
+ "epoch": 0.28,
549
+ "learning_rate": 0.00019662804169236225,
550
+ "loss": 1.5482,
551
+ "step": 89
552
+ },
553
+ {
554
+ "epoch": 0.28,
555
+ "learning_rate": 0.00019654263379793773,
556
+ "loss": 1.5781,
557
+ "step": 90
558
+ },
559
+ {
560
+ "epoch": 0.28,
561
+ "learning_rate": 0.00019645617676532963,
562
+ "loss": 1.5954,
563
+ "step": 91
564
+ },
565
+ {
566
+ "epoch": 0.29,
567
+ "learning_rate": 0.000196368671534075,
568
+ "loss": 1.619,
569
+ "step": 92
570
+ },
571
+ {
572
+ "epoch": 0.29,
573
+ "learning_rate": 0.0001962801190551016,
574
+ "loss": 1.6153,
575
+ "step": 93
576
+ },
577
+ {
578
+ "epoch": 0.29,
579
+ "learning_rate": 0.0001961905202907179,
580
+ "loss": 1.6008,
581
+ "step": 94
582
+ },
583
+ {
584
+ "epoch": 0.3,
585
+ "learning_rate": 0.00019609987621460232,
586
+ "loss": 1.5891,
587
+ "step": 95
588
+ },
589
+ {
590
+ "epoch": 0.3,
591
+ "learning_rate": 0.0001960081878117929,
592
+ "loss": 1.6438,
593
+ "step": 96
594
+ },
595
+ {
596
+ "epoch": 0.3,
597
+ "learning_rate": 0.0001959154560786764,
598
+ "loss": 1.5576,
599
+ "step": 97
600
+ },
601
+ {
602
+ "epoch": 0.3,
603
+ "learning_rate": 0.00019582168202297758,
604
+ "loss": 1.646,
605
+ "step": 98
606
+ },
607
+ {
608
+ "epoch": 0.31,
609
+ "learning_rate": 0.00019572686666374822,
610
+ "loss": 1.6269,
611
+ "step": 99
612
+ },
613
+ {
614
+ "epoch": 0.31,
615
+ "learning_rate": 0.00019563101103135602,
616
+ "loss": 1.6288,
617
+ "step": 100
618
+ },
619
+ {
620
+ "epoch": 0.31,
621
+ "eval_loss": 1.6143836975097656,
622
+ "eval_runtime": 233.6412,
623
+ "eval_samples_per_second": 16.363,
624
+ "eval_steps_per_second": 4.092,
625
+ "step": 100
626
+ },
627
+ {
628
+ "epoch": 0.31,
629
+ "learning_rate": 0.00019553411616747348,
630
+ "loss": 1.5667,
631
+ "step": 101
632
+ },
633
+ {
634
+ "epoch": 0.32,
635
+ "learning_rate": 0.00019543618312506647,
636
+ "loss": 1.6221,
637
+ "step": 102
638
+ },
639
+ {
640
+ "epoch": 0.32,
641
+ "learning_rate": 0.0001953372129683829,
642
+ "loss": 1.5992,
643
+ "step": 103
644
+ },
645
+ {
646
+ "epoch": 0.32,
647
+ "learning_rate": 0.0001952372067729411,
648
+ "loss": 1.6138,
649
+ "step": 104
650
+ },
651
+ {
652
+ "epoch": 0.33,
653
+ "learning_rate": 0.00019513616562551807,
654
+ "loss": 1.51,
655
+ "step": 105
656
+ },
657
+ {
658
+ "epoch": 0.33,
659
+ "learning_rate": 0.00019503409062413782,
660
+ "loss": 1.6227,
661
+ "step": 106
662
+ },
663
+ {
664
+ "epoch": 0.33,
665
+ "learning_rate": 0.00019493098287805927,
666
+ "loss": 1.6014,
667
+ "step": 107
668
+ },
669
+ {
670
+ "epoch": 0.34,
671
+ "learning_rate": 0.00019482684350776434,
672
+ "loss": 1.625,
673
+ "step": 108
674
+ },
675
+ {
676
+ "epoch": 0.34,
677
+ "learning_rate": 0.0001947216736449457,
678
+ "loss": 1.6109,
679
+ "step": 109
680
+ },
681
+ {
682
+ "epoch": 0.34,
683
+ "learning_rate": 0.0001946154744324945,
684
+ "loss": 1.62,
685
+ "step": 110
686
+ },
687
+ {
688
+ "epoch": 0.35,
689
+ "learning_rate": 0.00019450824702448778,
690
+ "loss": 1.5878,
691
+ "step": 111
692
+ },
693
+ {
694
+ "epoch": 0.35,
695
+ "learning_rate": 0.0001943999925861763,
696
+ "loss": 1.6264,
697
+ "step": 112
698
+ },
699
+ {
700
+ "epoch": 0.35,
701
+ "learning_rate": 0.00019429071229397157,
702
+ "loss": 1.6186,
703
+ "step": 113
704
+ },
705
+ {
706
+ "epoch": 0.35,
707
+ "learning_rate": 0.0001941804073354331,
708
+ "loss": 1.6363,
709
+ "step": 114
710
+ },
711
+ {
712
+ "epoch": 0.36,
713
+ "learning_rate": 0.00019406907890925562,
714
+ "loss": 1.5341,
715
+ "step": 115
716
+ },
717
+ {
718
+ "epoch": 0.36,
719
+ "learning_rate": 0.00019395672822525593,
720
+ "loss": 1.5986,
721
+ "step": 116
722
+ },
723
+ {
724
+ "epoch": 0.36,
725
+ "learning_rate": 0.00019384335650435985,
726
+ "loss": 1.6181,
727
+ "step": 117
728
+ },
729
+ {
730
+ "epoch": 0.37,
731
+ "learning_rate": 0.0001937289649785889,
732
+ "loss": 1.6118,
733
+ "step": 118
734
+ },
735
+ {
736
+ "epoch": 0.37,
737
+ "learning_rate": 0.0001936135548910469,
738
+ "loss": 1.6404,
739
+ "step": 119
740
+ },
741
+ {
742
+ "epoch": 0.37,
743
+ "learning_rate": 0.00019349712749590649,
744
+ "loss": 1.583,
745
+ "step": 120
746
+ },
747
+ {
748
+ "epoch": 0.38,
749
+ "learning_rate": 0.00019337968405839547,
750
+ "loss": 1.5827,
751
+ "step": 121
752
+ },
753
+ {
754
+ "epoch": 0.38,
755
+ "learning_rate": 0.00019326122585478308,
756
+ "loss": 1.6392,
757
+ "step": 122
758
+ },
759
+ {
760
+ "epoch": 0.38,
761
+ "learning_rate": 0.00019314175417236616,
762
+ "loss": 1.5861,
763
+ "step": 123
764
+ },
765
+ {
766
+ "epoch": 0.39,
767
+ "learning_rate": 0.00019302127030945508,
768
+ "loss": 1.5738,
769
+ "step": 124
770
+ },
771
+ {
772
+ "epoch": 0.39,
773
+ "learning_rate": 0.0001928997755753597,
774
+ "loss": 1.5915,
775
+ "step": 125
776
+ },
777
+ {
778
+ "epoch": 0.39,
779
+ "learning_rate": 0.00019277727129037508,
780
+ "loss": 1.617,
781
+ "step": 126
782
+ },
783
+ {
784
+ "epoch": 0.4,
785
+ "learning_rate": 0.0001926537587857672,
786
+ "loss": 1.5582,
787
+ "step": 127
788
+ },
789
+ {
790
+ "epoch": 0.4,
791
+ "learning_rate": 0.00019252923940375844,
792
+ "loss": 1.6294,
793
+ "step": 128
794
+ },
795
+ {
796
+ "epoch": 0.4,
797
+ "learning_rate": 0.00019240371449751306,
798
+ "loss": 1.6087,
799
+ "step": 129
800
+ },
801
+ {
802
+ "epoch": 0.4,
803
+ "learning_rate": 0.00019227718543112236,
804
+ "loss": 1.5749,
805
+ "step": 130
806
+ },
807
+ {
808
+ "epoch": 0.41,
809
+ "learning_rate": 0.00019214965357959005,
810
+ "loss": 1.6041,
811
+ "step": 131
812
+ },
813
+ {
814
+ "epoch": 0.41,
815
+ "learning_rate": 0.00019202112032881715,
816
+ "loss": 1.6106,
817
+ "step": 132
818
+ },
819
+ {
820
+ "epoch": 0.41,
821
+ "learning_rate": 0.00019189158707558695,
822
+ "loss": 1.5553,
823
+ "step": 133
824
+ },
825
+ {
826
+ "epoch": 0.42,
827
+ "learning_rate": 0.00019176105522754995,
828
+ "loss": 1.5638,
829
+ "step": 134
830
+ },
831
+ {
832
+ "epoch": 0.42,
833
+ "learning_rate": 0.0001916295262032084,
834
+ "loss": 1.5921,
835
+ "step": 135
836
+ },
837
+ {
838
+ "epoch": 0.42,
839
+ "learning_rate": 0.00019149700143190096,
840
+ "loss": 1.5837,
841
+ "step": 136
842
+ },
843
+ {
844
+ "epoch": 0.43,
845
+ "learning_rate": 0.00019136348235378726,
846
+ "loss": 1.6341,
847
+ "step": 137
848
+ },
849
+ {
850
+ "epoch": 0.43,
851
+ "learning_rate": 0.00019122897041983205,
852
+ "loss": 1.5678,
853
+ "step": 138
854
+ },
855
+ {
856
+ "epoch": 0.43,
857
+ "learning_rate": 0.00019109346709178963,
858
+ "loss": 1.6137,
859
+ "step": 139
860
+ },
861
+ {
862
+ "epoch": 0.44,
863
+ "learning_rate": 0.0001909569738421878,
864
+ "loss": 1.6324,
865
+ "step": 140
866
+ },
867
+ {
868
+ "epoch": 0.44,
869
+ "learning_rate": 0.00019081949215431194,
870
+ "loss": 1.612,
871
+ "step": 141
872
+ },
873
+ {
874
+ "epoch": 0.44,
875
+ "learning_rate": 0.00019068102352218897,
876
+ "loss": 1.5908,
877
+ "step": 142
878
+ },
879
+ {
880
+ "epoch": 0.44,
881
+ "learning_rate": 0.00019054156945057097,
882
+ "loss": 1.6087,
883
+ "step": 143
884
+ },
885
+ {
886
+ "epoch": 0.45,
887
+ "learning_rate": 0.00019040113145491887,
888
+ "loss": 1.5613,
889
+ "step": 144
890
+ },
891
+ {
892
+ "epoch": 0.45,
893
+ "learning_rate": 0.000190259711061386,
894
+ "loss": 1.6072,
895
+ "step": 145
896
+ },
897
+ {
898
+ "epoch": 0.45,
899
+ "learning_rate": 0.00019011730980680156,
900
+ "loss": 1.5722,
901
+ "step": 146
902
+ },
903
+ {
904
+ "epoch": 0.46,
905
+ "learning_rate": 0.0001899739292386538,
906
+ "loss": 1.5961,
907
+ "step": 147
908
+ },
909
+ {
910
+ "epoch": 0.46,
911
+ "learning_rate": 0.00018982957091507325,
912
+ "loss": 1.5409,
913
+ "step": 148
914
+ },
915
+ {
916
+ "epoch": 0.46,
917
+ "learning_rate": 0.0001896842364048159,
918
+ "loss": 1.6557,
919
+ "step": 149
920
+ },
921
+ {
922
+ "epoch": 0.47,
923
+ "learning_rate": 0.000189537927287246,
924
+ "loss": 1.5725,
925
+ "step": 150
926
+ },
927
+ {
928
+ "epoch": 0.47,
929
+ "eval_loss": 1.6101970672607422,
930
+ "eval_runtime": 233.5313,
931
+ "eval_samples_per_second": 16.37,
932
+ "eval_steps_per_second": 4.094,
933
+ "step": 150
934
+ },
935
+ {
936
+ "epoch": 0.47,
937
+ "learning_rate": 0.00018939064515231888,
938
+ "loss": 1.5949,
939
+ "step": 151
940
+ },
941
+ {
942
+ "epoch": 0.47,
943
+ "learning_rate": 0.0001892423916005639,
944
+ "loss": 1.6191,
945
+ "step": 152
946
+ },
947
+ {
948
+ "epoch": 0.48,
949
+ "learning_rate": 0.00018909316824306674,
950
+ "loss": 1.5487,
951
+ "step": 153
952
+ },
953
+ {
954
+ "epoch": 0.48,
955
+ "learning_rate": 0.00018894297670145216,
956
+ "loss": 1.5104,
957
+ "step": 154
958
+ },
959
+ {
960
+ "epoch": 0.48,
961
+ "learning_rate": 0.00018879181860786623,
962
+ "loss": 1.6392,
963
+ "step": 155
964
+ },
965
+ {
966
+ "epoch": 0.49,
967
+ "learning_rate": 0.00018863969560495866,
968
+ "loss": 1.5932,
969
+ "step": 156
970
+ },
971
+ {
972
+ "epoch": 0.49,
973
+ "learning_rate": 0.00018848660934586491,
974
+ "loss": 1.6213,
975
+ "step": 157
976
+ },
977
+ {
978
+ "epoch": 0.49,
979
+ "learning_rate": 0.0001883325614941882,
980
+ "loss": 1.5515,
981
+ "step": 158
982
+ },
983
+ {
984
+ "epoch": 0.49,
985
+ "learning_rate": 0.00018817755372398155,
986
+ "loss": 1.6166,
987
+ "step": 159
988
+ },
989
+ {
990
+ "epoch": 0.5,
991
+ "learning_rate": 0.00018802158771972943,
992
+ "loss": 1.6552,
993
+ "step": 160
994
+ },
995
+ {
996
+ "epoch": 0.5,
997
+ "learning_rate": 0.00018786466517632956,
998
+ "loss": 1.6378,
999
+ "step": 161
1000
+ },
1001
+ {
1002
+ "epoch": 0.5,
1003
+ "learning_rate": 0.00018770678779907448,
1004
+ "loss": 1.5176,
1005
+ "step": 162
1006
+ },
1007
+ {
1008
+ "epoch": 0.51,
1009
+ "learning_rate": 0.00018754795730363302,
1010
+ "loss": 1.5793,
1011
+ "step": 163
1012
+ },
1013
+ {
1014
+ "epoch": 0.51,
1015
+ "learning_rate": 0.00018738817541603156,
1016
+ "loss": 1.6616,
1017
+ "step": 164
1018
+ },
1019
+ {
1020
+ "epoch": 0.51,
1021
+ "learning_rate": 0.00018722744387263544,
1022
+ "loss": 1.6055,
1023
+ "step": 165
1024
+ },
1025
+ {
1026
+ "epoch": 0.52,
1027
+ "learning_rate": 0.00018706576442012994,
1028
+ "loss": 1.6204,
1029
+ "step": 166
1030
+ },
1031
+ {
1032
+ "epoch": 0.52,
1033
+ "learning_rate": 0.00018690313881550137,
1034
+ "loss": 1.5952,
1035
+ "step": 167
1036
+ },
1037
+ {
1038
+ "epoch": 0.52,
1039
+ "learning_rate": 0.00018673956882601803,
1040
+ "loss": 1.6271,
1041
+ "step": 168
1042
+ },
1043
+ {
1044
+ "epoch": 0.53,
1045
+ "learning_rate": 0.00018657505622921082,
1046
+ "loss": 1.538,
1047
+ "step": 169
1048
+ },
1049
+ {
1050
+ "epoch": 0.53,
1051
+ "learning_rate": 0.00018640960281285417,
1052
+ "loss": 1.5874,
1053
+ "step": 170
1054
+ },
1055
+ {
1056
+ "epoch": 0.53,
1057
+ "learning_rate": 0.0001862432103749464,
1058
+ "loss": 1.5694,
1059
+ "step": 171
1060
+ },
1061
+ {
1062
+ "epoch": 0.53,
1063
+ "learning_rate": 0.00018607588072369033,
1064
+ "loss": 1.583,
1065
+ "step": 172
1066
+ },
1067
+ {
1068
+ "epoch": 0.54,
1069
+ "learning_rate": 0.00018590761567747354,
1070
+ "loss": 1.5961,
1071
+ "step": 173
1072
+ },
1073
+ {
1074
+ "epoch": 0.54,
1075
+ "learning_rate": 0.00018573841706484866,
1076
+ "loss": 1.582,
1077
+ "step": 174
1078
+ },
1079
+ {
1080
+ "epoch": 0.54,
1081
+ "learning_rate": 0.0001855682867245134,
1082
+ "loss": 1.6427,
1083
+ "step": 175
1084
+ },
1085
+ {
1086
+ "epoch": 0.55,
1087
+ "learning_rate": 0.00018539722650529075,
1088
+ "loss": 1.604,
1089
+ "step": 176
1090
+ },
1091
+ {
1092
+ "epoch": 0.55,
1093
+ "learning_rate": 0.00018522523826610868,
1094
+ "loss": 1.577,
1095
+ "step": 177
1096
+ },
1097
+ {
1098
+ "epoch": 0.55,
1099
+ "learning_rate": 0.00018505232387598018,
1100
+ "loss": 1.6339,
1101
+ "step": 178
1102
+ },
1103
+ {
1104
+ "epoch": 0.56,
1105
+ "learning_rate": 0.00018487848521398265,
1106
+ "loss": 1.5993,
1107
+ "step": 179
1108
+ },
1109
+ {
1110
+ "epoch": 0.56,
1111
+ "learning_rate": 0.0001847037241692378,
1112
+ "loss": 1.6286,
1113
+ "step": 180
1114
+ },
1115
+ {
1116
+ "epoch": 0.56,
1117
+ "learning_rate": 0.00018452804264089084,
1118
+ "loss": 1.5963,
1119
+ "step": 181
1120
+ },
1121
+ {
1122
+ "epoch": 0.57,
1123
+ "learning_rate": 0.00018435144253809,
1124
+ "loss": 1.5856,
1125
+ "step": 182
1126
+ },
1127
+ {
1128
+ "epoch": 0.57,
1129
+ "learning_rate": 0.00018417392577996578,
1130
+ "loss": 1.5787,
1131
+ "step": 183
1132
+ },
1133
+ {
1134
+ "epoch": 0.57,
1135
+ "learning_rate": 0.00018399549429561006,
1136
+ "loss": 1.5876,
1137
+ "step": 184
1138
+ },
1139
+ {
1140
+ "epoch": 0.58,
1141
+ "learning_rate": 0.00018381615002405509,
1142
+ "loss": 1.5565,
1143
+ "step": 185
1144
+ },
1145
+ {
1146
+ "epoch": 0.58,
1147
+ "learning_rate": 0.00018363589491425248,
1148
+ "loss": 1.5897,
1149
+ "step": 186
1150
+ },
1151
+ {
1152
+ "epoch": 0.58,
1153
+ "learning_rate": 0.0001834547309250521,
1154
+ "loss": 1.5951,
1155
+ "step": 187
1156
+ },
1157
+ {
1158
+ "epoch": 0.58,
1159
+ "learning_rate": 0.00018327266002518056,
1160
+ "loss": 1.5447,
1161
+ "step": 188
1162
+ },
1163
+ {
1164
+ "epoch": 0.59,
1165
+ "learning_rate": 0.00018308968419322003,
1166
+ "loss": 1.6087,
1167
+ "step": 189
1168
+ },
1169
+ {
1170
+ "epoch": 0.59,
1171
+ "learning_rate": 0.00018290580541758668,
1172
+ "loss": 1.5946,
1173
+ "step": 190
1174
+ },
1175
+ {
1176
+ "epoch": 0.59,
1177
+ "learning_rate": 0.00018272102569650905,
1178
+ "loss": 1.6148,
1179
+ "step": 191
1180
+ },
1181
+ {
1182
+ "epoch": 0.6,
1183
+ "learning_rate": 0.00018253534703800627,
1184
+ "loss": 1.649,
1185
+ "step": 192
1186
+ },
1187
+ {
1188
+ "epoch": 0.6,
1189
+ "learning_rate": 0.0001823487714598664,
1190
+ "loss": 1.6312,
1191
+ "step": 193
1192
+ },
1193
+ {
1194
+ "epoch": 0.6,
1195
+ "learning_rate": 0.0001821613009896244,
1196
+ "loss": 1.5858,
1197
+ "step": 194
1198
+ },
1199
+ {
1200
+ "epoch": 0.61,
1201
+ "learning_rate": 0.00018197293766454003,
1202
+ "loss": 1.5925,
1203
+ "step": 195
1204
+ },
1205
+ {
1206
+ "epoch": 0.61,
1207
+ "learning_rate": 0.0001817836835315759,
1208
+ "loss": 1.5604,
1209
+ "step": 196
1210
+ },
1211
+ {
1212
+ "epoch": 0.61,
1213
+ "learning_rate": 0.00018159354064737506,
1214
+ "loss": 1.6125,
1215
+ "step": 197
1216
+ },
1217
+ {
1218
+ "epoch": 0.62,
1219
+ "learning_rate": 0.0001814025110782387,
1220
+ "loss": 1.5954,
1221
+ "step": 198
1222
+ },
1223
+ {
1224
+ "epoch": 0.62,
1225
+ "learning_rate": 0.00018121059690010368,
1226
+ "loss": 1.5937,
1227
+ "step": 199
1228
+ },
1229
+ {
1230
+ "epoch": 0.62,
1231
+ "learning_rate": 0.00018101780019852008,
1232
+ "loss": 1.5582,
1233
+ "step": 200
1234
+ },
1235
+ {
1236
+ "epoch": 0.62,
1237
+ "eval_loss": 1.6065257787704468,
1238
+ "eval_runtime": 233.7919,
1239
+ "eval_samples_per_second": 16.352,
1240
+ "eval_steps_per_second": 4.089,
1241
+ "step": 200
1242
+ },
1243
+ {
1244
+ "epoch": 0.63,
1245
+ "learning_rate": 0.00018082412306862837,
1246
+ "loss": 1.5628,
1247
+ "step": 201
1248
+ },
1249
+ {
1250
+ "epoch": 0.63,
1251
+ "learning_rate": 0.00018062956761513675,
1252
+ "loss": 1.5735,
1253
+ "step": 202
1254
+ },
1255
+ {
1256
+ "epoch": 0.63,
1257
+ "learning_rate": 0.00018043413595229818,
1258
+ "loss": 1.6011,
1259
+ "step": 203
1260
+ },
1261
+ {
1262
+ "epoch": 0.63,
1263
+ "learning_rate": 0.00018023783020388763,
1264
+ "loss": 1.5434,
1265
+ "step": 204
1266
+ },
1267
+ {
1268
+ "epoch": 0.64,
1269
+ "learning_rate": 0.00018004065250317868,
1270
+ "loss": 1.5533,
1271
+ "step": 205
1272
+ },
1273
+ {
1274
+ "epoch": 0.64,
1275
+ "learning_rate": 0.00017984260499292058,
1276
+ "loss": 1.6074,
1277
+ "step": 206
1278
+ },
1279
+ {
1280
+ "epoch": 0.64,
1281
+ "learning_rate": 0.00017964368982531487,
1282
+ "loss": 1.5286,
1283
+ "step": 207
1284
+ },
1285
+ {
1286
+ "epoch": 0.65,
1287
+ "learning_rate": 0.00017944390916199203,
1288
+ "loss": 1.5161,
1289
+ "step": 208
1290
+ },
1291
+ {
1292
+ "epoch": 0.65,
1293
+ "learning_rate": 0.00017924326517398793,
1294
+ "loss": 1.6024,
1295
+ "step": 209
1296
+ },
1297
+ {
1298
+ "epoch": 0.65,
1299
+ "learning_rate": 0.00017904176004172027,
1300
+ "loss": 1.5727,
1301
+ "step": 210
1302
+ },
1303
+ {
1304
+ "epoch": 0.66,
1305
+ "learning_rate": 0.0001788393959549649,
1306
+ "loss": 1.5752,
1307
+ "step": 211
1308
+ },
1309
+ {
1310
+ "epoch": 0.66,
1311
+ "learning_rate": 0.00017863617511283203,
1312
+ "loss": 1.5845,
1313
+ "step": 212
1314
+ },
1315
+ {
1316
+ "epoch": 0.66,
1317
+ "learning_rate": 0.00017843209972374233,
1318
+ "loss": 1.6082,
1319
+ "step": 213
1320
+ },
1321
+ {
1322
+ "epoch": 0.67,
1323
+ "learning_rate": 0.00017822717200540283,
1324
+ "loss": 1.5895,
1325
+ "step": 214
1326
+ },
1327
+ {
1328
+ "epoch": 0.67,
1329
+ "learning_rate": 0.00017802139418478298,
1330
+ "loss": 1.5836,
1331
+ "step": 215
1332
+ },
1333
+ {
1334
+ "epoch": 0.67,
1335
+ "learning_rate": 0.00017781476849809038,
1336
+ "loss": 1.5996,
1337
+ "step": 216
1338
+ },
1339
+ {
1340
+ "epoch": 0.67,
1341
+ "learning_rate": 0.00017760729719074644,
1342
+ "loss": 1.6256,
1343
+ "step": 217
1344
+ },
1345
+ {
1346
+ "epoch": 0.68,
1347
+ "learning_rate": 0.000177398982517362,
1348
+ "loss": 1.628,
1349
+ "step": 218
1350
+ },
1351
+ {
1352
+ "epoch": 0.68,
1353
+ "learning_rate": 0.00017718982674171284,
1354
+ "loss": 1.5543,
1355
+ "step": 219
1356
+ },
1357
+ {
1358
+ "epoch": 0.68,
1359
+ "learning_rate": 0.00017697983213671515,
1360
+ "loss": 1.5732,
1361
+ "step": 220
1362
+ },
1363
+ {
1364
+ "epoch": 0.69,
1365
+ "learning_rate": 0.0001767690009844007,
1366
+ "loss": 1.5892,
1367
+ "step": 221
1368
+ },
1369
+ {
1370
+ "epoch": 0.69,
1371
+ "learning_rate": 0.0001765573355758921,
1372
+ "loss": 1.6524,
1373
+ "step": 222
1374
+ },
1375
+ {
1376
+ "epoch": 0.69,
1377
+ "learning_rate": 0.00017634483821137787,
1378
+ "loss": 1.5694,
1379
+ "step": 223
1380
+ },
1381
+ {
1382
+ "epoch": 0.7,
1383
+ "learning_rate": 0.0001761315112000876,
1384
+ "loss": 1.6006,
1385
+ "step": 224
1386
+ },
1387
+ {
1388
+ "epoch": 0.7,
1389
+ "learning_rate": 0.00017591735686026661,
1390
+ "loss": 1.6161,
1391
+ "step": 225
1392
+ },
1393
+ {
1394
+ "epoch": 0.7,
1395
+ "learning_rate": 0.00017570237751915092,
1396
+ "loss": 1.595,
1397
+ "step": 226
1398
+ },
1399
+ {
1400
+ "epoch": 0.71,
1401
+ "learning_rate": 0.00017548657551294192,
1402
+ "loss": 1.6072,
1403
+ "step": 227
1404
+ },
1405
+ {
1406
+ "epoch": 0.71,
1407
+ "learning_rate": 0.000175269953186781,
1408
+ "loss": 1.5855,
1409
+ "step": 228
1410
+ },
1411
+ {
1412
+ "epoch": 0.71,
1413
+ "learning_rate": 0.00017505251289472406,
1414
+ "loss": 1.597,
1415
+ "step": 229
1416
+ },
1417
+ {
1418
+ "epoch": 0.72,
1419
+ "learning_rate": 0.0001748342569997158,
1420
+ "loss": 1.5837,
1421
+ "step": 230
1422
+ },
1423
+ {
1424
+ "epoch": 0.72,
1425
+ "learning_rate": 0.00017461518787356432,
1426
+ "loss": 1.5422,
1427
+ "step": 231
1428
+ },
1429
+ {
1430
+ "epoch": 0.72,
1431
+ "learning_rate": 0.00017439530789691506,
1432
+ "loss": 1.5837,
1433
+ "step": 232
1434
+ },
1435
+ {
1436
+ "epoch": 0.72,
1437
+ "learning_rate": 0.0001741746194592251,
1438
+ "loss": 1.6038,
1439
+ "step": 233
1440
+ },
1441
+ {
1442
+ "epoch": 0.73,
1443
+ "learning_rate": 0.00017395312495873717,
1444
+ "loss": 1.5882,
1445
+ "step": 234
1446
+ },
1447
+ {
1448
+ "epoch": 0.73,
1449
+ "learning_rate": 0.00017373082680245347,
1450
+ "loss": 1.5763,
1451
+ "step": 235
1452
+ },
1453
+ {
1454
+ "epoch": 0.73,
1455
+ "learning_rate": 0.00017350772740610976,
1456
+ "loss": 1.6046,
1457
+ "step": 236
1458
+ },
1459
+ {
1460
+ "epoch": 0.74,
1461
+ "learning_rate": 0.00017328382919414877,
1462
+ "loss": 1.594,
1463
+ "step": 237
1464
+ },
1465
+ {
1466
+ "epoch": 0.74,
1467
+ "learning_rate": 0.00017305913459969414,
1468
+ "loss": 1.5903,
1469
+ "step": 238
1470
+ },
1471
+ {
1472
+ "epoch": 0.74,
1473
+ "learning_rate": 0.00017283364606452396,
1474
+ "loss": 1.5704,
1475
+ "step": 239
1476
+ },
1477
+ {
1478
+ "epoch": 0.75,
1479
+ "learning_rate": 0.0001726073660390439,
1480
+ "loss": 1.588,
1481
+ "step": 240
1482
+ },
1483
+ {
1484
+ "epoch": 0.75,
1485
+ "learning_rate": 0.00017238029698226113,
1486
+ "loss": 1.6273,
1487
+ "step": 241
1488
+ },
1489
+ {
1490
+ "epoch": 0.75,
1491
+ "learning_rate": 0.00017215244136175705,
1492
+ "loss": 1.5166,
1493
+ "step": 242
1494
+ },
1495
+ {
1496
+ "epoch": 0.76,
1497
+ "learning_rate": 0.00017192380165366092,
1498
+ "loss": 1.5813,
1499
+ "step": 243
1500
+ },
1501
+ {
1502
+ "epoch": 0.76,
1503
+ "learning_rate": 0.0001716943803426226,
1504
+ "loss": 1.5654,
1505
+ "step": 244
1506
+ },
1507
+ {
1508
+ "epoch": 0.76,
1509
+ "learning_rate": 0.0001714641799217858,
1510
+ "loss": 1.5548,
1511
+ "step": 245
1512
+ },
1513
+ {
1514
+ "epoch": 0.77,
1515
+ "learning_rate": 0.00017123320289276085,
1516
+ "loss": 1.5491,
1517
+ "step": 246
1518
+ },
1519
+ {
1520
+ "epoch": 0.77,
1521
+ "learning_rate": 0.0001710014517655976,
1522
+ "loss": 1.5903,
1523
+ "step": 247
1524
+ },
1525
+ {
1526
+ "epoch": 0.77,
1527
+ "learning_rate": 0.00017076892905875806,
1528
+ "loss": 1.5687,
1529
+ "step": 248
1530
+ },
1531
+ {
1532
+ "epoch": 0.77,
1533
+ "learning_rate": 0.00017053563729908905,
1534
+ "loss": 1.5975,
1535
+ "step": 249
1536
+ },
1537
+ {
1538
+ "epoch": 0.78,
1539
+ "learning_rate": 0.00017030157902179485,
1540
+ "loss": 1.6055,
1541
+ "step": 250
1542
+ },
1543
+ {
1544
+ "epoch": 0.78,
1545
+ "eval_loss": 1.60513174533844,
1546
+ "eval_runtime": 233.7813,
1547
+ "eval_samples_per_second": 16.353,
1548
+ "eval_steps_per_second": 4.089,
1549
+ "step": 250
1550
+ },
1551
+ {
1552
+ "epoch": 0.78,
1553
+ "learning_rate": 0.00017006675677040946,
1554
+ "loss": 1.4661,
1555
+ "step": 251
1556
+ },
1557
+ {
1558
+ "epoch": 0.78,
1559
+ "learning_rate": 0.00016983117309676908,
1560
+ "loss": 1.6071,
1561
+ "step": 252
1562
+ },
1563
+ {
1564
+ "epoch": 0.79,
1565
+ "learning_rate": 0.00016959483056098445,
1566
+ "loss": 1.5664,
1567
+ "step": 253
1568
+ },
1569
+ {
1570
+ "epoch": 0.79,
1571
+ "learning_rate": 0.0001693577317314129,
1572
+ "loss": 1.5189,
1573
+ "step": 254
1574
+ },
1575
+ {
1576
+ "epoch": 0.79,
1577
+ "learning_rate": 0.00016911987918463034,
1578
+ "loss": 1.5488,
1579
+ "step": 255
1580
+ },
1581
+ {
1582
+ "epoch": 0.8,
1583
+ "learning_rate": 0.0001688812755054036,
1584
+ "loss": 1.6153,
1585
+ "step": 256
1586
+ },
1587
+ {
1588
+ "epoch": 0.8,
1589
+ "learning_rate": 0.00016864192328666202,
1590
+ "loss": 1.536,
1591
+ "step": 257
1592
+ },
1593
+ {
1594
+ "epoch": 0.8,
1595
+ "learning_rate": 0.00016840182512946943,
1596
+ "loss": 1.624,
1597
+ "step": 258
1598
+ },
1599
+ {
1600
+ "epoch": 0.81,
1601
+ "learning_rate": 0.00016816098364299582,
1602
+ "loss": 1.569,
1603
+ "step": 259
1604
+ },
1605
+ {
1606
+ "epoch": 0.81,
1607
+ "learning_rate": 0.00016791940144448902,
1608
+ "loss": 1.588,
1609
+ "step": 260
1610
+ },
1611
+ {
1612
+ "epoch": 0.81,
1613
+ "learning_rate": 0.0001676770811592463,
1614
+ "loss": 1.5626,
1615
+ "step": 261
1616
+ },
1617
+ {
1618
+ "epoch": 0.81,
1619
+ "learning_rate": 0.00016743402542058572,
1620
+ "loss": 1.5836,
1621
+ "step": 262
1622
+ },
1623
+ {
1624
+ "epoch": 0.82,
1625
+ "learning_rate": 0.00016719023686981763,
1626
+ "loss": 1.5573,
1627
+ "step": 263
1628
+ },
1629
+ {
1630
+ "epoch": 0.82,
1631
+ "learning_rate": 0.00016694571815621586,
1632
+ "loss": 1.5815,
1633
+ "step": 264
1634
+ },
1635
+ {
1636
+ "epoch": 0.82,
1637
+ "learning_rate": 0.00016670047193698912,
1638
+ "loss": 1.64,
1639
+ "step": 265
1640
+ },
1641
+ {
1642
+ "epoch": 0.83,
1643
+ "learning_rate": 0.0001664545008772518,
1644
+ "loss": 1.6395,
1645
+ "step": 266
1646
+ },
1647
+ {
1648
+ "epoch": 0.83,
1649
+ "learning_rate": 0.00016620780764999536,
1650
+ "loss": 1.5927,
1651
+ "step": 267
1652
+ },
1653
+ {
1654
+ "epoch": 0.83,
1655
+ "learning_rate": 0.00016596039493605913,
1656
+ "loss": 1.605,
1657
+ "step": 268
1658
+ },
1659
+ {
1660
+ "epoch": 0.84,
1661
+ "learning_rate": 0.000165712265424101,
1662
+ "loss": 1.6219,
1663
+ "step": 269
1664
+ },
1665
+ {
1666
+ "epoch": 0.84,
1667
+ "learning_rate": 0.0001654634218105686,
1668
+ "loss": 1.5458,
1669
+ "step": 270
1670
+ },
1671
+ {
1672
+ "epoch": 0.84,
1673
+ "learning_rate": 0.0001652138667996696,
1674
+ "loss": 1.59,
1675
+ "step": 271
1676
+ },
1677
+ {
1678
+ "epoch": 0.85,
1679
+ "learning_rate": 0.00016496360310334253,
1680
+ "loss": 1.633,
1681
+ "step": 272
1682
+ },
1683
+ {
1684
+ "epoch": 0.85,
1685
+ "learning_rate": 0.0001647126334412274,
1686
+ "loss": 1.6108,
1687
+ "step": 273
1688
+ },
1689
+ {
1690
+ "epoch": 0.85,
1691
+ "learning_rate": 0.0001644609605406358,
1692
+ "loss": 1.5747,
1693
+ "step": 274
1694
+ },
1695
+ {
1696
+ "epoch": 0.86,
1697
+ "learning_rate": 0.0001642085871365217,
1698
+ "loss": 1.5393,
1699
+ "step": 275
1700
+ },
1701
+ {
1702
+ "epoch": 0.86,
1703
+ "learning_rate": 0.00016395551597145133,
1704
+ "loss": 1.5768,
1705
+ "step": 276
1706
+ },
1707
+ {
1708
+ "epoch": 0.86,
1709
+ "learning_rate": 0.00016370174979557368,
1710
+ "loss": 1.6278,
1711
+ "step": 277
1712
+ },
1713
+ {
1714
+ "epoch": 0.86,
1715
+ "learning_rate": 0.0001634472913665904,
1716
+ "loss": 1.5983,
1717
+ "step": 278
1718
+ },
1719
+ {
1720
+ "epoch": 0.87,
1721
+ "learning_rate": 0.00016319214344972602,
1722
+ "loss": 1.5701,
1723
+ "step": 279
1724
+ },
1725
+ {
1726
+ "epoch": 0.87,
1727
+ "learning_rate": 0.00016293630881769773,
1728
+ "loss": 1.5874,
1729
+ "step": 280
1730
+ },
1731
+ {
1732
+ "epoch": 0.87,
1733
+ "learning_rate": 0.0001626797902506853,
1734
+ "loss": 1.5412,
1735
+ "step": 281
1736
+ },
1737
+ {
1738
+ "epoch": 0.88,
1739
+ "learning_rate": 0.000162422590536301,
1740
+ "loss": 1.5733,
1741
+ "step": 282
1742
+ },
1743
+ {
1744
+ "epoch": 0.88,
1745
+ "learning_rate": 0.00016216471246955906,
1746
+ "loss": 1.6245,
1747
+ "step": 283
1748
+ },
1749
+ {
1750
+ "epoch": 0.88,
1751
+ "learning_rate": 0.00016190615885284553,
1752
+ "loss": 1.5743,
1753
+ "step": 284
1754
+ },
1755
+ {
1756
+ "epoch": 0.89,
1757
+ "learning_rate": 0.00016164693249588768,
1758
+ "loss": 1.5793,
1759
+ "step": 285
1760
+ },
1761
+ {
1762
+ "epoch": 0.89,
1763
+ "learning_rate": 0.00016138703621572346,
1764
+ "loss": 1.5672,
1765
+ "step": 286
1766
+ },
1767
+ {
1768
+ "epoch": 0.89,
1769
+ "learning_rate": 0.0001611264728366711,
1770
+ "loss": 1.5442,
1771
+ "step": 287
1772
+ },
1773
+ {
1774
+ "epoch": 0.9,
1775
+ "learning_rate": 0.0001608652451902981,
1776
+ "loss": 1.5765,
1777
+ "step": 288
1778
+ },
1779
+ {
1780
+ "epoch": 0.9,
1781
+ "learning_rate": 0.00016060335611539072,
1782
+ "loss": 1.6058,
1783
+ "step": 289
1784
+ },
1785
+ {
1786
+ "epoch": 0.9,
1787
+ "learning_rate": 0.00016034080845792295,
1788
+ "loss": 1.6156,
1789
+ "step": 290
1790
+ },
1791
+ {
1792
+ "epoch": 0.91,
1793
+ "learning_rate": 0.0001600776050710257,
1794
+ "loss": 1.6179,
1795
+ "step": 291
1796
+ },
1797
+ {
1798
+ "epoch": 0.91,
1799
+ "learning_rate": 0.0001598137488149558,
1800
+ "loss": 1.5747,
1801
+ "step": 292
1802
+ },
1803
+ {
1804
+ "epoch": 0.91,
1805
+ "learning_rate": 0.00015954924255706478,
1806
+ "loss": 1.5772,
1807
+ "step": 293
1808
+ },
1809
+ {
1810
+ "epoch": 0.91,
1811
+ "learning_rate": 0.00015928408917176786,
1812
+ "loss": 1.6064,
1813
+ "step": 294
1814
+ },
1815
+ {
1816
+ "epoch": 0.92,
1817
+ "learning_rate": 0.00015901829154051265,
1818
+ "loss": 1.6082,
1819
+ "step": 295
1820
+ },
1821
+ {
1822
+ "epoch": 0.92,
1823
+ "learning_rate": 0.00015875185255174787,
1824
+ "loss": 1.5768,
1825
+ "step": 296
1826
+ },
1827
+ {
1828
+ "epoch": 0.92,
1829
+ "learning_rate": 0.0001584847751008918,
1830
+ "loss": 1.5466,
1831
+ "step": 297
1832
+ },
1833
+ {
1834
+ "epoch": 0.93,
1835
+ "learning_rate": 0.00015821706209030118,
1836
+ "loss": 1.5127,
1837
+ "step": 298
1838
+ },
1839
+ {
1840
+ "epoch": 0.93,
1841
+ "learning_rate": 0.00015794871642923927,
1842
+ "loss": 1.5745,
1843
+ "step": 299
1844
+ },
1845
+ {
1846
+ "epoch": 0.93,
1847
+ "learning_rate": 0.00015767974103384443,
1848
+ "loss": 1.5733,
1849
+ "step": 300
1850
+ },
1851
+ {
1852
+ "epoch": 0.93,
1853
+ "eval_loss": 1.6023043394088745,
1854
+ "eval_runtime": 233.7298,
1855
+ "eval_samples_per_second": 16.356,
1856
+ "eval_steps_per_second": 4.09,
1857
+ "step": 300
1858
+ }
1859
+ ],
1860
+ "logging_steps": 1,
1861
+ "max_steps": 963,
1862
+ "num_train_epochs": 3,
1863
+ "save_steps": 50,
1864
+ "total_flos": 8.412814972551168e+17,
1865
+ "trial_name": null,
1866
+ "trial_params": null
1867
+ }
checkpoint-300/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f61cafb89242b653e455003b5517e685ecccfa6180af5fb7d0bfb35b4fc77a4
3
+ size 4475
checkpoint-350/README.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ ---
4
+ ## Training procedure
5
+
6
+
7
+ The following `bitsandbytes` quantization config was used during training:
8
+ - quant_method: bitsandbytes
9
+ - load_in_8bit: False
10
+ - load_in_4bit: True
11
+ - llm_int8_threshold: 6.0
12
+ - llm_int8_skip_modules: None
13
+ - llm_int8_enable_fp32_cpu_offload: False
14
+ - llm_int8_has_fp16_weight: False
15
+ - bnb_4bit_quant_type: nf4
16
+ - bnb_4bit_use_double_quant: True
17
+ - bnb_4bit_compute_dtype: bfloat16
18
+ ### Framework versions
19
+
20
+
21
+ - PEFT 0.6.0.dev0
checkpoint-350/adapter_config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "./mistralai_Mistral-7B-v0.1",
5
+ "bias": "none",
6
+ "fan_in_fan_out": null,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "lora_alpha": 16,
12
+ "lora_dropout": 0.05,
13
+ "modules_to_save": null,
14
+ "peft_type": "LORA",
15
+ "r": 8,
16
+ "rank_pattern": {},
17
+ "revision": null,
18
+ "target_modules": [
19
+ "o_proj",
20
+ "k_proj",
21
+ "up_proj",
22
+ "v_proj",
23
+ "q_proj",
24
+ "gate_proj",
25
+ "down_proj"
26
+ ],
27
+ "task_type": "CAUSAL_LM"
28
+ }
checkpoint-350/adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b2acf0965b728b9f39ff875af1a0bb542f43cde79c4a44aec31d97b0a6d6b82
3
+ size 84046925
checkpoint-350/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11cb718fc6f4900ddc9d4ea78c1d177248d5b82b9a7a5e9ba5cab022b06f42c3
3
+ size 168039557
checkpoint-350/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:baa9a901b79ce6d4ba427b92cd20bcdf077fd9a1e8d53d67f2028575762241a6
3
+ size 14575
checkpoint-350/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:daca21bf7aaa413d1759b6b33dab777d4a7f3e9ba04816c3ce7a0f4e706b5cdb
3
+ size 627