Hugging Face's logo

officialthinkingstack
/
Sentence-Matcher-v0.5

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:1554
loss:ContrastiveLoss
text-embeddings-inference
Model card Files
xet
 Community
Praveenkr-ts commited on
Commit
90357e1
·
verified ·
1 Parent(s): 4bbc2fb

Upload folder using huggingface_hub

Browse files
Files changed (16) hide show
  1. 1_Pooling/config.json +10 -0
  2. README.md +934 -0
  3. config.json +25 -0
  4. config_sentence_transformers.json +10 -0
  5. model.safetensors +3 -0
  6. modules.json +20 -0
  7. optimizer.pt +3 -0
  8. rng_state.pth +3 -0
  9. scheduler.pt +3 -0
  10. sentence_bert_config.json +4 -0
  11. special_tokens_map.json +37 -0
  12. tokenizer.json +0 -0
  13. tokenizer_config.json +65 -0
  14. trainer_state.json +3464 -0
  15. training_args.bin +3 -0
  16. vocab.txt +0 -0
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 384,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,934 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - generated_from_trainer
7
+ - dataset_size:1554
8
+ - loss:ContrastiveLoss
9
+ base_model: sentence-transformers/all-MiniLM-L6-v2
10
+ widget:
11
+ - source_sentence: 'Patient reports increased joint pain, swelling, and morning stiffness
12
+ associated with Rheumatoid Arthritis.
13
+
14
+ The patient describes worsening joint pain in the hands and knees, along with
15
+ increased swelling and morning stiffness lasting over 60 minutes, over the past
16
+ month.
17
+
18
+ Musculoskeletal: Increased joint pain, swelling, morning stiffness.
19
+
20
+ Rheumatoid Arthritis, managed with disease-modifying antirheumatic drugs (DMARDs).
21
+
22
+ Worsening symptoms in a patient with Rheumatoid Arthritis.'
23
+ sentences:
24
+ - Chronic Obstructive Pulmonary Disease With (Acute) Exacerbation
25
+ - Enteroviral Vesicular Stomatitis With Exanthem
26
+ - Unspecified Juvenile Rheumatoid Arthritis Of Unspecified Site
27
+ - source_sentence: 'Follow-up for well-controlled type 2 diabetes mellitus.
28
+
29
+ Type 2 Diabetes Mellitus (ICD-10 code: E11.9)
30
+
31
+ Well-controlled diabetes.
32
+
33
+ Adjust diabetes medication.
34
+
35
+ Order HbA1c levels.
36
+
37
+ Take adjusted medication as directed.
38
+
39
+ Monitor blood glucose regularly.'
40
+ sentences:
41
+ - Type 2 Diabetes Mellitus Without Complications
42
+ - Type 2 diabetes mellitus without complications
43
+ - Diverticulitis Of Intestine, Part Unspecified, Without Perforation Or Abscess
44
+ Without Bleeding
45
+ - source_sentence: 'A 55-year-old female patient with a history of hypertension presented
46
+ to the clinic for a follow-up appointment.
47
+
48
+ Her blood pressure has been poorly controlled despite taking two different antihypertensive
49
+ medications.
50
+
51
+ Past Medical History (PMH): Hypertension
52
+
53
+ Vital signs: Temperature 98.6°F, heart rate 90 beats per minute, respiratory rate
54
+ 18 breaths per minute, blood pressure 165/95 mmHg
55
+
56
+ Assessment: Uncontrolled hypertension
57
+
58
+ Plan: Review adherence to medication regimen
59
+
60
+ Plan: Adjust medication dosages or switch to different medications
61
+
62
+ Plan: Reinforce lifestyle modification recommendations
63
+
64
+ Chief Complaint: Poorly controlled hypertension'
65
+ sentences:
66
+ - Hypothyroidism, Unspecified
67
+ - Hyperlipidemia, Unspecified
68
+ - Essential (Primary) Hypertension
69
+ - source_sentence: 'Presenting with fatigue, weight gain, and cold intolerance over
70
+ the past few months.
71
+
72
+ Positive for constipation, dry skin, and brittle nails.
73
+
74
+ Thyroid enlargement (goiter).
75
+
76
+ Diminished reflexes.
77
+
78
+ Hypothyroidism
79
+
80
+ Initiate levothyroxine therapy.
81
+
82
+ Schedule follow-up for thyroid function monitoring.'
83
+ sentences:
84
+ - Abnormal Weight Loss
85
+ - Acute Upper Respiratory Infection, Unspecified
86
+ - Hypothyroidism, Unspecified
87
+ - source_sentence: 'A 45-year-old male patient presented to the emergency department
88
+ with a chief complaint of severe lower back pain.
89
+
90
+ He reports that the pain started suddenly about two hours ago while lifting a
91
+ heavy object.
92
+
93
+ The pain is located in the lower back, radiates down the right leg, and is described
94
+ as sharp and stabbing.
95
+
96
+ He also reports numbness and tingling in the right leg.
97
+
98
+ Musculoskeletal: Severe lower back pain radiating down the right leg, numbness
99
+ and tingling in the right leg
100
+
101
+ Musculoskeletal: Limited range of motion in the lumbar spine due to pain.
102
+
103
+ Muscle guarding is present in the paraspinal muscles.
104
+
105
+ Motor strength and sensation are decreased in the right leg compared to the left.
106
+
107
+ Suspected lumbar spine herniated disc
108
+
109
+ X-rays of the lumbar spine to confirm the diagnosis
110
+
111
+ MRI of the lumbar spine if X-rays are inconclusive
112
+
113
+ Neurological consultation for further evaluation and management
114
+
115
+ Pain management with medication and physical therapy'
116
+ sentences:
117
+ - Postconcussional Syndrome
118
+ - Other Intervertebral Disc Displacement, Lumbar Region
119
+ - Encounter For Immunization
120
+ pipeline_tag: sentence-similarity
121
+ library_name: sentence-transformers
122
+ ---
123
+
124
+ # SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
125
+
126
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
127
+
128
+ ## Model Details
129
+
130
+ ### Model Description
131
+ - **Model Type:** Sentence Transformer
132
+ - **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->
133
+ - **Maximum Sequence Length:** 256 tokens
134
+ - **Output Dimensionality:** 384 dimensions
135
+ - **Similarity Function:** Cosine Similarity
136
+ <!-- - **Training Dataset:** Unknown -->
137
+ <!-- - **Language:** Unknown -->
138
+ <!-- - **License:** Unknown -->
139
+
140
+ ### Model Sources
141
+
142
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
143
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
144
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
145
+
146
+ ### Full Model Architecture
147
+
148
+ ```
149
+ SentenceTransformer(
150
+ (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
151
+ (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
152
+ (2): Normalize()
153
+ )
154
+ ```
155
+
156
+ ## Usage
157
+
158
+ ### Direct Usage (Sentence Transformers)
159
+
160
+ First install the Sentence Transformers library:
161
+
162
+ ```bash
163
+ pip install -U sentence-transformers
164
+ ```
165
+
166
+ Then you can load this model and run inference.
167
+ ```python
168
+ from sentence_transformers import SentenceTransformer
169
+
170
+ # Download from the 🤗 Hub
171
+ model = SentenceTransformer("sentence_transformers_model_id")
172
+ # Run inference
173
+ sentences = [
174
+ 'A 45-year-old male patient presented to the emergency department with a chief complaint of severe lower back pain.\nHe reports that the pain started suddenly about two hours ago while lifting a heavy object.\nThe pain is located in the lower back, radiates down the right leg, and is described as sharp and stabbing.\nHe also reports numbness and tingling in the right leg.\nMusculoskeletal: Severe lower back pain radiating down the right leg, numbness and tingling in the right leg\nMusculoskeletal: Limited range of motion in the lumbar spine due to pain.\nMuscle guarding is present in the paraspinal muscles.\nMotor strength and sensation are decreased in the right leg compared to the left.\nSuspected lumbar spine herniated disc\nX-rays of the lumbar spine to confirm the diagnosis\nMRI of the lumbar spine if X-rays are inconclusive\nNeurological consultation for further evaluation and management\nPain management with medication and physical therapy',
175
+ 'Other Intervertebral Disc Displacement, Lumbar Region',
176
+ 'Postconcussional Syndrome',
177
+ ]
178
+ embeddings = model.encode(sentences)
179
+ print(embeddings.shape)
180
+ # [3, 384]
181
+
182
+ # Get the similarity scores for the embeddings
183
+ similarities = model.similarity(embeddings, embeddings)
184
+ print(similarities.shape)
185
+ # [3, 3]
186
+ ```
187
+
188
+ <!--
189
+ ### Direct Usage (Transformers)
190
+
191
+ <details><summary>Click to see the direct usage in Transformers</summary>
192
+
193
+ </details>
194
+ -->
195
+
196
+ <!--
197
+ ### Downstream Usage (Sentence Transformers)
198
+
199
+ You can finetune this model on your own dataset.
200
+
201
+ <details><summary>Click to expand</summary>
202
+
203
+ </details>
204
+ -->
205
+
206
+ <!--
207
+ ### Out-of-Scope Use
208
+
209
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
210
+ -->
211
+
212
+ <!--
213
+ ## Bias, Risks and Limitations
214
+
215
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
216
+ -->
217
+
218
+ <!--
219
+ ### Recommendations
220
+
221
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
222
+ -->
223
+
224
+ ## Training Details
225
+
226
+ ### Training Dataset
227
+
228
+ #### Unnamed Dataset
229
+
230
+ * Size: 1,554 training samples
231
+ * Columns: <code>anchor</code>, <code>positive</code>, and <code>label</code>
232
+ * Approximate statistics based on the first 1000 samples:
233
+ | | anchor | positive | label |
234
+ |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:--------------------------------------------------------------|
235
+ | type | string | string | float |
236
+ | details | <ul><li>min: 3 tokens</li><li>mean: 83.3 tokens</li><li>max: 256 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 10.12 tokens</li><li>max: 40 tokens</li></ul> | <ul><li>min: 1.0</li><li>mean: 1.0</li><li>max: 1.0</li></ul> |
237
+ * Samples:
238
+ | anchor | positive | label |
239
+ |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:-----------------|
240
+ | <code>Worsening sciatic pain despite conservative treatment<br>She reports that her sciatic pain has been worsening despite conservative treatment with rest, ice, over-the-counter pain relievers, and lumbar spine exercises.<br>She says that the pain is now radiating down her entire leg, from her lower back to her foot.<br>She also reports that she has numbness and tingling in her toes.<br>The pain is worse when she sits or stands for long periods of time.<br>Musculoskeletal: Worsening sciatic pain radiating down the entire leg<br>Neurological: Numbness and tingling in the toes of the affected leg<br>Exacerbation of sciatica</code> | <code>Sciatica, Unspecified Side</code> | <code>1.0</code> |
241
+ | <code>Chief Complaint: Malaria<br>Patient presents with fever, chills, and headache.<br>Reports a sudden onset of symptoms after returning from a recent trip to a malaria-endemic region.<br>Positive for fever and chills.<br>Recent travel to a malaria-endemic area.<br>Fever (38.5°C), chills, and generalized body aches.<br>High suspicion for malaria infection.<br>Initiate prompt diagnostic testing and antimalarial treatment.<br>Blood smear for malaria parasites.<br>Prescribe antimalarial medications based on species identification.<br>Consideration of hospitalization if severe symptoms or complications arise.<br>Follow-up in two weeks for reassessment and treatment response.<br>Blood smear for Plasmodium species identification.<br>Complete blood count (CBC) for assessment of anemia and thrombocytopenia.<br>Liver function tests to assess for malaria-related complications.</code> | <code>Unspecified Malaria</code> | <code>1.0</code> |
242
+ | <code>The patient returns for a follow-up visit after being diagnosed with psychosis.<br>He reports partial improvement in symptoms with medication but notes persistent paranoia and difficulty concentrating.<br>Partial improvement in sleep but persistent paranoia.<br>Some reduction in auditory hallucinations, persistent racing thoughts.<br>Ongoing diagnosis of psychosis and mood disorder.<br>Observable signs of residual psychosis and disorganized thoughts.<br>Partial improvement in psychosis.<br>Adjust medication dosage or consider alternative medications.<br>Continue individual therapy for ongoing management.<br>Monitor for side effects and therapeutic response.<br>Attend scheduled therapy sessions.<br>Take prescribed medications as directed.<br>Report any new or worsening symptoms promptly.</code> | <code>Psychotic Disorder With Delusions Due To Known Physiological Condition</code> | <code>1.0</code> |
243
+ * Loss: [<code>ContrastiveLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#contrastiveloss) with these parameters:
244
+ ```json
245
+ {
246
+ "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
247
+ "margin": 0.5,
248
+ "size_average": true
249
+ }
250
+ ```
251
+
252
+ ### Training Hyperparameters
253
+ #### Non-Default Hyperparameters
254
+
255
+ - `per_device_train_batch_size`: 16
256
+ - `per_device_eval_batch_size`: 16
257
+ - `learning_rate`: 2e-05
258
+ - `num_train_epochs`: 5
259
+ - `warmup_ratio`: 0.1
260
+
261
+ #### All Hyperparameters
262
+ <details><summary>Click to expand</summary>
263
+
264
+ - `overwrite_output_dir`: False
265
+ - `do_predict`: False
266
+ - `eval_strategy`: no
267
+ - `prediction_loss_only`: True
268
+ - `per_device_train_batch_size`: 16
269
+ - `per_device_eval_batch_size`: 16
270
+ - `per_gpu_train_batch_size`: None
271
+ - `per_gpu_eval_batch_size`: None
272
+ - `gradient_accumulation_steps`: 1
273
+ - `eval_accumulation_steps`: None
274
+ - `torch_empty_cache_steps`: None
275
+ - `learning_rate`: 2e-05
276
+ - `weight_decay`: 0.0
277
+ - `adam_beta1`: 0.9
278
+ - `adam_beta2`: 0.999
279
+ - `adam_epsilon`: 1e-08
280
+ - `max_grad_norm`: 1.0
281
+ - `num_train_epochs`: 5
282
+ - `max_steps`: -1
283
+ - `lr_scheduler_type`: linear
284
+ - `lr_scheduler_kwargs`: {}
285
+ - `warmup_ratio`: 0.1
286
+ - `warmup_steps`: 0
287
+ - `log_level`: passive
288
+ - `log_level_replica`: warning
289
+ - `log_on_each_node`: True
290
+ - `logging_nan_inf_filter`: True
291
+ - `save_safetensors`: True
292
+ - `save_on_each_node`: False
293
+ - `save_only_model`: False
294
+ - `restore_callback_states_from_checkpoint`: False
295
+ - `no_cuda`: False
296
+ - `use_cpu`: False
297
+ - `use_mps_device`: False
298
+ - `seed`: 42
299
+ - `data_seed`: None
300
+ - `jit_mode_eval`: False
301
+ - `use_ipex`: False
302
+ - `bf16`: False
303
+ - `fp16`: False
304
+ - `fp16_opt_level`: O1
305
+ - `half_precision_backend`: auto
306
+ - `bf16_full_eval`: False
307
+ - `fp16_full_eval`: False
308
+ - `tf32`: None
309
+ - `local_rank`: 0
310
+ - `ddp_backend`: None
311
+ - `tpu_num_cores`: None
312
+ - `tpu_metrics_debug`: False
313
+ - `debug`: []
314
+ - `dataloader_drop_last`: False
315
+ - `dataloader_num_workers`: 0
316
+ - `dataloader_prefetch_factor`: None
317
+ - `past_index`: -1
318
+ - `disable_tqdm`: False
319
+ - `remove_unused_columns`: True
320
+ - `label_names`: None
321
+ - `load_best_model_at_end`: False
322
+ - `ignore_data_skip`: False
323
+ - `fsdp`: []
324
+ - `fsdp_min_num_params`: 0
325
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
326
+ - `fsdp_transformer_layer_cls_to_wrap`: None
327
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
328
+ - `deepspeed`: None
329
+ - `label_smoothing_factor`: 0.0
330
+ - `optim`: adamw_torch
331
+ - `optim_args`: None
332
+ - `adafactor`: False
333
+ - `group_by_length`: False
334
+ - `length_column_name`: length
335
+ - `ddp_find_unused_parameters`: None
336
+ - `ddp_bucket_cap_mb`: None
337
+ - `ddp_broadcast_buffers`: False
338
+ - `dataloader_pin_memory`: True
339
+ - `dataloader_persistent_workers`: False
340
+ - `skip_memory_metrics`: True
341
+ - `use_legacy_prediction_loop`: False
342
+ - `push_to_hub`: False
343
+ - `resume_from_checkpoint`: None
344
+ - `hub_model_id`: None
345
+ - `hub_strategy`: every_save
346
+ - `hub_private_repo`: None
347
+ - `hub_always_push`: False
348
+ - `gradient_checkpointing`: False
349
+ - `gradient_checkpointing_kwargs`: None
350
+ - `include_inputs_for_metrics`: False
351
+ - `include_for_metrics`: []
352
+ - `eval_do_concat_batches`: True
353
+ - `fp16_backend`: auto
354
+ - `push_to_hub_model_id`: None
355
+ - `push_to_hub_organization`: None
356
+ - `mp_parameters`:
357
+ - `auto_find_batch_size`: False
358
+ - `full_determinism`: False
359
+ - `torchdynamo`: None
360
+ - `ray_scope`: last
361
+ - `ddp_timeout`: 1800
362
+ - `torch_compile`: False
363
+ - `torch_compile_backend`: None
364
+ - `torch_compile_mode`: None
365
+ - `include_tokens_per_second`: False
366
+ - `include_num_input_tokens_seen`: False
367
+ - `neftune_noise_alpha`: None
368
+ - `optim_target_modules`: None
369
+ - `batch_eval_metrics`: False
370
+ - `eval_on_start`: False
371
+ - `use_liger_kernel`: False
372
+ - `eval_use_gather_object`: False
373
+ - `average_tokens_across_devices`: False
374
+ - `prompts`: None
375
+ - `batch_sampler`: batch_sampler
376
+ - `multi_dataset_batch_sampler`: proportional
377
+
378
+ </details>
379
+
380
+ ### Training Logs
381
+ <details><summary>Click to expand</summary>
382
+
383
+ | Epoch | Step | Training Loss |
384
+ |:------:|:----:|:-------------:|
385
+ | 0.0102 | 1 | 0.1624 |
386
+ | 0.0204 | 2 | 0.1369 |
387
+ | 0.0306 | 3 | 0.1151 |
388
+ | 0.0408 | 4 | 0.1031 |
389
+ | 0.0510 | 5 | 0.097 |
390
+ | 0.0612 | 6 | 0.095 |
391
+ | 0.0714 | 7 | 0.1236 |
392
+ | 0.0816 | 8 | 0.1177 |
393
+ | 0.0918 | 9 | 0.0931 |
394
+ | 0.1020 | 10 | 0.1049 |
395
+ | 0.1122 | 11 | 0.0757 |
396
+ | 0.1224 | 12 | 0.0936 |
397
+ | 0.1327 | 13 | 0.0797 |
398
+ | 0.1429 | 14 | 0.0855 |
399
+ | 0.1531 | 15 | 0.079 |
400
+ | 0.1633 | 16 | 0.0666 |
401
+ | 0.1735 | 17 | 0.073 |
402
+ | 0.1837 | 18 | 0.0669 |
403
+ | 0.1939 | 19 | 0.0517 |
404
+ | 0.2041 | 20 | 0.0667 |
405
+ | 0.2143 | 21 | 0.0639 |
406
+ | 0.2245 | 22 | 0.0729 |
407
+ | 0.2347 | 23 | 0.0565 |
408
+ | 0.2449 | 24 | 0.0501 |
409
+ | 0.2551 | 25 | 0.0596 |
410
+ | 0.2653 | 26 | 0.0478 |
411
+ | 0.2755 | 27 | 0.0306 |
412
+ | 0.2857 | 28 | 0.0509 |
413
+ | 0.2959 | 29 | 0.0415 |
414
+ | 0.3061 | 30 | 0.0396 |
415
+ | 0.3163 | 31 | 0.0215 |
416
+ | 0.3265 | 32 | 0.0402 |
417
+ | 0.3367 | 33 | 0.0692 |
418
+ | 0.3469 | 34 | 0.0602 |
419
+ | 0.3571 | 35 | 0.0215 |
420
+ | 0.3673 | 36 | 0.0274 |
421
+ | 0.3776 | 37 | 0.0212 |
422
+ | 0.3878 | 38 | 0.0231 |
423
+ | 0.3980 | 39 | 0.0159 |
424
+ | 0.4082 | 40 | 0.0154 |
425
+ | 0.4184 | 41 | 0.013 |
426
+ | 0.4286 | 42 | 0.0144 |
427
+ | 0.4388 | 43 | 0.0353 |
428
+ | 0.4490 | 44 | 0.0169 |
429
+ | 0.4592 | 45 | 0.0055 |
430
+ | 0.4694 | 46 | 0.0098 |
431
+ | 0.4796 | 47 | 0.0071 |
432
+ | 0.4898 | 48 | 0.0167 |
433
+ | 0.5 | 49 | 0.0062 |
434
+ | 0.5102 | 50 | 0.0064 |
435
+ | 0.5204 | 51 | 0.0125 |
436
+ | 0.5306 | 52 | 0.0044 |
437
+ | 0.5408 | 53 | 0.0193 |
438
+ | 0.5510 | 54 | 0.0058 |
439
+ | 0.5612 | 55 | 0.0043 |
440
+ | 0.5714 | 56 | 0.0036 |
441
+ | 0.5816 | 57 | 0.0018 |
442
+ | 0.5918 | 58 | 0.0039 |
443
+ | 0.6020 | 59 | 0.0031 |
444
+ | 0.6122 | 60 | 0.0019 |
445
+ | 0.6224 | 61 | 0.003 |
446
+ | 0.6327 | 62 | 0.003 |
447
+ | 0.6429 | 63 | 0.0039 |
448
+ | 0.6531 | 64 | 0.0048 |
449
+ | 0.6633 | 65 | 0.0013 |
450
+ | 0.6735 | 66 | 0.0039 |
451
+ | 0.6837 | 67 | 0.0113 |
452
+ | 0.6939 | 68 | 0.0042 |
453
+ | 0.7041 | 69 | 0.0029 |
454
+ | 0.7143 | 70 | 0.0014 |
455
+ | 0.7245 | 71 | 0.0012 |
456
+ | 0.7347 | 72 | 0.001 |
457
+ | 0.7449 | 73 | 0.0128 |
458
+ | 0.7551 | 74 | 0.0076 |
459
+ | 0.7653 | 75 | 0.0031 |
460
+ | 0.7755 | 76 | 0.0012 |
461
+ | 0.7857 | 77 | 0.0014 |
462
+ | 0.7959 | 78 | 0.0015 |
463
+ | 0.8061 | 79 | 0.0017 |
464
+ | 0.8163 | 80 | 0.0014 |
465
+ | 0.8265 | 81 | 0.0015 |
466
+ | 0.8367 | 82 | 0.0013 |
467
+ | 0.8469 | 83 | 0.001 |
468
+ | 0.8571 | 84 | 0.0021 |
469
+ | 0.8673 | 85 | 0.0008 |
470
+ | 0.8776 | 86 | 0.0009 |
471
+ | 0.8878 | 87 | 0.0117 |
472
+ | 0.8980 | 88 | 0.003 |
473
+ | 0.9082 | 89 | 0.0008 |
474
+ | 0.9184 | 90 | 0.0068 |
475
+ | 0.9286 | 91 | 0.0014 |
476
+ | 0.9388 | 92 | 0.0014 |
477
+ | 0.9490 | 93 | 0.0007 |
478
+ | 0.9592 | 94 | 0.0011 |
479
+ | 0.9694 | 95 | 0.0009 |
480
+ | 0.9796 | 96 | 0.0008 |
481
+ | 0.9898 | 97 | 0.0011 |
482
+ | 1.0 | 98 | 0.0011 |
483
+ | 1.0102 | 99 | 0.0005 |
484
+ | 1.0204 | 100 | 0.0005 |
485
+ | 1.0306 | 101 | 0.0012 |
486
+ | 1.0408 | 102 | 0.0008 |
487
+ | 1.0510 | 103 | 0.0016 |
488
+ | 1.0612 | 104 | 0.0005 |
489
+ | 1.0714 | 105 | 0.0015 |
490
+ | 1.0816 | 106 | 0.0005 |
491
+ | 1.0918 | 107 | 0.0018 |
492
+ | 1.1020 | 108 | 0.0006 |
493
+ | 1.1122 | 109 | 0.0006 |
494
+ | 1.1224 | 110 | 0.0043 |
495
+ | 1.1327 | 111 | 0.0007 |
496
+ | 1.1429 | 112 | 0.0009 |
497
+ | 1.1531 | 113 | 0.0007 |
498
+ | 1.1633 | 114 | 0.0019 |
499
+ | 1.1735 | 115 | 0.0032 |
500
+ | 1.1837 | 116 | 0.0004 |
501
+ | 1.1939 | 117 | 0.0005 |
502
+ | 1.2041 | 118 | 0.0005 |
503
+ | 1.2143 | 119 | 0.0009 |
504
+ | 1.2245 | 120 | 0.0018 |
505
+ | 1.2347 | 121 | 0.0006 |
506
+ | 1.2449 | 122 | 0.0004 |
507
+ | 1.2551 | 123 | 0.0004 |
508
+ | 1.2653 | 124 | 0.0008 |
509
+ | 1.2755 | 125 | 0.0007 |
510
+ | 1.2857 | 126 | 0.0006 |
511
+ | 1.2959 | 127 | 0.0004 |
512
+ | 1.3061 | 128 | 0.0032 |
513
+ | 1.3163 | 129 | 0.0011 |
514
+ | 1.3265 | 130 | 0.0008 |
515
+ | 1.3367 | 131 | 0.0006 |
516
+ | 1.3469 | 132 | 0.0004 |
517
+ | 1.3571 | 133 | 0.0005 |
518
+ | 1.3673 | 134 | 0.0003 |
519
+ | 1.3776 | 135 | 0.0006 |
520
+ | 1.3878 | 136 | 0.0009 |
521
+ | 1.3980 | 137 | 0.0003 |
522
+ | 1.4082 | 138 | 0.0003 |
523
+ | 1.4184 | 139 | 0.0005 |
524
+ | 1.4286 | 140 | 0.0005 |
525
+ | 1.4388 | 141 | 0.0005 |
526
+ | 1.4490 | 142 | 0.0006 |
527
+ | 1.4592 | 143 | 0.0022 |
528
+ | 1.4694 | 144 | 0.0004 |
529
+ | 1.4796 | 145 | 0.0012 |
530
+ | 1.4898 | 146 | 0.0006 |
531
+ | 1.5 | 147 | 0.0003 |
532
+ | 1.5102 | 148 | 0.0008 |
533
+ | 1.5204 | 149 | 0.0004 |
534
+ | 1.5306 | 150 | 0.0004 |
535
+ | 1.5408 | 151 | 0.0004 |
536
+ | 1.5510 | 152 | 0.0004 |
537
+ | 1.5612 | 153 | 0.0007 |
538
+ | 1.5714 | 154 | 0.0022 |
539
+ | 1.5816 | 155 | 0.0005 |
540
+ | 1.5918 | 156 | 0.0003 |
541
+ | 1.6020 | 157 | 0.0005 |
542
+ | 1.6122 | 158 | 0.0003 |
543
+ | 1.6224 | 159 | 0.0004 |
544
+ | 1.6327 | 160 | 0.0004 |
545
+ | 1.6429 | 161 | 0.0002 |
546
+ | 1.6531 | 162 | 0.0005 |
547
+ | 1.6633 | 163 | 0.0005 |
548
+ | 1.6735 | 164 | 0.0003 |
549
+ | 1.6837 | 165 | 0.0005 |
550
+ | 1.6939 | 166 | 0.0005 |
551
+ | 1.7041 | 167 | 0.0004 |
552
+ | 1.7143 | 168 | 0.0003 |
553
+ | 1.7245 | 169 | 0.0003 |
554
+ | 1.7347 | 170 | 0.0003 |
555
+ | 1.7449 | 171 | 0.0005 |
556
+ | 1.7551 | 172 | 0.0005 |
557
+ | 1.7653 | 173 | 0.0002 |
558
+ | 1.7755 | 174 | 0.0005 |
559
+ | 1.7857 | 175 | 0.0003 |
560
+ | 1.7959 | 176 | 0.0006 |
561
+ | 1.8061 | 177 | 0.0003 |
562
+ | 1.8163 | 178 | 0.0004 |
563
+ | 1.8265 | 179 | 0.0004 |
564
+ | 1.8367 | 180 | 0.0002 |
565
+ | 1.8469 | 181 | 0.0002 |
566
+ | 1.8571 | 182 | 0.0005 |
567
+ | 1.8673 | 183 | 0.0003 |
568
+ | 1.8776 | 184 | 0.0003 |
569
+ | 1.8878 | 185 | 0.0002 |
570
+ | 1.8980 | 186 | 0.0003 |
571
+ | 1.9082 | 187 | 0.0032 |
572
+ | 1.9184 | 188 | 0.0006 |
573
+ | 1.9286 | 189 | 0.0003 |
574
+ | 1.9388 | 190 | 0.0005 |
575
+ | 1.9490 | 191 | 0.0003 |
576
+ | 1.9592 | 192 | 0.0004 |
577
+ | 1.9694 | 193 | 0.0004 |
578
+ | 1.9796 | 194 | 0.0004 |
579
+ | 1.9898 | 195 | 0.0003 |
580
+ | 2.0 | 196 | 0.0001 |
581
+ | 2.0102 | 197 | 0.0003 |
582
+ | 2.0204 | 198 | 0.0003 |
583
+ | 2.0306 | 199 | 0.0002 |
584
+ | 2.0408 | 200 | 0.0002 |
585
+ | 2.0510 | 201 | 0.0003 |
586
+ | 2.0612 | 202 | 0.0002 |
587
+ | 2.0714 | 203 | 0.0002 |
588
+ | 2.0816 | 204 | 0.0003 |
589
+ | 2.0918 | 205 | 0.0003 |
590
+ | 2.1020 | 206 | 0.0008 |
591
+ | 2.1122 | 207 | 0.0004 |
592
+ | 2.1224 | 208 | 0.0004 |
593
+ | 2.1327 | 209 | 0.0004 |
594
+ | 2.1429 | 210 | 0.0003 |
595
+ | 2.1531 | 211 | 0.0004 |
596
+ | 2.1633 | 212 | 0.0002 |
597
+ | 2.1735 | 213 | 0.0002 |
598
+ | 2.1837 | 214 | 0.0002 |
599
+ | 2.1939 | 215 | 0.0002 |
600
+ | 2.2041 | 216 | 0.0002 |
601
+ | 2.2143 | 217 | 0.0003 |
602
+ | 2.2245 | 218 | 0.0004 |
603
+ | 2.2347 | 219 | 0.0003 |
604
+ | 2.2449 | 220 | 0.0002 |
605
+ | 2.2551 | 221 | 0.0002 |
606
+ | 2.2653 | 222 | 0.0003 |
607
+ | 2.2755 | 223 | 0.0002 |
608
+ | 2.2857 | 224 | 0.0003 |
609
+ | 2.2959 | 225 | 0.0002 |
610
+ | 2.3061 | 226 | 0.0003 |
611
+ | 2.3163 | 227 | 0.0002 |
612
+ | 2.3265 | 228 | 0.0004 |
613
+ | 2.3367 | 229 | 0.0002 |
614
+ | 2.3469 | 230 | 0.0002 |
615
+ | 2.3571 | 231 | 0.001 |
616
+ | 2.3673 | 232 | 0.0002 |
617
+ | 2.3776 | 233 | 0.0006 |
618
+ | 2.3878 | 234 | 0.0003 |
619
+ | 2.3980 | 235 | 0.0003 |
620
+ | 2.4082 | 236 | 0.0005 |
621
+ | 2.4184 | 237 | 0.0004 |
622
+ | 2.4286 | 238 | 0.0011 |
623
+ | 2.4388 | 239 | 0.0009 |
624
+ | 2.4490 | 240 | 0.0004 |
625
+ | 2.4592 | 241 | 0.0003 |
626
+ | 2.4694 | 242 | 0.0003 |
627
+ | 2.4796 | 243 | 0.0002 |
628
+ | 2.4898 | 244 | 0.0004 |
629
+ | 2.5 | 245 | 0.0002 |
630
+ | 2.5102 | 246 | 0.0002 |
631
+ | 2.5204 | 247 | 0.0004 |
632
+ | 2.5306 | 248 | 0.0003 |
633
+ | 2.5408 | 249 | 0.0002 |
634
+ | 2.5510 | 250 | 0.0006 |
635
+ | 2.5612 | 251 | 0.0002 |
636
+ | 2.5714 | 252 | 0.0002 |
637
+ | 2.5816 | 253 | 0.0002 |
638
+ | 2.5918 | 254 | 0.0002 |
639
+ | 2.6020 | 255 | 0.0013 |
640
+ | 2.6122 | 256 | 0.0002 |
641
+ | 2.6224 | 257 | 0.0012 |
642
+ | 2.6327 | 258 | 0.0003 |
643
+ | 2.6429 | 259 | 0.0002 |
644
+ | 2.6531 | 260 | 0.0003 |
645
+ | 2.6633 | 261 | 0.0002 |
646
+ | 2.6735 | 262 | 0.0011 |
647
+ | 2.6837 | 263 | 0.0003 |
648
+ | 2.6939 | 264 | 0.0003 |
649
+ | 2.7041 | 265 | 0.0004 |
650
+ | 2.7143 | 266 | 0.0003 |
651
+ | 2.7245 | 267 | 0.0001 |
652
+ | 2.7347 | 268 | 0.0002 |
653
+ | 2.7449 | 269 | 0.0002 |
654
+ | 2.7551 | 270 | 0.0003 |
655
+ | 2.7653 | 271 | 0.0002 |
656
+ | 2.7755 | 272 | 0.0002 |
657
+ | 2.7857 | 273 | 0.0002 |
658
+ | 2.7959 | 274 | 0.0004 |
659
+ | 2.8061 | 275 | 0.0002 |
660
+ | 2.8163 | 276 | 0.0003 |
661
+ | 2.8265 | 277 | 0.0002 |
662
+ | 2.8367 | 278 | 0.0002 |
663
+ | 2.8469 | 279 | 0.0004 |
664
+ | 2.8571 | 280 | 0.0004 |
665
+ | 2.8673 | 281 | 0.0004 |
666
+ | 2.8776 | 282 | 0.0002 |
667
+ | 2.8878 | 283 | 0.0002 |
668
+ | 2.8980 | 284 | 0.0004 |
669
+ | 2.9082 | 285 | 0.0002 |
670
+ | 2.9184 | 286 | 0.0002 |
671
+ | 2.9286 | 287 | 0.0004 |
672
+ | 2.9388 | 288 | 0.0003 |
673
+ | 2.9490 | 289 | 0.0002 |
674
+ | 2.9592 | 290 | 0.0006 |
675
+ | 2.9694 | 291 | 0.0002 |
676
+ | 2.9796 | 292 | 0.0003 |
677
+ | 2.9898 | 293 | 0.0003 |
678
+ | 3.0 | 294 | 0.0002 |
679
+ | 3.0102 | 295 | 0.0002 |
680
+ | 3.0204 | 296 | 0.0001 |
681
+ | 3.0306 | 297 | 0.0002 |
682
+ | 3.0408 | 298 | 0.0005 |
683
+ | 3.0510 | 299 | 0.0004 |
684
+ | 3.0612 | 300 | 0.0005 |
685
+ | 3.0714 | 301 | 0.0002 |
686
+ | 3.0816 | 302 | 0.0002 |
687
+ | 3.0918 | 303 | 0.0002 |
688
+ | 3.1020 | 304 | 0.0004 |
689
+ | 3.1122 | 305 | 0.0002 |
690
+ | 3.1224 | 306 | 0.0002 |
691
+ | 3.1327 | 307 | 0.0002 |
692
+ | 3.1429 | 308 | 0.0002 |
693
+ | 3.1531 | 309 | 0.0003 |
694
+ | 3.1633 | 310 | 0.0003 |
695
+ | 3.1735 | 311 | 0.0002 |
696
+ | 3.1837 | 312 | 0.0004 |
697
+ | 3.1939 | 313 | 0.0002 |
698
+ | 3.2041 | 314 | 0.0001 |
699
+ | 3.2143 | 315 | 0.0002 |
700
+ | 3.2245 | 316 | 0.0004 |
701
+ | 3.2347 | 317 | 0.0004 |
702
+ | 3.2449 | 318 | 0.0003 |
703
+ | 3.2551 | 319 | 0.0002 |
704
+ | 3.2653 | 320 | 0.0002 |
705
+ | 3.2755 | 321 | 0.0002 |
706
+ | 3.2857 | 322 | 0.0003 |
707
+ | 3.2959 | 323 | 0.0003 |
708
+ | 3.3061 | 324 | 0.0003 |
709
+ | 3.3163 | 325 | 0.0002 |
710
+ | 3.3265 | 326 | 0.0002 |
711
+ | 3.3367 | 327 | 0.0001 |
712
+ | 3.3469 | 328 | 0.0002 |
713
+ | 3.3571 | 329 | 0.0004 |
714
+ | 3.3673 | 330 | 0.0002 |
715
+ | 3.3776 | 331 | 0.0002 |
716
+ | 3.3878 | 332 | 0.0002 |
717
+ | 3.3980 | 333 | 0.0001 |
718
+ | 3.4082 | 334 | 0.0002 |
719
+ | 3.4184 | 335 | 0.0002 |
720
+ | 3.4286 | 336 | 0.0001 |
721
+ | 3.4388 | 337 | 0.0005 |
722
+ | 3.4490 | 338 | 0.0001 |
723
+ | 3.4592 | 339 | 0.0003 |
724
+ | 3.4694 | 340 | 0.0003 |
725
+ | 3.4796 | 341 | 0.0002 |
726
+ | 3.4898 | 342 | 0.0002 |
727
+ | 3.5 | 343 | 0.0001 |
728
+ | 3.5102 | 344 | 0.0002 |
729
+ | 3.5204 | 345 | 0.0008 |
730
+ | 3.5306 | 346 | 0.0002 |
731
+ | 3.5408 | 347 | 0.0003 |
732
+ | 3.5510 | 348 | 0.0003 |
733
+ | 3.5612 | 349 | 0.0003 |
734
+ | 3.5714 | 350 | 0.0002 |
735
+ | 3.5816 | 351 | 0.0002 |
736
+ | 3.5918 | 352 | 0.0002 |
737
+ | 3.6020 | 353 | 0.0001 |
738
+ | 3.6122 | 354 | 0.0002 |
739
+ | 3.6224 | 355 | 0.0001 |
740
+ | 3.6327 | 356 | 0.0002 |
741
+ | 3.6429 | 357 | 0.0001 |
742
+ | 3.6531 | 358 | 0.0001 |
743
+ | 3.6633 | 359 | 0.0003 |
744
+ | 3.6735 | 360 | 0.0003 |
745
+ | 3.6837 | 361 | 0.0002 |
746
+ | 3.6939 | 362 | 0.0002 |
747
+ | 3.7041 | 363 | 0.0001 |
748
+ | 3.7143 | 364 | 0.0003 |
749
+ | 3.7245 | 365 | 0.0003 |
750
+ | 3.7347 | 366 | 0.0002 |
751
+ | 3.7449 | 367 | 0.0006 |
752
+ | 3.7551 | 368 | 0.0003 |
753
+ | 3.7653 | 369 | 0.0002 |
754
+ | 3.7755 | 370 | 0.0002 |
755
+ | 3.7857 | 371 | 0.0001 |
756
+ | 3.7959 | 372 | 0.0002 |
757
+ | 3.8061 | 373 | 0.0002 |
758
+ | 3.8163 | 374 | 0.0003 |
759
+ | 3.8265 | 375 | 0.0001 |
760
+ | 3.8367 | 376 | 0.0002 |
761
+ | 3.8469 | 377 | 0.0004 |
762
+ | 3.8571 | 378 | 0.0002 |
763
+ | 3.8673 | 379 | 0.0003 |
764
+ | 3.8776 | 380 | 0.0001 |
765
+ | 3.8878 | 381 | 0.0003 |
766
+ | 3.8980 | 382 | 0.0001 |
767
+ | 3.9082 | 383 | 0.0002 |
768
+ | 3.9184 | 384 | 0.0002 |
769
+ | 3.9286 | 385 | 0.0002 |
770
+ | 3.9388 | 386 | 0.0003 |
771
+ | 3.9490 | 387 | 0.0002 |
772
+ | 3.9592 | 388 | 0.0002 |
773
+ | 3.9694 | 389 | 0.0001 |
774
+ | 3.9796 | 390 | 0.0002 |
775
+ | 3.9898 | 391 | 0.0001 |
776
+ | 4.0 | 392 | 0.0001 |
777
+ | 4.0102 | 393 | 0.0001 |
778
+ | 4.0204 | 394 | 0.0002 |
779
+ | 4.0306 | 395 | 0.0001 |
780
+ | 4.0408 | 396 | 0.0007 |
781
+ | 4.0510 | 397 | 0.0002 |
782
+ | 4.0612 | 398 | 0.0002 |
783
+ | 4.0714 | 399 | 0.0001 |
784
+ | 4.0816 | 400 | 0.0001 |
785
+ | 4.0918 | 401 | 0.0002 |
786
+ | 4.1020 | 402 | 0.0002 |
787
+ | 4.1122 | 403 | 0.0001 |
788
+ | 4.1224 | 404 | 0.0001 |
789
+ | 4.1327 | 405 | 0.0002 |
790
+ | 4.1429 | 406 | 0.0004 |
791
+ | 4.1531 | 407 | 0.0004 |
792
+ | 4.1633 | 408 | 0.0006 |
793
+ | 4.1735 | 409 | 0.0001 |
794
+ | 4.1837 | 410 | 0.0002 |
795
+ | 4.1939 | 411 | 0.0002 |
796
+ | 4.2041 | 412 | 0.0001 |
797
+ | 4.2143 | 413 | 0.0001 |
798
+ | 4.2245 | 414 | 0.0001 |
799
+ | 4.2347 | 415 | 0.0001 |
800
+ | 4.2449 | 416 | 0.0003 |
801
+ | 4.2551 | 417 | 0.0001 |
802
+ | 4.2653 | 418 | 0.0002 |
803
+ | 4.2755 | 419 | 0.0001 |
804
+ | 4.2857 | 420 | 0.0002 |
805
+ | 4.2959 | 421 | 0.0003 |
806
+ | 4.3061 | 422 | 0.0004 |
807
+ | 4.3163 | 423 | 0.0002 |
808
+ | 4.3265 | 424 | 0.0003 |
809
+ | 4.3367 | 425 | 0.0001 |
810
+ | 4.3469 | 426 | 0.0001 |
811
+ | 4.3571 | 427 | 0.0002 |
812
+ | 4.3673 | 428 | 0.0002 |
813
+ | 4.3776 | 429 | 0.0002 |
814
+ | 4.3878 | 430 | 0.0002 |
815
+ | 4.3980 | 431 | 0.0002 |
816
+ | 4.4082 | 432 | 0.0001 |
817
+ | 4.4184 | 433 | 0.0003 |
818
+ | 4.4286 | 434 | 0.0002 |
819
+ | 4.4388 | 435 | 0.0003 |
820
+ | 4.4490 | 436 | 0.0003 |
821
+ | 4.4592 | 437 | 0.0003 |
822
+ | 4.4694 | 438 | 0.0001 |
823
+ | 4.4796 | 439 | 0.0002 |
824
+ | 4.4898 | 440 | 0.0002 |
825
+ | 4.5 | 441 | 0.0002 |
826
+ | 4.5102 | 442 | 0.0003 |
827
+ | 4.5204 | 443 | 0.0003 |
828
+ | 4.5306 | 444 | 0.0002 |
829
+ | 4.5408 | 445 | 0.0002 |
830
+ | 4.5510 | 446 | 0.0001 |
831
+ | 4.5612 | 447 | 0.0002 |
832
+ | 4.5714 | 448 | 0.0002 |
833
+ | 4.5816 | 449 | 0.0001 |
834
+ | 4.5918 | 450 | 0.0002 |
835
+ | 4.6020 | 451 | 0.0002 |
836
+ | 4.6122 | 452 | 0.0001 |
837
+ | 4.6224 | 453 | 0.0003 |
838
+ | 4.6327 | 454 | 0.0002 |
839
+ | 4.6429 | 455 | 0.0001 |
840
+ | 4.6531 | 456 | 0.0001 |
841
+ | 4.6633 | 457 | 0.0001 |
842
+ | 4.6735 | 458 | 0.0001 |
843
+ | 4.6837 | 459 | 0.0002 |
844
+ | 4.6939 | 460 | 0.0001 |
845
+ | 4.7041 | 461 | 0.0002 |
846
+ | 4.7143 | 462 | 0.0001 |
847
+ | 4.7245 | 463 | 0.0001 |
848
+ | 4.7347 | 464 | 0.0002 |
849
+ | 4.7449 | 465 | 0.0002 |
850
+ | 4.7551 | 466 | 0.0001 |
851
+ | 4.7653 | 467 | 0.0002 |
852
+ | 4.7755 | 468 | 0.0002 |
853
+ | 4.7857 | 469 | 0.0002 |
854
+ | 4.7959 | 470 | 0.0002 |
855
+ | 4.8061 | 471 | 0.0007 |
856
+ | 4.8163 | 472 | 0.0002 |
857
+ | 4.8265 | 473 | 0.0006 |
858
+ | 4.8367 | 474 | 0.0002 |
859
+ | 4.8469 | 475 | 0.0001 |
860
+ | 4.8571 | 476 | 0.0002 |
861
+ | 4.8673 | 477 | 0.0001 |
862
+ | 4.8776 | 478 | 0.0002 |
863
+ | 4.8878 | 479 | 0.0002 |
864
+ | 4.8980 | 480 | 0.0003 |
865
+ | 4.9082 | 481 | 0.0002 |
866
+ | 4.9184 | 482 | 0.0001 |
867
+ | 4.9286 | 483 | 0.0002 |
868
+ | 4.9388 | 484 | 0.0002 |
869
+ | 4.9490 | 485 | 0.0002 |
870
+ | 4.9592 | 486 | 0.0002 |
871
+ | 4.9694 | 487 | 0.0002 |
872
+ | 4.9796 | 488 | 0.0002 |
873
+ | 4.9898 | 489 | 0.0004 |
874
+ | 5.0 | 490 | 0.0002 |
875
+
876
+ </details>
877
+
878
+ ### Framework Versions
879
+ - Python: 3.11.13
880
+ - Sentence Transformers: 4.1.0
881
+ - Transformers: 4.52.3
882
+ - PyTorch: 2.6.0+cu124
883
+ - Accelerate: 1.7.0
884
+ - Datasets: 2.14.4
885
+ - Tokenizers: 0.21.1
886
+
887
+ ## Citation
888
+
889
+ ### BibTeX
890
+
891
+ #### Sentence Transformers
892
+ ```bibtex
893
+ @inproceedings{reimers-2019-sentence-bert,
894
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
895
+ author = "Reimers, Nils and Gurevych, Iryna",
896
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
897
+ month = "11",
898
+ year = "2019",
899
+ publisher = "Association for Computational Linguistics",
900
+ url = "https://arxiv.org/abs/1908.10084",
901
+ }
902
+ ```
903
+
904
+ #### ContrastiveLoss
905
+ ```bibtex
906
+ @inproceedings{hadsell2006dimensionality,
907
+ author={Hadsell, R. and Chopra, S. and LeCun, Y.},
908
+ booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
909
+ title={Dimensionality Reduction by Learning an Invariant Mapping},
910
+ year={2006},
911
+ volume={2},
912
+ number={},
913
+ pages={1735-1742},
914
+ doi={10.1109/CVPR.2006.100}
915
+ }
916
+ ```
917
+
918
+ <!--
919
+ ## Glossary
920
+
921
+ *Clearly define terms in order to be accessible across audiences.*
922
+ -->
923
+
924
+ <!--
925
+ ## Model Card Authors
926
+
927
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
928
+ -->
929
+
930
+ <!--
931
+ ## Model Card Contact
932
+
933
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
934
+ -->
config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "BertModel"
4
+ ],
5
+ "attention_probs_dropout_prob": 0.1,
6
+ "classifier_dropout": null,
7
+ "gradient_checkpointing": false,
8
+ "hidden_act": "gelu",
9
+ "hidden_dropout_prob": 0.1,
10
+ "hidden_size": 384,
11
+ "initializer_range": 0.02,
12
+ "intermediate_size": 1536,
13
+ "layer_norm_eps": 1e-12,
14
+ "max_position_embeddings": 512,
15
+ "model_type": "bert",
16
+ "num_attention_heads": 12,
17
+ "num_hidden_layers": 6,
18
+ "pad_token_id": 0,
19
+ "position_embedding_type": "absolute",
20
+ "torch_dtype": "float32",
21
+ "transformers_version": "4.52.3",
22
+ "type_vocab_size": 2,
23
+ "use_cache": true,
24
+ "vocab_size": 30522
25
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "4.1.0",
4
+ "transformers": "4.52.3",
5
+ "pytorch": "2.6.0+cu124"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": "cosine"
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b93f892eeea63dc29d6444fccb7b49dcb24b3ef812cca5de2feaf29d4013a6ff
3
+ size 90864192
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:26c9567e7b109b4075c2347b7dc95fff319a91d2f65fa92640d126d7d0b94825
3
+ size 180607738
rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dec8a60243016005f50a7bd1a167a7153fa63d679bfcca2028470ff1969256e2
3
+ size 14244
scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27a1f4a8ff132f06024125d5dca678171d935f9e2017641c33f10c350a9027d2
3
+ size 1064
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_basic_tokenize": true,
47
+ "do_lower_case": true,
48
+ "extra_special_tokens": {},
49
+ "mask_token": "[MASK]",
50
+ "max_length": 128,
51
+ "model_max_length": 256,
52
+ "never_split": null,
53
+ "pad_to_multiple_of": null,
54
+ "pad_token": "[PAD]",
55
+ "pad_token_type_id": 0,
56
+ "padding_side": "right",
57
+ "sep_token": "[SEP]",
58
+ "stride": 0,
59
+ "strip_accents": null,
60
+ "tokenize_chinese_chars": true,
61
+ "tokenizer_class": "BertTokenizer",
62
+ "truncation_side": "right",
63
+ "truncation_strategy": "longest_first",
64
+ "unk_token": "[UNK]"
65
+ }
trainer_state.json ADDED
@@ -0,0 +1,3464 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 5.0,
6
+ "eval_steps": 500,
7
+ "global_step": 490,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.01020408163265306,
14
+ "grad_norm": 1.5778175592422485,
15
+ "learning_rate": 0.0,
16
+ "loss": 0.1624,
17
+ "step": 1
18
+ },
19
+ {
20
+ "epoch": 0.02040816326530612,
21
+ "grad_norm": 1.3363069295883179,
22
+ "learning_rate": 4.0816326530612243e-07,
23
+ "loss": 0.1369,
24
+ "step": 2
25
+ },
26
+ {
27
+ "epoch": 0.030612244897959183,
28
+ "grad_norm": 1.248914361000061,
29
+ "learning_rate": 8.163265306122449e-07,
30
+ "loss": 0.1151,
31
+ "step": 3
32
+ },
33
+ {
34
+ "epoch": 0.04081632653061224,
35
+ "grad_norm": 1.4152894020080566,
36
+ "learning_rate": 1.2244897959183673e-06,
37
+ "loss": 0.1031,
38
+ "step": 4
39
+ },
40
+ {
41
+ "epoch": 0.05102040816326531,
42
+ "grad_norm": 1.2651602029800415,
43
+ "learning_rate": 1.6326530612244897e-06,
44
+ "loss": 0.097,
45
+ "step": 5
46
+ },
47
+ {
48
+ "epoch": 0.061224489795918366,
49
+ "grad_norm": 1.289127230644226,
50
+ "learning_rate": 2.0408163265306125e-06,
51
+ "loss": 0.095,
52
+ "step": 6
53
+ },
54
+ {
55
+ "epoch": 0.07142857142857142,
56
+ "grad_norm": 1.1998013257980347,
57
+ "learning_rate": 2.4489795918367347e-06,
58
+ "loss": 0.1236,
59
+ "step": 7
60
+ },
61
+ {
62
+ "epoch": 0.08163265306122448,
63
+ "grad_norm": 1.538970947265625,
64
+ "learning_rate": 2.8571428571428573e-06,
65
+ "loss": 0.1177,
66
+ "step": 8
67
+ },
68
+ {
69
+ "epoch": 0.09183673469387756,
70
+ "grad_norm": 1.2269916534423828,
71
+ "learning_rate": 3.2653061224489794e-06,
72
+ "loss": 0.0931,
73
+ "step": 9
74
+ },
75
+ {
76
+ "epoch": 0.10204081632653061,
77
+ "grad_norm": 1.2894030809402466,
78
+ "learning_rate": 3.6734693877551024e-06,
79
+ "loss": 0.1049,
80
+ "step": 10
81
+ },
82
+ {
83
+ "epoch": 0.11224489795918367,
84
+ "grad_norm": 0.9413341283798218,
85
+ "learning_rate": 4.081632653061225e-06,
86
+ "loss": 0.0757,
87
+ "step": 11
88
+ },
89
+ {
90
+ "epoch": 0.12244897959183673,
91
+ "grad_norm": 1.1163856983184814,
92
+ "learning_rate": 4.489795918367348e-06,
93
+ "loss": 0.0936,
94
+ "step": 12
95
+ },
96
+ {
97
+ "epoch": 0.1326530612244898,
98
+ "grad_norm": 0.997565507888794,
99
+ "learning_rate": 4.897959183673469e-06,
100
+ "loss": 0.0797,
101
+ "step": 13
102
+ },
103
+ {
104
+ "epoch": 0.14285714285714285,
105
+ "grad_norm": 1.0046685934066772,
106
+ "learning_rate": 5.306122448979593e-06,
107
+ "loss": 0.0855,
108
+ "step": 14
109
+ },
110
+ {
111
+ "epoch": 0.15306122448979592,
112
+ "grad_norm": 0.9205936789512634,
113
+ "learning_rate": 5.7142857142857145e-06,
114
+ "loss": 0.079,
115
+ "step": 15
116
+ },
117
+ {
118
+ "epoch": 0.16326530612244897,
119
+ "grad_norm": 0.8712719678878784,
120
+ "learning_rate": 6.122448979591837e-06,
121
+ "loss": 0.0666,
122
+ "step": 16
123
+ },
124
+ {
125
+ "epoch": 0.17346938775510204,
126
+ "grad_norm": 0.9426755905151367,
127
+ "learning_rate": 6.530612244897959e-06,
128
+ "loss": 0.073,
129
+ "step": 17
130
+ },
131
+ {
132
+ "epoch": 0.1836734693877551,
133
+ "grad_norm": 0.8015092611312866,
134
+ "learning_rate": 6.938775510204082e-06,
135
+ "loss": 0.0669,
136
+ "step": 18
137
+ },
138
+ {
139
+ "epoch": 0.19387755102040816,
140
+ "grad_norm": 0.7584081292152405,
141
+ "learning_rate": 7.346938775510205e-06,
142
+ "loss": 0.0517,
143
+ "step": 19
144
+ },
145
+ {
146
+ "epoch": 0.20408163265306123,
147
+ "grad_norm": 0.7984261512756348,
148
+ "learning_rate": 7.755102040816327e-06,
149
+ "loss": 0.0667,
150
+ "step": 20
151
+ },
152
+ {
153
+ "epoch": 0.21428571428571427,
154
+ "grad_norm": 0.7533179521560669,
155
+ "learning_rate": 8.16326530612245e-06,
156
+ "loss": 0.0639,
157
+ "step": 21
158
+ },
159
+ {
160
+ "epoch": 0.22448979591836735,
161
+ "grad_norm": 0.9156713485717773,
162
+ "learning_rate": 8.571428571428571e-06,
163
+ "loss": 0.0729,
164
+ "step": 22
165
+ },
166
+ {
167
+ "epoch": 0.23469387755102042,
168
+ "grad_norm": 0.6727928519248962,
169
+ "learning_rate": 8.979591836734695e-06,
170
+ "loss": 0.0565,
171
+ "step": 23
172
+ },
173
+ {
174
+ "epoch": 0.24489795918367346,
175
+ "grad_norm": 0.5908196568489075,
176
+ "learning_rate": 9.387755102040818e-06,
177
+ "loss": 0.0501,
178
+ "step": 24
179
+ },
180
+ {
181
+ "epoch": 0.25510204081632654,
182
+ "grad_norm": 0.5994157195091248,
183
+ "learning_rate": 9.795918367346939e-06,
184
+ "loss": 0.0596,
185
+ "step": 25
186
+ },
187
+ {
188
+ "epoch": 0.2653061224489796,
189
+ "grad_norm": 0.5699151754379272,
190
+ "learning_rate": 1.0204081632653063e-05,
191
+ "loss": 0.0478,
192
+ "step": 26
193
+ },
194
+ {
195
+ "epoch": 0.2755102040816326,
196
+ "grad_norm": 0.41071373224258423,
197
+ "learning_rate": 1.0612244897959186e-05,
198
+ "loss": 0.0306,
199
+ "step": 27
200
+ },
201
+ {
202
+ "epoch": 0.2857142857142857,
203
+ "grad_norm": 0.6520228981971741,
204
+ "learning_rate": 1.1020408163265306e-05,
205
+ "loss": 0.0509,
206
+ "step": 28
207
+ },
208
+ {
209
+ "epoch": 0.29591836734693877,
210
+ "grad_norm": 0.5062035918235779,
211
+ "learning_rate": 1.1428571428571429e-05,
212
+ "loss": 0.0415,
213
+ "step": 29
214
+ },
215
+ {
216
+ "epoch": 0.30612244897959184,
217
+ "grad_norm": 0.5349479913711548,
218
+ "learning_rate": 1.1836734693877552e-05,
219
+ "loss": 0.0396,
220
+ "step": 30
221
+ },
222
+ {
223
+ "epoch": 0.3163265306122449,
224
+ "grad_norm": 0.29080551862716675,
225
+ "learning_rate": 1.2244897959183674e-05,
226
+ "loss": 0.0215,
227
+ "step": 31
228
+ },
229
+ {
230
+ "epoch": 0.32653061224489793,
231
+ "grad_norm": 0.5437124371528625,
232
+ "learning_rate": 1.2653061224489798e-05,
233
+ "loss": 0.0402,
234
+ "step": 32
235
+ },
236
+ {
237
+ "epoch": 0.336734693877551,
238
+ "grad_norm": 0.7592443823814392,
239
+ "learning_rate": 1.3061224489795918e-05,
240
+ "loss": 0.0692,
241
+ "step": 33
242
+ },
243
+ {
244
+ "epoch": 0.3469387755102041,
245
+ "grad_norm": 0.6122593283653259,
246
+ "learning_rate": 1.3469387755102042e-05,
247
+ "loss": 0.0602,
248
+ "step": 34
249
+ },
250
+ {
251
+ "epoch": 0.35714285714285715,
252
+ "grad_norm": 0.31229618191719055,
253
+ "learning_rate": 1.3877551020408165e-05,
254
+ "loss": 0.0215,
255
+ "step": 35
256
+ },
257
+ {
258
+ "epoch": 0.3673469387755102,
259
+ "grad_norm": 0.37940043210983276,
260
+ "learning_rate": 1.4285714285714287e-05,
261
+ "loss": 0.0274,
262
+ "step": 36
263
+ },
264
+ {
265
+ "epoch": 0.37755102040816324,
266
+ "grad_norm": 0.2848958969116211,
267
+ "learning_rate": 1.469387755102041e-05,
268
+ "loss": 0.0212,
269
+ "step": 37
270
+ },
271
+ {
272
+ "epoch": 0.3877551020408163,
273
+ "grad_norm": 0.3313491940498352,
274
+ "learning_rate": 1.510204081632653e-05,
275
+ "loss": 0.0231,
276
+ "step": 38
277
+ },
278
+ {
279
+ "epoch": 0.3979591836734694,
280
+ "grad_norm": 0.24816128611564636,
281
+ "learning_rate": 1.5510204081632655e-05,
282
+ "loss": 0.0159,
283
+ "step": 39
284
+ },
285
+ {
286
+ "epoch": 0.40816326530612246,
287
+ "grad_norm": 0.2411227524280548,
288
+ "learning_rate": 1.5918367346938776e-05,
289
+ "loss": 0.0154,
290
+ "step": 40
291
+ },
292
+ {
293
+ "epoch": 0.41836734693877553,
294
+ "grad_norm": 0.20029953122138977,
295
+ "learning_rate": 1.63265306122449e-05,
296
+ "loss": 0.013,
297
+ "step": 41
298
+ },
299
+ {
300
+ "epoch": 0.42857142857142855,
301
+ "grad_norm": 0.23513872921466827,
302
+ "learning_rate": 1.673469387755102e-05,
303
+ "loss": 0.0144,
304
+ "step": 42
305
+ },
306
+ {
307
+ "epoch": 0.4387755102040816,
308
+ "grad_norm": 0.4002116918563843,
309
+ "learning_rate": 1.7142857142857142e-05,
310
+ "loss": 0.0353,
311
+ "step": 43
312
+ },
313
+ {
314
+ "epoch": 0.4489795918367347,
315
+ "grad_norm": 0.27058476209640503,
316
+ "learning_rate": 1.7551020408163266e-05,
317
+ "loss": 0.0169,
318
+ "step": 44
319
+ },
320
+ {
321
+ "epoch": 0.45918367346938777,
322
+ "grad_norm": 0.09177622944116592,
323
+ "learning_rate": 1.795918367346939e-05,
324
+ "loss": 0.0055,
325
+ "step": 45
326
+ },
327
+ {
328
+ "epoch": 0.46938775510204084,
329
+ "grad_norm": 0.16117192804813385,
330
+ "learning_rate": 1.836734693877551e-05,
331
+ "loss": 0.0098,
332
+ "step": 46
333
+ },
334
+ {
335
+ "epoch": 0.47959183673469385,
336
+ "grad_norm": 0.1534506231546402,
337
+ "learning_rate": 1.8775510204081636e-05,
338
+ "loss": 0.0071,
339
+ "step": 47
340
+ },
341
+ {
342
+ "epoch": 0.4897959183673469,
343
+ "grad_norm": 0.2407277524471283,
344
+ "learning_rate": 1.9183673469387756e-05,
345
+ "loss": 0.0167,
346
+ "step": 48
347
+ },
348
+ {
349
+ "epoch": 0.5,
350
+ "grad_norm": 0.1129893809556961,
351
+ "learning_rate": 1.9591836734693877e-05,
352
+ "loss": 0.0062,
353
+ "step": 49
354
+ },
355
+ {
356
+ "epoch": 0.5102040816326531,
357
+ "grad_norm": 0.15642525255680084,
358
+ "learning_rate": 2e-05,
359
+ "loss": 0.0064,
360
+ "step": 50
361
+ },
362
+ {
363
+ "epoch": 0.5204081632653061,
364
+ "grad_norm": 0.30184754729270935,
365
+ "learning_rate": 1.9954648526077098e-05,
366
+ "loss": 0.0125,
367
+ "step": 51
368
+ },
369
+ {
370
+ "epoch": 0.5306122448979592,
371
+ "grad_norm": 0.07106052339076996,
372
+ "learning_rate": 1.9909297052154198e-05,
373
+ "loss": 0.0044,
374
+ "step": 52
375
+ },
376
+ {
377
+ "epoch": 0.5408163265306123,
378
+ "grad_norm": 0.26723626255989075,
379
+ "learning_rate": 1.9863945578231295e-05,
380
+ "loss": 0.0193,
381
+ "step": 53
382
+ },
383
+ {
384
+ "epoch": 0.5510204081632653,
385
+ "grad_norm": 0.09843797981739044,
386
+ "learning_rate": 1.981859410430839e-05,
387
+ "loss": 0.0058,
388
+ "step": 54
389
+ },
390
+ {
391
+ "epoch": 0.5612244897959183,
392
+ "grad_norm": 0.07936914265155792,
393
+ "learning_rate": 1.977324263038549e-05,
394
+ "loss": 0.0043,
395
+ "step": 55
396
+ },
397
+ {
398
+ "epoch": 0.5714285714285714,
399
+ "grad_norm": 0.06574582308530807,
400
+ "learning_rate": 1.9727891156462588e-05,
401
+ "loss": 0.0036,
402
+ "step": 56
403
+ },
404
+ {
405
+ "epoch": 0.5816326530612245,
406
+ "grad_norm": 0.030592354014515877,
407
+ "learning_rate": 1.9682539682539684e-05,
408
+ "loss": 0.0018,
409
+ "step": 57
410
+ },
411
+ {
412
+ "epoch": 0.5918367346938775,
413
+ "grad_norm": 0.07422778010368347,
414
+ "learning_rate": 1.963718820861678e-05,
415
+ "loss": 0.0039,
416
+ "step": 58
417
+ },
418
+ {
419
+ "epoch": 0.6020408163265306,
420
+ "grad_norm": 0.05910489708185196,
421
+ "learning_rate": 1.9591836734693877e-05,
422
+ "loss": 0.0031,
423
+ "step": 59
424
+ },
425
+ {
426
+ "epoch": 0.6122448979591837,
427
+ "grad_norm": 0.035012971609830856,
428
+ "learning_rate": 1.9546485260770977e-05,
429
+ "loss": 0.0019,
430
+ "step": 60
431
+ },
432
+ {
433
+ "epoch": 0.6224489795918368,
434
+ "grad_norm": 0.05377289652824402,
435
+ "learning_rate": 1.9501133786848074e-05,
436
+ "loss": 0.003,
437
+ "step": 61
438
+ },
439
+ {
440
+ "epoch": 0.6326530612244898,
441
+ "grad_norm": 0.059411946684122086,
442
+ "learning_rate": 1.945578231292517e-05,
443
+ "loss": 0.003,
444
+ "step": 62
445
+ },
446
+ {
447
+ "epoch": 0.6428571428571429,
448
+ "grad_norm": 0.07829850167036057,
449
+ "learning_rate": 1.941043083900227e-05,
450
+ "loss": 0.0039,
451
+ "step": 63
452
+ },
453
+ {
454
+ "epoch": 0.6530612244897959,
455
+ "grad_norm": 0.1004122868180275,
456
+ "learning_rate": 1.9365079365079367e-05,
457
+ "loss": 0.0048,
458
+ "step": 64
459
+ },
460
+ {
461
+ "epoch": 0.6632653061224489,
462
+ "grad_norm": 0.022585352882742882,
463
+ "learning_rate": 1.9319727891156463e-05,
464
+ "loss": 0.0013,
465
+ "step": 65
466
+ },
467
+ {
468
+ "epoch": 0.673469387755102,
469
+ "grad_norm": 0.08342932909727097,
470
+ "learning_rate": 1.9274376417233563e-05,
471
+ "loss": 0.0039,
472
+ "step": 66
473
+ },
474
+ {
475
+ "epoch": 0.6836734693877551,
476
+ "grad_norm": 0.3428645730018616,
477
+ "learning_rate": 1.922902494331066e-05,
478
+ "loss": 0.0113,
479
+ "step": 67
480
+ },
481
+ {
482
+ "epoch": 0.6938775510204082,
483
+ "grad_norm": 0.08267664909362793,
484
+ "learning_rate": 1.9183673469387756e-05,
485
+ "loss": 0.0042,
486
+ "step": 68
487
+ },
488
+ {
489
+ "epoch": 0.7040816326530612,
490
+ "grad_norm": 0.07195252925157547,
491
+ "learning_rate": 1.9138321995464853e-05,
492
+ "loss": 0.0029,
493
+ "step": 69
494
+ },
495
+ {
496
+ "epoch": 0.7142857142857143,
497
+ "grad_norm": 0.024874132126569748,
498
+ "learning_rate": 1.9092970521541953e-05,
499
+ "loss": 0.0014,
500
+ "step": 70
501
+ },
502
+ {
503
+ "epoch": 0.7244897959183674,
504
+ "grad_norm": 0.03532341867685318,
505
+ "learning_rate": 1.904761904761905e-05,
506
+ "loss": 0.0012,
507
+ "step": 71
508
+ },
509
+ {
510
+ "epoch": 0.7346938775510204,
511
+ "grad_norm": 0.01858861930668354,
512
+ "learning_rate": 1.9002267573696146e-05,
513
+ "loss": 0.001,
514
+ "step": 72
515
+ },
516
+ {
517
+ "epoch": 0.7448979591836735,
518
+ "grad_norm": 0.36321982741355896,
519
+ "learning_rate": 1.8956916099773243e-05,
520
+ "loss": 0.0128,
521
+ "step": 73
522
+ },
523
+ {
524
+ "epoch": 0.7551020408163265,
525
+ "grad_norm": 0.20222659409046173,
526
+ "learning_rate": 1.8911564625850343e-05,
527
+ "loss": 0.0076,
528
+ "step": 74
529
+ },
530
+ {
531
+ "epoch": 0.7653061224489796,
532
+ "grad_norm": 0.07980707287788391,
533
+ "learning_rate": 1.886621315192744e-05,
534
+ "loss": 0.0031,
535
+ "step": 75
536
+ },
537
+ {
538
+ "epoch": 0.7755102040816326,
539
+ "grad_norm": 0.020555464550852776,
540
+ "learning_rate": 1.8820861678004536e-05,
541
+ "loss": 0.0012,
542
+ "step": 76
543
+ },
544
+ {
545
+ "epoch": 0.7857142857142857,
546
+ "grad_norm": 0.02769128419458866,
547
+ "learning_rate": 1.8775510204081636e-05,
548
+ "loss": 0.0014,
549
+ "step": 77
550
+ },
551
+ {
552
+ "epoch": 0.7959183673469388,
553
+ "grad_norm": 0.030886279419064522,
554
+ "learning_rate": 1.8730158730158732e-05,
555
+ "loss": 0.0015,
556
+ "step": 78
557
+ },
558
+ {
559
+ "epoch": 0.8061224489795918,
560
+ "grad_norm": 0.04239689186215401,
561
+ "learning_rate": 1.868480725623583e-05,
562
+ "loss": 0.0017,
563
+ "step": 79
564
+ },
565
+ {
566
+ "epoch": 0.8163265306122449,
567
+ "grad_norm": 0.03217000514268875,
568
+ "learning_rate": 1.863945578231293e-05,
569
+ "loss": 0.0014,
570
+ "step": 80
571
+ },
572
+ {
573
+ "epoch": 0.826530612244898,
574
+ "grad_norm": 0.029874522238969803,
575
+ "learning_rate": 1.8594104308390025e-05,
576
+ "loss": 0.0015,
577
+ "step": 81
578
+ },
579
+ {
580
+ "epoch": 0.8367346938775511,
581
+ "grad_norm": 0.02627841755747795,
582
+ "learning_rate": 1.8548752834467122e-05,
583
+ "loss": 0.0013,
584
+ "step": 82
585
+ },
586
+ {
587
+ "epoch": 0.8469387755102041,
588
+ "grad_norm": 0.019378235563635826,
589
+ "learning_rate": 1.8503401360544218e-05,
590
+ "loss": 0.001,
591
+ "step": 83
592
+ },
593
+ {
594
+ "epoch": 0.8571428571428571,
595
+ "grad_norm": 0.08690612763166428,
596
+ "learning_rate": 1.8458049886621315e-05,
597
+ "loss": 0.0021,
598
+ "step": 84
599
+ },
600
+ {
601
+ "epoch": 0.8673469387755102,
602
+ "grad_norm": 0.014376318082213402,
603
+ "learning_rate": 1.8412698412698415e-05,
604
+ "loss": 0.0008,
605
+ "step": 85
606
+ },
607
+ {
608
+ "epoch": 0.8775510204081632,
609
+ "grad_norm": 0.01724099926650524,
610
+ "learning_rate": 1.836734693877551e-05,
611
+ "loss": 0.0009,
612
+ "step": 86
613
+ },
614
+ {
615
+ "epoch": 0.8877551020408163,
616
+ "grad_norm": 0.3265489339828491,
617
+ "learning_rate": 1.8321995464852608e-05,
618
+ "loss": 0.0117,
619
+ "step": 87
620
+ },
621
+ {
622
+ "epoch": 0.8979591836734694,
623
+ "grad_norm": 0.09740184992551804,
624
+ "learning_rate": 1.8276643990929708e-05,
625
+ "loss": 0.003,
626
+ "step": 88
627
+ },
628
+ {
629
+ "epoch": 0.9081632653061225,
630
+ "grad_norm": 0.015478034503757954,
631
+ "learning_rate": 1.8231292517006804e-05,
632
+ "loss": 0.0008,
633
+ "step": 89
634
+ },
635
+ {
636
+ "epoch": 0.9183673469387755,
637
+ "grad_norm": 0.18761862814426422,
638
+ "learning_rate": 1.81859410430839e-05,
639
+ "loss": 0.0068,
640
+ "step": 90
641
+ },
642
+ {
643
+ "epoch": 0.9285714285714286,
644
+ "grad_norm": 0.038408756256103516,
645
+ "learning_rate": 1.8140589569161e-05,
646
+ "loss": 0.0014,
647
+ "step": 91
648
+ },
649
+ {
650
+ "epoch": 0.9387755102040817,
651
+ "grad_norm": 0.03130817040801048,
652
+ "learning_rate": 1.8095238095238097e-05,
653
+ "loss": 0.0014,
654
+ "step": 92
655
+ },
656
+ {
657
+ "epoch": 0.9489795918367347,
658
+ "grad_norm": 0.014020106755197048,
659
+ "learning_rate": 1.8049886621315194e-05,
660
+ "loss": 0.0007,
661
+ "step": 93
662
+ },
663
+ {
664
+ "epoch": 0.9591836734693877,
665
+ "grad_norm": 0.02029995806515217,
666
+ "learning_rate": 1.8004535147392294e-05,
667
+ "loss": 0.0011,
668
+ "step": 94
669
+ },
670
+ {
671
+ "epoch": 0.9693877551020408,
672
+ "grad_norm": 0.021185798570513725,
673
+ "learning_rate": 1.795918367346939e-05,
674
+ "loss": 0.0009,
675
+ "step": 95
676
+ },
677
+ {
678
+ "epoch": 0.9795918367346939,
679
+ "grad_norm": 0.014589856378734112,
680
+ "learning_rate": 1.7913832199546487e-05,
681
+ "loss": 0.0008,
682
+ "step": 96
683
+ },
684
+ {
685
+ "epoch": 0.9897959183673469,
686
+ "grad_norm": 0.022265039384365082,
687
+ "learning_rate": 1.7868480725623583e-05,
688
+ "loss": 0.0011,
689
+ "step": 97
690
+ },
691
+ {
692
+ "epoch": 1.0,
693
+ "grad_norm": 0.0408700592815876,
694
+ "learning_rate": 1.782312925170068e-05,
695
+ "loss": 0.0011,
696
+ "step": 98
697
+ },
698
+ {
699
+ "epoch": 1.010204081632653,
700
+ "grad_norm": 0.009026318788528442,
701
+ "learning_rate": 1.7777777777777777e-05,
702
+ "loss": 0.0005,
703
+ "step": 99
704
+ },
705
+ {
706
+ "epoch": 1.0204081632653061,
707
+ "grad_norm": 0.007882497273385525,
708
+ "learning_rate": 1.7732426303854877e-05,
709
+ "loss": 0.0005,
710
+ "step": 100
711
+ },
712
+ {
713
+ "epoch": 1.030612244897959,
714
+ "grad_norm": 0.025666292756795883,
715
+ "learning_rate": 1.7687074829931973e-05,
716
+ "loss": 0.0012,
717
+ "step": 101
718
+ },
719
+ {
720
+ "epoch": 1.0408163265306123,
721
+ "grad_norm": 0.01795661635696888,
722
+ "learning_rate": 1.7641723356009073e-05,
723
+ "loss": 0.0008,
724
+ "step": 102
725
+ },
726
+ {
727
+ "epoch": 1.0510204081632653,
728
+ "grad_norm": 0.04071149602532387,
729
+ "learning_rate": 1.759637188208617e-05,
730
+ "loss": 0.0016,
731
+ "step": 103
732
+ },
733
+ {
734
+ "epoch": 1.0612244897959184,
735
+ "grad_norm": 0.007932674139738083,
736
+ "learning_rate": 1.7551020408163266e-05,
737
+ "loss": 0.0005,
738
+ "step": 104
739
+ },
740
+ {
741
+ "epoch": 1.0714285714285714,
742
+ "grad_norm": 0.03695099800825119,
743
+ "learning_rate": 1.7505668934240366e-05,
744
+ "loss": 0.0015,
745
+ "step": 105
746
+ },
747
+ {
748
+ "epoch": 1.0816326530612246,
749
+ "grad_norm": 0.008060461841523647,
750
+ "learning_rate": 1.7460317460317463e-05,
751
+ "loss": 0.0005,
752
+ "step": 106
753
+ },
754
+ {
755
+ "epoch": 1.0918367346938775,
756
+ "grad_norm": 0.04425932839512825,
757
+ "learning_rate": 1.741496598639456e-05,
758
+ "loss": 0.0018,
759
+ "step": 107
760
+ },
761
+ {
762
+ "epoch": 1.1020408163265305,
763
+ "grad_norm": 0.010241498239338398,
764
+ "learning_rate": 1.736961451247166e-05,
765
+ "loss": 0.0006,
766
+ "step": 108
767
+ },
768
+ {
769
+ "epoch": 1.1122448979591837,
770
+ "grad_norm": 0.010430874302983284,
771
+ "learning_rate": 1.7324263038548756e-05,
772
+ "loss": 0.0006,
773
+ "step": 109
774
+ },
775
+ {
776
+ "epoch": 1.1224489795918366,
777
+ "grad_norm": 0.16115950047969818,
778
+ "learning_rate": 1.7278911564625852e-05,
779
+ "loss": 0.0043,
780
+ "step": 110
781
+ },
782
+ {
783
+ "epoch": 1.1326530612244898,
784
+ "grad_norm": 0.018837768584489822,
785
+ "learning_rate": 1.723356009070295e-05,
786
+ "loss": 0.0007,
787
+ "step": 111
788
+ },
789
+ {
790
+ "epoch": 1.1428571428571428,
791
+ "grad_norm": 0.016730893403291702,
792
+ "learning_rate": 1.7188208616780045e-05,
793
+ "loss": 0.0009,
794
+ "step": 112
795
+ },
796
+ {
797
+ "epoch": 1.153061224489796,
798
+ "grad_norm": 0.011841993778944016,
799
+ "learning_rate": 1.7142857142857142e-05,
800
+ "loss": 0.0007,
801
+ "step": 113
802
+ },
803
+ {
804
+ "epoch": 1.163265306122449,
805
+ "grad_norm": 0.045097168534994125,
806
+ "learning_rate": 1.7097505668934242e-05,
807
+ "loss": 0.0019,
808
+ "step": 114
809
+ },
810
+ {
811
+ "epoch": 1.1734693877551021,
812
+ "grad_norm": 0.09953276813030243,
813
+ "learning_rate": 1.705215419501134e-05,
814
+ "loss": 0.0032,
815
+ "step": 115
816
+ },
817
+ {
818
+ "epoch": 1.183673469387755,
819
+ "grad_norm": 0.007014868780970573,
820
+ "learning_rate": 1.7006802721088435e-05,
821
+ "loss": 0.0004,
822
+ "step": 116
823
+ },
824
+ {
825
+ "epoch": 1.193877551020408,
826
+ "grad_norm": 0.009045367129147053,
827
+ "learning_rate": 1.6961451247165535e-05,
828
+ "loss": 0.0005,
829
+ "step": 117
830
+ },
831
+ {
832
+ "epoch": 1.2040816326530612,
833
+ "grad_norm": 0.007859342731535435,
834
+ "learning_rate": 1.691609977324263e-05,
835
+ "loss": 0.0005,
836
+ "step": 118
837
+ },
838
+ {
839
+ "epoch": 1.2142857142857142,
840
+ "grad_norm": 0.032524097710847855,
841
+ "learning_rate": 1.687074829931973e-05,
842
+ "loss": 0.0009,
843
+ "step": 119
844
+ },
845
+ {
846
+ "epoch": 1.2244897959183674,
847
+ "grad_norm": 0.05151795968413353,
848
+ "learning_rate": 1.6825396825396828e-05,
849
+ "loss": 0.0018,
850
+ "step": 120
851
+ },
852
+ {
853
+ "epoch": 1.2346938775510203,
854
+ "grad_norm": 0.010988794267177582,
855
+ "learning_rate": 1.6780045351473924e-05,
856
+ "loss": 0.0006,
857
+ "step": 121
858
+ },
859
+ {
860
+ "epoch": 1.2448979591836735,
861
+ "grad_norm": 0.006904716603457928,
862
+ "learning_rate": 1.673469387755102e-05,
863
+ "loss": 0.0004,
864
+ "step": 122
865
+ },
866
+ {
867
+ "epoch": 1.2551020408163265,
868
+ "grad_norm": 0.006797518581151962,
869
+ "learning_rate": 1.668934240362812e-05,
870
+ "loss": 0.0004,
871
+ "step": 123
872
+ },
873
+ {
874
+ "epoch": 1.2653061224489797,
875
+ "grad_norm": 0.01896447129547596,
876
+ "learning_rate": 1.6643990929705217e-05,
877
+ "loss": 0.0008,
878
+ "step": 124
879
+ },
880
+ {
881
+ "epoch": 1.2755102040816326,
882
+ "grad_norm": 0.01258290559053421,
883
+ "learning_rate": 1.6598639455782314e-05,
884
+ "loss": 0.0007,
885
+ "step": 125
886
+ },
887
+ {
888
+ "epoch": 1.2857142857142856,
889
+ "grad_norm": 0.015102504752576351,
890
+ "learning_rate": 1.655328798185941e-05,
891
+ "loss": 0.0006,
892
+ "step": 126
893
+ },
894
+ {
895
+ "epoch": 1.2959183673469388,
896
+ "grad_norm": 0.005591754335910082,
897
+ "learning_rate": 1.6507936507936507e-05,
898
+ "loss": 0.0004,
899
+ "step": 127
900
+ },
901
+ {
902
+ "epoch": 1.306122448979592,
903
+ "grad_norm": 0.10714168101549149,
904
+ "learning_rate": 1.6462585034013607e-05,
905
+ "loss": 0.0032,
906
+ "step": 128
907
+ },
908
+ {
909
+ "epoch": 1.316326530612245,
910
+ "grad_norm": 0.030577057972550392,
911
+ "learning_rate": 1.6417233560090704e-05,
912
+ "loss": 0.0011,
913
+ "step": 129
914
+ },
915
+ {
916
+ "epoch": 1.3265306122448979,
917
+ "grad_norm": 0.017115091904997826,
918
+ "learning_rate": 1.63718820861678e-05,
919
+ "loss": 0.0008,
920
+ "step": 130
921
+ },
922
+ {
923
+ "epoch": 1.336734693877551,
924
+ "grad_norm": 0.011700804345309734,
925
+ "learning_rate": 1.63265306122449e-05,
926
+ "loss": 0.0006,
927
+ "step": 131
928
+ },
929
+ {
930
+ "epoch": 1.346938775510204,
931
+ "grad_norm": 0.008858302608132362,
932
+ "learning_rate": 1.6281179138321997e-05,
933
+ "loss": 0.0004,
934
+ "step": 132
935
+ },
936
+ {
937
+ "epoch": 1.3571428571428572,
938
+ "grad_norm": 0.00907884445041418,
939
+ "learning_rate": 1.6235827664399097e-05,
940
+ "loss": 0.0005,
941
+ "step": 133
942
+ },
943
+ {
944
+ "epoch": 1.3673469387755102,
945
+ "grad_norm": 0.00645515276119113,
946
+ "learning_rate": 1.6190476190476193e-05,
947
+ "loss": 0.0003,
948
+ "step": 134
949
+ },
950
+ {
951
+ "epoch": 1.3775510204081631,
952
+ "grad_norm": 0.01644102856516838,
953
+ "learning_rate": 1.614512471655329e-05,
954
+ "loss": 0.0006,
955
+ "step": 135
956
+ },
957
+ {
958
+ "epoch": 1.3877551020408163,
959
+ "grad_norm": 0.023088015615940094,
960
+ "learning_rate": 1.6099773242630386e-05,
961
+ "loss": 0.0009,
962
+ "step": 136
963
+ },
964
+ {
965
+ "epoch": 1.3979591836734695,
966
+ "grad_norm": 0.004741874989122152,
967
+ "learning_rate": 1.6054421768707483e-05,
968
+ "loss": 0.0003,
969
+ "step": 137
970
+ },
971
+ {
972
+ "epoch": 1.4081632653061225,
973
+ "grad_norm": 0.005127857904881239,
974
+ "learning_rate": 1.6009070294784583e-05,
975
+ "loss": 0.0003,
976
+ "step": 138
977
+ },
978
+ {
979
+ "epoch": 1.4183673469387754,
980
+ "grad_norm": 0.009942681528627872,
981
+ "learning_rate": 1.596371882086168e-05,
982
+ "loss": 0.0005,
983
+ "step": 139
984
+ },
985
+ {
986
+ "epoch": 1.4285714285714286,
987
+ "grad_norm": 0.009013745002448559,
988
+ "learning_rate": 1.5918367346938776e-05,
989
+ "loss": 0.0005,
990
+ "step": 140
991
+ },
992
+ {
993
+ "epoch": 1.4387755102040816,
994
+ "grad_norm": 0.008382913656532764,
995
+ "learning_rate": 1.5873015873015872e-05,
996
+ "loss": 0.0005,
997
+ "step": 141
998
+ },
999
+ {
1000
+ "epoch": 1.4489795918367347,
1001
+ "grad_norm": 0.01376293320208788,
1002
+ "learning_rate": 1.5827664399092972e-05,
1003
+ "loss": 0.0006,
1004
+ "step": 142
1005
+ },
1006
+ {
1007
+ "epoch": 1.4591836734693877,
1008
+ "grad_norm": 0.07127456367015839,
1009
+ "learning_rate": 1.578231292517007e-05,
1010
+ "loss": 0.0022,
1011
+ "step": 143
1012
+ },
1013
+ {
1014
+ "epoch": 1.469387755102041,
1015
+ "grad_norm": 0.006247013341635466,
1016
+ "learning_rate": 1.5736961451247165e-05,
1017
+ "loss": 0.0004,
1018
+ "step": 144
1019
+ },
1020
+ {
1021
+ "epoch": 1.4795918367346939,
1022
+ "grad_norm": 0.03836556524038315,
1023
+ "learning_rate": 1.5691609977324265e-05,
1024
+ "loss": 0.0012,
1025
+ "step": 145
1026
+ },
1027
+ {
1028
+ "epoch": 1.489795918367347,
1029
+ "grad_norm": 0.011062193661928177,
1030
+ "learning_rate": 1.5646258503401362e-05,
1031
+ "loss": 0.0006,
1032
+ "step": 146
1033
+ },
1034
+ {
1035
+ "epoch": 1.5,
1036
+ "grad_norm": 0.005953874904662371,
1037
+ "learning_rate": 1.5600907029478462e-05,
1038
+ "loss": 0.0003,
1039
+ "step": 147
1040
+ },
1041
+ {
1042
+ "epoch": 1.510204081632653,
1043
+ "grad_norm": 0.014096422120928764,
1044
+ "learning_rate": 1.555555555555556e-05,
1045
+ "loss": 0.0008,
1046
+ "step": 148
1047
+ },
1048
+ {
1049
+ "epoch": 1.5204081632653061,
1050
+ "grad_norm": 0.0064276340417563915,
1051
+ "learning_rate": 1.5510204081632655e-05,
1052
+ "loss": 0.0004,
1053
+ "step": 149
1054
+ },
1055
+ {
1056
+ "epoch": 1.5306122448979593,
1057
+ "grad_norm": 0.007169738411903381,
1058
+ "learning_rate": 1.546485260770975e-05,
1059
+ "loss": 0.0004,
1060
+ "step": 150
1061
+ },
1062
+ {
1063
+ "epoch": 1.5408163265306123,
1064
+ "grad_norm": 0.005434677470475435,
1065
+ "learning_rate": 1.5419501133786848e-05,
1066
+ "loss": 0.0004,
1067
+ "step": 151
1068
+ },
1069
+ {
1070
+ "epoch": 1.5510204081632653,
1071
+ "grad_norm": 0.008770623244345188,
1072
+ "learning_rate": 1.5374149659863945e-05,
1073
+ "loss": 0.0004,
1074
+ "step": 152
1075
+ },
1076
+ {
1077
+ "epoch": 1.5612244897959182,
1078
+ "grad_norm": 0.011159502901136875,
1079
+ "learning_rate": 1.5328798185941044e-05,
1080
+ "loss": 0.0007,
1081
+ "step": 153
1082
+ },
1083
+ {
1084
+ "epoch": 1.5714285714285714,
1085
+ "grad_norm": 0.10471421480178833,
1086
+ "learning_rate": 1.528344671201814e-05,
1087
+ "loss": 0.0022,
1088
+ "step": 154
1089
+ },
1090
+ {
1091
+ "epoch": 1.5816326530612246,
1092
+ "grad_norm": 0.008612933568656445,
1093
+ "learning_rate": 1.523809523809524e-05,
1094
+ "loss": 0.0005,
1095
+ "step": 155
1096
+ },
1097
+ {
1098
+ "epoch": 1.5918367346938775,
1099
+ "grad_norm": 0.006375262048095465,
1100
+ "learning_rate": 1.5192743764172338e-05,
1101
+ "loss": 0.0003,
1102
+ "step": 156
1103
+ },
1104
+ {
1105
+ "epoch": 1.6020408163265305,
1106
+ "grad_norm": 0.00903844740241766,
1107
+ "learning_rate": 1.5147392290249434e-05,
1108
+ "loss": 0.0005,
1109
+ "step": 157
1110
+ },
1111
+ {
1112
+ "epoch": 1.6122448979591837,
1113
+ "grad_norm": 0.005267101805657148,
1114
+ "learning_rate": 1.510204081632653e-05,
1115
+ "loss": 0.0003,
1116
+ "step": 158
1117
+ },
1118
+ {
1119
+ "epoch": 1.6224489795918369,
1120
+ "grad_norm": 0.006081985309720039,
1121
+ "learning_rate": 1.505668934240363e-05,
1122
+ "loss": 0.0004,
1123
+ "step": 159
1124
+ },
1125
+ {
1126
+ "epoch": 1.6326530612244898,
1127
+ "grad_norm": 0.0072037833742797375,
1128
+ "learning_rate": 1.5011337868480727e-05,
1129
+ "loss": 0.0004,
1130
+ "step": 160
1131
+ },
1132
+ {
1133
+ "epoch": 1.6428571428571428,
1134
+ "grad_norm": 0.0033731532748788595,
1135
+ "learning_rate": 1.4965986394557825e-05,
1136
+ "loss": 0.0002,
1137
+ "step": 161
1138
+ },
1139
+ {
1140
+ "epoch": 1.6530612244897958,
1141
+ "grad_norm": 0.0077390824444592,
1142
+ "learning_rate": 1.4920634920634922e-05,
1143
+ "loss": 0.0005,
1144
+ "step": 162
1145
+ },
1146
+ {
1147
+ "epoch": 1.663265306122449,
1148
+ "grad_norm": 0.009692452847957611,
1149
+ "learning_rate": 1.4875283446712018e-05,
1150
+ "loss": 0.0005,
1151
+ "step": 163
1152
+ },
1153
+ {
1154
+ "epoch": 1.6734693877551021,
1155
+ "grad_norm": 0.006450532004237175,
1156
+ "learning_rate": 1.4829931972789118e-05,
1157
+ "loss": 0.0003,
1158
+ "step": 164
1159
+ },
1160
+ {
1161
+ "epoch": 1.683673469387755,
1162
+ "grad_norm": 0.009719816036522388,
1163
+ "learning_rate": 1.4784580498866215e-05,
1164
+ "loss": 0.0005,
1165
+ "step": 165
1166
+ },
1167
+ {
1168
+ "epoch": 1.693877551020408,
1169
+ "grad_norm": 0.010457034222781658,
1170
+ "learning_rate": 1.4739229024943311e-05,
1171
+ "loss": 0.0005,
1172
+ "step": 166
1173
+ },
1174
+ {
1175
+ "epoch": 1.7040816326530612,
1176
+ "grad_norm": 0.008015105500817299,
1177
+ "learning_rate": 1.469387755102041e-05,
1178
+ "loss": 0.0004,
1179
+ "step": 167
1180
+ },
1181
+ {
1182
+ "epoch": 1.7142857142857144,
1183
+ "grad_norm": 0.006307144183665514,
1184
+ "learning_rate": 1.4648526077097506e-05,
1185
+ "loss": 0.0003,
1186
+ "step": 168
1187
+ },
1188
+ {
1189
+ "epoch": 1.7244897959183674,
1190
+ "grad_norm": 0.005334992427378893,
1191
+ "learning_rate": 1.4603174603174603e-05,
1192
+ "loss": 0.0003,
1193
+ "step": 169
1194
+ },
1195
+ {
1196
+ "epoch": 1.7346938775510203,
1197
+ "grad_norm": 0.008067265152931213,
1198
+ "learning_rate": 1.4557823129251703e-05,
1199
+ "loss": 0.0003,
1200
+ "step": 170
1201
+ },
1202
+ {
1203
+ "epoch": 1.7448979591836735,
1204
+ "grad_norm": 0.012681787833571434,
1205
+ "learning_rate": 1.45124716553288e-05,
1206
+ "loss": 0.0005,
1207
+ "step": 171
1208
+ },
1209
+ {
1210
+ "epoch": 1.7551020408163265,
1211
+ "grad_norm": 0.01536930724978447,
1212
+ "learning_rate": 1.4467120181405896e-05,
1213
+ "loss": 0.0005,
1214
+ "step": 172
1215
+ },
1216
+ {
1217
+ "epoch": 1.7653061224489797,
1218
+ "grad_norm": 0.0037332891952246428,
1219
+ "learning_rate": 1.4421768707482994e-05,
1220
+ "loss": 0.0002,
1221
+ "step": 173
1222
+ },
1223
+ {
1224
+ "epoch": 1.7755102040816326,
1225
+ "grad_norm": 0.010341755114495754,
1226
+ "learning_rate": 1.4376417233560092e-05,
1227
+ "loss": 0.0005,
1228
+ "step": 174
1229
+ },
1230
+ {
1231
+ "epoch": 1.7857142857142856,
1232
+ "grad_norm": 0.0045587471686303616,
1233
+ "learning_rate": 1.433106575963719e-05,
1234
+ "loss": 0.0003,
1235
+ "step": 175
1236
+ },
1237
+ {
1238
+ "epoch": 1.7959183673469388,
1239
+ "grad_norm": 0.016639186069369316,
1240
+ "learning_rate": 1.4285714285714287e-05,
1241
+ "loss": 0.0006,
1242
+ "step": 176
1243
+ },
1244
+ {
1245
+ "epoch": 1.806122448979592,
1246
+ "grad_norm": 0.005003046710044146,
1247
+ "learning_rate": 1.4240362811791384e-05,
1248
+ "loss": 0.0003,
1249
+ "step": 177
1250
+ },
1251
+ {
1252
+ "epoch": 1.816326530612245,
1253
+ "grad_norm": 0.009210484102368355,
1254
+ "learning_rate": 1.4195011337868484e-05,
1255
+ "loss": 0.0004,
1256
+ "step": 178
1257
+ },
1258
+ {
1259
+ "epoch": 1.8265306122448979,
1260
+ "grad_norm": 0.007876208052039146,
1261
+ "learning_rate": 1.414965986394558e-05,
1262
+ "loss": 0.0004,
1263
+ "step": 179
1264
+ },
1265
+ {
1266
+ "epoch": 1.836734693877551,
1267
+ "grad_norm": 0.0038002703804522753,
1268
+ "learning_rate": 1.4104308390022677e-05,
1269
+ "loss": 0.0002,
1270
+ "step": 180
1271
+ },
1272
+ {
1273
+ "epoch": 1.8469387755102042,
1274
+ "grad_norm": 0.00423433817923069,
1275
+ "learning_rate": 1.4058956916099775e-05,
1276
+ "loss": 0.0002,
1277
+ "step": 181
1278
+ },
1279
+ {
1280
+ "epoch": 1.8571428571428572,
1281
+ "grad_norm": 0.008944015018641949,
1282
+ "learning_rate": 1.4013605442176872e-05,
1283
+ "loss": 0.0005,
1284
+ "step": 182
1285
+ },
1286
+ {
1287
+ "epoch": 1.8673469387755102,
1288
+ "grad_norm": 0.004832221195101738,
1289
+ "learning_rate": 1.3968253968253968e-05,
1290
+ "loss": 0.0003,
1291
+ "step": 183
1292
+ },
1293
+ {
1294
+ "epoch": 1.8775510204081631,
1295
+ "grad_norm": 0.005358612630516291,
1296
+ "learning_rate": 1.3922902494331068e-05,
1297
+ "loss": 0.0003,
1298
+ "step": 184
1299
+ },
1300
+ {
1301
+ "epoch": 1.8877551020408163,
1302
+ "grad_norm": 0.004266591276973486,
1303
+ "learning_rate": 1.3877551020408165e-05,
1304
+ "loss": 0.0002,
1305
+ "step": 185
1306
+ },
1307
+ {
1308
+ "epoch": 1.8979591836734695,
1309
+ "grad_norm": 0.004511923063546419,
1310
+ "learning_rate": 1.3832199546485261e-05,
1311
+ "loss": 0.0003,
1312
+ "step": 186
1313
+ },
1314
+ {
1315
+ "epoch": 1.9081632653061225,
1316
+ "grad_norm": 0.12353862076997757,
1317
+ "learning_rate": 1.378684807256236e-05,
1318
+ "loss": 0.0032,
1319
+ "step": 187
1320
+ },
1321
+ {
1322
+ "epoch": 1.9183673469387754,
1323
+ "grad_norm": 0.009472350589931011,
1324
+ "learning_rate": 1.3741496598639456e-05,
1325
+ "loss": 0.0006,
1326
+ "step": 188
1327
+ },
1328
+ {
1329
+ "epoch": 1.9285714285714286,
1330
+ "grad_norm": 0.005253692157566547,
1331
+ "learning_rate": 1.3696145124716554e-05,
1332
+ "loss": 0.0003,
1333
+ "step": 189
1334
+ },
1335
+ {
1336
+ "epoch": 1.9387755102040818,
1337
+ "grad_norm": 0.01199701614677906,
1338
+ "learning_rate": 1.3650793650793652e-05,
1339
+ "loss": 0.0005,
1340
+ "step": 190
1341
+ },
1342
+ {
1343
+ "epoch": 1.9489795918367347,
1344
+ "grad_norm": 0.006006367038935423,
1345
+ "learning_rate": 1.3605442176870749e-05,
1346
+ "loss": 0.0003,
1347
+ "step": 191
1348
+ },
1349
+ {
1350
+ "epoch": 1.9591836734693877,
1351
+ "grad_norm": 0.010423636995255947,
1352
+ "learning_rate": 1.3560090702947847e-05,
1353
+ "loss": 0.0004,
1354
+ "step": 192
1355
+ },
1356
+ {
1357
+ "epoch": 1.9693877551020407,
1358
+ "grad_norm": 0.006484678015112877,
1359
+ "learning_rate": 1.3514739229024945e-05,
1360
+ "loss": 0.0004,
1361
+ "step": 193
1362
+ },
1363
+ {
1364
+ "epoch": 1.9795918367346939,
1365
+ "grad_norm": 0.007823942229151726,
1366
+ "learning_rate": 1.3469387755102042e-05,
1367
+ "loss": 0.0004,
1368
+ "step": 194
1369
+ },
1370
+ {
1371
+ "epoch": 1.989795918367347,
1372
+ "grad_norm": 0.005013170652091503,
1373
+ "learning_rate": 1.342403628117914e-05,
1374
+ "loss": 0.0003,
1375
+ "step": 195
1376
+ },
1377
+ {
1378
+ "epoch": 2.0,
1379
+ "grad_norm": 0.003286719787865877,
1380
+ "learning_rate": 1.3378684807256237e-05,
1381
+ "loss": 0.0001,
1382
+ "step": 196
1383
+ },
1384
+ {
1385
+ "epoch": 2.010204081632653,
1386
+ "grad_norm": 0.007182662840932608,
1387
+ "learning_rate": 1.3333333333333333e-05,
1388
+ "loss": 0.0003,
1389
+ "step": 197
1390
+ },
1391
+ {
1392
+ "epoch": 2.020408163265306,
1393
+ "grad_norm": 0.007124132476747036,
1394
+ "learning_rate": 1.3287981859410433e-05,
1395
+ "loss": 0.0003,
1396
+ "step": 198
1397
+ },
1398
+ {
1399
+ "epoch": 2.0306122448979593,
1400
+ "grad_norm": 0.0033109758514910936,
1401
+ "learning_rate": 1.324263038548753e-05,
1402
+ "loss": 0.0002,
1403
+ "step": 199
1404
+ },
1405
+ {
1406
+ "epoch": 2.0408163265306123,
1407
+ "grad_norm": 0.0030752080492675304,
1408
+ "learning_rate": 1.3197278911564626e-05,
1409
+ "loss": 0.0002,
1410
+ "step": 200
1411
+ },
1412
+ {
1413
+ "epoch": 2.0510204081632653,
1414
+ "grad_norm": 0.003937916364520788,
1415
+ "learning_rate": 1.3151927437641725e-05,
1416
+ "loss": 0.0003,
1417
+ "step": 201
1418
+ },
1419
+ {
1420
+ "epoch": 2.061224489795918,
1421
+ "grad_norm": 0.003902744734659791,
1422
+ "learning_rate": 1.3106575963718821e-05,
1423
+ "loss": 0.0002,
1424
+ "step": 202
1425
+ },
1426
+ {
1427
+ "epoch": 2.0714285714285716,
1428
+ "grad_norm": 0.003365420503541827,
1429
+ "learning_rate": 1.3061224489795918e-05,
1430
+ "loss": 0.0002,
1431
+ "step": 203
1432
+ },
1433
+ {
1434
+ "epoch": 2.0816326530612246,
1435
+ "grad_norm": 0.005177025683224201,
1436
+ "learning_rate": 1.3015873015873018e-05,
1437
+ "loss": 0.0003,
1438
+ "step": 204
1439
+ },
1440
+ {
1441
+ "epoch": 2.0918367346938775,
1442
+ "grad_norm": 0.004482835531234741,
1443
+ "learning_rate": 1.2970521541950114e-05,
1444
+ "loss": 0.0003,
1445
+ "step": 205
1446
+ },
1447
+ {
1448
+ "epoch": 2.1020408163265305,
1449
+ "grad_norm": 0.03137246519327164,
1450
+ "learning_rate": 1.2925170068027212e-05,
1451
+ "loss": 0.0008,
1452
+ "step": 206
1453
+ },
1454
+ {
1455
+ "epoch": 2.1122448979591835,
1456
+ "grad_norm": 0.0061664879322052,
1457
+ "learning_rate": 1.2879818594104309e-05,
1458
+ "loss": 0.0004,
1459
+ "step": 207
1460
+ },
1461
+ {
1462
+ "epoch": 2.122448979591837,
1463
+ "grad_norm": 0.007974425330758095,
1464
+ "learning_rate": 1.2834467120181407e-05,
1465
+ "loss": 0.0004,
1466
+ "step": 208
1467
+ },
1468
+ {
1469
+ "epoch": 2.13265306122449,
1470
+ "grad_norm": 0.008234084583818913,
1471
+ "learning_rate": 1.2789115646258505e-05,
1472
+ "loss": 0.0004,
1473
+ "step": 209
1474
+ },
1475
+ {
1476
+ "epoch": 2.142857142857143,
1477
+ "grad_norm": 0.005270775873214006,
1478
+ "learning_rate": 1.2743764172335602e-05,
1479
+ "loss": 0.0003,
1480
+ "step": 210
1481
+ },
1482
+ {
1483
+ "epoch": 2.1530612244897958,
1484
+ "grad_norm": 0.008359711617231369,
1485
+ "learning_rate": 1.2698412698412699e-05,
1486
+ "loss": 0.0004,
1487
+ "step": 211
1488
+ },
1489
+ {
1490
+ "epoch": 2.163265306122449,
1491
+ "grad_norm": 0.004007325973361731,
1492
+ "learning_rate": 1.2653061224489798e-05,
1493
+ "loss": 0.0002,
1494
+ "step": 212
1495
+ },
1496
+ {
1497
+ "epoch": 2.173469387755102,
1498
+ "grad_norm": 0.004752746783196926,
1499
+ "learning_rate": 1.2607709750566895e-05,
1500
+ "loss": 0.0002,
1501
+ "step": 213
1502
+ },
1503
+ {
1504
+ "epoch": 2.183673469387755,
1505
+ "grad_norm": 0.0031563639640808105,
1506
+ "learning_rate": 1.2562358276643992e-05,
1507
+ "loss": 0.0002,
1508
+ "step": 214
1509
+ },
1510
+ {
1511
+ "epoch": 2.193877551020408,
1512
+ "grad_norm": 0.003636228386312723,
1513
+ "learning_rate": 1.251700680272109e-05,
1514
+ "loss": 0.0002,
1515
+ "step": 215
1516
+ },
1517
+ {
1518
+ "epoch": 2.204081632653061,
1519
+ "grad_norm": 0.0034094173461198807,
1520
+ "learning_rate": 1.2471655328798186e-05,
1521
+ "loss": 0.0002,
1522
+ "step": 216
1523
+ },
1524
+ {
1525
+ "epoch": 2.2142857142857144,
1526
+ "grad_norm": 0.004791253712028265,
1527
+ "learning_rate": 1.2426303854875283e-05,
1528
+ "loss": 0.0003,
1529
+ "step": 217
1530
+ },
1531
+ {
1532
+ "epoch": 2.2244897959183674,
1533
+ "grad_norm": 0.010279831476509571,
1534
+ "learning_rate": 1.2380952380952383e-05,
1535
+ "loss": 0.0004,
1536
+ "step": 218
1537
+ },
1538
+ {
1539
+ "epoch": 2.2346938775510203,
1540
+ "grad_norm": 0.006269859150052071,
1541
+ "learning_rate": 1.233560090702948e-05,
1542
+ "loss": 0.0003,
1543
+ "step": 219
1544
+ },
1545
+ {
1546
+ "epoch": 2.2448979591836733,
1547
+ "grad_norm": 0.003878034185618162,
1548
+ "learning_rate": 1.2290249433106578e-05,
1549
+ "loss": 0.0002,
1550
+ "step": 220
1551
+ },
1552
+ {
1553
+ "epoch": 2.2551020408163267,
1554
+ "grad_norm": 0.0031356397084891796,
1555
+ "learning_rate": 1.2244897959183674e-05,
1556
+ "loss": 0.0002,
1557
+ "step": 221
1558
+ },
1559
+ {
1560
+ "epoch": 2.2653061224489797,
1561
+ "grad_norm": 0.004956800024956465,
1562
+ "learning_rate": 1.219954648526077e-05,
1563
+ "loss": 0.0003,
1564
+ "step": 222
1565
+ },
1566
+ {
1567
+ "epoch": 2.2755102040816326,
1568
+ "grad_norm": 0.0036491460632532835,
1569
+ "learning_rate": 1.215419501133787e-05,
1570
+ "loss": 0.0002,
1571
+ "step": 223
1572
+ },
1573
+ {
1574
+ "epoch": 2.2857142857142856,
1575
+ "grad_norm": 0.005171376280486584,
1576
+ "learning_rate": 1.2108843537414967e-05,
1577
+ "loss": 0.0003,
1578
+ "step": 224
1579
+ },
1580
+ {
1581
+ "epoch": 2.295918367346939,
1582
+ "grad_norm": 0.0029648093041032553,
1583
+ "learning_rate": 1.2063492063492064e-05,
1584
+ "loss": 0.0002,
1585
+ "step": 225
1586
+ },
1587
+ {
1588
+ "epoch": 2.306122448979592,
1589
+ "grad_norm": 0.006329487543553114,
1590
+ "learning_rate": 1.2018140589569162e-05,
1591
+ "loss": 0.0003,
1592
+ "step": 226
1593
+ },
1594
+ {
1595
+ "epoch": 2.316326530612245,
1596
+ "grad_norm": 0.0031556261237710714,
1597
+ "learning_rate": 1.197278911564626e-05,
1598
+ "loss": 0.0002,
1599
+ "step": 227
1600
+ },
1601
+ {
1602
+ "epoch": 2.326530612244898,
1603
+ "grad_norm": 0.009794807992875576,
1604
+ "learning_rate": 1.1927437641723357e-05,
1605
+ "loss": 0.0004,
1606
+ "step": 228
1607
+ },
1608
+ {
1609
+ "epoch": 2.336734693877551,
1610
+ "grad_norm": 0.003714526304975152,
1611
+ "learning_rate": 1.1882086167800455e-05,
1612
+ "loss": 0.0002,
1613
+ "step": 229
1614
+ },
1615
+ {
1616
+ "epoch": 2.3469387755102042,
1617
+ "grad_norm": 0.0031528149265795946,
1618
+ "learning_rate": 1.1836734693877552e-05,
1619
+ "loss": 0.0002,
1620
+ "step": 230
1621
+ },
1622
+ {
1623
+ "epoch": 2.357142857142857,
1624
+ "grad_norm": 0.024612465873360634,
1625
+ "learning_rate": 1.1791383219954648e-05,
1626
+ "loss": 0.001,
1627
+ "step": 231
1628
+ },
1629
+ {
1630
+ "epoch": 2.36734693877551,
1631
+ "grad_norm": 0.00424389261752367,
1632
+ "learning_rate": 1.1746031746031748e-05,
1633
+ "loss": 0.0002,
1634
+ "step": 232
1635
+ },
1636
+ {
1637
+ "epoch": 2.377551020408163,
1638
+ "grad_norm": 0.01282750815153122,
1639
+ "learning_rate": 1.1700680272108845e-05,
1640
+ "loss": 0.0006,
1641
+ "step": 233
1642
+ },
1643
+ {
1644
+ "epoch": 2.387755102040816,
1645
+ "grad_norm": 0.006169433705508709,
1646
+ "learning_rate": 1.1655328798185943e-05,
1647
+ "loss": 0.0003,
1648
+ "step": 234
1649
+ },
1650
+ {
1651
+ "epoch": 2.3979591836734695,
1652
+ "grad_norm": 0.005562425125390291,
1653
+ "learning_rate": 1.160997732426304e-05,
1654
+ "loss": 0.0003,
1655
+ "step": 235
1656
+ },
1657
+ {
1658
+ "epoch": 2.4081632653061225,
1659
+ "grad_norm": 0.01002059318125248,
1660
+ "learning_rate": 1.1564625850340136e-05,
1661
+ "loss": 0.0005,
1662
+ "step": 236
1663
+ },
1664
+ {
1665
+ "epoch": 2.4183673469387754,
1666
+ "grad_norm": 0.007645392790436745,
1667
+ "learning_rate": 1.1519274376417236e-05,
1668
+ "loss": 0.0004,
1669
+ "step": 237
1670
+ },
1671
+ {
1672
+ "epoch": 2.4285714285714284,
1673
+ "grad_norm": 0.03314538300037384,
1674
+ "learning_rate": 1.1473922902494332e-05,
1675
+ "loss": 0.0011,
1676
+ "step": 238
1677
+ },
1678
+ {
1679
+ "epoch": 2.438775510204082,
1680
+ "grad_norm": 0.04838201776146889,
1681
+ "learning_rate": 1.1428571428571429e-05,
1682
+ "loss": 0.0009,
1683
+ "step": 239
1684
+ },
1685
+ {
1686
+ "epoch": 2.4489795918367347,
1687
+ "grad_norm": 0.006126615218818188,
1688
+ "learning_rate": 1.1383219954648527e-05,
1689
+ "loss": 0.0004,
1690
+ "step": 240
1691
+ },
1692
+ {
1693
+ "epoch": 2.4591836734693877,
1694
+ "grad_norm": 0.005240059457719326,
1695
+ "learning_rate": 1.1337868480725624e-05,
1696
+ "loss": 0.0003,
1697
+ "step": 241
1698
+ },
1699
+ {
1700
+ "epoch": 2.4693877551020407,
1701
+ "grad_norm": 0.006122751161456108,
1702
+ "learning_rate": 1.1292517006802722e-05,
1703
+ "loss": 0.0003,
1704
+ "step": 242
1705
+ },
1706
+ {
1707
+ "epoch": 2.479591836734694,
1708
+ "grad_norm": 0.0024781699758023024,
1709
+ "learning_rate": 1.124716553287982e-05,
1710
+ "loss": 0.0002,
1711
+ "step": 243
1712
+ },
1713
+ {
1714
+ "epoch": 2.489795918367347,
1715
+ "grad_norm": 0.00649678660556674,
1716
+ "learning_rate": 1.1201814058956917e-05,
1717
+ "loss": 0.0004,
1718
+ "step": 244
1719
+ },
1720
+ {
1721
+ "epoch": 2.5,
1722
+ "grad_norm": 0.003478443017229438,
1723
+ "learning_rate": 1.1156462585034013e-05,
1724
+ "loss": 0.0002,
1725
+ "step": 245
1726
+ },
1727
+ {
1728
+ "epoch": 2.510204081632653,
1729
+ "grad_norm": 0.003858257783576846,
1730
+ "learning_rate": 1.1111111111111113e-05,
1731
+ "loss": 0.0002,
1732
+ "step": 246
1733
+ },
1734
+ {
1735
+ "epoch": 2.520408163265306,
1736
+ "grad_norm": 0.006577960215508938,
1737
+ "learning_rate": 1.106575963718821e-05,
1738
+ "loss": 0.0004,
1739
+ "step": 247
1740
+ },
1741
+ {
1742
+ "epoch": 2.5306122448979593,
1743
+ "grad_norm": 0.004543396644294262,
1744
+ "learning_rate": 1.1020408163265306e-05,
1745
+ "loss": 0.0003,
1746
+ "step": 248
1747
+ },
1748
+ {
1749
+ "epoch": 2.5408163265306123,
1750
+ "grad_norm": 0.0032837800681591034,
1751
+ "learning_rate": 1.0975056689342405e-05,
1752
+ "loss": 0.0002,
1753
+ "step": 249
1754
+ },
1755
+ {
1756
+ "epoch": 2.5510204081632653,
1757
+ "grad_norm": 0.012741784565150738,
1758
+ "learning_rate": 1.0929705215419501e-05,
1759
+ "loss": 0.0006,
1760
+ "step": 250
1761
+ },
1762
+ {
1763
+ "epoch": 2.561224489795918,
1764
+ "grad_norm": 0.002741026459261775,
1765
+ "learning_rate": 1.0884353741496601e-05,
1766
+ "loss": 0.0002,
1767
+ "step": 251
1768
+ },
1769
+ {
1770
+ "epoch": 2.571428571428571,
1771
+ "grad_norm": 0.0027930692303925753,
1772
+ "learning_rate": 1.0839002267573698e-05,
1773
+ "loss": 0.0002,
1774
+ "step": 252
1775
+ },
1776
+ {
1777
+ "epoch": 2.5816326530612246,
1778
+ "grad_norm": 0.003156407503411174,
1779
+ "learning_rate": 1.0793650793650794e-05,
1780
+ "loss": 0.0002,
1781
+ "step": 253
1782
+ },
1783
+ {
1784
+ "epoch": 2.5918367346938775,
1785
+ "grad_norm": 0.0036412908229976892,
1786
+ "learning_rate": 1.0748299319727893e-05,
1787
+ "loss": 0.0002,
1788
+ "step": 254
1789
+ },
1790
+ {
1791
+ "epoch": 2.6020408163265305,
1792
+ "grad_norm": 0.04034988954663277,
1793
+ "learning_rate": 1.0702947845804989e-05,
1794
+ "loss": 0.0013,
1795
+ "step": 255
1796
+ },
1797
+ {
1798
+ "epoch": 2.612244897959184,
1799
+ "grad_norm": 0.0034895159769803286,
1800
+ "learning_rate": 1.0657596371882086e-05,
1801
+ "loss": 0.0002,
1802
+ "step": 256
1803
+ },
1804
+ {
1805
+ "epoch": 2.622448979591837,
1806
+ "grad_norm": 0.04325950890779495,
1807
+ "learning_rate": 1.0612244897959186e-05,
1808
+ "loss": 0.0012,
1809
+ "step": 257
1810
+ },
1811
+ {
1812
+ "epoch": 2.63265306122449,
1813
+ "grad_norm": 0.004671269562095404,
1814
+ "learning_rate": 1.0566893424036282e-05,
1815
+ "loss": 0.0003,
1816
+ "step": 258
1817
+ },
1818
+ {
1819
+ "epoch": 2.642857142857143,
1820
+ "grad_norm": 0.002534637926146388,
1821
+ "learning_rate": 1.0521541950113379e-05,
1822
+ "loss": 0.0002,
1823
+ "step": 259
1824
+ },
1825
+ {
1826
+ "epoch": 2.6530612244897958,
1827
+ "grad_norm": 0.004457306116819382,
1828
+ "learning_rate": 1.0476190476190477e-05,
1829
+ "loss": 0.0003,
1830
+ "step": 260
1831
+ },
1832
+ {
1833
+ "epoch": 2.663265306122449,
1834
+ "grad_norm": 0.004050545394420624,
1835
+ "learning_rate": 1.0430839002267575e-05,
1836
+ "loss": 0.0002,
1837
+ "step": 261
1838
+ },
1839
+ {
1840
+ "epoch": 2.673469387755102,
1841
+ "grad_norm": 0.04582836106419563,
1842
+ "learning_rate": 1.0385487528344672e-05,
1843
+ "loss": 0.0011,
1844
+ "step": 262
1845
+ },
1846
+ {
1847
+ "epoch": 2.683673469387755,
1848
+ "grad_norm": 0.004835136700421572,
1849
+ "learning_rate": 1.034013605442177e-05,
1850
+ "loss": 0.0003,
1851
+ "step": 263
1852
+ },
1853
+ {
1854
+ "epoch": 2.693877551020408,
1855
+ "grad_norm": 0.008025884628295898,
1856
+ "learning_rate": 1.0294784580498866e-05,
1857
+ "loss": 0.0003,
1858
+ "step": 264
1859
+ },
1860
+ {
1861
+ "epoch": 2.704081632653061,
1862
+ "grad_norm": 0.007876653224229813,
1863
+ "learning_rate": 1.0249433106575966e-05,
1864
+ "loss": 0.0004,
1865
+ "step": 265
1866
+ },
1867
+ {
1868
+ "epoch": 2.7142857142857144,
1869
+ "grad_norm": 0.006527318619191647,
1870
+ "learning_rate": 1.0204081632653063e-05,
1871
+ "loss": 0.0003,
1872
+ "step": 266
1873
+ },
1874
+ {
1875
+ "epoch": 2.7244897959183674,
1876
+ "grad_norm": 0.002143925055861473,
1877
+ "learning_rate": 1.015873015873016e-05,
1878
+ "loss": 0.0001,
1879
+ "step": 267
1880
+ },
1881
+ {
1882
+ "epoch": 2.7346938775510203,
1883
+ "grad_norm": 0.003183850785717368,
1884
+ "learning_rate": 1.0113378684807258e-05,
1885
+ "loss": 0.0002,
1886
+ "step": 268
1887
+ },
1888
+ {
1889
+ "epoch": 2.7448979591836737,
1890
+ "grad_norm": 0.003816920565441251,
1891
+ "learning_rate": 1.0068027210884354e-05,
1892
+ "loss": 0.0002,
1893
+ "step": 269
1894
+ },
1895
+ {
1896
+ "epoch": 2.7551020408163263,
1897
+ "grad_norm": 0.005489765666425228,
1898
+ "learning_rate": 1.0022675736961451e-05,
1899
+ "loss": 0.0003,
1900
+ "step": 270
1901
+ },
1902
+ {
1903
+ "epoch": 2.7653061224489797,
1904
+ "grad_norm": 0.002469045575708151,
1905
+ "learning_rate": 9.977324263038549e-06,
1906
+ "loss": 0.0002,
1907
+ "step": 271
1908
+ },
1909
+ {
1910
+ "epoch": 2.7755102040816326,
1911
+ "grad_norm": 0.0031796926632523537,
1912
+ "learning_rate": 9.931972789115647e-06,
1913
+ "loss": 0.0002,
1914
+ "step": 272
1915
+ },
1916
+ {
1917
+ "epoch": 2.7857142857142856,
1918
+ "grad_norm": 0.003679267829284072,
1919
+ "learning_rate": 9.886621315192746e-06,
1920
+ "loss": 0.0002,
1921
+ "step": 273
1922
+ },
1923
+ {
1924
+ "epoch": 2.795918367346939,
1925
+ "grad_norm": 0.010035431012511253,
1926
+ "learning_rate": 9.841269841269842e-06,
1927
+ "loss": 0.0004,
1928
+ "step": 274
1929
+ },
1930
+ {
1931
+ "epoch": 2.806122448979592,
1932
+ "grad_norm": 0.0031564754899591208,
1933
+ "learning_rate": 9.795918367346939e-06,
1934
+ "loss": 0.0002,
1935
+ "step": 275
1936
+ },
1937
+ {
1938
+ "epoch": 2.816326530612245,
1939
+ "grad_norm": 0.005581808276474476,
1940
+ "learning_rate": 9.750566893424037e-06,
1941
+ "loss": 0.0003,
1942
+ "step": 276
1943
+ },
1944
+ {
1945
+ "epoch": 2.826530612244898,
1946
+ "grad_norm": 0.002813218394294381,
1947
+ "learning_rate": 9.705215419501135e-06,
1948
+ "loss": 0.0002,
1949
+ "step": 277
1950
+ },
1951
+ {
1952
+ "epoch": 2.836734693877551,
1953
+ "grad_norm": 0.003005703678354621,
1954
+ "learning_rate": 9.659863945578232e-06,
1955
+ "loss": 0.0002,
1956
+ "step": 278
1957
+ },
1958
+ {
1959
+ "epoch": 2.8469387755102042,
1960
+ "grad_norm": 0.0068191043101251125,
1961
+ "learning_rate": 9.61451247165533e-06,
1962
+ "loss": 0.0004,
1963
+ "step": 279
1964
+ },
1965
+ {
1966
+ "epoch": 2.857142857142857,
1967
+ "grad_norm": 0.008098878897726536,
1968
+ "learning_rate": 9.569160997732427e-06,
1969
+ "loss": 0.0004,
1970
+ "step": 280
1971
+ },
1972
+ {
1973
+ "epoch": 2.86734693877551,
1974
+ "grad_norm": 0.014086649753153324,
1975
+ "learning_rate": 9.523809523809525e-06,
1976
+ "loss": 0.0004,
1977
+ "step": 281
1978
+ },
1979
+ {
1980
+ "epoch": 2.877551020408163,
1981
+ "grad_norm": 0.004192018415778875,
1982
+ "learning_rate": 9.478458049886621e-06,
1983
+ "loss": 0.0002,
1984
+ "step": 282
1985
+ },
1986
+ {
1987
+ "epoch": 2.887755102040816,
1988
+ "grad_norm": 0.0025980896316468716,
1989
+ "learning_rate": 9.43310657596372e-06,
1990
+ "loss": 0.0002,
1991
+ "step": 283
1992
+ },
1993
+ {
1994
+ "epoch": 2.8979591836734695,
1995
+ "grad_norm": 0.010852901265025139,
1996
+ "learning_rate": 9.387755102040818e-06,
1997
+ "loss": 0.0004,
1998
+ "step": 284
1999
+ },
2000
+ {
2001
+ "epoch": 2.9081632653061225,
2002
+ "grad_norm": 0.003038214286789298,
2003
+ "learning_rate": 9.342403628117914e-06,
2004
+ "loss": 0.0002,
2005
+ "step": 285
2006
+ },
2007
+ {
2008
+ "epoch": 2.9183673469387754,
2009
+ "grad_norm": 0.0038609837647527456,
2010
+ "learning_rate": 9.297052154195013e-06,
2011
+ "loss": 0.0002,
2012
+ "step": 286
2013
+ },
2014
+ {
2015
+ "epoch": 2.928571428571429,
2016
+ "grad_norm": 0.0073219058103859425,
2017
+ "learning_rate": 9.251700680272109e-06,
2018
+ "loss": 0.0004,
2019
+ "step": 287
2020
+ },
2021
+ {
2022
+ "epoch": 2.938775510204082,
2023
+ "grad_norm": 0.004274952691048384,
2024
+ "learning_rate": 9.206349206349207e-06,
2025
+ "loss": 0.0003,
2026
+ "step": 288
2027
+ },
2028
+ {
2029
+ "epoch": 2.9489795918367347,
2030
+ "grad_norm": 0.003952549304813147,
2031
+ "learning_rate": 9.160997732426304e-06,
2032
+ "loss": 0.0002,
2033
+ "step": 289
2034
+ },
2035
+ {
2036
+ "epoch": 2.9591836734693877,
2037
+ "grad_norm": 0.03005502186715603,
2038
+ "learning_rate": 9.115646258503402e-06,
2039
+ "loss": 0.0006,
2040
+ "step": 290
2041
+ },
2042
+ {
2043
+ "epoch": 2.9693877551020407,
2044
+ "grad_norm": 0.0023858973290771246,
2045
+ "learning_rate": 9.0702947845805e-06,
2046
+ "loss": 0.0002,
2047
+ "step": 291
2048
+ },
2049
+ {
2050
+ "epoch": 2.979591836734694,
2051
+ "grad_norm": 0.004339877981692553,
2052
+ "learning_rate": 9.024943310657597e-06,
2053
+ "loss": 0.0003,
2054
+ "step": 292
2055
+ },
2056
+ {
2057
+ "epoch": 2.989795918367347,
2058
+ "grad_norm": 0.006001957226544619,
2059
+ "learning_rate": 8.979591836734695e-06,
2060
+ "loss": 0.0003,
2061
+ "step": 293
2062
+ },
2063
+ {
2064
+ "epoch": 3.0,
2065
+ "grad_norm": 0.007597202900797129,
2066
+ "learning_rate": 8.934240362811792e-06,
2067
+ "loss": 0.0002,
2068
+ "step": 294
2069
+ },
2070
+ {
2071
+ "epoch": 3.010204081632653,
2072
+ "grad_norm": 0.003272986738011241,
2073
+ "learning_rate": 8.888888888888888e-06,
2074
+ "loss": 0.0002,
2075
+ "step": 295
2076
+ },
2077
+ {
2078
+ "epoch": 3.020408163265306,
2079
+ "grad_norm": 0.0023373092990368605,
2080
+ "learning_rate": 8.843537414965987e-06,
2081
+ "loss": 0.0001,
2082
+ "step": 296
2083
+ },
2084
+ {
2085
+ "epoch": 3.0306122448979593,
2086
+ "grad_norm": 0.0037628381978720427,
2087
+ "learning_rate": 8.798185941043085e-06,
2088
+ "loss": 0.0002,
2089
+ "step": 297
2090
+ },
2091
+ {
2092
+ "epoch": 3.0408163265306123,
2093
+ "grad_norm": 0.011344632133841515,
2094
+ "learning_rate": 8.752834467120183e-06,
2095
+ "loss": 0.0005,
2096
+ "step": 298
2097
+ },
2098
+ {
2099
+ "epoch": 3.0510204081632653,
2100
+ "grad_norm": 0.009169838391244411,
2101
+ "learning_rate": 8.70748299319728e-06,
2102
+ "loss": 0.0004,
2103
+ "step": 299
2104
+ },
2105
+ {
2106
+ "epoch": 3.061224489795918,
2107
+ "grad_norm": 0.019571438431739807,
2108
+ "learning_rate": 8.662131519274378e-06,
2109
+ "loss": 0.0005,
2110
+ "step": 300
2111
+ },
2112
+ {
2113
+ "epoch": 3.0714285714285716,
2114
+ "grad_norm": 0.0035050984006375074,
2115
+ "learning_rate": 8.616780045351474e-06,
2116
+ "loss": 0.0002,
2117
+ "step": 301
2118
+ },
2119
+ {
2120
+ "epoch": 3.0816326530612246,
2121
+ "grad_norm": 0.004051654599606991,
2122
+ "learning_rate": 8.571428571428571e-06,
2123
+ "loss": 0.0002,
2124
+ "step": 302
2125
+ },
2126
+ {
2127
+ "epoch": 3.0918367346938775,
2128
+ "grad_norm": 0.002926639514043927,
2129
+ "learning_rate": 8.52607709750567e-06,
2130
+ "loss": 0.0002,
2131
+ "step": 303
2132
+ },
2133
+ {
2134
+ "epoch": 3.1020408163265305,
2135
+ "grad_norm": 0.013055351562798023,
2136
+ "learning_rate": 8.480725623582767e-06,
2137
+ "loss": 0.0004,
2138
+ "step": 304
2139
+ },
2140
+ {
2141
+ "epoch": 3.1122448979591835,
2142
+ "grad_norm": 0.004692048765718937,
2143
+ "learning_rate": 8.435374149659866e-06,
2144
+ "loss": 0.0002,
2145
+ "step": 305
2146
+ },
2147
+ {
2148
+ "epoch": 3.122448979591837,
2149
+ "grad_norm": 0.0025202229153364897,
2150
+ "learning_rate": 8.390022675736962e-06,
2151
+ "loss": 0.0002,
2152
+ "step": 306
2153
+ },
2154
+ {
2155
+ "epoch": 3.13265306122449,
2156
+ "grad_norm": 0.005598872900009155,
2157
+ "learning_rate": 8.34467120181406e-06,
2158
+ "loss": 0.0002,
2159
+ "step": 307
2160
+ },
2161
+ {
2162
+ "epoch": 3.142857142857143,
2163
+ "grad_norm": 0.0032469748985022306,
2164
+ "learning_rate": 8.299319727891157e-06,
2165
+ "loss": 0.0002,
2166
+ "step": 308
2167
+ },
2168
+ {
2169
+ "epoch": 3.1530612244897958,
2170
+ "grad_norm": 0.004803687799721956,
2171
+ "learning_rate": 8.253968253968254e-06,
2172
+ "loss": 0.0003,
2173
+ "step": 309
2174
+ },
2175
+ {
2176
+ "epoch": 3.163265306122449,
2177
+ "grad_norm": 0.0046676271595060825,
2178
+ "learning_rate": 8.208616780045352e-06,
2179
+ "loss": 0.0003,
2180
+ "step": 310
2181
+ },
2182
+ {
2183
+ "epoch": 3.173469387755102,
2184
+ "grad_norm": 0.002468443475663662,
2185
+ "learning_rate": 8.16326530612245e-06,
2186
+ "loss": 0.0002,
2187
+ "step": 311
2188
+ },
2189
+ {
2190
+ "epoch": 3.183673469387755,
2191
+ "grad_norm": 0.006342902779579163,
2192
+ "learning_rate": 8.117913832199548e-06,
2193
+ "loss": 0.0004,
2194
+ "step": 312
2195
+ },
2196
+ {
2197
+ "epoch": 3.193877551020408,
2198
+ "grad_norm": 0.0023443913087248802,
2199
+ "learning_rate": 8.072562358276645e-06,
2200
+ "loss": 0.0002,
2201
+ "step": 313
2202
+ },
2203
+ {
2204
+ "epoch": 3.204081632653061,
2205
+ "grad_norm": 0.0020017994102090597,
2206
+ "learning_rate": 8.027210884353741e-06,
2207
+ "loss": 0.0001,
2208
+ "step": 314
2209
+ },
2210
+ {
2211
+ "epoch": 3.2142857142857144,
2212
+ "grad_norm": 0.0026365304365754128,
2213
+ "learning_rate": 7.98185941043084e-06,
2214
+ "loss": 0.0002,
2215
+ "step": 315
2216
+ },
2217
+ {
2218
+ "epoch": 3.2244897959183674,
2219
+ "grad_norm": 0.0056705656461417675,
2220
+ "learning_rate": 7.936507936507936e-06,
2221
+ "loss": 0.0004,
2222
+ "step": 316
2223
+ },
2224
+ {
2225
+ "epoch": 3.2346938775510203,
2226
+ "grad_norm": 0.009689562022686005,
2227
+ "learning_rate": 7.891156462585034e-06,
2228
+ "loss": 0.0004,
2229
+ "step": 317
2230
+ },
2231
+ {
2232
+ "epoch": 3.2448979591836733,
2233
+ "grad_norm": 0.008008199743926525,
2234
+ "learning_rate": 7.845804988662133e-06,
2235
+ "loss": 0.0003,
2236
+ "step": 318
2237
+ },
2238
+ {
2239
+ "epoch": 3.2551020408163267,
2240
+ "grad_norm": 0.0026869464199990034,
2241
+ "learning_rate": 7.800453514739231e-06,
2242
+ "loss": 0.0002,
2243
+ "step": 319
2244
+ },
2245
+ {
2246
+ "epoch": 3.2653061224489797,
2247
+ "grad_norm": 0.0031625712290406227,
2248
+ "learning_rate": 7.755102040816327e-06,
2249
+ "loss": 0.0002,
2250
+ "step": 320
2251
+ },
2252
+ {
2253
+ "epoch": 3.2755102040816326,
2254
+ "grad_norm": 0.0028163609094917774,
2255
+ "learning_rate": 7.709750566893424e-06,
2256
+ "loss": 0.0002,
2257
+ "step": 321
2258
+ },
2259
+ {
2260
+ "epoch": 3.2857142857142856,
2261
+ "grad_norm": 0.005660755559802055,
2262
+ "learning_rate": 7.664399092970522e-06,
2263
+ "loss": 0.0003,
2264
+ "step": 322
2265
+ },
2266
+ {
2267
+ "epoch": 3.295918367346939,
2268
+ "grad_norm": 0.0039995694532990456,
2269
+ "learning_rate": 7.61904761904762e-06,
2270
+ "loss": 0.0003,
2271
+ "step": 323
2272
+ },
2273
+ {
2274
+ "epoch": 3.306122448979592,
2275
+ "grad_norm": 0.005670357029885054,
2276
+ "learning_rate": 7.573696145124717e-06,
2277
+ "loss": 0.0003,
2278
+ "step": 324
2279
+ },
2280
+ {
2281
+ "epoch": 3.316326530612245,
2282
+ "grad_norm": 0.0025813328102231026,
2283
+ "learning_rate": 7.528344671201815e-06,
2284
+ "loss": 0.0002,
2285
+ "step": 325
2286
+ },
2287
+ {
2288
+ "epoch": 3.326530612244898,
2289
+ "grad_norm": 0.0030447246972471476,
2290
+ "learning_rate": 7.482993197278913e-06,
2291
+ "loss": 0.0002,
2292
+ "step": 326
2293
+ },
2294
+ {
2295
+ "epoch": 3.336734693877551,
2296
+ "grad_norm": 0.0019142641685903072,
2297
+ "learning_rate": 7.437641723356009e-06,
2298
+ "loss": 0.0001,
2299
+ "step": 327
2300
+ },
2301
+ {
2302
+ "epoch": 3.3469387755102042,
2303
+ "grad_norm": 0.004176548682153225,
2304
+ "learning_rate": 7.3922902494331075e-06,
2305
+ "loss": 0.0002,
2306
+ "step": 328
2307
+ },
2308
+ {
2309
+ "epoch": 3.357142857142857,
2310
+ "grad_norm": 0.009249200113117695,
2311
+ "learning_rate": 7.346938775510205e-06,
2312
+ "loss": 0.0004,
2313
+ "step": 329
2314
+ },
2315
+ {
2316
+ "epoch": 3.36734693877551,
2317
+ "grad_norm": 0.005048077553510666,
2318
+ "learning_rate": 7.301587301587301e-06,
2319
+ "loss": 0.0002,
2320
+ "step": 330
2321
+ },
2322
+ {
2323
+ "epoch": 3.377551020408163,
2324
+ "grad_norm": 0.0024696551263332367,
2325
+ "learning_rate": 7.2562358276644e-06,
2326
+ "loss": 0.0002,
2327
+ "step": 331
2328
+ },
2329
+ {
2330
+ "epoch": 3.387755102040816,
2331
+ "grad_norm": 0.002270912518724799,
2332
+ "learning_rate": 7.210884353741497e-06,
2333
+ "loss": 0.0002,
2334
+ "step": 332
2335
+ },
2336
+ {
2337
+ "epoch": 3.3979591836734695,
2338
+ "grad_norm": 0.0020916208159178495,
2339
+ "learning_rate": 7.165532879818595e-06,
2340
+ "loss": 0.0001,
2341
+ "step": 333
2342
+ },
2343
+ {
2344
+ "epoch": 3.4081632653061225,
2345
+ "grad_norm": 0.002925699343904853,
2346
+ "learning_rate": 7.120181405895692e-06,
2347
+ "loss": 0.0002,
2348
+ "step": 334
2349
+ },
2350
+ {
2351
+ "epoch": 3.4183673469387754,
2352
+ "grad_norm": 0.003520503407344222,
2353
+ "learning_rate": 7.07482993197279e-06,
2354
+ "loss": 0.0002,
2355
+ "step": 335
2356
+ },
2357
+ {
2358
+ "epoch": 3.4285714285714284,
2359
+ "grad_norm": 0.0024117021821439266,
2360
+ "learning_rate": 7.0294784580498875e-06,
2361
+ "loss": 0.0001,
2362
+ "step": 336
2363
+ },
2364
+ {
2365
+ "epoch": 3.438775510204082,
2366
+ "grad_norm": 0.011653084307909012,
2367
+ "learning_rate": 6.984126984126984e-06,
2368
+ "loss": 0.0005,
2369
+ "step": 337
2370
+ },
2371
+ {
2372
+ "epoch": 3.4489795918367347,
2373
+ "grad_norm": 0.0021838736720383167,
2374
+ "learning_rate": 6.938775510204082e-06,
2375
+ "loss": 0.0001,
2376
+ "step": 338
2377
+ },
2378
+ {
2379
+ "epoch": 3.4591836734693877,
2380
+ "grad_norm": 0.005142625421285629,
2381
+ "learning_rate": 6.89342403628118e-06,
2382
+ "loss": 0.0003,
2383
+ "step": 339
2384
+ },
2385
+ {
2386
+ "epoch": 3.4693877551020407,
2387
+ "grad_norm": 0.009561867453157902,
2388
+ "learning_rate": 6.848072562358277e-06,
2389
+ "loss": 0.0003,
2390
+ "step": 340
2391
+ },
2392
+ {
2393
+ "epoch": 3.479591836734694,
2394
+ "grad_norm": 0.0029086614958941936,
2395
+ "learning_rate": 6.8027210884353745e-06,
2396
+ "loss": 0.0002,
2397
+ "step": 341
2398
+ },
2399
+ {
2400
+ "epoch": 3.489795918367347,
2401
+ "grad_norm": 0.004574024584144354,
2402
+ "learning_rate": 6.757369614512473e-06,
2403
+ "loss": 0.0002,
2404
+ "step": 342
2405
+ },
2406
+ {
2407
+ "epoch": 3.5,
2408
+ "grad_norm": 0.002138937823474407,
2409
+ "learning_rate": 6.71201814058957e-06,
2410
+ "loss": 0.0001,
2411
+ "step": 343
2412
+ },
2413
+ {
2414
+ "epoch": 3.510204081632653,
2415
+ "grad_norm": 0.003517791396006942,
2416
+ "learning_rate": 6.666666666666667e-06,
2417
+ "loss": 0.0002,
2418
+ "step": 344
2419
+ },
2420
+ {
2421
+ "epoch": 3.520408163265306,
2422
+ "grad_norm": 0.03443054482340813,
2423
+ "learning_rate": 6.621315192743765e-06,
2424
+ "loss": 0.0008,
2425
+ "step": 345
2426
+ },
2427
+ {
2428
+ "epoch": 3.5306122448979593,
2429
+ "grad_norm": 0.008042026311159134,
2430
+ "learning_rate": 6.575963718820862e-06,
2431
+ "loss": 0.0002,
2432
+ "step": 346
2433
+ },
2434
+ {
2435
+ "epoch": 3.5408163265306123,
2436
+ "grad_norm": 0.0047872308641672134,
2437
+ "learning_rate": 6.530612244897959e-06,
2438
+ "loss": 0.0003,
2439
+ "step": 347
2440
+ },
2441
+ {
2442
+ "epoch": 3.5510204081632653,
2443
+ "grad_norm": 0.010120042599737644,
2444
+ "learning_rate": 6.485260770975057e-06,
2445
+ "loss": 0.0003,
2446
+ "step": 348
2447
+ },
2448
+ {
2449
+ "epoch": 3.561224489795918,
2450
+ "grad_norm": 0.004412388429045677,
2451
+ "learning_rate": 6.4399092970521545e-06,
2452
+ "loss": 0.0003,
2453
+ "step": 349
2454
+ },
2455
+ {
2456
+ "epoch": 3.571428571428571,
2457
+ "grad_norm": 0.003939002752304077,
2458
+ "learning_rate": 6.394557823129253e-06,
2459
+ "loss": 0.0002,
2460
+ "step": 350
2461
+ },
2462
+ {
2463
+ "epoch": 3.5816326530612246,
2464
+ "grad_norm": 0.003072823630645871,
2465
+ "learning_rate": 6.349206349206349e-06,
2466
+ "loss": 0.0002,
2467
+ "step": 351
2468
+ },
2469
+ {
2470
+ "epoch": 3.5918367346938775,
2471
+ "grad_norm": 0.002594695193693042,
2472
+ "learning_rate": 6.3038548752834475e-06,
2473
+ "loss": 0.0002,
2474
+ "step": 352
2475
+ },
2476
+ {
2477
+ "epoch": 3.6020408163265305,
2478
+ "grad_norm": 0.0021781930699944496,
2479
+ "learning_rate": 6.258503401360545e-06,
2480
+ "loss": 0.0001,
2481
+ "step": 353
2482
+ },
2483
+ {
2484
+ "epoch": 3.612244897959184,
2485
+ "grad_norm": 0.004957903642207384,
2486
+ "learning_rate": 6.2131519274376415e-06,
2487
+ "loss": 0.0002,
2488
+ "step": 354
2489
+ },
2490
+ {
2491
+ "epoch": 3.622448979591837,
2492
+ "grad_norm": 0.0020664699841290712,
2493
+ "learning_rate": 6.16780045351474e-06,
2494
+ "loss": 0.0001,
2495
+ "step": 355
2496
+ },
2497
+ {
2498
+ "epoch": 3.63265306122449,
2499
+ "grad_norm": 0.00455419672653079,
2500
+ "learning_rate": 6.122448979591837e-06,
2501
+ "loss": 0.0002,
2502
+ "step": 356
2503
+ },
2504
+ {
2505
+ "epoch": 3.642857142857143,
2506
+ "grad_norm": 0.001891249674372375,
2507
+ "learning_rate": 6.077097505668935e-06,
2508
+ "loss": 0.0001,
2509
+ "step": 357
2510
+ },
2511
+ {
2512
+ "epoch": 3.6530612244897958,
2513
+ "grad_norm": 0.0015174165600910783,
2514
+ "learning_rate": 6.031746031746032e-06,
2515
+ "loss": 0.0001,
2516
+ "step": 358
2517
+ },
2518
+ {
2519
+ "epoch": 3.663265306122449,
2520
+ "grad_norm": 0.008895975537598133,
2521
+ "learning_rate": 5.98639455782313e-06,
2522
+ "loss": 0.0003,
2523
+ "step": 359
2524
+ },
2525
+ {
2526
+ "epoch": 3.673469387755102,
2527
+ "grad_norm": 0.010570279322564602,
2528
+ "learning_rate": 5.9410430839002275e-06,
2529
+ "loss": 0.0003,
2530
+ "step": 360
2531
+ },
2532
+ {
2533
+ "epoch": 3.683673469387755,
2534
+ "grad_norm": 0.005755205638706684,
2535
+ "learning_rate": 5.895691609977324e-06,
2536
+ "loss": 0.0002,
2537
+ "step": 361
2538
+ },
2539
+ {
2540
+ "epoch": 3.693877551020408,
2541
+ "grad_norm": 0.00319477915763855,
2542
+ "learning_rate": 5.850340136054422e-06,
2543
+ "loss": 0.0002,
2544
+ "step": 362
2545
+ },
2546
+ {
2547
+ "epoch": 3.704081632653061,
2548
+ "grad_norm": 0.0023295124992728233,
2549
+ "learning_rate": 5.80498866213152e-06,
2550
+ "loss": 0.0001,
2551
+ "step": 363
2552
+ },
2553
+ {
2554
+ "epoch": 3.7142857142857144,
2555
+ "grad_norm": 0.0038169752806425095,
2556
+ "learning_rate": 5.759637188208618e-06,
2557
+ "loss": 0.0003,
2558
+ "step": 364
2559
+ },
2560
+ {
2561
+ "epoch": 3.7244897959183674,
2562
+ "grad_norm": 0.007799374870955944,
2563
+ "learning_rate": 5.7142857142857145e-06,
2564
+ "loss": 0.0003,
2565
+ "step": 365
2566
+ },
2567
+ {
2568
+ "epoch": 3.7346938775510203,
2569
+ "grad_norm": 0.002488058526068926,
2570
+ "learning_rate": 5.668934240362812e-06,
2571
+ "loss": 0.0002,
2572
+ "step": 366
2573
+ },
2574
+ {
2575
+ "epoch": 3.7448979591836737,
2576
+ "grad_norm": 0.01512609887868166,
2577
+ "learning_rate": 5.62358276643991e-06,
2578
+ "loss": 0.0006,
2579
+ "step": 367
2580
+ },
2581
+ {
2582
+ "epoch": 3.7551020408163263,
2583
+ "grad_norm": 0.004572188016027212,
2584
+ "learning_rate": 5.578231292517007e-06,
2585
+ "loss": 0.0003,
2586
+ "step": 368
2587
+ },
2588
+ {
2589
+ "epoch": 3.7653061224489797,
2590
+ "grad_norm": 0.0024051195941865444,
2591
+ "learning_rate": 5.532879818594105e-06,
2592
+ "loss": 0.0002,
2593
+ "step": 369
2594
+ },
2595
+ {
2596
+ "epoch": 3.7755102040816326,
2597
+ "grad_norm": 0.0032509195152670145,
2598
+ "learning_rate": 5.487528344671202e-06,
2599
+ "loss": 0.0002,
2600
+ "step": 370
2601
+ },
2602
+ {
2603
+ "epoch": 3.7857142857142856,
2604
+ "grad_norm": 0.0019066549139097333,
2605
+ "learning_rate": 5.442176870748301e-06,
2606
+ "loss": 0.0001,
2607
+ "step": 371
2608
+ },
2609
+ {
2610
+ "epoch": 3.795918367346939,
2611
+ "grad_norm": 0.004059778060764074,
2612
+ "learning_rate": 5.396825396825397e-06,
2613
+ "loss": 0.0002,
2614
+ "step": 372
2615
+ },
2616
+ {
2617
+ "epoch": 3.806122448979592,
2618
+ "grad_norm": 0.003823889186605811,
2619
+ "learning_rate": 5.3514739229024945e-06,
2620
+ "loss": 0.0002,
2621
+ "step": 373
2622
+ },
2623
+ {
2624
+ "epoch": 3.816326530612245,
2625
+ "grad_norm": 0.005696111358702183,
2626
+ "learning_rate": 5.306122448979593e-06,
2627
+ "loss": 0.0003,
2628
+ "step": 374
2629
+ },
2630
+ {
2631
+ "epoch": 3.826530612244898,
2632
+ "grad_norm": 0.002276304177939892,
2633
+ "learning_rate": 5.260770975056689e-06,
2634
+ "loss": 0.0001,
2635
+ "step": 375
2636
+ },
2637
+ {
2638
+ "epoch": 3.836734693877551,
2639
+ "grad_norm": 0.003423569956794381,
2640
+ "learning_rate": 5.2154195011337876e-06,
2641
+ "loss": 0.0002,
2642
+ "step": 376
2643
+ },
2644
+ {
2645
+ "epoch": 3.8469387755102042,
2646
+ "grad_norm": 0.009261609055101871,
2647
+ "learning_rate": 5.170068027210885e-06,
2648
+ "loss": 0.0004,
2649
+ "step": 377
2650
+ },
2651
+ {
2652
+ "epoch": 3.857142857142857,
2653
+ "grad_norm": 0.0026830616407096386,
2654
+ "learning_rate": 5.124716553287983e-06,
2655
+ "loss": 0.0002,
2656
+ "step": 378
2657
+ },
2658
+ {
2659
+ "epoch": 3.86734693877551,
2660
+ "grad_norm": 0.007292145863175392,
2661
+ "learning_rate": 5.07936507936508e-06,
2662
+ "loss": 0.0003,
2663
+ "step": 379
2664
+ },
2665
+ {
2666
+ "epoch": 3.877551020408163,
2667
+ "grad_norm": 0.001463556895032525,
2668
+ "learning_rate": 5.034013605442177e-06,
2669
+ "loss": 0.0001,
2670
+ "step": 380
2671
+ },
2672
+ {
2673
+ "epoch": 3.887755102040816,
2674
+ "grad_norm": 0.0077773998491466045,
2675
+ "learning_rate": 4.9886621315192745e-06,
2676
+ "loss": 0.0003,
2677
+ "step": 381
2678
+ },
2679
+ {
2680
+ "epoch": 3.8979591836734695,
2681
+ "grad_norm": 0.001333568710833788,
2682
+ "learning_rate": 4.943310657596373e-06,
2683
+ "loss": 0.0001,
2684
+ "step": 382
2685
+ },
2686
+ {
2687
+ "epoch": 3.9081632653061225,
2688
+ "grad_norm": 0.0033744387328624725,
2689
+ "learning_rate": 4.897959183673469e-06,
2690
+ "loss": 0.0002,
2691
+ "step": 383
2692
+ },
2693
+ {
2694
+ "epoch": 3.9183673469387754,
2695
+ "grad_norm": 0.0031404553446918726,
2696
+ "learning_rate": 4.852607709750568e-06,
2697
+ "loss": 0.0002,
2698
+ "step": 384
2699
+ },
2700
+ {
2701
+ "epoch": 3.928571428571429,
2702
+ "grad_norm": 0.002246819669380784,
2703
+ "learning_rate": 4.807256235827665e-06,
2704
+ "loss": 0.0002,
2705
+ "step": 385
2706
+ },
2707
+ {
2708
+ "epoch": 3.938775510204082,
2709
+ "grad_norm": 0.006392229348421097,
2710
+ "learning_rate": 4.761904761904762e-06,
2711
+ "loss": 0.0003,
2712
+ "step": 386
2713
+ },
2714
+ {
2715
+ "epoch": 3.9489795918367347,
2716
+ "grad_norm": 0.002482037292793393,
2717
+ "learning_rate": 4.71655328798186e-06,
2718
+ "loss": 0.0002,
2719
+ "step": 387
2720
+ },
2721
+ {
2722
+ "epoch": 3.9591836734693877,
2723
+ "grad_norm": 0.0029472103342413902,
2724
+ "learning_rate": 4.671201814058957e-06,
2725
+ "loss": 0.0002,
2726
+ "step": 388
2727
+ },
2728
+ {
2729
+ "epoch": 3.9693877551020407,
2730
+ "grad_norm": 0.001341557246632874,
2731
+ "learning_rate": 4.6258503401360546e-06,
2732
+ "loss": 0.0001,
2733
+ "step": 389
2734
+ },
2735
+ {
2736
+ "epoch": 3.979591836734694,
2737
+ "grad_norm": 0.003789098234847188,
2738
+ "learning_rate": 4.580498866213152e-06,
2739
+ "loss": 0.0002,
2740
+ "step": 390
2741
+ },
2742
+ {
2743
+ "epoch": 3.989795918367347,
2744
+ "grad_norm": 0.0022696068044751883,
2745
+ "learning_rate": 4.53514739229025e-06,
2746
+ "loss": 0.0001,
2747
+ "step": 391
2748
+ },
2749
+ {
2750
+ "epoch": 4.0,
2751
+ "grad_norm": 0.0036536771804094315,
2752
+ "learning_rate": 4.489795918367348e-06,
2753
+ "loss": 0.0001,
2754
+ "step": 392
2755
+ },
2756
+ {
2757
+ "epoch": 4.010204081632653,
2758
+ "grad_norm": 0.0017391832079738379,
2759
+ "learning_rate": 4.444444444444444e-06,
2760
+ "loss": 0.0001,
2761
+ "step": 393
2762
+ },
2763
+ {
2764
+ "epoch": 4.020408163265306,
2765
+ "grad_norm": 0.003093178616836667,
2766
+ "learning_rate": 4.399092970521542e-06,
2767
+ "loss": 0.0002,
2768
+ "step": 394
2769
+ },
2770
+ {
2771
+ "epoch": 4.030612244897959,
2772
+ "grad_norm": 0.0017501730471849442,
2773
+ "learning_rate": 4.35374149659864e-06,
2774
+ "loss": 0.0001,
2775
+ "step": 395
2776
+ },
2777
+ {
2778
+ "epoch": 4.040816326530612,
2779
+ "grad_norm": 0.03281351551413536,
2780
+ "learning_rate": 4.308390022675737e-06,
2781
+ "loss": 0.0007,
2782
+ "step": 396
2783
+ },
2784
+ {
2785
+ "epoch": 4.051020408163265,
2786
+ "grad_norm": 0.002625512657687068,
2787
+ "learning_rate": 4.263038548752835e-06,
2788
+ "loss": 0.0002,
2789
+ "step": 397
2790
+ },
2791
+ {
2792
+ "epoch": 4.061224489795919,
2793
+ "grad_norm": 0.003758464241400361,
2794
+ "learning_rate": 4.217687074829933e-06,
2795
+ "loss": 0.0002,
2796
+ "step": 398
2797
+ },
2798
+ {
2799
+ "epoch": 4.071428571428571,
2800
+ "grad_norm": 0.0021065385080873966,
2801
+ "learning_rate": 4.17233560090703e-06,
2802
+ "loss": 0.0001,
2803
+ "step": 399
2804
+ },
2805
+ {
2806
+ "epoch": 4.081632653061225,
2807
+ "grad_norm": 0.0022317173425108194,
2808
+ "learning_rate": 4.126984126984127e-06,
2809
+ "loss": 0.0001,
2810
+ "step": 400
2811
+ },
2812
+ {
2813
+ "epoch": 4.091836734693878,
2814
+ "grad_norm": 0.0025758843403309584,
2815
+ "learning_rate": 4.081632653061225e-06,
2816
+ "loss": 0.0002,
2817
+ "step": 401
2818
+ },
2819
+ {
2820
+ "epoch": 4.1020408163265305,
2821
+ "grad_norm": 0.003262228099629283,
2822
+ "learning_rate": 4.036281179138322e-06,
2823
+ "loss": 0.0002,
2824
+ "step": 402
2825
+ },
2826
+ {
2827
+ "epoch": 4.112244897959184,
2828
+ "grad_norm": 0.002355805365368724,
2829
+ "learning_rate": 3.99092970521542e-06,
2830
+ "loss": 0.0001,
2831
+ "step": 403
2832
+ },
2833
+ {
2834
+ "epoch": 4.122448979591836,
2835
+ "grad_norm": 0.002239174908027053,
2836
+ "learning_rate": 3.945578231292517e-06,
2837
+ "loss": 0.0001,
2838
+ "step": 404
2839
+ },
2840
+ {
2841
+ "epoch": 4.13265306122449,
2842
+ "grad_norm": 0.003173491917550564,
2843
+ "learning_rate": 3.9002267573696154e-06,
2844
+ "loss": 0.0002,
2845
+ "step": 405
2846
+ },
2847
+ {
2848
+ "epoch": 4.142857142857143,
2849
+ "grad_norm": 0.009472887963056564,
2850
+ "learning_rate": 3.854875283446712e-06,
2851
+ "loss": 0.0004,
2852
+ "step": 406
2853
+ },
2854
+ {
2855
+ "epoch": 4.153061224489796,
2856
+ "grad_norm": 0.010682443156838417,
2857
+ "learning_rate": 3.80952380952381e-06,
2858
+ "loss": 0.0004,
2859
+ "step": 407
2860
+ },
2861
+ {
2862
+ "epoch": 4.163265306122449,
2863
+ "grad_norm": 0.01789182610809803,
2864
+ "learning_rate": 3.7641723356009076e-06,
2865
+ "loss": 0.0006,
2866
+ "step": 408
2867
+ },
2868
+ {
2869
+ "epoch": 4.173469387755102,
2870
+ "grad_norm": 0.002530967351049185,
2871
+ "learning_rate": 3.7188208616780046e-06,
2872
+ "loss": 0.0001,
2873
+ "step": 409
2874
+ },
2875
+ {
2876
+ "epoch": 4.183673469387755,
2877
+ "grad_norm": 0.0029371839482337236,
2878
+ "learning_rate": 3.6734693877551024e-06,
2879
+ "loss": 0.0002,
2880
+ "step": 410
2881
+ },
2882
+ {
2883
+ "epoch": 4.1938775510204085,
2884
+ "grad_norm": 0.004367890767753124,
2885
+ "learning_rate": 3.6281179138322e-06,
2886
+ "loss": 0.0002,
2887
+ "step": 411
2888
+ },
2889
+ {
2890
+ "epoch": 4.204081632653061,
2891
+ "grad_norm": 0.0021538427099585533,
2892
+ "learning_rate": 3.5827664399092976e-06,
2893
+ "loss": 0.0001,
2894
+ "step": 412
2895
+ },
2896
+ {
2897
+ "epoch": 4.214285714285714,
2898
+ "grad_norm": 0.0021221789065748453,
2899
+ "learning_rate": 3.537414965986395e-06,
2900
+ "loss": 0.0001,
2901
+ "step": 413
2902
+ },
2903
+ {
2904
+ "epoch": 4.224489795918367,
2905
+ "grad_norm": 0.0021122246980667114,
2906
+ "learning_rate": 3.492063492063492e-06,
2907
+ "loss": 0.0001,
2908
+ "step": 414
2909
+ },
2910
+ {
2911
+ "epoch": 4.23469387755102,
2912
+ "grad_norm": 0.0025512792635709047,
2913
+ "learning_rate": 3.44671201814059e-06,
2914
+ "loss": 0.0001,
2915
+ "step": 415
2916
+ },
2917
+ {
2918
+ "epoch": 4.244897959183674,
2919
+ "grad_norm": 0.0041538686491549015,
2920
+ "learning_rate": 3.4013605442176872e-06,
2921
+ "loss": 0.0003,
2922
+ "step": 416
2923
+ },
2924
+ {
2925
+ "epoch": 4.255102040816326,
2926
+ "grad_norm": 0.0017510091420263052,
2927
+ "learning_rate": 3.356009070294785e-06,
2928
+ "loss": 0.0001,
2929
+ "step": 417
2930
+ },
2931
+ {
2932
+ "epoch": 4.26530612244898,
2933
+ "grad_norm": 0.0024814323987811804,
2934
+ "learning_rate": 3.3106575963718824e-06,
2935
+ "loss": 0.0002,
2936
+ "step": 418
2937
+ },
2938
+ {
2939
+ "epoch": 4.275510204081632,
2940
+ "grad_norm": 0.00235186074860394,
2941
+ "learning_rate": 3.2653061224489794e-06,
2942
+ "loss": 0.0001,
2943
+ "step": 419
2944
+ },
2945
+ {
2946
+ "epoch": 4.285714285714286,
2947
+ "grad_norm": 0.0030012091156095266,
2948
+ "learning_rate": 3.2199546485260772e-06,
2949
+ "loss": 0.0002,
2950
+ "step": 420
2951
+ },
2952
+ {
2953
+ "epoch": 4.295918367346939,
2954
+ "grad_norm": 0.004921985324472189,
2955
+ "learning_rate": 3.1746031746031746e-06,
2956
+ "loss": 0.0003,
2957
+ "step": 421
2958
+ },
2959
+ {
2960
+ "epoch": 4.3061224489795915,
2961
+ "grad_norm": 0.009844960644841194,
2962
+ "learning_rate": 3.1292517006802725e-06,
2963
+ "loss": 0.0004,
2964
+ "step": 422
2965
+ },
2966
+ {
2967
+ "epoch": 4.316326530612245,
2968
+ "grad_norm": 0.003105542156845331,
2969
+ "learning_rate": 3.08390022675737e-06,
2970
+ "loss": 0.0002,
2971
+ "step": 423
2972
+ },
2973
+ {
2974
+ "epoch": 4.326530612244898,
2975
+ "grad_norm": 0.005888419691473246,
2976
+ "learning_rate": 3.0385487528344677e-06,
2977
+ "loss": 0.0003,
2978
+ "step": 424
2979
+ },
2980
+ {
2981
+ "epoch": 4.336734693877551,
2982
+ "grad_norm": 0.002076453994959593,
2983
+ "learning_rate": 2.993197278911565e-06,
2984
+ "loss": 0.0001,
2985
+ "step": 425
2986
+ },
2987
+ {
2988
+ "epoch": 4.346938775510204,
2989
+ "grad_norm": 0.0016607132274657488,
2990
+ "learning_rate": 2.947845804988662e-06,
2991
+ "loss": 0.0001,
2992
+ "step": 426
2993
+ },
2994
+ {
2995
+ "epoch": 4.357142857142857,
2996
+ "grad_norm": 0.0024275570176541805,
2997
+ "learning_rate": 2.90249433106576e-06,
2998
+ "loss": 0.0002,
2999
+ "step": 427
3000
+ },
3001
+ {
3002
+ "epoch": 4.36734693877551,
3003
+ "grad_norm": 0.003902298165485263,
3004
+ "learning_rate": 2.8571428571428573e-06,
3005
+ "loss": 0.0002,
3006
+ "step": 428
3007
+ },
3008
+ {
3009
+ "epoch": 4.377551020408164,
3010
+ "grad_norm": 0.0023378883488476276,
3011
+ "learning_rate": 2.811791383219955e-06,
3012
+ "loss": 0.0002,
3013
+ "step": 429
3014
+ },
3015
+ {
3016
+ "epoch": 4.387755102040816,
3017
+ "grad_norm": 0.0051103937439620495,
3018
+ "learning_rate": 2.7664399092970525e-06,
3019
+ "loss": 0.0002,
3020
+ "step": 430
3021
+ },
3022
+ {
3023
+ "epoch": 4.3979591836734695,
3024
+ "grad_norm": 0.0026863350067287683,
3025
+ "learning_rate": 2.7210884353741503e-06,
3026
+ "loss": 0.0002,
3027
+ "step": 431
3028
+ },
3029
+ {
3030
+ "epoch": 4.408163265306122,
3031
+ "grad_norm": 0.002001287881284952,
3032
+ "learning_rate": 2.6757369614512473e-06,
3033
+ "loss": 0.0001,
3034
+ "step": 432
3035
+ },
3036
+ {
3037
+ "epoch": 4.418367346938775,
3038
+ "grad_norm": 0.008789247833192348,
3039
+ "learning_rate": 2.6303854875283447e-06,
3040
+ "loss": 0.0003,
3041
+ "step": 433
3042
+ },
3043
+ {
3044
+ "epoch": 4.428571428571429,
3045
+ "grad_norm": 0.004610543139278889,
3046
+ "learning_rate": 2.5850340136054425e-06,
3047
+ "loss": 0.0002,
3048
+ "step": 434
3049
+ },
3050
+ {
3051
+ "epoch": 4.438775510204081,
3052
+ "grad_norm": 0.00599480327218771,
3053
+ "learning_rate": 2.53968253968254e-06,
3054
+ "loss": 0.0003,
3055
+ "step": 435
3056
+ },
3057
+ {
3058
+ "epoch": 4.448979591836735,
3059
+ "grad_norm": 0.00554778054356575,
3060
+ "learning_rate": 2.4943310657596373e-06,
3061
+ "loss": 0.0003,
3062
+ "step": 436
3063
+ },
3064
+ {
3065
+ "epoch": 4.459183673469388,
3066
+ "grad_norm": 0.00669802725315094,
3067
+ "learning_rate": 2.4489795918367347e-06,
3068
+ "loss": 0.0003,
3069
+ "step": 437
3070
+ },
3071
+ {
3072
+ "epoch": 4.469387755102041,
3073
+ "grad_norm": 0.0016791113885119557,
3074
+ "learning_rate": 2.4036281179138325e-06,
3075
+ "loss": 0.0001,
3076
+ "step": 438
3077
+ },
3078
+ {
3079
+ "epoch": 4.479591836734694,
3080
+ "grad_norm": 0.0024550866801291704,
3081
+ "learning_rate": 2.35827664399093e-06,
3082
+ "loss": 0.0002,
3083
+ "step": 439
3084
+ },
3085
+ {
3086
+ "epoch": 4.489795918367347,
3087
+ "grad_norm": 0.0024079831782728434,
3088
+ "learning_rate": 2.3129251700680273e-06,
3089
+ "loss": 0.0002,
3090
+ "step": 440
3091
+ },
3092
+ {
3093
+ "epoch": 4.5,
3094
+ "grad_norm": 0.0028663375414907932,
3095
+ "learning_rate": 2.267573696145125e-06,
3096
+ "loss": 0.0002,
3097
+ "step": 441
3098
+ },
3099
+ {
3100
+ "epoch": 4.510204081632653,
3101
+ "grad_norm": 0.008847690187394619,
3102
+ "learning_rate": 2.222222222222222e-06,
3103
+ "loss": 0.0003,
3104
+ "step": 442
3105
+ },
3106
+ {
3107
+ "epoch": 4.520408163265306,
3108
+ "grad_norm": 0.005266358610242605,
3109
+ "learning_rate": 2.17687074829932e-06,
3110
+ "loss": 0.0003,
3111
+ "step": 443
3112
+ },
3113
+ {
3114
+ "epoch": 4.530612244897959,
3115
+ "grad_norm": 0.0024195676669478416,
3116
+ "learning_rate": 2.1315192743764173e-06,
3117
+ "loss": 0.0002,
3118
+ "step": 444
3119
+ },
3120
+ {
3121
+ "epoch": 4.540816326530612,
3122
+ "grad_norm": 0.00421124929562211,
3123
+ "learning_rate": 2.086167800453515e-06,
3124
+ "loss": 0.0002,
3125
+ "step": 445
3126
+ },
3127
+ {
3128
+ "epoch": 4.551020408163265,
3129
+ "grad_norm": 0.0020824300590902567,
3130
+ "learning_rate": 2.0408163265306125e-06,
3131
+ "loss": 0.0001,
3132
+ "step": 446
3133
+ },
3134
+ {
3135
+ "epoch": 4.561224489795919,
3136
+ "grad_norm": 0.005051845218986273,
3137
+ "learning_rate": 1.99546485260771e-06,
3138
+ "loss": 0.0002,
3139
+ "step": 447
3140
+ },
3141
+ {
3142
+ "epoch": 4.571428571428571,
3143
+ "grad_norm": 0.0022977020125836134,
3144
+ "learning_rate": 1.9501133786848077e-06,
3145
+ "loss": 0.0002,
3146
+ "step": 448
3147
+ },
3148
+ {
3149
+ "epoch": 4.581632653061225,
3150
+ "grad_norm": 0.001990046352148056,
3151
+ "learning_rate": 1.904761904761905e-06,
3152
+ "loss": 0.0001,
3153
+ "step": 449
3154
+ },
3155
+ {
3156
+ "epoch": 4.591836734693878,
3157
+ "grad_norm": 0.0031708430033177137,
3158
+ "learning_rate": 1.8594104308390023e-06,
3159
+ "loss": 0.0002,
3160
+ "step": 450
3161
+ },
3162
+ {
3163
+ "epoch": 4.6020408163265305,
3164
+ "grad_norm": 0.0034788285847753286,
3165
+ "learning_rate": 1.8140589569161e-06,
3166
+ "loss": 0.0002,
3167
+ "step": 451
3168
+ },
3169
+ {
3170
+ "epoch": 4.612244897959184,
3171
+ "grad_norm": 0.0018601809861138463,
3172
+ "learning_rate": 1.7687074829931975e-06,
3173
+ "loss": 0.0001,
3174
+ "step": 452
3175
+ },
3176
+ {
3177
+ "epoch": 4.622448979591836,
3178
+ "grad_norm": 0.016590220853686333,
3179
+ "learning_rate": 1.723356009070295e-06,
3180
+ "loss": 0.0003,
3181
+ "step": 453
3182
+ },
3183
+ {
3184
+ "epoch": 4.63265306122449,
3185
+ "grad_norm": 0.003050972009077668,
3186
+ "learning_rate": 1.6780045351473925e-06,
3187
+ "loss": 0.0002,
3188
+ "step": 454
3189
+ },
3190
+ {
3191
+ "epoch": 4.642857142857143,
3192
+ "grad_norm": 0.002196480752900243,
3193
+ "learning_rate": 1.6326530612244897e-06,
3194
+ "loss": 0.0001,
3195
+ "step": 455
3196
+ },
3197
+ {
3198
+ "epoch": 4.653061224489796,
3199
+ "grad_norm": 0.0025891121476888657,
3200
+ "learning_rate": 1.5873015873015873e-06,
3201
+ "loss": 0.0001,
3202
+ "step": 456
3203
+ },
3204
+ {
3205
+ "epoch": 4.663265306122449,
3206
+ "grad_norm": 0.002245939103886485,
3207
+ "learning_rate": 1.541950113378685e-06,
3208
+ "loss": 0.0001,
3209
+ "step": 457
3210
+ },
3211
+ {
3212
+ "epoch": 4.673469387755102,
3213
+ "grad_norm": 0.0021706093102693558,
3214
+ "learning_rate": 1.4965986394557825e-06,
3215
+ "loss": 0.0001,
3216
+ "step": 458
3217
+ },
3218
+ {
3219
+ "epoch": 4.683673469387755,
3220
+ "grad_norm": 0.0034395295660942793,
3221
+ "learning_rate": 1.45124716553288e-06,
3222
+ "loss": 0.0002,
3223
+ "step": 459
3224
+ },
3225
+ {
3226
+ "epoch": 4.6938775510204085,
3227
+ "grad_norm": 0.0023007583804428577,
3228
+ "learning_rate": 1.4058956916099775e-06,
3229
+ "loss": 0.0001,
3230
+ "step": 460
3231
+ },
3232
+ {
3233
+ "epoch": 4.704081632653061,
3234
+ "grad_norm": 0.004494468215852976,
3235
+ "learning_rate": 1.3605442176870751e-06,
3236
+ "loss": 0.0002,
3237
+ "step": 461
3238
+ },
3239
+ {
3240
+ "epoch": 4.714285714285714,
3241
+ "grad_norm": 0.0017052018083631992,
3242
+ "learning_rate": 1.3151927437641723e-06,
3243
+ "loss": 0.0001,
3244
+ "step": 462
3245
+ },
3246
+ {
3247
+ "epoch": 4.724489795918368,
3248
+ "grad_norm": 0.0016002283664420247,
3249
+ "learning_rate": 1.26984126984127e-06,
3250
+ "loss": 0.0001,
3251
+ "step": 463
3252
+ },
3253
+ {
3254
+ "epoch": 4.73469387755102,
3255
+ "grad_norm": 0.0033771556336432695,
3256
+ "learning_rate": 1.2244897959183673e-06,
3257
+ "loss": 0.0002,
3258
+ "step": 464
3259
+ },
3260
+ {
3261
+ "epoch": 4.744897959183674,
3262
+ "grad_norm": 0.002580232685431838,
3263
+ "learning_rate": 1.179138321995465e-06,
3264
+ "loss": 0.0002,
3265
+ "step": 465
3266
+ },
3267
+ {
3268
+ "epoch": 4.755102040816326,
3269
+ "grad_norm": 0.0020106956362724304,
3270
+ "learning_rate": 1.1337868480725626e-06,
3271
+ "loss": 0.0001,
3272
+ "step": 466
3273
+ },
3274
+ {
3275
+ "epoch": 4.76530612244898,
3276
+ "grad_norm": 0.0036402051337063313,
3277
+ "learning_rate": 1.08843537414966e-06,
3278
+ "loss": 0.0002,
3279
+ "step": 467
3280
+ },
3281
+ {
3282
+ "epoch": 4.775510204081632,
3283
+ "grad_norm": 0.004601712804287672,
3284
+ "learning_rate": 1.0430839002267576e-06,
3285
+ "loss": 0.0002,
3286
+ "step": 468
3287
+ },
3288
+ {
3289
+ "epoch": 4.785714285714286,
3290
+ "grad_norm": 0.002733904868364334,
3291
+ "learning_rate": 9.97732426303855e-07,
3292
+ "loss": 0.0002,
3293
+ "step": 469
3294
+ },
3295
+ {
3296
+ "epoch": 4.795918367346939,
3297
+ "grad_norm": 0.0027253010775893927,
3298
+ "learning_rate": 9.523809523809525e-07,
3299
+ "loss": 0.0002,
3300
+ "step": 470
3301
+ },
3302
+ {
3303
+ "epoch": 4.8061224489795915,
3304
+ "grad_norm": 0.02664267271757126,
3305
+ "learning_rate": 9.0702947845805e-07,
3306
+ "loss": 0.0007,
3307
+ "step": 471
3308
+ },
3309
+ {
3310
+ "epoch": 4.816326530612245,
3311
+ "grad_norm": 0.002349977381527424,
3312
+ "learning_rate": 8.616780045351475e-07,
3313
+ "loss": 0.0002,
3314
+ "step": 472
3315
+ },
3316
+ {
3317
+ "epoch": 4.826530612244898,
3318
+ "grad_norm": 0.01735026389360428,
3319
+ "learning_rate": 8.163265306122449e-07,
3320
+ "loss": 0.0006,
3321
+ "step": 473
3322
+ },
3323
+ {
3324
+ "epoch": 4.836734693877551,
3325
+ "grad_norm": 0.003476122161373496,
3326
+ "learning_rate": 7.709750566893425e-07,
3327
+ "loss": 0.0002,
3328
+ "step": 474
3329
+ },
3330
+ {
3331
+ "epoch": 4.846938775510204,
3332
+ "grad_norm": 0.0016622812254354358,
3333
+ "learning_rate": 7.2562358276644e-07,
3334
+ "loss": 0.0001,
3335
+ "step": 475
3336
+ },
3337
+ {
3338
+ "epoch": 4.857142857142857,
3339
+ "grad_norm": 0.0042258999310433865,
3340
+ "learning_rate": 6.802721088435376e-07,
3341
+ "loss": 0.0002,
3342
+ "step": 476
3343
+ },
3344
+ {
3345
+ "epoch": 4.86734693877551,
3346
+ "grad_norm": 0.0023899853695183992,
3347
+ "learning_rate": 6.34920634920635e-07,
3348
+ "loss": 0.0001,
3349
+ "step": 477
3350
+ },
3351
+ {
3352
+ "epoch": 4.877551020408164,
3353
+ "grad_norm": 0.0037495435681194067,
3354
+ "learning_rate": 5.895691609977325e-07,
3355
+ "loss": 0.0002,
3356
+ "step": 478
3357
+ },
3358
+ {
3359
+ "epoch": 4.887755102040816,
3360
+ "grad_norm": 0.00362688978202641,
3361
+ "learning_rate": 5.4421768707483e-07,
3362
+ "loss": 0.0002,
3363
+ "step": 479
3364
+ },
3365
+ {
3366
+ "epoch": 4.8979591836734695,
3367
+ "grad_norm": 0.005572126246988773,
3368
+ "learning_rate": 4.988662131519275e-07,
3369
+ "loss": 0.0003,
3370
+ "step": 480
3371
+ },
3372
+ {
3373
+ "epoch": 4.908163265306122,
3374
+ "grad_norm": 0.0033278209157288074,
3375
+ "learning_rate": 4.53514739229025e-07,
3376
+ "loss": 0.0002,
3377
+ "step": 481
3378
+ },
3379
+ {
3380
+ "epoch": 4.918367346938775,
3381
+ "grad_norm": 0.0017971718916669488,
3382
+ "learning_rate": 4.0816326530612243e-07,
3383
+ "loss": 0.0001,
3384
+ "step": 482
3385
+ },
3386
+ {
3387
+ "epoch": 4.928571428571429,
3388
+ "grad_norm": 0.003246983280405402,
3389
+ "learning_rate": 3.6281179138322e-07,
3390
+ "loss": 0.0002,
3391
+ "step": 483
3392
+ },
3393
+ {
3394
+ "epoch": 4.938775510204081,
3395
+ "grad_norm": 0.002570765558630228,
3396
+ "learning_rate": 3.174603174603175e-07,
3397
+ "loss": 0.0002,
3398
+ "step": 484
3399
+ },
3400
+ {
3401
+ "epoch": 4.948979591836735,
3402
+ "grad_norm": 0.003006896935403347,
3403
+ "learning_rate": 2.72108843537415e-07,
3404
+ "loss": 0.0002,
3405
+ "step": 485
3406
+ },
3407
+ {
3408
+ "epoch": 4.959183673469388,
3409
+ "grad_norm": 0.003843962447717786,
3410
+ "learning_rate": 2.267573696145125e-07,
3411
+ "loss": 0.0002,
3412
+ "step": 486
3413
+ },
3414
+ {
3415
+ "epoch": 4.969387755102041,
3416
+ "grad_norm": 0.005520265083760023,
3417
+ "learning_rate": 1.8140589569161e-07,
3418
+ "loss": 0.0002,
3419
+ "step": 487
3420
+ },
3421
+ {
3422
+ "epoch": 4.979591836734694,
3423
+ "grad_norm": 0.0035644182935357094,
3424
+ "learning_rate": 1.360544217687075e-07,
3425
+ "loss": 0.0002,
3426
+ "step": 488
3427
+ },
3428
+ {
3429
+ "epoch": 4.989795918367347,
3430
+ "grad_norm": 0.009897944517433643,
3431
+ "learning_rate": 9.0702947845805e-08,
3432
+ "loss": 0.0004,
3433
+ "step": 489
3434
+ },
3435
+ {
3436
+ "epoch": 5.0,
3437
+ "grad_norm": 0.005771194584667683,
3438
+ "learning_rate": 4.53514739229025e-08,
3439
+ "loss": 0.0002,
3440
+ "step": 490
3441
+ }
3442
+ ],
3443
+ "logging_steps": 1,
3444
+ "max_steps": 490,
3445
+ "num_input_tokens_seen": 0,
3446
+ "num_train_epochs": 5,
3447
+ "save_steps": 100,
3448
+ "stateful_callbacks": {
3449
+ "TrainerControl": {
3450
+ "args": {
3451
+ "should_epoch_stop": false,
3452
+ "should_evaluate": false,
3453
+ "should_log": false,
3454
+ "should_save": true,
3455
+ "should_training_stop": true
3456
+ },
3457
+ "attributes": {}
3458
+ }
3459
+ },
3460
+ "total_flos": 0.0,
3461
+ "train_batch_size": 16,
3462
+ "trial_name": null,
3463
+ "trial_params": null
3464
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0199643b6e066221cff14d871ab010e3f6f02c49589430a769f473fe7bbd48f6
3
+ size 5560
vocab.txt ADDED
The diff for this file is too large to render. See raw diff