slackingfred commited on
Commit
c3e0053
1 Parent(s): d6d2e80

After 5 stages / 150K steps

Browse files
README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ base_model: deepseek-ai/deepseek-coder-1.3b-instruct
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.9.1.dev0
adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "deepseek-ai/deepseek-coder-1.3b-instruct",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 64,
14
+ "lora_dropout": 0.05,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": null,
18
+ "peft_type": "LORA",
19
+ "r": 32,
20
+ "rank_pattern": {},
21
+ "revision": null,
22
+ "target_modules": [
23
+ "gate_proj",
24
+ "k_proj",
25
+ "down_proj",
26
+ "q_proj",
27
+ "v_proj",
28
+ "o_proj",
29
+ "up_proj"
30
+ ],
31
+ "task_type": "CAUSAL_LM",
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf8d5799a19426dda0836405b58f22ac985f2cb9375ffe97e9e245a737a9b94b
3
+ size 119975656
optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d125a26ddb220e10d32bdf14c9d0eaf1247560ab803358890c35a2d9edf9e0cd
3
+ size 240145026
rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fbc9d904ed26e815db2d9a93ce55caf3f4748d632162cc71db4f38fe9ed69e5e
3
+ size 14244
scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d8902004d681d6ba488d5a81c9b9232056455215d2aeb00f396dec2c2744859
3
+ size 1064
trainer_state.json ADDED
@@ -0,0 +1,2361 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 2.5,
5
+ "eval_steps": 1000,
6
+ "global_step": 30000,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.01,
13
+ "grad_norm": 6.840716361999512,
14
+ "learning_rate": 1.9600000000000002e-05,
15
+ "loss": 3.0263,
16
+ "step": 100
17
+ },
18
+ {
19
+ "epoch": 0.02,
20
+ "grad_norm": 3.3429949283599854,
21
+ "learning_rate": 3.960000000000001e-05,
22
+ "loss": 1.6744,
23
+ "step": 200
24
+ },
25
+ {
26
+ "epoch": 0.03,
27
+ "grad_norm": 5.441232681274414,
28
+ "learning_rate": 5.96e-05,
29
+ "loss": 1.5933,
30
+ "step": 300
31
+ },
32
+ {
33
+ "epoch": 0.03,
34
+ "grad_norm": 2.98654842376709,
35
+ "learning_rate": 7.960000000000001e-05,
36
+ "loss": 1.5397,
37
+ "step": 400
38
+ },
39
+ {
40
+ "epoch": 0.04,
41
+ "grad_norm": 1.9203338623046875,
42
+ "learning_rate": 9.960000000000001e-05,
43
+ "loss": 1.5443,
44
+ "step": 500
45
+ },
46
+ {
47
+ "epoch": 0.05,
48
+ "grad_norm": 2.706545352935791,
49
+ "learning_rate": 9.966779661016949e-05,
50
+ "loss": 1.5593,
51
+ "step": 600
52
+ },
53
+ {
54
+ "epoch": 0.06,
55
+ "grad_norm": 1.7645795345306396,
56
+ "learning_rate": 9.932881355932204e-05,
57
+ "loss": 1.569,
58
+ "step": 700
59
+ },
60
+ {
61
+ "epoch": 0.07,
62
+ "grad_norm": 1.51504647731781,
63
+ "learning_rate": 9.898983050847458e-05,
64
+ "loss": 1.5094,
65
+ "step": 800
66
+ },
67
+ {
68
+ "epoch": 0.07,
69
+ "grad_norm": 3.9701757431030273,
70
+ "learning_rate": 9.865084745762713e-05,
71
+ "loss": 1.5007,
72
+ "step": 900
73
+ },
74
+ {
75
+ "epoch": 0.08,
76
+ "grad_norm": 2.04455304145813,
77
+ "learning_rate": 9.831186440677966e-05,
78
+ "loss": 1.5154,
79
+ "step": 1000
80
+ },
81
+ {
82
+ "epoch": 0.08,
83
+ "eval_loss": 1.4852064847946167,
84
+ "eval_runtime": 93.0778,
85
+ "eval_samples_per_second": 10.744,
86
+ "eval_steps_per_second": 2.686,
87
+ "step": 1000
88
+ },
89
+ {
90
+ "epoch": 0.09,
91
+ "grad_norm": 1.8170721530914307,
92
+ "learning_rate": 9.797288135593221e-05,
93
+ "loss": 1.4842,
94
+ "step": 1100
95
+ },
96
+ {
97
+ "epoch": 0.1,
98
+ "grad_norm": 1.3413264751434326,
99
+ "learning_rate": 9.763389830508474e-05,
100
+ "loss": 1.4884,
101
+ "step": 1200
102
+ },
103
+ {
104
+ "epoch": 0.11,
105
+ "grad_norm": 1.753246784210205,
106
+ "learning_rate": 9.72949152542373e-05,
107
+ "loss": 1.464,
108
+ "step": 1300
109
+ },
110
+ {
111
+ "epoch": 0.12,
112
+ "grad_norm": 3.4083104133605957,
113
+ "learning_rate": 9.695593220338983e-05,
114
+ "loss": 1.4739,
115
+ "step": 1400
116
+ },
117
+ {
118
+ "epoch": 0.12,
119
+ "grad_norm": 2.3532192707061768,
120
+ "learning_rate": 9.661694915254238e-05,
121
+ "loss": 1.5031,
122
+ "step": 1500
123
+ },
124
+ {
125
+ "epoch": 0.13,
126
+ "grad_norm": 2.6274845600128174,
127
+ "learning_rate": 9.627796610169491e-05,
128
+ "loss": 1.4936,
129
+ "step": 1600
130
+ },
131
+ {
132
+ "epoch": 0.14,
133
+ "grad_norm": 3.692795991897583,
134
+ "learning_rate": 9.593898305084746e-05,
135
+ "loss": 1.4842,
136
+ "step": 1700
137
+ },
138
+ {
139
+ "epoch": 0.15,
140
+ "grad_norm": 2.3271782398223877,
141
+ "learning_rate": 9.56e-05,
142
+ "loss": 1.527,
143
+ "step": 1800
144
+ },
145
+ {
146
+ "epoch": 0.16,
147
+ "grad_norm": 1.4074125289916992,
148
+ "learning_rate": 9.526101694915255e-05,
149
+ "loss": 1.4918,
150
+ "step": 1900
151
+ },
152
+ {
153
+ "epoch": 0.17,
154
+ "grad_norm": 1.9484646320343018,
155
+ "learning_rate": 9.492203389830509e-05,
156
+ "loss": 1.459,
157
+ "step": 2000
158
+ },
159
+ {
160
+ "epoch": 0.17,
161
+ "eval_loss": 1.489809274673462,
162
+ "eval_runtime": 93.0028,
163
+ "eval_samples_per_second": 10.752,
164
+ "eval_steps_per_second": 2.688,
165
+ "step": 2000
166
+ },
167
+ {
168
+ "epoch": 0.17,
169
+ "grad_norm": 1.8115673065185547,
170
+ "learning_rate": 9.458305084745763e-05,
171
+ "loss": 1.4285,
172
+ "step": 2100
173
+ },
174
+ {
175
+ "epoch": 0.18,
176
+ "grad_norm": 1.803442358970642,
177
+ "learning_rate": 9.424406779661016e-05,
178
+ "loss": 1.5765,
179
+ "step": 2200
180
+ },
181
+ {
182
+ "epoch": 0.19,
183
+ "grad_norm": 1.2882965803146362,
184
+ "learning_rate": 9.39084745762712e-05,
185
+ "loss": 1.4512,
186
+ "step": 2300
187
+ },
188
+ {
189
+ "epoch": 0.2,
190
+ "grad_norm": 1.9917691946029663,
191
+ "learning_rate": 9.356949152542374e-05,
192
+ "loss": 1.5322,
193
+ "step": 2400
194
+ },
195
+ {
196
+ "epoch": 0.21,
197
+ "grad_norm": 1.6726502180099487,
198
+ "learning_rate": 9.323050847457627e-05,
199
+ "loss": 1.5219,
200
+ "step": 2500
201
+ },
202
+ {
203
+ "epoch": 0.22,
204
+ "grad_norm": 3.595461845397949,
205
+ "learning_rate": 9.289152542372882e-05,
206
+ "loss": 1.444,
207
+ "step": 2600
208
+ },
209
+ {
210
+ "epoch": 0.23,
211
+ "grad_norm": 2.5381546020507812,
212
+ "learning_rate": 9.255254237288135e-05,
213
+ "loss": 1.4474,
214
+ "step": 2700
215
+ },
216
+ {
217
+ "epoch": 0.23,
218
+ "grad_norm": 3.1601195335388184,
219
+ "learning_rate": 9.22135593220339e-05,
220
+ "loss": 1.4111,
221
+ "step": 2800
222
+ },
223
+ {
224
+ "epoch": 0.24,
225
+ "grad_norm": 2.147916078567505,
226
+ "learning_rate": 9.187457627118645e-05,
227
+ "loss": 1.4551,
228
+ "step": 2900
229
+ },
230
+ {
231
+ "epoch": 0.25,
232
+ "grad_norm": 1.6436530351638794,
233
+ "learning_rate": 9.153559322033899e-05,
234
+ "loss": 1.4915,
235
+ "step": 3000
236
+ },
237
+ {
238
+ "epoch": 0.25,
239
+ "eval_loss": 1.458221197128296,
240
+ "eval_runtime": 93.0923,
241
+ "eval_samples_per_second": 10.742,
242
+ "eval_steps_per_second": 2.686,
243
+ "step": 3000
244
+ },
245
+ {
246
+ "epoch": 0.26,
247
+ "grad_norm": 2.2148969173431396,
248
+ "learning_rate": 9.119661016949152e-05,
249
+ "loss": 1.4816,
250
+ "step": 3100
251
+ },
252
+ {
253
+ "epoch": 0.27,
254
+ "grad_norm": 2.3700318336486816,
255
+ "learning_rate": 9.085762711864407e-05,
256
+ "loss": 1.4501,
257
+ "step": 3200
258
+ },
259
+ {
260
+ "epoch": 0.28,
261
+ "grad_norm": 1.5613566637039185,
262
+ "learning_rate": 9.05186440677966e-05,
263
+ "loss": 1.4646,
264
+ "step": 3300
265
+ },
266
+ {
267
+ "epoch": 0.28,
268
+ "grad_norm": 2.175220251083374,
269
+ "learning_rate": 9.017966101694916e-05,
270
+ "loss": 1.5036,
271
+ "step": 3400
272
+ },
273
+ {
274
+ "epoch": 0.29,
275
+ "grad_norm": 12.032076835632324,
276
+ "learning_rate": 8.98406779661017e-05,
277
+ "loss": 1.4563,
278
+ "step": 3500
279
+ },
280
+ {
281
+ "epoch": 0.3,
282
+ "grad_norm": 1.2785927057266235,
283
+ "learning_rate": 8.950169491525424e-05,
284
+ "loss": 1.5178,
285
+ "step": 3600
286
+ },
287
+ {
288
+ "epoch": 0.31,
289
+ "grad_norm": 1.5785298347473145,
290
+ "learning_rate": 8.916271186440679e-05,
291
+ "loss": 1.4257,
292
+ "step": 3700
293
+ },
294
+ {
295
+ "epoch": 0.32,
296
+ "grad_norm": 3.1845338344573975,
297
+ "learning_rate": 8.882372881355932e-05,
298
+ "loss": 1.4742,
299
+ "step": 3800
300
+ },
301
+ {
302
+ "epoch": 0.33,
303
+ "grad_norm": 2.2537214756011963,
304
+ "learning_rate": 8.848474576271187e-05,
305
+ "loss": 1.4538,
306
+ "step": 3900
307
+ },
308
+ {
309
+ "epoch": 0.33,
310
+ "grad_norm": 2.092677354812622,
311
+ "learning_rate": 8.814576271186441e-05,
312
+ "loss": 1.4436,
313
+ "step": 4000
314
+ },
315
+ {
316
+ "epoch": 0.33,
317
+ "eval_loss": 1.4766532182693481,
318
+ "eval_runtime": 93.0059,
319
+ "eval_samples_per_second": 10.752,
320
+ "eval_steps_per_second": 2.688,
321
+ "step": 4000
322
+ },
323
+ {
324
+ "epoch": 0.34,
325
+ "grad_norm": 2.3880198001861572,
326
+ "learning_rate": 8.780677966101696e-05,
327
+ "loss": 1.4706,
328
+ "step": 4100
329
+ },
330
+ {
331
+ "epoch": 0.35,
332
+ "grad_norm": 1.7497975826263428,
333
+ "learning_rate": 8.746779661016949e-05,
334
+ "loss": 1.4361,
335
+ "step": 4200
336
+ },
337
+ {
338
+ "epoch": 0.36,
339
+ "grad_norm": 2.1048712730407715,
340
+ "learning_rate": 8.713220338983051e-05,
341
+ "loss": 1.4484,
342
+ "step": 4300
343
+ },
344
+ {
345
+ "epoch": 0.37,
346
+ "grad_norm": 2.206047534942627,
347
+ "learning_rate": 8.679322033898306e-05,
348
+ "loss": 1.4685,
349
+ "step": 4400
350
+ },
351
+ {
352
+ "epoch": 0.38,
353
+ "grad_norm": 2.012110471725464,
354
+ "learning_rate": 8.64542372881356e-05,
355
+ "loss": 1.429,
356
+ "step": 4500
357
+ },
358
+ {
359
+ "epoch": 0.38,
360
+ "grad_norm": 2.0992116928100586,
361
+ "learning_rate": 8.611525423728813e-05,
362
+ "loss": 1.5028,
363
+ "step": 4600
364
+ },
365
+ {
366
+ "epoch": 0.39,
367
+ "grad_norm": 4.423650741577148,
368
+ "learning_rate": 8.577627118644068e-05,
369
+ "loss": 1.4644,
370
+ "step": 4700
371
+ },
372
+ {
373
+ "epoch": 0.4,
374
+ "grad_norm": 1.7697961330413818,
375
+ "learning_rate": 8.543728813559323e-05,
376
+ "loss": 1.4537,
377
+ "step": 4800
378
+ },
379
+ {
380
+ "epoch": 0.41,
381
+ "grad_norm": 3.925752878189087,
382
+ "learning_rate": 8.509830508474576e-05,
383
+ "loss": 1.4298,
384
+ "step": 4900
385
+ },
386
+ {
387
+ "epoch": 0.42,
388
+ "grad_norm": 1.335164189338684,
389
+ "learning_rate": 8.475932203389832e-05,
390
+ "loss": 1.4534,
391
+ "step": 5000
392
+ },
393
+ {
394
+ "epoch": 0.42,
395
+ "eval_loss": 1.4249181747436523,
396
+ "eval_runtime": 93.056,
397
+ "eval_samples_per_second": 10.746,
398
+ "eval_steps_per_second": 2.687,
399
+ "step": 5000
400
+ },
401
+ {
402
+ "epoch": 0.42,
403
+ "grad_norm": 2.5609335899353027,
404
+ "learning_rate": 8.442033898305085e-05,
405
+ "loss": 1.4444,
406
+ "step": 5100
407
+ },
408
+ {
409
+ "epoch": 0.43,
410
+ "grad_norm": 1.4093282222747803,
411
+ "learning_rate": 8.40813559322034e-05,
412
+ "loss": 1.4737,
413
+ "step": 5200
414
+ },
415
+ {
416
+ "epoch": 0.44,
417
+ "grad_norm": 1.9211537837982178,
418
+ "learning_rate": 8.374237288135593e-05,
419
+ "loss": 1.4108,
420
+ "step": 5300
421
+ },
422
+ {
423
+ "epoch": 0.45,
424
+ "grad_norm": 1.1686183214187622,
425
+ "learning_rate": 8.340338983050848e-05,
426
+ "loss": 1.4603,
427
+ "step": 5400
428
+ },
429
+ {
430
+ "epoch": 0.46,
431
+ "grad_norm": 2.1233601570129395,
432
+ "learning_rate": 8.306440677966102e-05,
433
+ "loss": 1.414,
434
+ "step": 5500
435
+ },
436
+ {
437
+ "epoch": 0.47,
438
+ "grad_norm": 1.9452950954437256,
439
+ "learning_rate": 8.272881355932204e-05,
440
+ "loss": 1.3891,
441
+ "step": 5600
442
+ },
443
+ {
444
+ "epoch": 0.47,
445
+ "grad_norm": 2.8304834365844727,
446
+ "learning_rate": 8.238983050847457e-05,
447
+ "loss": 1.4634,
448
+ "step": 5700
449
+ },
450
+ {
451
+ "epoch": 0.48,
452
+ "grad_norm": 2.8340227603912354,
453
+ "learning_rate": 8.205084745762712e-05,
454
+ "loss": 1.4711,
455
+ "step": 5800
456
+ },
457
+ {
458
+ "epoch": 0.49,
459
+ "grad_norm": 1.5472098588943481,
460
+ "learning_rate": 8.171186440677967e-05,
461
+ "loss": 1.4079,
462
+ "step": 5900
463
+ },
464
+ {
465
+ "epoch": 0.5,
466
+ "grad_norm": 1.8752516508102417,
467
+ "learning_rate": 8.137288135593221e-05,
468
+ "loss": 1.4255,
469
+ "step": 6000
470
+ },
471
+ {
472
+ "epoch": 0.5,
473
+ "eval_loss": 1.433774471282959,
474
+ "eval_runtime": 93.0758,
475
+ "eval_samples_per_second": 10.744,
476
+ "eval_steps_per_second": 2.686,
477
+ "step": 6000
478
+ },
479
+ {
480
+ "epoch": 0.51,
481
+ "grad_norm": 4.256100177764893,
482
+ "learning_rate": 8.103389830508476e-05,
483
+ "loss": 1.4102,
484
+ "step": 6100
485
+ },
486
+ {
487
+ "epoch": 0.52,
488
+ "grad_norm": 1.966732144355774,
489
+ "learning_rate": 8.069491525423729e-05,
490
+ "loss": 1.4337,
491
+ "step": 6200
492
+ },
493
+ {
494
+ "epoch": 0.53,
495
+ "grad_norm": 0.9053303599357605,
496
+ "learning_rate": 8.035593220338984e-05,
497
+ "loss": 1.4517,
498
+ "step": 6300
499
+ },
500
+ {
501
+ "epoch": 0.53,
502
+ "grad_norm": 2.6552069187164307,
503
+ "learning_rate": 8.001694915254237e-05,
504
+ "loss": 1.459,
505
+ "step": 6400
506
+ },
507
+ {
508
+ "epoch": 0.54,
509
+ "grad_norm": 2.2912251949310303,
510
+ "learning_rate": 7.967796610169492e-05,
511
+ "loss": 1.3857,
512
+ "step": 6500
513
+ },
514
+ {
515
+ "epoch": 0.55,
516
+ "grad_norm": 2.4170544147491455,
517
+ "learning_rate": 7.933898305084746e-05,
518
+ "loss": 1.4032,
519
+ "step": 6600
520
+ },
521
+ {
522
+ "epoch": 0.56,
523
+ "grad_norm": 1.7848442792892456,
524
+ "learning_rate": 7.900000000000001e-05,
525
+ "loss": 1.4374,
526
+ "step": 6700
527
+ },
528
+ {
529
+ "epoch": 0.57,
530
+ "grad_norm": 4.551065444946289,
531
+ "learning_rate": 7.866101694915254e-05,
532
+ "loss": 1.4481,
533
+ "step": 6800
534
+ },
535
+ {
536
+ "epoch": 0.57,
537
+ "grad_norm": 1.3571339845657349,
538
+ "learning_rate": 7.832203389830509e-05,
539
+ "loss": 1.4213,
540
+ "step": 6900
541
+ },
542
+ {
543
+ "epoch": 0.58,
544
+ "grad_norm": 1.5277926921844482,
545
+ "learning_rate": 7.798305084745763e-05,
546
+ "loss": 1.429,
547
+ "step": 7000
548
+ },
549
+ {
550
+ "epoch": 0.58,
551
+ "eval_loss": 1.394229769706726,
552
+ "eval_runtime": 93.0895,
553
+ "eval_samples_per_second": 10.742,
554
+ "eval_steps_per_second": 2.686,
555
+ "step": 7000
556
+ },
557
+ {
558
+ "epoch": 0.59,
559
+ "grad_norm": 2.962740421295166,
560
+ "learning_rate": 7.764406779661018e-05,
561
+ "loss": 1.4396,
562
+ "step": 7100
563
+ },
564
+ {
565
+ "epoch": 0.6,
566
+ "grad_norm": 3.3276052474975586,
567
+ "learning_rate": 7.730508474576271e-05,
568
+ "loss": 1.4628,
569
+ "step": 7200
570
+ },
571
+ {
572
+ "epoch": 0.61,
573
+ "grad_norm": 0.9193934202194214,
574
+ "learning_rate": 7.696610169491526e-05,
575
+ "loss": 1.4234,
576
+ "step": 7300
577
+ },
578
+ {
579
+ "epoch": 0.62,
580
+ "grad_norm": 1.859925627708435,
581
+ "learning_rate": 7.662711864406779e-05,
582
+ "loss": 1.4427,
583
+ "step": 7400
584
+ },
585
+ {
586
+ "epoch": 0.62,
587
+ "grad_norm": 2.7623603343963623,
588
+ "learning_rate": 7.628813559322034e-05,
589
+ "loss": 1.4046,
590
+ "step": 7500
591
+ },
592
+ {
593
+ "epoch": 0.63,
594
+ "grad_norm": 2.3409364223480225,
595
+ "learning_rate": 7.594915254237289e-05,
596
+ "loss": 1.4351,
597
+ "step": 7600
598
+ },
599
+ {
600
+ "epoch": 0.64,
601
+ "grad_norm": 1.8644472360610962,
602
+ "learning_rate": 7.561016949152543e-05,
603
+ "loss": 1.4281,
604
+ "step": 7700
605
+ },
606
+ {
607
+ "epoch": 0.65,
608
+ "grad_norm": 2.663987398147583,
609
+ "learning_rate": 7.527118644067796e-05,
610
+ "loss": 1.3743,
611
+ "step": 7800
612
+ },
613
+ {
614
+ "epoch": 0.66,
615
+ "grad_norm": 1.3779089450836182,
616
+ "learning_rate": 7.493220338983051e-05,
617
+ "loss": 1.4531,
618
+ "step": 7900
619
+ },
620
+ {
621
+ "epoch": 0.67,
622
+ "grad_norm": 1.2310881614685059,
623
+ "learning_rate": 7.459322033898304e-05,
624
+ "loss": 1.427,
625
+ "step": 8000
626
+ },
627
+ {
628
+ "epoch": 0.67,
629
+ "eval_loss": 1.4101628065109253,
630
+ "eval_runtime": 92.9756,
631
+ "eval_samples_per_second": 10.756,
632
+ "eval_steps_per_second": 2.689,
633
+ "step": 8000
634
+ },
635
+ {
636
+ "epoch": 0.68,
637
+ "grad_norm": 2.426481246948242,
638
+ "learning_rate": 7.425762711864407e-05,
639
+ "loss": 1.4215,
640
+ "step": 8100
641
+ },
642
+ {
643
+ "epoch": 0.68,
644
+ "grad_norm": 3.706364154815674,
645
+ "learning_rate": 7.391864406779662e-05,
646
+ "loss": 1.3344,
647
+ "step": 8200
648
+ },
649
+ {
650
+ "epoch": 0.69,
651
+ "grad_norm": 2.511159658432007,
652
+ "learning_rate": 7.357966101694915e-05,
653
+ "loss": 1.4132,
654
+ "step": 8300
655
+ },
656
+ {
657
+ "epoch": 0.7,
658
+ "grad_norm": 2.31131911277771,
659
+ "learning_rate": 7.32406779661017e-05,
660
+ "loss": 1.4357,
661
+ "step": 8400
662
+ },
663
+ {
664
+ "epoch": 0.71,
665
+ "grad_norm": 2.3403029441833496,
666
+ "learning_rate": 7.290169491525423e-05,
667
+ "loss": 1.455,
668
+ "step": 8500
669
+ },
670
+ {
671
+ "epoch": 0.72,
672
+ "grad_norm": 2.386819362640381,
673
+ "learning_rate": 7.256271186440679e-05,
674
+ "loss": 1.374,
675
+ "step": 8600
676
+ },
677
+ {
678
+ "epoch": 0.72,
679
+ "grad_norm": 12.603606224060059,
680
+ "learning_rate": 7.222372881355932e-05,
681
+ "loss": 1.461,
682
+ "step": 8700
683
+ },
684
+ {
685
+ "epoch": 0.73,
686
+ "grad_norm": 2.5920355319976807,
687
+ "learning_rate": 7.188474576271187e-05,
688
+ "loss": 1.423,
689
+ "step": 8800
690
+ },
691
+ {
692
+ "epoch": 0.74,
693
+ "grad_norm": 3.7238869667053223,
694
+ "learning_rate": 7.15457627118644e-05,
695
+ "loss": 1.3845,
696
+ "step": 8900
697
+ },
698
+ {
699
+ "epoch": 0.75,
700
+ "grad_norm": 4.266007900238037,
701
+ "learning_rate": 7.120677966101695e-05,
702
+ "loss": 1.4293,
703
+ "step": 9000
704
+ },
705
+ {
706
+ "epoch": 0.75,
707
+ "eval_loss": 1.4198664426803589,
708
+ "eval_runtime": 93.0273,
709
+ "eval_samples_per_second": 10.75,
710
+ "eval_steps_per_second": 2.687,
711
+ "step": 9000
712
+ },
713
+ {
714
+ "epoch": 0.76,
715
+ "grad_norm": 2.2583582401275635,
716
+ "learning_rate": 7.08677966101695e-05,
717
+ "loss": 1.4032,
718
+ "step": 9100
719
+ },
720
+ {
721
+ "epoch": 0.77,
722
+ "grad_norm": 3.374390125274658,
723
+ "learning_rate": 7.052881355932204e-05,
724
+ "loss": 1.3469,
725
+ "step": 9200
726
+ },
727
+ {
728
+ "epoch": 0.78,
729
+ "grad_norm": 2.431976318359375,
730
+ "learning_rate": 7.018983050847457e-05,
731
+ "loss": 1.3861,
732
+ "step": 9300
733
+ },
734
+ {
735
+ "epoch": 0.78,
736
+ "grad_norm": 2.953474760055542,
737
+ "learning_rate": 6.985084745762712e-05,
738
+ "loss": 1.3406,
739
+ "step": 9400
740
+ },
741
+ {
742
+ "epoch": 0.79,
743
+ "grad_norm": 1.3457430601119995,
744
+ "learning_rate": 6.951186440677967e-05,
745
+ "loss": 1.4132,
746
+ "step": 9500
747
+ },
748
+ {
749
+ "epoch": 0.8,
750
+ "grad_norm": 3.713928699493408,
751
+ "learning_rate": 6.91728813559322e-05,
752
+ "loss": 1.4555,
753
+ "step": 9600
754
+ },
755
+ {
756
+ "epoch": 0.81,
757
+ "grad_norm": 3.0444705486297607,
758
+ "learning_rate": 6.883389830508476e-05,
759
+ "loss": 1.3372,
760
+ "step": 9700
761
+ },
762
+ {
763
+ "epoch": 0.82,
764
+ "grad_norm": 2.5544724464416504,
765
+ "learning_rate": 6.849491525423729e-05,
766
+ "loss": 1.4265,
767
+ "step": 9800
768
+ },
769
+ {
770
+ "epoch": 0.82,
771
+ "grad_norm": 3.4634218215942383,
772
+ "learning_rate": 6.815593220338984e-05,
773
+ "loss": 1.4021,
774
+ "step": 9900
775
+ },
776
+ {
777
+ "epoch": 0.83,
778
+ "grad_norm": 2.940263509750366,
779
+ "learning_rate": 6.781694915254237e-05,
780
+ "loss": 1.3975,
781
+ "step": 10000
782
+ },
783
+ {
784
+ "epoch": 0.83,
785
+ "eval_loss": 1.3409844636917114,
786
+ "eval_runtime": 92.9614,
787
+ "eval_samples_per_second": 10.757,
788
+ "eval_steps_per_second": 2.689,
789
+ "step": 10000
790
+ },
791
+ {
792
+ "epoch": 0.84,
793
+ "grad_norm": 2.2668895721435547,
794
+ "learning_rate": 6.748135593220339e-05,
795
+ "loss": 1.3879,
796
+ "step": 10100
797
+ },
798
+ {
799
+ "epoch": 0.85,
800
+ "grad_norm": 2.5967912673950195,
801
+ "learning_rate": 6.714237288135593e-05,
802
+ "loss": 1.4197,
803
+ "step": 10200
804
+ },
805
+ {
806
+ "epoch": 0.86,
807
+ "grad_norm": 4.100217819213867,
808
+ "learning_rate": 6.680338983050848e-05,
809
+ "loss": 1.4081,
810
+ "step": 10300
811
+ },
812
+ {
813
+ "epoch": 0.87,
814
+ "grad_norm": 1.7082469463348389,
815
+ "learning_rate": 6.646440677966101e-05,
816
+ "loss": 1.4103,
817
+ "step": 10400
818
+ },
819
+ {
820
+ "epoch": 0.88,
821
+ "grad_norm": 2.1128108501434326,
822
+ "learning_rate": 6.612542372881356e-05,
823
+ "loss": 1.44,
824
+ "step": 10500
825
+ },
826
+ {
827
+ "epoch": 0.88,
828
+ "grad_norm": 2.1471731662750244,
829
+ "learning_rate": 6.57864406779661e-05,
830
+ "loss": 1.412,
831
+ "step": 10600
832
+ },
833
+ {
834
+ "epoch": 0.89,
835
+ "grad_norm": 3.2873263359069824,
836
+ "learning_rate": 6.544745762711865e-05,
837
+ "loss": 1.4407,
838
+ "step": 10700
839
+ },
840
+ {
841
+ "epoch": 0.9,
842
+ "grad_norm": 3.7035319805145264,
843
+ "learning_rate": 6.51084745762712e-05,
844
+ "loss": 1.3792,
845
+ "step": 10800
846
+ },
847
+ {
848
+ "epoch": 0.91,
849
+ "grad_norm": 2.7462477684020996,
850
+ "learning_rate": 6.476949152542373e-05,
851
+ "loss": 1.3676,
852
+ "step": 10900
853
+ },
854
+ {
855
+ "epoch": 0.92,
856
+ "grad_norm": 1.895967721939087,
857
+ "learning_rate": 6.443050847457628e-05,
858
+ "loss": 1.4092,
859
+ "step": 11000
860
+ },
861
+ {
862
+ "epoch": 0.92,
863
+ "eval_loss": 1.4003005027770996,
864
+ "eval_runtime": 93.0106,
865
+ "eval_samples_per_second": 10.751,
866
+ "eval_steps_per_second": 2.688,
867
+ "step": 11000
868
+ },
869
+ {
870
+ "epoch": 0.93,
871
+ "grad_norm": 2.039684295654297,
872
+ "learning_rate": 6.409152542372881e-05,
873
+ "loss": 1.4015,
874
+ "step": 11100
875
+ },
876
+ {
877
+ "epoch": 0.93,
878
+ "grad_norm": 1.625591516494751,
879
+ "learning_rate": 6.375254237288136e-05,
880
+ "loss": 1.4362,
881
+ "step": 11200
882
+ },
883
+ {
884
+ "epoch": 0.94,
885
+ "grad_norm": 3.4086389541625977,
886
+ "learning_rate": 6.34135593220339e-05,
887
+ "loss": 1.4192,
888
+ "step": 11300
889
+ },
890
+ {
891
+ "epoch": 0.95,
892
+ "grad_norm": 1.9734954833984375,
893
+ "learning_rate": 6.307457627118645e-05,
894
+ "loss": 1.4413,
895
+ "step": 11400
896
+ },
897
+ {
898
+ "epoch": 0.96,
899
+ "grad_norm": 3.1704461574554443,
900
+ "learning_rate": 6.273559322033898e-05,
901
+ "loss": 1.3856,
902
+ "step": 11500
903
+ },
904
+ {
905
+ "epoch": 0.97,
906
+ "grad_norm": 3.659949541091919,
907
+ "learning_rate": 6.239661016949153e-05,
908
+ "loss": 1.4044,
909
+ "step": 11600
910
+ },
911
+ {
912
+ "epoch": 0.97,
913
+ "grad_norm": 1.8390787839889526,
914
+ "learning_rate": 6.205762711864406e-05,
915
+ "loss": 1.3704,
916
+ "step": 11700
917
+ },
918
+ {
919
+ "epoch": 0.98,
920
+ "grad_norm": 1.7644896507263184,
921
+ "learning_rate": 6.171864406779662e-05,
922
+ "loss": 1.4389,
923
+ "step": 11800
924
+ },
925
+ {
926
+ "epoch": 0.99,
927
+ "grad_norm": 2.0395686626434326,
928
+ "learning_rate": 6.137966101694915e-05,
929
+ "loss": 1.397,
930
+ "step": 11900
931
+ },
932
+ {
933
+ "epoch": 1.0,
934
+ "grad_norm": 1.8224962949752808,
935
+ "learning_rate": 6.10406779661017e-05,
936
+ "loss": 1.351,
937
+ "step": 12000
938
+ },
939
+ {
940
+ "epoch": 1.0,
941
+ "eval_loss": 1.4065361022949219,
942
+ "eval_runtime": 93.0021,
943
+ "eval_samples_per_second": 10.752,
944
+ "eval_steps_per_second": 2.688,
945
+ "step": 12000
946
+ },
947
+ {
948
+ "epoch": 1.01,
949
+ "grad_norm": 3.049715995788574,
950
+ "learning_rate": 6.070169491525424e-05,
951
+ "loss": 1.3315,
952
+ "step": 12100
953
+ },
954
+ {
955
+ "epoch": 1.02,
956
+ "grad_norm": 2.025336503982544,
957
+ "learning_rate": 6.0362711864406786e-05,
958
+ "loss": 1.3722,
959
+ "step": 12200
960
+ },
961
+ {
962
+ "epoch": 1.02,
963
+ "grad_norm": 2.9472439289093018,
964
+ "learning_rate": 6.002372881355932e-05,
965
+ "loss": 1.3185,
966
+ "step": 12300
967
+ },
968
+ {
969
+ "epoch": 1.03,
970
+ "grad_norm": 3.2052855491638184,
971
+ "learning_rate": 5.968474576271187e-05,
972
+ "loss": 1.3029,
973
+ "step": 12400
974
+ },
975
+ {
976
+ "epoch": 1.04,
977
+ "grad_norm": 3.9783244132995605,
978
+ "learning_rate": 5.9345762711864405e-05,
979
+ "loss": 1.3617,
980
+ "step": 12500
981
+ },
982
+ {
983
+ "epoch": 1.05,
984
+ "grad_norm": 6.090597152709961,
985
+ "learning_rate": 5.900677966101695e-05,
986
+ "loss": 1.4188,
987
+ "step": 12600
988
+ },
989
+ {
990
+ "epoch": 1.06,
991
+ "grad_norm": 2.486492156982422,
992
+ "learning_rate": 5.866779661016949e-05,
993
+ "loss": 1.3771,
994
+ "step": 12700
995
+ },
996
+ {
997
+ "epoch": 1.07,
998
+ "grad_norm": 2.2206759452819824,
999
+ "learning_rate": 5.832881355932204e-05,
1000
+ "loss": 1.3583,
1001
+ "step": 12800
1002
+ },
1003
+ {
1004
+ "epoch": 1.07,
1005
+ "grad_norm": 2.092623710632324,
1006
+ "learning_rate": 5.7989830508474577e-05,
1007
+ "loss": 1.344,
1008
+ "step": 12900
1009
+ },
1010
+ {
1011
+ "epoch": 1.08,
1012
+ "grad_norm": 1.9661636352539062,
1013
+ "learning_rate": 5.765084745762712e-05,
1014
+ "loss": 1.3364,
1015
+ "step": 13000
1016
+ },
1017
+ {
1018
+ "epoch": 1.08,
1019
+ "eval_loss": 1.3436501026153564,
1020
+ "eval_runtime": 92.9061,
1021
+ "eval_samples_per_second": 10.764,
1022
+ "eval_steps_per_second": 2.691,
1023
+ "step": 13000
1024
+ },
1025
+ {
1026
+ "epoch": 1.09,
1027
+ "grad_norm": 2.299858331680298,
1028
+ "learning_rate": 5.7311864406779656e-05,
1029
+ "loss": 1.327,
1030
+ "step": 13100
1031
+ },
1032
+ {
1033
+ "epoch": 1.1,
1034
+ "grad_norm": 1.999542474746704,
1035
+ "learning_rate": 5.697288135593221e-05,
1036
+ "loss": 1.3852,
1037
+ "step": 13200
1038
+ },
1039
+ {
1040
+ "epoch": 1.11,
1041
+ "grad_norm": 4.422431468963623,
1042
+ "learning_rate": 5.6633898305084755e-05,
1043
+ "loss": 1.3812,
1044
+ "step": 13300
1045
+ },
1046
+ {
1047
+ "epoch": 1.12,
1048
+ "grad_norm": 1.9813380241394043,
1049
+ "learning_rate": 5.629491525423729e-05,
1050
+ "loss": 1.3517,
1051
+ "step": 13400
1052
+ },
1053
+ {
1054
+ "epoch": 1.12,
1055
+ "grad_norm": 1.8180012702941895,
1056
+ "learning_rate": 5.595593220338984e-05,
1057
+ "loss": 1.3715,
1058
+ "step": 13500
1059
+ },
1060
+ {
1061
+ "epoch": 1.13,
1062
+ "grad_norm": 3.0727899074554443,
1063
+ "learning_rate": 5.5616949152542374e-05,
1064
+ "loss": 1.3278,
1065
+ "step": 13600
1066
+ },
1067
+ {
1068
+ "epoch": 1.14,
1069
+ "grad_norm": 1.4579344987869263,
1070
+ "learning_rate": 5.527796610169492e-05,
1071
+ "loss": 1.3649,
1072
+ "step": 13700
1073
+ },
1074
+ {
1075
+ "epoch": 1.15,
1076
+ "grad_norm": 1.9728525876998901,
1077
+ "learning_rate": 5.493898305084746e-05,
1078
+ "loss": 1.2894,
1079
+ "step": 13800
1080
+ },
1081
+ {
1082
+ "epoch": 1.16,
1083
+ "grad_norm": 3.2770729064941406,
1084
+ "learning_rate": 5.4600000000000006e-05,
1085
+ "loss": 1.3748,
1086
+ "step": 13900
1087
+ },
1088
+ {
1089
+ "epoch": 1.17,
1090
+ "grad_norm": 4.356542587280273,
1091
+ "learning_rate": 5.4261016949152545e-05,
1092
+ "loss": 1.3153,
1093
+ "step": 14000
1094
+ },
1095
+ {
1096
+ "epoch": 1.17,
1097
+ "eval_loss": 1.3724653720855713,
1098
+ "eval_runtime": 92.8763,
1099
+ "eval_samples_per_second": 10.767,
1100
+ "eval_steps_per_second": 2.692,
1101
+ "step": 14000
1102
+ },
1103
+ {
1104
+ "epoch": 1.18,
1105
+ "grad_norm": 1.5841625928878784,
1106
+ "learning_rate": 5.392203389830509e-05,
1107
+ "loss": 1.3295,
1108
+ "step": 14100
1109
+ },
1110
+ {
1111
+ "epoch": 1.18,
1112
+ "grad_norm": 8.736273765563965,
1113
+ "learning_rate": 5.35864406779661e-05,
1114
+ "loss": 1.3146,
1115
+ "step": 14200
1116
+ },
1117
+ {
1118
+ "epoch": 1.19,
1119
+ "grad_norm": 3.8798937797546387,
1120
+ "learning_rate": 5.324745762711865e-05,
1121
+ "loss": 1.2944,
1122
+ "step": 14300
1123
+ },
1124
+ {
1125
+ "epoch": 1.2,
1126
+ "grad_norm": 3.1168253421783447,
1127
+ "learning_rate": 5.2908474576271194e-05,
1128
+ "loss": 1.2995,
1129
+ "step": 14400
1130
+ },
1131
+ {
1132
+ "epoch": 1.21,
1133
+ "grad_norm": 1.7689605951309204,
1134
+ "learning_rate": 5.256949152542373e-05,
1135
+ "loss": 1.3166,
1136
+ "step": 14500
1137
+ },
1138
+ {
1139
+ "epoch": 1.22,
1140
+ "grad_norm": 2.00744891166687,
1141
+ "learning_rate": 5.223050847457628e-05,
1142
+ "loss": 1.3385,
1143
+ "step": 14600
1144
+ },
1145
+ {
1146
+ "epoch": 1.23,
1147
+ "grad_norm": 1.9501924514770508,
1148
+ "learning_rate": 5.189152542372881e-05,
1149
+ "loss": 1.3564,
1150
+ "step": 14700
1151
+ },
1152
+ {
1153
+ "epoch": 1.23,
1154
+ "grad_norm": 2.614849328994751,
1155
+ "learning_rate": 5.1552542372881366e-05,
1156
+ "loss": 1.3326,
1157
+ "step": 14800
1158
+ },
1159
+ {
1160
+ "epoch": 1.24,
1161
+ "grad_norm": 2.0598857402801514,
1162
+ "learning_rate": 5.12135593220339e-05,
1163
+ "loss": 1.2322,
1164
+ "step": 14900
1165
+ },
1166
+ {
1167
+ "epoch": 1.25,
1168
+ "grad_norm": 2.12611985206604,
1169
+ "learning_rate": 5.0874576271186445e-05,
1170
+ "loss": 1.354,
1171
+ "step": 15000
1172
+ },
1173
+ {
1174
+ "epoch": 1.25,
1175
+ "eval_loss": 1.368465542793274,
1176
+ "eval_runtime": 93.0272,
1177
+ "eval_samples_per_second": 10.75,
1178
+ "eval_steps_per_second": 2.687,
1179
+ "step": 15000
1180
+ },
1181
+ {
1182
+ "epoch": 1.26,
1183
+ "grad_norm": 1.287361979484558,
1184
+ "learning_rate": 5.0535593220338984e-05,
1185
+ "loss": 1.3178,
1186
+ "step": 15100
1187
+ },
1188
+ {
1189
+ "epoch": 1.27,
1190
+ "grad_norm": 3.340104579925537,
1191
+ "learning_rate": 5.019661016949153e-05,
1192
+ "loss": 1.3893,
1193
+ "step": 15200
1194
+ },
1195
+ {
1196
+ "epoch": 1.27,
1197
+ "grad_norm": 2.4543209075927734,
1198
+ "learning_rate": 4.985762711864407e-05,
1199
+ "loss": 1.3597,
1200
+ "step": 15300
1201
+ },
1202
+ {
1203
+ "epoch": 1.28,
1204
+ "grad_norm": 2.6554243564605713,
1205
+ "learning_rate": 4.951864406779661e-05,
1206
+ "loss": 1.2903,
1207
+ "step": 15400
1208
+ },
1209
+ {
1210
+ "epoch": 1.29,
1211
+ "grad_norm": 1.685389518737793,
1212
+ "learning_rate": 4.917966101694915e-05,
1213
+ "loss": 1.2902,
1214
+ "step": 15500
1215
+ },
1216
+ {
1217
+ "epoch": 1.3,
1218
+ "grad_norm": 1.9464205503463745,
1219
+ "learning_rate": 4.88406779661017e-05,
1220
+ "loss": 1.3333,
1221
+ "step": 15600
1222
+ },
1223
+ {
1224
+ "epoch": 1.31,
1225
+ "grad_norm": 2.7877588272094727,
1226
+ "learning_rate": 4.850169491525424e-05,
1227
+ "loss": 1.3016,
1228
+ "step": 15700
1229
+ },
1230
+ {
1231
+ "epoch": 1.32,
1232
+ "grad_norm": 1.88291335105896,
1233
+ "learning_rate": 4.816271186440678e-05,
1234
+ "loss": 1.382,
1235
+ "step": 15800
1236
+ },
1237
+ {
1238
+ "epoch": 1.32,
1239
+ "grad_norm": 4.62669038772583,
1240
+ "learning_rate": 4.782372881355933e-05,
1241
+ "loss": 1.3343,
1242
+ "step": 15900
1243
+ },
1244
+ {
1245
+ "epoch": 1.33,
1246
+ "grad_norm": 2.525221824645996,
1247
+ "learning_rate": 4.748474576271187e-05,
1248
+ "loss": 1.2822,
1249
+ "step": 16000
1250
+ },
1251
+ {
1252
+ "epoch": 1.33,
1253
+ "eval_loss": 1.3313556909561157,
1254
+ "eval_runtime": 93.0605,
1255
+ "eval_samples_per_second": 10.746,
1256
+ "eval_steps_per_second": 2.686,
1257
+ "step": 16000
1258
+ },
1259
+ {
1260
+ "epoch": 1.34,
1261
+ "grad_norm": 2.8074324131011963,
1262
+ "learning_rate": 4.714576271186441e-05,
1263
+ "loss": 1.3487,
1264
+ "step": 16100
1265
+ },
1266
+ {
1267
+ "epoch": 1.35,
1268
+ "grad_norm": 3.902251958847046,
1269
+ "learning_rate": 4.680677966101695e-05,
1270
+ "loss": 1.2796,
1271
+ "step": 16200
1272
+ },
1273
+ {
1274
+ "epoch": 1.36,
1275
+ "grad_norm": 3.72412109375,
1276
+ "learning_rate": 4.646779661016949e-05,
1277
+ "loss": 1.3161,
1278
+ "step": 16300
1279
+ },
1280
+ {
1281
+ "epoch": 1.37,
1282
+ "grad_norm": 1.7499101161956787,
1283
+ "learning_rate": 4.612881355932204e-05,
1284
+ "loss": 1.3477,
1285
+ "step": 16400
1286
+ },
1287
+ {
1288
+ "epoch": 1.38,
1289
+ "grad_norm": 2.252373695373535,
1290
+ "learning_rate": 4.578983050847458e-05,
1291
+ "loss": 1.3288,
1292
+ "step": 16500
1293
+ },
1294
+ {
1295
+ "epoch": 1.38,
1296
+ "grad_norm": 3.353426933288574,
1297
+ "learning_rate": 4.5450847457627125e-05,
1298
+ "loss": 1.3472,
1299
+ "step": 16600
1300
+ },
1301
+ {
1302
+ "epoch": 1.39,
1303
+ "grad_norm": 2.308408260345459,
1304
+ "learning_rate": 4.5111864406779664e-05,
1305
+ "loss": 1.3213,
1306
+ "step": 16700
1307
+ },
1308
+ {
1309
+ "epoch": 1.4,
1310
+ "grad_norm": 4.369659423828125,
1311
+ "learning_rate": 4.4772881355932204e-05,
1312
+ "loss": 1.2767,
1313
+ "step": 16800
1314
+ },
1315
+ {
1316
+ "epoch": 1.41,
1317
+ "grad_norm": 1.9831510782241821,
1318
+ "learning_rate": 4.443389830508475e-05,
1319
+ "loss": 1.2989,
1320
+ "step": 16900
1321
+ },
1322
+ {
1323
+ "epoch": 1.42,
1324
+ "grad_norm": 1.2581738233566284,
1325
+ "learning_rate": 4.409491525423729e-05,
1326
+ "loss": 1.3224,
1327
+ "step": 17000
1328
+ },
1329
+ {
1330
+ "epoch": 1.42,
1331
+ "eval_loss": 1.3554507493972778,
1332
+ "eval_runtime": 92.9689,
1333
+ "eval_samples_per_second": 10.756,
1334
+ "eval_steps_per_second": 2.689,
1335
+ "step": 17000
1336
+ },
1337
+ {
1338
+ "epoch": 1.43,
1339
+ "grad_norm": 5.903542518615723,
1340
+ "learning_rate": 4.375593220338983e-05,
1341
+ "loss": 1.3135,
1342
+ "step": 17100
1343
+ },
1344
+ {
1345
+ "epoch": 1.43,
1346
+ "grad_norm": 2.0982234477996826,
1347
+ "learning_rate": 4.3416949152542375e-05,
1348
+ "loss": 1.3036,
1349
+ "step": 17200
1350
+ },
1351
+ {
1352
+ "epoch": 1.44,
1353
+ "grad_norm": 1.5168031454086304,
1354
+ "learning_rate": 4.3077966101694915e-05,
1355
+ "loss": 1.3223,
1356
+ "step": 17300
1357
+ },
1358
+ {
1359
+ "epoch": 1.45,
1360
+ "grad_norm": 2.655869722366333,
1361
+ "learning_rate": 4.273898305084746e-05,
1362
+ "loss": 1.3036,
1363
+ "step": 17400
1364
+ },
1365
+ {
1366
+ "epoch": 1.46,
1367
+ "grad_norm": 3.1604421138763428,
1368
+ "learning_rate": 4.24e-05,
1369
+ "loss": 1.3172,
1370
+ "step": 17500
1371
+ },
1372
+ {
1373
+ "epoch": 1.47,
1374
+ "grad_norm": 1.9133104085922241,
1375
+ "learning_rate": 4.206101694915254e-05,
1376
+ "loss": 1.3596,
1377
+ "step": 17600
1378
+ },
1379
+ {
1380
+ "epoch": 1.48,
1381
+ "grad_norm": 5.270731449127197,
1382
+ "learning_rate": 4.1722033898305086e-05,
1383
+ "loss": 1.3021,
1384
+ "step": 17700
1385
+ },
1386
+ {
1387
+ "epoch": 1.48,
1388
+ "grad_norm": 1.63645339012146,
1389
+ "learning_rate": 4.1383050847457626e-05,
1390
+ "loss": 1.3197,
1391
+ "step": 17800
1392
+ },
1393
+ {
1394
+ "epoch": 1.49,
1395
+ "grad_norm": 3.240687370300293,
1396
+ "learning_rate": 4.104406779661017e-05,
1397
+ "loss": 1.347,
1398
+ "step": 17900
1399
+ },
1400
+ {
1401
+ "epoch": 1.5,
1402
+ "grad_norm": 2.892014265060425,
1403
+ "learning_rate": 4.070508474576271e-05,
1404
+ "loss": 1.2844,
1405
+ "step": 18000
1406
+ },
1407
+ {
1408
+ "epoch": 1.5,
1409
+ "eval_loss": 1.3619292974472046,
1410
+ "eval_runtime": 92.9703,
1411
+ "eval_samples_per_second": 10.756,
1412
+ "eval_steps_per_second": 2.689,
1413
+ "step": 18000
1414
+ },
1415
+ {
1416
+ "epoch": 1.51,
1417
+ "grad_norm": 1.62366783618927,
1418
+ "learning_rate": 4.036949152542373e-05,
1419
+ "loss": 1.3391,
1420
+ "step": 18100
1421
+ },
1422
+ {
1423
+ "epoch": 1.52,
1424
+ "grad_norm": 3.9142508506774902,
1425
+ "learning_rate": 4.0030508474576275e-05,
1426
+ "loss": 1.2751,
1427
+ "step": 18200
1428
+ },
1429
+ {
1430
+ "epoch": 1.52,
1431
+ "grad_norm": 3.78511381149292,
1432
+ "learning_rate": 3.9691525423728814e-05,
1433
+ "loss": 1.3169,
1434
+ "step": 18300
1435
+ },
1436
+ {
1437
+ "epoch": 1.53,
1438
+ "grad_norm": 2.912883758544922,
1439
+ "learning_rate": 3.9352542372881354e-05,
1440
+ "loss": 1.3224,
1441
+ "step": 18400
1442
+ },
1443
+ {
1444
+ "epoch": 1.54,
1445
+ "grad_norm": 2.096040725708008,
1446
+ "learning_rate": 3.90135593220339e-05,
1447
+ "loss": 1.2708,
1448
+ "step": 18500
1449
+ },
1450
+ {
1451
+ "epoch": 1.55,
1452
+ "grad_norm": 5.406456470489502,
1453
+ "learning_rate": 3.867457627118644e-05,
1454
+ "loss": 1.3068,
1455
+ "step": 18600
1456
+ },
1457
+ {
1458
+ "epoch": 1.56,
1459
+ "grad_norm": 1.6677026748657227,
1460
+ "learning_rate": 3.8335593220338986e-05,
1461
+ "loss": 1.3202,
1462
+ "step": 18700
1463
+ },
1464
+ {
1465
+ "epoch": 1.57,
1466
+ "grad_norm": 2.0527544021606445,
1467
+ "learning_rate": 3.7996610169491525e-05,
1468
+ "loss": 1.2886,
1469
+ "step": 18800
1470
+ },
1471
+ {
1472
+ "epoch": 1.57,
1473
+ "grad_norm": 3.8199574947357178,
1474
+ "learning_rate": 3.7657627118644065e-05,
1475
+ "loss": 1.3367,
1476
+ "step": 18900
1477
+ },
1478
+ {
1479
+ "epoch": 1.58,
1480
+ "grad_norm": 2.5120060443878174,
1481
+ "learning_rate": 3.731864406779661e-05,
1482
+ "loss": 1.2945,
1483
+ "step": 19000
1484
+ },
1485
+ {
1486
+ "epoch": 1.58,
1487
+ "eval_loss": 1.3452004194259644,
1488
+ "eval_runtime": 93.0461,
1489
+ "eval_samples_per_second": 10.747,
1490
+ "eval_steps_per_second": 2.687,
1491
+ "step": 19000
1492
+ },
1493
+ {
1494
+ "epoch": 1.59,
1495
+ "grad_norm": 1.0111308097839355,
1496
+ "learning_rate": 3.697966101694915e-05,
1497
+ "loss": 1.3713,
1498
+ "step": 19100
1499
+ },
1500
+ {
1501
+ "epoch": 1.6,
1502
+ "grad_norm": 2.966914176940918,
1503
+ "learning_rate": 3.66406779661017e-05,
1504
+ "loss": 1.3124,
1505
+ "step": 19200
1506
+ },
1507
+ {
1508
+ "epoch": 1.61,
1509
+ "grad_norm": 4.211293697357178,
1510
+ "learning_rate": 3.630508474576271e-05,
1511
+ "loss": 1.3617,
1512
+ "step": 19300
1513
+ },
1514
+ {
1515
+ "epoch": 1.62,
1516
+ "grad_norm": 1.8540898561477661,
1517
+ "learning_rate": 3.596610169491525e-05,
1518
+ "loss": 1.3393,
1519
+ "step": 19400
1520
+ },
1521
+ {
1522
+ "epoch": 1.62,
1523
+ "grad_norm": 4.376151084899902,
1524
+ "learning_rate": 3.56271186440678e-05,
1525
+ "loss": 1.304,
1526
+ "step": 19500
1527
+ },
1528
+ {
1529
+ "epoch": 1.63,
1530
+ "grad_norm": 3.297484874725342,
1531
+ "learning_rate": 3.528813559322034e-05,
1532
+ "loss": 1.276,
1533
+ "step": 19600
1534
+ },
1535
+ {
1536
+ "epoch": 1.64,
1537
+ "grad_norm": 3.258236885070801,
1538
+ "learning_rate": 3.4949152542372885e-05,
1539
+ "loss": 1.3191,
1540
+ "step": 19700
1541
+ },
1542
+ {
1543
+ "epoch": 1.65,
1544
+ "grad_norm": 4.905130863189697,
1545
+ "learning_rate": 3.4610169491525425e-05,
1546
+ "loss": 1.3228,
1547
+ "step": 19800
1548
+ },
1549
+ {
1550
+ "epoch": 1.66,
1551
+ "grad_norm": 2.611107110977173,
1552
+ "learning_rate": 3.4271186440677964e-05,
1553
+ "loss": 1.2464,
1554
+ "step": 19900
1555
+ },
1556
+ {
1557
+ "epoch": 1.67,
1558
+ "grad_norm": 4.044171333312988,
1559
+ "learning_rate": 3.393220338983051e-05,
1560
+ "loss": 1.2852,
1561
+ "step": 20000
1562
+ },
1563
+ {
1564
+ "epoch": 1.67,
1565
+ "eval_loss": 1.322507619857788,
1566
+ "eval_runtime": 93.0207,
1567
+ "eval_samples_per_second": 10.75,
1568
+ "eval_steps_per_second": 2.688,
1569
+ "step": 20000
1570
+ },
1571
+ {
1572
+ "epoch": 1.68,
1573
+ "grad_norm": 2.803262710571289,
1574
+ "learning_rate": 3.359322033898305e-05,
1575
+ "loss": 1.32,
1576
+ "step": 20100
1577
+ },
1578
+ {
1579
+ "epoch": 1.68,
1580
+ "grad_norm": 1.6002579927444458,
1581
+ "learning_rate": 3.325423728813559e-05,
1582
+ "loss": 1.2831,
1583
+ "step": 20200
1584
+ },
1585
+ {
1586
+ "epoch": 1.69,
1587
+ "grad_norm": 5.442947864532471,
1588
+ "learning_rate": 3.2915254237288136e-05,
1589
+ "loss": 1.2849,
1590
+ "step": 20300
1591
+ },
1592
+ {
1593
+ "epoch": 1.7,
1594
+ "grad_norm": 3.7041471004486084,
1595
+ "learning_rate": 3.257627118644068e-05,
1596
+ "loss": 1.2846,
1597
+ "step": 20400
1598
+ },
1599
+ {
1600
+ "epoch": 1.71,
1601
+ "grad_norm": 4.188296794891357,
1602
+ "learning_rate": 3.223728813559322e-05,
1603
+ "loss": 1.3026,
1604
+ "step": 20500
1605
+ },
1606
+ {
1607
+ "epoch": 1.72,
1608
+ "grad_norm": 3.6854124069213867,
1609
+ "learning_rate": 3.189830508474577e-05,
1610
+ "loss": 1.2429,
1611
+ "step": 20600
1612
+ },
1613
+ {
1614
+ "epoch": 1.73,
1615
+ "grad_norm": 3.1174068450927734,
1616
+ "learning_rate": 3.155932203389831e-05,
1617
+ "loss": 1.2728,
1618
+ "step": 20700
1619
+ },
1620
+ {
1621
+ "epoch": 1.73,
1622
+ "grad_norm": 1.2928777933120728,
1623
+ "learning_rate": 3.1220338983050854e-05,
1624
+ "loss": 1.291,
1625
+ "step": 20800
1626
+ },
1627
+ {
1628
+ "epoch": 1.74,
1629
+ "grad_norm": 3.058570146560669,
1630
+ "learning_rate": 3.088135593220339e-05,
1631
+ "loss": 1.2786,
1632
+ "step": 20900
1633
+ },
1634
+ {
1635
+ "epoch": 1.75,
1636
+ "grad_norm": 3.311408042907715,
1637
+ "learning_rate": 3.054237288135593e-05,
1638
+ "loss": 1.2607,
1639
+ "step": 21000
1640
+ },
1641
+ {
1642
+ "epoch": 1.75,
1643
+ "eval_loss": 1.2871655225753784,
1644
+ "eval_runtime": 93.0004,
1645
+ "eval_samples_per_second": 10.753,
1646
+ "eval_steps_per_second": 2.688,
1647
+ "step": 21000
1648
+ },
1649
+ {
1650
+ "epoch": 1.76,
1651
+ "grad_norm": 3.0149199962615967,
1652
+ "learning_rate": 3.0203389830508476e-05,
1653
+ "loss": 1.2897,
1654
+ "step": 21100
1655
+ },
1656
+ {
1657
+ "epoch": 1.77,
1658
+ "grad_norm": 3.8012924194335938,
1659
+ "learning_rate": 2.986440677966102e-05,
1660
+ "loss": 1.3003,
1661
+ "step": 21200
1662
+ },
1663
+ {
1664
+ "epoch": 1.77,
1665
+ "grad_norm": 1.5058399438858032,
1666
+ "learning_rate": 2.952542372881356e-05,
1667
+ "loss": 1.245,
1668
+ "step": 21300
1669
+ },
1670
+ {
1671
+ "epoch": 1.78,
1672
+ "grad_norm": 5.085143089294434,
1673
+ "learning_rate": 2.9186440677966104e-05,
1674
+ "loss": 1.2702,
1675
+ "step": 21400
1676
+ },
1677
+ {
1678
+ "epoch": 1.79,
1679
+ "grad_norm": 1.6351230144500732,
1680
+ "learning_rate": 2.8847457627118647e-05,
1681
+ "loss": 1.28,
1682
+ "step": 21500
1683
+ },
1684
+ {
1685
+ "epoch": 1.8,
1686
+ "grad_norm": 3.3916802406311035,
1687
+ "learning_rate": 2.8508474576271187e-05,
1688
+ "loss": 1.2621,
1689
+ "step": 21600
1690
+ },
1691
+ {
1692
+ "epoch": 1.81,
1693
+ "grad_norm": 2.4449045658111572,
1694
+ "learning_rate": 2.816949152542373e-05,
1695
+ "loss": 1.2968,
1696
+ "step": 21700
1697
+ },
1698
+ {
1699
+ "epoch": 1.82,
1700
+ "grad_norm": 2.8779613971710205,
1701
+ "learning_rate": 2.7830508474576273e-05,
1702
+ "loss": 1.3546,
1703
+ "step": 21800
1704
+ },
1705
+ {
1706
+ "epoch": 1.82,
1707
+ "grad_norm": 1.7994945049285889,
1708
+ "learning_rate": 2.7491525423728816e-05,
1709
+ "loss": 1.2766,
1710
+ "step": 21900
1711
+ },
1712
+ {
1713
+ "epoch": 1.83,
1714
+ "grad_norm": 3.81644344329834,
1715
+ "learning_rate": 2.7152542372881355e-05,
1716
+ "loss": 1.2648,
1717
+ "step": 22000
1718
+ },
1719
+ {
1720
+ "epoch": 1.83,
1721
+ "eval_loss": 1.322618007659912,
1722
+ "eval_runtime": 92.9974,
1723
+ "eval_samples_per_second": 10.753,
1724
+ "eval_steps_per_second": 2.688,
1725
+ "step": 22000
1726
+ },
1727
+ {
1728
+ "epoch": 1.84,
1729
+ "grad_norm": 3.8521928787231445,
1730
+ "learning_rate": 2.6813559322033898e-05,
1731
+ "loss": 1.2689,
1732
+ "step": 22100
1733
+ },
1734
+ {
1735
+ "epoch": 1.85,
1736
+ "grad_norm": 3.50951886177063,
1737
+ "learning_rate": 2.647457627118644e-05,
1738
+ "loss": 1.3133,
1739
+ "step": 22200
1740
+ },
1741
+ {
1742
+ "epoch": 1.86,
1743
+ "grad_norm": 2.187098979949951,
1744
+ "learning_rate": 2.6135593220338984e-05,
1745
+ "loss": 1.2842,
1746
+ "step": 22300
1747
+ },
1748
+ {
1749
+ "epoch": 1.87,
1750
+ "grad_norm": 2.276773691177368,
1751
+ "learning_rate": 2.5796610169491527e-05,
1752
+ "loss": 1.252,
1753
+ "step": 22400
1754
+ },
1755
+ {
1756
+ "epoch": 1.88,
1757
+ "grad_norm": 1.8711966276168823,
1758
+ "learning_rate": 2.5457627118644066e-05,
1759
+ "loss": 1.2994,
1760
+ "step": 22500
1761
+ },
1762
+ {
1763
+ "epoch": 1.88,
1764
+ "grad_norm": 2.4090774059295654,
1765
+ "learning_rate": 2.511864406779661e-05,
1766
+ "loss": 1.2686,
1767
+ "step": 22600
1768
+ },
1769
+ {
1770
+ "epoch": 1.89,
1771
+ "grad_norm": 4.18037748336792,
1772
+ "learning_rate": 2.4779661016949156e-05,
1773
+ "loss": 1.318,
1774
+ "step": 22700
1775
+ },
1776
+ {
1777
+ "epoch": 1.9,
1778
+ "grad_norm": 3.396822452545166,
1779
+ "learning_rate": 2.4440677966101695e-05,
1780
+ "loss": 1.27,
1781
+ "step": 22800
1782
+ },
1783
+ {
1784
+ "epoch": 1.91,
1785
+ "grad_norm": 3.4309394359588623,
1786
+ "learning_rate": 2.4101694915254238e-05,
1787
+ "loss": 1.2888,
1788
+ "step": 22900
1789
+ },
1790
+ {
1791
+ "epoch": 1.92,
1792
+ "grad_norm": 1.140385627746582,
1793
+ "learning_rate": 2.376271186440678e-05,
1794
+ "loss": 1.2936,
1795
+ "step": 23000
1796
+ },
1797
+ {
1798
+ "epoch": 1.92,
1799
+ "eval_loss": 1.3380318880081177,
1800
+ "eval_runtime": 93.0298,
1801
+ "eval_samples_per_second": 10.749,
1802
+ "eval_steps_per_second": 2.687,
1803
+ "step": 23000
1804
+ },
1805
+ {
1806
+ "epoch": 1.93,
1807
+ "grad_norm": 1.4485564231872559,
1808
+ "learning_rate": 2.3423728813559324e-05,
1809
+ "loss": 1.2744,
1810
+ "step": 23100
1811
+ },
1812
+ {
1813
+ "epoch": 1.93,
1814
+ "grad_norm": 2.8173444271087646,
1815
+ "learning_rate": 2.3084745762711867e-05,
1816
+ "loss": 1.2774,
1817
+ "step": 23200
1818
+ },
1819
+ {
1820
+ "epoch": 1.94,
1821
+ "grad_norm": 2.9735991954803467,
1822
+ "learning_rate": 2.2745762711864406e-05,
1823
+ "loss": 1.3241,
1824
+ "step": 23300
1825
+ },
1826
+ {
1827
+ "epoch": 1.95,
1828
+ "grad_norm": 2.011720657348633,
1829
+ "learning_rate": 2.240677966101695e-05,
1830
+ "loss": 1.2912,
1831
+ "step": 23400
1832
+ },
1833
+ {
1834
+ "epoch": 1.96,
1835
+ "grad_norm": 2.2044718265533447,
1836
+ "learning_rate": 2.2067796610169492e-05,
1837
+ "loss": 1.2904,
1838
+ "step": 23500
1839
+ },
1840
+ {
1841
+ "epoch": 1.97,
1842
+ "grad_norm": 3.7060163021087646,
1843
+ "learning_rate": 2.1728813559322035e-05,
1844
+ "loss": 1.2289,
1845
+ "step": 23600
1846
+ },
1847
+ {
1848
+ "epoch": 1.98,
1849
+ "grad_norm": 4.124302864074707,
1850
+ "learning_rate": 2.1389830508474575e-05,
1851
+ "loss": 1.2544,
1852
+ "step": 23700
1853
+ },
1854
+ {
1855
+ "epoch": 1.98,
1856
+ "grad_norm": 3.379523992538452,
1857
+ "learning_rate": 2.105084745762712e-05,
1858
+ "loss": 1.2604,
1859
+ "step": 23800
1860
+ },
1861
+ {
1862
+ "epoch": 1.99,
1863
+ "grad_norm": 7.474296569824219,
1864
+ "learning_rate": 2.0711864406779664e-05,
1865
+ "loss": 1.274,
1866
+ "step": 23900
1867
+ },
1868
+ {
1869
+ "epoch": 2.0,
1870
+ "grad_norm": 5.122653007507324,
1871
+ "learning_rate": 2.0372881355932207e-05,
1872
+ "loss": 1.2985,
1873
+ "step": 24000
1874
+ },
1875
+ {
1876
+ "epoch": 2.0,
1877
+ "eval_loss": 1.274635910987854,
1878
+ "eval_runtime": 93.0906,
1879
+ "eval_samples_per_second": 10.742,
1880
+ "eval_steps_per_second": 2.686,
1881
+ "step": 24000
1882
+ },
1883
+ {
1884
+ "epoch": 2.01,
1885
+ "grad_norm": 2.563617467880249,
1886
+ "learning_rate": 2.0033898305084746e-05,
1887
+ "loss": 1.1996,
1888
+ "step": 24100
1889
+ },
1890
+ {
1891
+ "epoch": 2.02,
1892
+ "grad_norm": 3.481621265411377,
1893
+ "learning_rate": 1.969491525423729e-05,
1894
+ "loss": 1.2156,
1895
+ "step": 24200
1896
+ },
1897
+ {
1898
+ "epoch": 2.02,
1899
+ "grad_norm": 2.6262869834899902,
1900
+ "learning_rate": 1.9355932203389832e-05,
1901
+ "loss": 1.221,
1902
+ "step": 24300
1903
+ },
1904
+ {
1905
+ "epoch": 2.03,
1906
+ "grad_norm": 3.437523603439331,
1907
+ "learning_rate": 1.9016949152542375e-05,
1908
+ "loss": 1.1814,
1909
+ "step": 24400
1910
+ },
1911
+ {
1912
+ "epoch": 2.04,
1913
+ "grad_norm": 4.948596477508545,
1914
+ "learning_rate": 1.8677966101694915e-05,
1915
+ "loss": 1.1972,
1916
+ "step": 24500
1917
+ },
1918
+ {
1919
+ "epoch": 2.05,
1920
+ "grad_norm": 3.0667967796325684,
1921
+ "learning_rate": 1.8338983050847458e-05,
1922
+ "loss": 1.2229,
1923
+ "step": 24600
1924
+ },
1925
+ {
1926
+ "epoch": 2.06,
1927
+ "grad_norm": 2.1678037643432617,
1928
+ "learning_rate": 1.8e-05,
1929
+ "loss": 1.182,
1930
+ "step": 24700
1931
+ },
1932
+ {
1933
+ "epoch": 2.07,
1934
+ "grad_norm": 2.7943167686462402,
1935
+ "learning_rate": 1.7661016949152543e-05,
1936
+ "loss": 1.1889,
1937
+ "step": 24800
1938
+ },
1939
+ {
1940
+ "epoch": 2.08,
1941
+ "grad_norm": 1.5044032335281372,
1942
+ "learning_rate": 1.7322033898305083e-05,
1943
+ "loss": 1.1796,
1944
+ "step": 24900
1945
+ },
1946
+ {
1947
+ "epoch": 2.08,
1948
+ "grad_norm": 3.931037425994873,
1949
+ "learning_rate": 1.6983050847457626e-05,
1950
+ "loss": 1.2159,
1951
+ "step": 25000
1952
+ },
1953
+ {
1954
+ "epoch": 2.08,
1955
+ "eval_loss": 1.33604097366333,
1956
+ "eval_runtime": 93.1418,
1957
+ "eval_samples_per_second": 10.736,
1958
+ "eval_steps_per_second": 2.684,
1959
+ "step": 25000
1960
+ },
1961
+ {
1962
+ "epoch": 2.09,
1963
+ "grad_norm": 5.764918804168701,
1964
+ "learning_rate": 1.6644067796610172e-05,
1965
+ "loss": 1.1995,
1966
+ "step": 25100
1967
+ },
1968
+ {
1969
+ "epoch": 2.1,
1970
+ "grad_norm": 2.17740535736084,
1971
+ "learning_rate": 1.6305084745762715e-05,
1972
+ "loss": 1.1873,
1973
+ "step": 25200
1974
+ },
1975
+ {
1976
+ "epoch": 2.11,
1977
+ "grad_norm": 3.145355224609375,
1978
+ "learning_rate": 1.5966101694915255e-05,
1979
+ "loss": 1.2101,
1980
+ "step": 25300
1981
+ },
1982
+ {
1983
+ "epoch": 2.12,
1984
+ "grad_norm": 5.353292942047119,
1985
+ "learning_rate": 1.563050847457627e-05,
1986
+ "loss": 1.2076,
1987
+ "step": 25400
1988
+ },
1989
+ {
1990
+ "epoch": 2.12,
1991
+ "grad_norm": 2.399566411972046,
1992
+ "learning_rate": 1.5291525423728814e-05,
1993
+ "loss": 1.2099,
1994
+ "step": 25500
1995
+ },
1996
+ {
1997
+ "epoch": 2.13,
1998
+ "grad_norm": 3.8052713871002197,
1999
+ "learning_rate": 1.4952542372881357e-05,
2000
+ "loss": 1.1932,
2001
+ "step": 25600
2002
+ },
2003
+ {
2004
+ "epoch": 2.14,
2005
+ "grad_norm": 4.528957843780518,
2006
+ "learning_rate": 1.4613559322033898e-05,
2007
+ "loss": 1.1838,
2008
+ "step": 25700
2009
+ },
2010
+ {
2011
+ "epoch": 2.15,
2012
+ "grad_norm": 3.597487449645996,
2013
+ "learning_rate": 1.4274576271186441e-05,
2014
+ "loss": 1.187,
2015
+ "step": 25800
2016
+ },
2017
+ {
2018
+ "epoch": 2.16,
2019
+ "grad_norm": 3.2705302238464355,
2020
+ "learning_rate": 1.3935593220338982e-05,
2021
+ "loss": 1.1829,
2022
+ "step": 25900
2023
+ },
2024
+ {
2025
+ "epoch": 2.17,
2026
+ "grad_norm": 3.251126289367676,
2027
+ "learning_rate": 1.3596610169491525e-05,
2028
+ "loss": 1.1888,
2029
+ "step": 26000
2030
+ },
2031
+ {
2032
+ "epoch": 2.17,
2033
+ "eval_loss": 1.3534026145935059,
2034
+ "eval_runtime": 93.1511,
2035
+ "eval_samples_per_second": 10.735,
2036
+ "eval_steps_per_second": 2.684,
2037
+ "step": 26000
2038
+ },
2039
+ {
2040
+ "epoch": 2.17,
2041
+ "grad_norm": 3.9193084239959717,
2042
+ "learning_rate": 1.325762711864407e-05,
2043
+ "loss": 1.2176,
2044
+ "step": 26100
2045
+ },
2046
+ {
2047
+ "epoch": 2.18,
2048
+ "grad_norm": 1.7747070789337158,
2049
+ "learning_rate": 1.2918644067796613e-05,
2050
+ "loss": 1.1845,
2051
+ "step": 26200
2052
+ },
2053
+ {
2054
+ "epoch": 2.19,
2055
+ "grad_norm": 1.7094638347625732,
2056
+ "learning_rate": 1.2579661016949154e-05,
2057
+ "loss": 1.2025,
2058
+ "step": 26300
2059
+ },
2060
+ {
2061
+ "epoch": 2.2,
2062
+ "grad_norm": 4.238964557647705,
2063
+ "learning_rate": 1.2240677966101695e-05,
2064
+ "loss": 1.1665,
2065
+ "step": 26400
2066
+ },
2067
+ {
2068
+ "epoch": 2.21,
2069
+ "grad_norm": 1.669313669204712,
2070
+ "learning_rate": 1.1901694915254238e-05,
2071
+ "loss": 1.1792,
2072
+ "step": 26500
2073
+ },
2074
+ {
2075
+ "epoch": 2.22,
2076
+ "grad_norm": 8.226383209228516,
2077
+ "learning_rate": 1.1562711864406781e-05,
2078
+ "loss": 1.1591,
2079
+ "step": 26600
2080
+ },
2081
+ {
2082
+ "epoch": 2.23,
2083
+ "grad_norm": 3.0040552616119385,
2084
+ "learning_rate": 1.1223728813559322e-05,
2085
+ "loss": 1.2112,
2086
+ "step": 26700
2087
+ },
2088
+ {
2089
+ "epoch": 2.23,
2090
+ "grad_norm": 1.9113960266113281,
2091
+ "learning_rate": 1.0884745762711865e-05,
2092
+ "loss": 1.164,
2093
+ "step": 26800
2094
+ },
2095
+ {
2096
+ "epoch": 2.24,
2097
+ "grad_norm": 2.2811169624328613,
2098
+ "learning_rate": 1.0545762711864406e-05,
2099
+ "loss": 1.1807,
2100
+ "step": 26900
2101
+ },
2102
+ {
2103
+ "epoch": 2.25,
2104
+ "grad_norm": 3.2554876804351807,
2105
+ "learning_rate": 1.020677966101695e-05,
2106
+ "loss": 1.2362,
2107
+ "step": 27000
2108
+ },
2109
+ {
2110
+ "epoch": 2.25,
2111
+ "eval_loss": 1.315274715423584,
2112
+ "eval_runtime": 93.1384,
2113
+ "eval_samples_per_second": 10.737,
2114
+ "eval_steps_per_second": 2.684,
2115
+ "step": 27000
2116
+ },
2117
+ {
2118
+ "epoch": 2.26,
2119
+ "grad_norm": 3.656125545501709,
2120
+ "learning_rate": 9.867796610169492e-06,
2121
+ "loss": 1.1712,
2122
+ "step": 27100
2123
+ },
2124
+ {
2125
+ "epoch": 2.27,
2126
+ "grad_norm": 4.444190502166748,
2127
+ "learning_rate": 9.528813559322035e-06,
2128
+ "loss": 1.2532,
2129
+ "step": 27200
2130
+ },
2131
+ {
2132
+ "epoch": 2.27,
2133
+ "grad_norm": 5.304549217224121,
2134
+ "learning_rate": 9.189830508474576e-06,
2135
+ "loss": 1.1848,
2136
+ "step": 27300
2137
+ },
2138
+ {
2139
+ "epoch": 2.28,
2140
+ "grad_norm": 4.003425121307373,
2141
+ "learning_rate": 8.85084745762712e-06,
2142
+ "loss": 1.1601,
2143
+ "step": 27400
2144
+ },
2145
+ {
2146
+ "epoch": 2.29,
2147
+ "grad_norm": 4.701766490936279,
2148
+ "learning_rate": 8.511864406779662e-06,
2149
+ "loss": 1.1682,
2150
+ "step": 27500
2151
+ },
2152
+ {
2153
+ "epoch": 2.3,
2154
+ "grad_norm": 4.16826057434082,
2155
+ "learning_rate": 8.172881355932203e-06,
2156
+ "loss": 1.1806,
2157
+ "step": 27600
2158
+ },
2159
+ {
2160
+ "epoch": 2.31,
2161
+ "grad_norm": 3.569227695465088,
2162
+ "learning_rate": 7.833898305084746e-06,
2163
+ "loss": 1.1295,
2164
+ "step": 27700
2165
+ },
2166
+ {
2167
+ "epoch": 2.32,
2168
+ "grad_norm": 6.8756890296936035,
2169
+ "learning_rate": 7.494915254237289e-06,
2170
+ "loss": 1.1863,
2171
+ "step": 27800
2172
+ },
2173
+ {
2174
+ "epoch": 2.33,
2175
+ "grad_norm": 4.841982364654541,
2176
+ "learning_rate": 7.155932203389831e-06,
2177
+ "loss": 1.1651,
2178
+ "step": 27900
2179
+ },
2180
+ {
2181
+ "epoch": 2.33,
2182
+ "grad_norm": 6.766154766082764,
2183
+ "learning_rate": 6.816949152542373e-06,
2184
+ "loss": 1.1831,
2185
+ "step": 28000
2186
+ },
2187
+ {
2188
+ "epoch": 2.33,
2189
+ "eval_loss": 1.3225314617156982,
2190
+ "eval_runtime": 93.1704,
2191
+ "eval_samples_per_second": 10.733,
2192
+ "eval_steps_per_second": 2.683,
2193
+ "step": 28000
2194
+ },
2195
+ {
2196
+ "epoch": 2.34,
2197
+ "grad_norm": 4.122812747955322,
2198
+ "learning_rate": 6.477966101694915e-06,
2199
+ "loss": 1.1934,
2200
+ "step": 28100
2201
+ },
2202
+ {
2203
+ "epoch": 2.35,
2204
+ "grad_norm": 2.2169628143310547,
2205
+ "learning_rate": 6.1389830508474575e-06,
2206
+ "loss": 1.1044,
2207
+ "step": 28200
2208
+ },
2209
+ {
2210
+ "epoch": 2.36,
2211
+ "grad_norm": 3.0423035621643066,
2212
+ "learning_rate": 5.8e-06,
2213
+ "loss": 1.1876,
2214
+ "step": 28300
2215
+ },
2216
+ {
2217
+ "epoch": 2.37,
2218
+ "grad_norm": 3.517133951187134,
2219
+ "learning_rate": 5.4610169491525424e-06,
2220
+ "loss": 1.2411,
2221
+ "step": 28400
2222
+ },
2223
+ {
2224
+ "epoch": 2.38,
2225
+ "grad_norm": 3.3230624198913574,
2226
+ "learning_rate": 5.1220338983050845e-06,
2227
+ "loss": 1.189,
2228
+ "step": 28500
2229
+ },
2230
+ {
2231
+ "epoch": 2.38,
2232
+ "grad_norm": 5.101288795471191,
2233
+ "learning_rate": 4.7830508474576274e-06,
2234
+ "loss": 1.1435,
2235
+ "step": 28600
2236
+ },
2237
+ {
2238
+ "epoch": 2.39,
2239
+ "grad_norm": 3.061377763748169,
2240
+ "learning_rate": 4.4440677966101695e-06,
2241
+ "loss": 1.1902,
2242
+ "step": 28700
2243
+ },
2244
+ {
2245
+ "epoch": 2.4,
2246
+ "grad_norm": 3.9341752529144287,
2247
+ "learning_rate": 4.1050847457627124e-06,
2248
+ "loss": 1.22,
2249
+ "step": 28800
2250
+ },
2251
+ {
2252
+ "epoch": 2.41,
2253
+ "grad_norm": 3.691490888595581,
2254
+ "learning_rate": 3.766101694915254e-06,
2255
+ "loss": 1.2105,
2256
+ "step": 28900
2257
+ },
2258
+ {
2259
+ "epoch": 2.42,
2260
+ "grad_norm": 3.67085599899292,
2261
+ "learning_rate": 3.427118644067797e-06,
2262
+ "loss": 1.1909,
2263
+ "step": 29000
2264
+ },
2265
+ {
2266
+ "epoch": 2.42,
2267
+ "eval_loss": 1.294958472251892,
2268
+ "eval_runtime": 93.09,
2269
+ "eval_samples_per_second": 10.742,
2270
+ "eval_steps_per_second": 2.686,
2271
+ "step": 29000
2272
+ },
2273
+ {
2274
+ "epoch": 2.42,
2275
+ "grad_norm": 4.373409748077393,
2276
+ "learning_rate": 3.088135593220339e-06,
2277
+ "loss": 1.2423,
2278
+ "step": 29100
2279
+ },
2280
+ {
2281
+ "epoch": 2.43,
2282
+ "grad_norm": 4.1964006423950195,
2283
+ "learning_rate": 2.7491525423728816e-06,
2284
+ "loss": 1.1482,
2285
+ "step": 29200
2286
+ },
2287
+ {
2288
+ "epoch": 2.44,
2289
+ "grad_norm": 4.233890056610107,
2290
+ "learning_rate": 2.4135593220338984e-06,
2291
+ "loss": 1.1824,
2292
+ "step": 29300
2293
+ },
2294
+ {
2295
+ "epoch": 2.45,
2296
+ "grad_norm": 3.9279072284698486,
2297
+ "learning_rate": 2.0745762711864404e-06,
2298
+ "loss": 1.1466,
2299
+ "step": 29400
2300
+ },
2301
+ {
2302
+ "epoch": 2.46,
2303
+ "grad_norm": 3.39454984664917,
2304
+ "learning_rate": 1.7355932203389831e-06,
2305
+ "loss": 1.1302,
2306
+ "step": 29500
2307
+ },
2308
+ {
2309
+ "epoch": 2.47,
2310
+ "grad_norm": 1.3635748624801636,
2311
+ "learning_rate": 1.3966101694915254e-06,
2312
+ "loss": 1.214,
2313
+ "step": 29600
2314
+ },
2315
+ {
2316
+ "epoch": 2.48,
2317
+ "grad_norm": 3.5587875843048096,
2318
+ "learning_rate": 1.057627118644068e-06,
2319
+ "loss": 1.1981,
2320
+ "step": 29700
2321
+ },
2322
+ {
2323
+ "epoch": 2.48,
2324
+ "grad_norm": 7.449352741241455,
2325
+ "learning_rate": 7.186440677966102e-07,
2326
+ "loss": 1.1196,
2327
+ "step": 29800
2328
+ },
2329
+ {
2330
+ "epoch": 2.49,
2331
+ "grad_norm": 2.7902631759643555,
2332
+ "learning_rate": 3.796610169491526e-07,
2333
+ "loss": 1.1452,
2334
+ "step": 29900
2335
+ },
2336
+ {
2337
+ "epoch": 2.5,
2338
+ "grad_norm": 4.423699855804443,
2339
+ "learning_rate": 4.067796610169492e-08,
2340
+ "loss": 1.197,
2341
+ "step": 30000
2342
+ },
2343
+ {
2344
+ "epoch": 2.5,
2345
+ "eval_loss": 1.2965623140335083,
2346
+ "eval_runtime": 93.1115,
2347
+ "eval_samples_per_second": 10.74,
2348
+ "eval_steps_per_second": 2.685,
2349
+ "step": 30000
2350
+ }
2351
+ ],
2352
+ "logging_steps": 100,
2353
+ "max_steps": 30000,
2354
+ "num_input_tokens_seen": 0,
2355
+ "num_train_epochs": 3,
2356
+ "save_steps": 2000,
2357
+ "total_flos": 4.8306377981952e+17,
2358
+ "train_batch_size": 4,
2359
+ "trial_name": null,
2360
+ "trial_params": null
2361
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2b88ba55a2975145e701c8ea2dfa9c2fa18c0f92e0e08a2c67c81e0266309c6
3
+ size 4856