Hugging Face's logo

cs552-mlp
/
phi3-lora-sciq3

PEFT
Safetensors
Model card Files Community
mikasenghaas commited on
Commit
aa54da2
•
1 Parent(s): 6210108

Upload folder using huggingface_hub

Browse files
Files changed (25) hide show
  1. README.md +202 -0
  2. adapter_config.json +31 -0
  3. adapter_model.safetensors +3 -0
  4. added_tokens.json +14 -0
  5. all_results.json +7 -0
  6. checkpoint-120/README.md +202 -0
  7. checkpoint-120/adapter_config.json +31 -0
  8. checkpoint-120/adapter_model.safetensors +3 -0
  9. checkpoint-120/added_tokens.json +14 -0
  10. checkpoint-120/optimizer.pt +3 -0
  11. checkpoint-120/rng_state.pth +3 -0
  12. checkpoint-120/scheduler.pt +3 -0
  13. checkpoint-120/special_tokens_map.json +24 -0
  14. checkpoint-120/tokenizer.json +0 -0
  15. checkpoint-120/tokenizer.model +3 -0
  16. checkpoint-120/tokenizer_config.json +139 -0
  17. checkpoint-120/trainer_state.json +1833 -0
  18. checkpoint-120/training_args.bin +3 -0
  19. special_tokens_map.json +24 -0
  20. tokenizer.json +0 -0
  21. tokenizer.model +3 -0
  22. tokenizer_config.json +139 -0
  23. train_results.json +7 -0
  24. trainer_state.json +1977 -0
  25. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ base_model: microsoft/Phi-3-mini-4k-instruct
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.11.1
adapter_config.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "microsoft/Phi-3-mini-4k-instruct",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 16,
14
+ "lora_dropout": 0.05,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": null,
18
+ "peft_type": "LORA",
19
+ "r": 32,
20
+ "rank_pattern": {},
21
+ "revision": null,
22
+ "target_modules": [
23
+ "down_proj",
24
+ "qkv_proj",
25
+ "o_proj",
26
+ "gate_up_proj"
27
+ ],
28
+ "task_type": "CAUSAL_LM",
29
+ "use_dora": false,
30
+ "use_rslora": false
31
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:972089b9330fa852f01cf692c54a7be88b386073f7d89f2c62f2f56432740621
3
+ size 201361312
added_tokens.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "<|assistant|>": 32001,
3
+ "<|endoftext|>": 32000,
4
+ "<|end|>": 32007,
5
+ "<|placeholder1|>": 32002,
6
+ "<|placeholder2|>": 32003,
7
+ "<|placeholder3|>": 32004,
8
+ "<|placeholder4|>": 32005,
9
+ "<|placeholder5|>": 32008,
10
+ "<|placeholder6|>": 32009,
11
+ "<|system|>": 32006,
12
+ "<|user|>": 32010,
13
+ "eos_token": 32011
14
+ }
all_results.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 3.0,
3
+ "train_loss": 1.1705420798109483,
4
+ "train_runtime": 2551.3345,
5
+ "train_samples_per_second": 1.594,
6
+ "train_steps_per_second": 0.051
7
+ }
checkpoint-120/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: peft
3
+ base_model: microsoft/Phi-3-mini-4k-instruct
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.11.1
checkpoint-120/adapter_config.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "microsoft/Phi-3-mini-4k-instruct",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 16,
14
+ "lora_dropout": 0.05,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": null,
18
+ "peft_type": "LORA",
19
+ "r": 32,
20
+ "rank_pattern": {},
21
+ "revision": null,
22
+ "target_modules": [
23
+ "down_proj",
24
+ "qkv_proj",
25
+ "o_proj",
26
+ "gate_up_proj"
27
+ ],
28
+ "task_type": "CAUSAL_LM",
29
+ "use_dora": false,
30
+ "use_rslora": false
31
+ }
checkpoint-120/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ca49c598d09db3db5ce23ae171ced4007d0a4eae01d8201a30847095b594734e
3
+ size 201361312
checkpoint-120/added_tokens.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "<|assistant|>": 32001,
3
+ "<|endoftext|>": 32000,
4
+ "<|end|>": 32007,
5
+ "<|placeholder1|>": 32002,
6
+ "<|placeholder2|>": 32003,
7
+ "<|placeholder3|>": 32004,
8
+ "<|placeholder4|>": 32005,
9
+ "<|placeholder5|>": 32008,
10
+ "<|placeholder6|>": 32009,
11
+ "<|system|>": 32006,
12
+ "<|user|>": 32010,
13
+ "eos_token": 32011
14
+ }
checkpoint-120/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2228c43d8ce0fc2ab5dc88c5cfb402f7c9f85e19a80dd0bdb9c3b5d84fe7105d
3
+ size 402807738
checkpoint-120/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d699b826ec47acd8786a3c713e091eb08badfacb60816a5399d04df532a267db
3
+ size 14180
checkpoint-120/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99b1ddfabaf30893fcf2ed98025417c4c80816fd291b59a16f4a987e2e626179
3
+ size 1064
checkpoint-120/special_tokens_map.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<|endoftext|>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": "<unk>",
17
+ "unk_token": {
18
+ "content": "<unk>",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ }
24
+ }
checkpoint-120/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-120/tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
checkpoint-120/tokenizer_config.json ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "0": {
6
+ "content": "<unk>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "1": {
14
+ "content": "<s>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "2": {
22
+ "content": "</s>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": true,
26
+ "single_word": false,
27
+ "special": false
28
+ },
29
+ "32000": {
30
+ "content": "<|endoftext|>",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ },
37
+ "32001": {
38
+ "content": "<|assistant|>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": true,
42
+ "single_word": false,
43
+ "special": true
44
+ },
45
+ "32002": {
46
+ "content": "<|placeholder1|>",
47
+ "lstrip": false,
48
+ "normalized": false,
49
+ "rstrip": true,
50
+ "single_word": false,
51
+ "special": true
52
+ },
53
+ "32003": {
54
+ "content": "<|placeholder2|>",
55
+ "lstrip": false,
56
+ "normalized": false,
57
+ "rstrip": true,
58
+ "single_word": false,
59
+ "special": true
60
+ },
61
+ "32004": {
62
+ "content": "<|placeholder3|>",
63
+ "lstrip": false,
64
+ "normalized": false,
65
+ "rstrip": true,
66
+ "single_word": false,
67
+ "special": true
68
+ },
69
+ "32005": {
70
+ "content": "<|placeholder4|>",
71
+ "lstrip": false,
72
+ "normalized": false,
73
+ "rstrip": true,
74
+ "single_word": false,
75
+ "special": true
76
+ },
77
+ "32006": {
78
+ "content": "<|system|>",
79
+ "lstrip": false,
80
+ "normalized": false,
81
+ "rstrip": true,
82
+ "single_word": false,
83
+ "special": true
84
+ },
85
+ "32007": {
86
+ "content": "<|end|>",
87
+ "lstrip": false,
88
+ "normalized": false,
89
+ "rstrip": true,
90
+ "single_word": false,
91
+ "special": true
92
+ },
93
+ "32008": {
94
+ "content": "<|placeholder5|>",
95
+ "lstrip": false,
96
+ "normalized": false,
97
+ "rstrip": true,
98
+ "single_word": false,
99
+ "special": true
100
+ },
101
+ "32009": {
102
+ "content": "<|placeholder6|>",
103
+ "lstrip": false,
104
+ "normalized": false,
105
+ "rstrip": true,
106
+ "single_word": false,
107
+ "special": true
108
+ },
109
+ "32010": {
110
+ "content": "<|user|>",
111
+ "lstrip": false,
112
+ "normalized": false,
113
+ "rstrip": true,
114
+ "single_word": false,
115
+ "special": true
116
+ },
117
+ "32011": {
118
+ "content": "eos_token",
119
+ "lstrip": false,
120
+ "normalized": false,
121
+ "rstrip": false,
122
+ "single_word": false,
123
+ "special": true
124
+ }
125
+ },
126
+ "bos_token": "<s>",
127
+ "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}",
128
+ "clean_up_tokenization_spaces": false,
129
+ "eos_token": "<|endoftext|>",
130
+ "legacy": false,
131
+ "model_max_length": 4096,
132
+ "pad_token": "<unk>",
133
+ "padding_side": "right",
134
+ "sp_model_kwargs": {},
135
+ "tokenizer_class": "LlamaTokenizer",
136
+ "truncation_side": "left",
137
+ "unk_token": "<unk>",
138
+ "use_default_system_prompt": false
139
+ }
checkpoint-120/trainer_state.json ADDED
@@ -0,0 +1,1833 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 1.0082221031188965,
3
+ "best_model_checkpoint": "./output/checkpoints/2024-06-11_15-07-46/checkpoint-120",
4
+ "epoch": 2.7906976744186047,
5
+ "eval_steps": 1,
6
+ "global_step": 120,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.023255813953488372,
13
+ "grad_norm": 3.230428695678711,
14
+ "learning_rate": 3.0769230769230774e-05,
15
+ "loss": 5.1513,
16
+ "step": 1
17
+ },
18
+ {
19
+ "epoch": 0.023255813953488372,
20
+ "eval_loss": 5.098567485809326,
21
+ "eval_runtime": 10.4425,
22
+ "eval_samples_per_second": 11.204,
23
+ "eval_steps_per_second": 0.766,
24
+ "step": 1
25
+ },
26
+ {
27
+ "epoch": 0.046511627906976744,
28
+ "grad_norm": 3.204726219177246,
29
+ "learning_rate": 6.153846153846155e-05,
30
+ "loss": 5.1556,
31
+ "step": 2
32
+ },
33
+ {
34
+ "epoch": 0.046511627906976744,
35
+ "eval_loss": 5.000132083892822,
36
+ "eval_runtime": 10.4412,
37
+ "eval_samples_per_second": 11.206,
38
+ "eval_steps_per_second": 0.766,
39
+ "step": 2
40
+ },
41
+ {
42
+ "epoch": 0.06976744186046512,
43
+ "grad_norm": 3.1961917877197266,
44
+ "learning_rate": 9.230769230769232e-05,
45
+ "loss": 4.9354,
46
+ "step": 3
47
+ },
48
+ {
49
+ "epoch": 0.06976744186046512,
50
+ "eval_loss": 4.540091514587402,
51
+ "eval_runtime": 10.4665,
52
+ "eval_samples_per_second": 11.178,
53
+ "eval_steps_per_second": 0.764,
54
+ "step": 3
55
+ },
56
+ {
57
+ "epoch": 0.09302325581395349,
58
+ "grad_norm": 3.3630478382110596,
59
+ "learning_rate": 0.0001230769230769231,
60
+ "loss": 4.5394,
61
+ "step": 4
62
+ },
63
+ {
64
+ "epoch": 0.09302325581395349,
65
+ "eval_loss": 3.6481263637542725,
66
+ "eval_runtime": 10.5,
67
+ "eval_samples_per_second": 11.143,
68
+ "eval_steps_per_second": 0.762,
69
+ "step": 4
70
+ },
71
+ {
72
+ "epoch": 0.11627906976744186,
73
+ "grad_norm": 3.4904136657714844,
74
+ "learning_rate": 0.00015384615384615385,
75
+ "loss": 3.6545,
76
+ "step": 5
77
+ },
78
+ {
79
+ "epoch": 0.11627906976744186,
80
+ "eval_loss": 2.6286702156066895,
81
+ "eval_runtime": 10.4841,
82
+ "eval_samples_per_second": 11.16,
83
+ "eval_steps_per_second": 0.763,
84
+ "step": 5
85
+ },
86
+ {
87
+ "epoch": 0.13953488372093023,
88
+ "grad_norm": 2.7318434715270996,
89
+ "learning_rate": 0.00018461538461538463,
90
+ "loss": 2.5645,
91
+ "step": 6
92
+ },
93
+ {
94
+ "epoch": 0.13953488372093023,
95
+ "eval_loss": 2.047227144241333,
96
+ "eval_runtime": 10.5229,
97
+ "eval_samples_per_second": 11.119,
98
+ "eval_steps_per_second": 0.76,
99
+ "step": 6
100
+ },
101
+ {
102
+ "epoch": 0.16279069767441862,
103
+ "grad_norm": 2.1703038215637207,
104
+ "learning_rate": 0.0002153846153846154,
105
+ "loss": 2.0145,
106
+ "step": 7
107
+ },
108
+ {
109
+ "epoch": 0.16279069767441862,
110
+ "eval_loss": 1.6754586696624756,
111
+ "eval_runtime": 10.513,
112
+ "eval_samples_per_second": 11.129,
113
+ "eval_steps_per_second": 0.761,
114
+ "step": 7
115
+ },
116
+ {
117
+ "epoch": 0.18604651162790697,
118
+ "grad_norm": 1.1857593059539795,
119
+ "learning_rate": 0.0002461538461538462,
120
+ "loss": 1.6469,
121
+ "step": 8
122
+ },
123
+ {
124
+ "epoch": 0.18604651162790697,
125
+ "eval_loss": 1.4785383939743042,
126
+ "eval_runtime": 10.4789,
127
+ "eval_samples_per_second": 11.165,
128
+ "eval_steps_per_second": 0.763,
129
+ "step": 8
130
+ },
131
+ {
132
+ "epoch": 0.20930232558139536,
133
+ "grad_norm": 0.94368976354599,
134
+ "learning_rate": 0.00027692307692307695,
135
+ "loss": 1.4676,
136
+ "step": 9
137
+ },
138
+ {
139
+ "epoch": 0.20930232558139536,
140
+ "eval_loss": 1.3956246376037598,
141
+ "eval_runtime": 10.5914,
142
+ "eval_samples_per_second": 11.047,
143
+ "eval_steps_per_second": 0.755,
144
+ "step": 9
145
+ },
146
+ {
147
+ "epoch": 0.23255813953488372,
148
+ "grad_norm": 1.3516250848770142,
149
+ "learning_rate": 0.0003076923076923077,
150
+ "loss": 1.377,
151
+ "step": 10
152
+ },
153
+ {
154
+ "epoch": 0.23255813953488372,
155
+ "eval_loss": 1.2937995195388794,
156
+ "eval_runtime": 10.5422,
157
+ "eval_samples_per_second": 11.098,
158
+ "eval_steps_per_second": 0.759,
159
+ "step": 10
160
+ },
161
+ {
162
+ "epoch": 0.2558139534883721,
163
+ "grad_norm": 0.21453160047531128,
164
+ "learning_rate": 0.00033846153846153846,
165
+ "loss": 1.2814,
166
+ "step": 11
167
+ },
168
+ {
169
+ "epoch": 0.2558139534883721,
170
+ "eval_loss": 1.258020281791687,
171
+ "eval_runtime": 10.4428,
172
+ "eval_samples_per_second": 11.204,
173
+ "eval_steps_per_second": 0.766,
174
+ "step": 11
175
+ },
176
+ {
177
+ "epoch": 0.27906976744186046,
178
+ "grad_norm": 0.1224498599767685,
179
+ "learning_rate": 0.00036923076923076927,
180
+ "loss": 1.2261,
181
+ "step": 12
182
+ },
183
+ {
184
+ "epoch": 0.27906976744186046,
185
+ "eval_loss": 1.228270411491394,
186
+ "eval_runtime": 10.5707,
187
+ "eval_samples_per_second": 11.068,
188
+ "eval_steps_per_second": 0.757,
189
+ "step": 12
190
+ },
191
+ {
192
+ "epoch": 0.3023255813953488,
193
+ "grad_norm": 0.11027181148529053,
194
+ "learning_rate": 0.0004,
195
+ "loss": 1.1848,
196
+ "step": 13
197
+ },
198
+ {
199
+ "epoch": 0.3023255813953488,
200
+ "eval_loss": 1.2002977132797241,
201
+ "eval_runtime": 10.5152,
202
+ "eval_samples_per_second": 11.127,
203
+ "eval_steps_per_second": 0.761,
204
+ "step": 13
205
+ },
206
+ {
207
+ "epoch": 0.32558139534883723,
208
+ "grad_norm": 0.14012101292610168,
209
+ "learning_rate": 0.0003965517241379311,
210
+ "loss": 1.1961,
211
+ "step": 14
212
+ },
213
+ {
214
+ "epoch": 0.32558139534883723,
215
+ "eval_loss": 1.1754467487335205,
216
+ "eval_runtime": 10.5013,
217
+ "eval_samples_per_second": 11.141,
218
+ "eval_steps_per_second": 0.762,
219
+ "step": 14
220
+ },
221
+ {
222
+ "epoch": 0.3488372093023256,
223
+ "grad_norm": 0.22193074226379395,
224
+ "learning_rate": 0.0003931034482758621,
225
+ "loss": 1.1407,
226
+ "step": 15
227
+ },
228
+ {
229
+ "epoch": 0.3488372093023256,
230
+ "eval_loss": 1.1523548364639282,
231
+ "eval_runtime": 10.4708,
232
+ "eval_samples_per_second": 11.174,
233
+ "eval_steps_per_second": 0.764,
234
+ "step": 15
235
+ },
236
+ {
237
+ "epoch": 0.37209302325581395,
238
+ "grad_norm": 0.15213920176029205,
239
+ "learning_rate": 0.00038965517241379313,
240
+ "loss": 1.1364,
241
+ "step": 16
242
+ },
243
+ {
244
+ "epoch": 0.37209302325581395,
245
+ "eval_loss": 1.1305698156356812,
246
+ "eval_runtime": 10.5494,
247
+ "eval_samples_per_second": 11.091,
248
+ "eval_steps_per_second": 0.758,
249
+ "step": 16
250
+ },
251
+ {
252
+ "epoch": 0.3953488372093023,
253
+ "grad_norm": 0.08833792805671692,
254
+ "learning_rate": 0.0003862068965517242,
255
+ "loss": 1.1057,
256
+ "step": 17
257
+ },
258
+ {
259
+ "epoch": 0.3953488372093023,
260
+ "eval_loss": 1.1136623620986938,
261
+ "eval_runtime": 10.6102,
262
+ "eval_samples_per_second": 11.027,
263
+ "eval_steps_per_second": 0.754,
264
+ "step": 17
265
+ },
266
+ {
267
+ "epoch": 0.4186046511627907,
268
+ "grad_norm": 0.07902093976736069,
269
+ "learning_rate": 0.0003827586206896552,
270
+ "loss": 1.089,
271
+ "step": 18
272
+ },
273
+ {
274
+ "epoch": 0.4186046511627907,
275
+ "eval_loss": 1.10086989402771,
276
+ "eval_runtime": 10.5681,
277
+ "eval_samples_per_second": 11.071,
278
+ "eval_steps_per_second": 0.757,
279
+ "step": 18
280
+ },
281
+ {
282
+ "epoch": 0.4418604651162791,
283
+ "grad_norm": 0.07194280624389648,
284
+ "learning_rate": 0.00037931034482758624,
285
+ "loss": 1.0849,
286
+ "step": 19
287
+ },
288
+ {
289
+ "epoch": 0.4418604651162791,
290
+ "eval_loss": 1.091064691543579,
291
+ "eval_runtime": 10.5436,
292
+ "eval_samples_per_second": 11.097,
293
+ "eval_steps_per_second": 0.759,
294
+ "step": 19
295
+ },
296
+ {
297
+ "epoch": 0.46511627906976744,
298
+ "grad_norm": 0.05419577285647392,
299
+ "learning_rate": 0.0003758620689655173,
300
+ "loss": 1.0729,
301
+ "step": 20
302
+ },
303
+ {
304
+ "epoch": 0.46511627906976744,
305
+ "eval_loss": 1.0844314098358154,
306
+ "eval_runtime": 10.46,
307
+ "eval_samples_per_second": 11.185,
308
+ "eval_steps_per_second": 0.765,
309
+ "step": 20
310
+ },
311
+ {
312
+ "epoch": 0.4883720930232558,
313
+ "grad_norm": 0.05308014526963234,
314
+ "learning_rate": 0.0003724137931034483,
315
+ "loss": 1.0576,
316
+ "step": 21
317
+ },
318
+ {
319
+ "epoch": 0.4883720930232558,
320
+ "eval_loss": 1.0792510509490967,
321
+ "eval_runtime": 10.5659,
322
+ "eval_samples_per_second": 11.073,
323
+ "eval_steps_per_second": 0.757,
324
+ "step": 21
325
+ },
326
+ {
327
+ "epoch": 0.5116279069767442,
328
+ "grad_norm": 0.05068132281303406,
329
+ "learning_rate": 0.00036896551724137935,
330
+ "loss": 1.0591,
331
+ "step": 22
332
+ },
333
+ {
334
+ "epoch": 0.5116279069767442,
335
+ "eval_loss": 1.074510931968689,
336
+ "eval_runtime": 10.5969,
337
+ "eval_samples_per_second": 11.041,
338
+ "eval_steps_per_second": 0.755,
339
+ "step": 22
340
+ },
341
+ {
342
+ "epoch": 0.5348837209302325,
343
+ "grad_norm": 0.05279520899057388,
344
+ "learning_rate": 0.00036551724137931036,
345
+ "loss": 1.0216,
346
+ "step": 23
347
+ },
348
+ {
349
+ "epoch": 0.5348837209302325,
350
+ "eval_loss": 1.0699939727783203,
351
+ "eval_runtime": 10.5288,
352
+ "eval_samples_per_second": 11.112,
353
+ "eval_steps_per_second": 0.76,
354
+ "step": 23
355
+ },
356
+ {
357
+ "epoch": 0.5581395348837209,
358
+ "grad_norm": 0.051589012145996094,
359
+ "learning_rate": 0.0003620689655172414,
360
+ "loss": 1.0384,
361
+ "step": 24
362
+ },
363
+ {
364
+ "epoch": 0.5581395348837209,
365
+ "eval_loss": 1.0660674571990967,
366
+ "eval_runtime": 10.4984,
367
+ "eval_samples_per_second": 11.145,
368
+ "eval_steps_per_second": 0.762,
369
+ "step": 24
370
+ },
371
+ {
372
+ "epoch": 0.5813953488372093,
373
+ "grad_norm": 0.04975143447518349,
374
+ "learning_rate": 0.0003586206896551724,
375
+ "loss": 1.0754,
376
+ "step": 25
377
+ },
378
+ {
379
+ "epoch": 0.5813953488372093,
380
+ "eval_loss": 1.0627647638320923,
381
+ "eval_runtime": 10.5768,
382
+ "eval_samples_per_second": 11.062,
383
+ "eval_steps_per_second": 0.756,
384
+ "step": 25
385
+ },
386
+ {
387
+ "epoch": 0.6046511627906976,
388
+ "grad_norm": 0.048871301114559174,
389
+ "learning_rate": 0.00035517241379310347,
390
+ "loss": 1.037,
391
+ "step": 26
392
+ },
393
+ {
394
+ "epoch": 0.6046511627906976,
395
+ "eval_loss": 1.0597318410873413,
396
+ "eval_runtime": 10.4738,
397
+ "eval_samples_per_second": 11.171,
398
+ "eval_steps_per_second": 0.764,
399
+ "step": 26
400
+ },
401
+ {
402
+ "epoch": 0.627906976744186,
403
+ "grad_norm": 0.05112334340810776,
404
+ "learning_rate": 0.00035172413793103447,
405
+ "loss": 1.0579,
406
+ "step": 27
407
+ },
408
+ {
409
+ "epoch": 0.627906976744186,
410
+ "eval_loss": 1.0566964149475098,
411
+ "eval_runtime": 10.5953,
412
+ "eval_samples_per_second": 11.043,
413
+ "eval_steps_per_second": 0.755,
414
+ "step": 27
415
+ },
416
+ {
417
+ "epoch": 0.6511627906976745,
418
+ "grad_norm": 0.08299978822469711,
419
+ "learning_rate": 0.0003482758620689655,
420
+ "loss": 1.039,
421
+ "step": 28
422
+ },
423
+ {
424
+ "epoch": 0.6511627906976745,
425
+ "eval_loss": 1.053879976272583,
426
+ "eval_runtime": 10.5064,
427
+ "eval_samples_per_second": 11.136,
428
+ "eval_steps_per_second": 0.761,
429
+ "step": 28
430
+ },
431
+ {
432
+ "epoch": 0.6744186046511628,
433
+ "grad_norm": 0.04733407124876976,
434
+ "learning_rate": 0.0003448275862068965,
435
+ "loss": 1.054,
436
+ "step": 29
437
+ },
438
+ {
439
+ "epoch": 0.6744186046511628,
440
+ "eval_loss": 1.0513161420822144,
441
+ "eval_runtime": 10.5869,
442
+ "eval_samples_per_second": 11.051,
443
+ "eval_steps_per_second": 0.756,
444
+ "step": 29
445
+ },
446
+ {
447
+ "epoch": 0.6976744186046512,
448
+ "grad_norm": 0.045666251331567764,
449
+ "learning_rate": 0.0003413793103448276,
450
+ "loss": 1.0237,
451
+ "step": 30
452
+ },
453
+ {
454
+ "epoch": 0.6976744186046512,
455
+ "eval_loss": 1.0490825176239014,
456
+ "eval_runtime": 10.5294,
457
+ "eval_samples_per_second": 11.112,
458
+ "eval_steps_per_second": 0.76,
459
+ "step": 30
460
+ },
461
+ {
462
+ "epoch": 0.7209302325581395,
463
+ "grad_norm": 0.04347428306937218,
464
+ "learning_rate": 0.00033793103448275864,
465
+ "loss": 1.0238,
466
+ "step": 31
467
+ },
468
+ {
469
+ "epoch": 0.7209302325581395,
470
+ "eval_loss": 1.0473451614379883,
471
+ "eval_runtime": 10.4778,
472
+ "eval_samples_per_second": 11.166,
473
+ "eval_steps_per_second": 0.764,
474
+ "step": 31
475
+ },
476
+ {
477
+ "epoch": 0.7441860465116279,
478
+ "grad_norm": 0.04594381898641586,
479
+ "learning_rate": 0.00033448275862068964,
480
+ "loss": 1.0339,
481
+ "step": 32
482
+ },
483
+ {
484
+ "epoch": 0.7441860465116279,
485
+ "eval_loss": 1.0460575819015503,
486
+ "eval_runtime": 10.4914,
487
+ "eval_samples_per_second": 11.152,
488
+ "eval_steps_per_second": 0.763,
489
+ "step": 32
490
+ },
491
+ {
492
+ "epoch": 0.7674418604651163,
493
+ "grad_norm": 0.049526289105415344,
494
+ "learning_rate": 0.0003310344827586207,
495
+ "loss": 1.0418,
496
+ "step": 33
497
+ },
498
+ {
499
+ "epoch": 0.7674418604651163,
500
+ "eval_loss": 1.04463791847229,
501
+ "eval_runtime": 10.4747,
502
+ "eval_samples_per_second": 11.17,
503
+ "eval_steps_per_second": 0.764,
504
+ "step": 33
505
+ },
506
+ {
507
+ "epoch": 0.7906976744186046,
508
+ "grad_norm": 0.04762812703847885,
509
+ "learning_rate": 0.00032758620689655175,
510
+ "loss": 1.0302,
511
+ "step": 34
512
+ },
513
+ {
514
+ "epoch": 0.7906976744186046,
515
+ "eval_loss": 1.042962670326233,
516
+ "eval_runtime": 10.6089,
517
+ "eval_samples_per_second": 11.028,
518
+ "eval_steps_per_second": 0.754,
519
+ "step": 34
520
+ },
521
+ {
522
+ "epoch": 0.813953488372093,
523
+ "grad_norm": 0.05331471934914589,
524
+ "learning_rate": 0.00032413793103448275,
525
+ "loss": 1.0401,
526
+ "step": 35
527
+ },
528
+ {
529
+ "epoch": 0.813953488372093,
530
+ "eval_loss": 1.0412102937698364,
531
+ "eval_runtime": 10.4972,
532
+ "eval_samples_per_second": 11.146,
533
+ "eval_steps_per_second": 0.762,
534
+ "step": 35
535
+ },
536
+ {
537
+ "epoch": 0.8372093023255814,
538
+ "grad_norm": 0.041780710220336914,
539
+ "learning_rate": 0.0003206896551724138,
540
+ "loss": 1.0038,
541
+ "step": 36
542
+ },
543
+ {
544
+ "epoch": 0.8372093023255814,
545
+ "eval_loss": 1.0397316217422485,
546
+ "eval_runtime": 10.4649,
547
+ "eval_samples_per_second": 11.18,
548
+ "eval_steps_per_second": 0.764,
549
+ "step": 36
550
+ },
551
+ {
552
+ "epoch": 0.8604651162790697,
553
+ "grad_norm": 0.04302731156349182,
554
+ "learning_rate": 0.00031724137931034486,
555
+ "loss": 1.0249,
556
+ "step": 37
557
+ },
558
+ {
559
+ "epoch": 0.8604651162790697,
560
+ "eval_loss": 1.0385717153549194,
561
+ "eval_runtime": 10.5101,
562
+ "eval_samples_per_second": 11.132,
563
+ "eval_steps_per_second": 0.761,
564
+ "step": 37
565
+ },
566
+ {
567
+ "epoch": 0.8837209302325582,
568
+ "grad_norm": 0.0549028106033802,
569
+ "learning_rate": 0.00031379310344827586,
570
+ "loss": 1.0128,
571
+ "step": 38
572
+ },
573
+ {
574
+ "epoch": 0.8837209302325582,
575
+ "eval_loss": 1.0375303030014038,
576
+ "eval_runtime": 10.479,
577
+ "eval_samples_per_second": 11.165,
578
+ "eval_steps_per_second": 0.763,
579
+ "step": 38
580
+ },
581
+ {
582
+ "epoch": 0.9069767441860465,
583
+ "grad_norm": 0.04620625823736191,
584
+ "learning_rate": 0.0003103448275862069,
585
+ "loss": 1.0278,
586
+ "step": 39
587
+ },
588
+ {
589
+ "epoch": 0.9069767441860465,
590
+ "eval_loss": 1.03622567653656,
591
+ "eval_runtime": 10.496,
592
+ "eval_samples_per_second": 11.147,
593
+ "eval_steps_per_second": 0.762,
594
+ "step": 39
595
+ },
596
+ {
597
+ "epoch": 0.9302325581395349,
598
+ "grad_norm": 0.04755247011780739,
599
+ "learning_rate": 0.00030689655172413797,
600
+ "loss": 1.0059,
601
+ "step": 40
602
+ },
603
+ {
604
+ "epoch": 0.9302325581395349,
605
+ "eval_loss": 1.0347106456756592,
606
+ "eval_runtime": 10.522,
607
+ "eval_samples_per_second": 11.12,
608
+ "eval_steps_per_second": 0.76,
609
+ "step": 40
610
+ },
611
+ {
612
+ "epoch": 0.9534883720930233,
613
+ "grad_norm": 0.04391615465283394,
614
+ "learning_rate": 0.00030344827586206897,
615
+ "loss": 1.0234,
616
+ "step": 41
617
+ },
618
+ {
619
+ "epoch": 0.9534883720930233,
620
+ "eval_loss": 1.033333420753479,
621
+ "eval_runtime": 10.5498,
622
+ "eval_samples_per_second": 11.09,
623
+ "eval_steps_per_second": 0.758,
624
+ "step": 41
625
+ },
626
+ {
627
+ "epoch": 0.9767441860465116,
628
+ "grad_norm": 0.04336484521627426,
629
+ "learning_rate": 0.00030000000000000003,
630
+ "loss": 0.9913,
631
+ "step": 42
632
+ },
633
+ {
634
+ "epoch": 0.9767441860465116,
635
+ "eval_loss": 1.0323785543441772,
636
+ "eval_runtime": 10.5082,
637
+ "eval_samples_per_second": 11.134,
638
+ "eval_steps_per_second": 0.761,
639
+ "step": 42
640
+ },
641
+ {
642
+ "epoch": 1.0,
643
+ "grad_norm": 0.07296782732009888,
644
+ "learning_rate": 0.0002965517241379311,
645
+ "loss": 0.9975,
646
+ "step": 43
647
+ },
648
+ {
649
+ "epoch": 1.0,
650
+ "eval_loss": 1.0313116312026978,
651
+ "eval_runtime": 10.5008,
652
+ "eval_samples_per_second": 11.142,
653
+ "eval_steps_per_second": 0.762,
654
+ "step": 43
655
+ },
656
+ {
657
+ "epoch": 1.0232558139534884,
658
+ "grad_norm": 0.041788019239902496,
659
+ "learning_rate": 0.0002931034482758621,
660
+ "loss": 0.9841,
661
+ "step": 44
662
+ },
663
+ {
664
+ "epoch": 1.0232558139534884,
665
+ "eval_loss": 1.0304553508758545,
666
+ "eval_runtime": 10.4812,
667
+ "eval_samples_per_second": 11.163,
668
+ "eval_steps_per_second": 0.763,
669
+ "step": 44
670
+ },
671
+ {
672
+ "epoch": 1.0465116279069768,
673
+ "grad_norm": 0.044039350003004074,
674
+ "learning_rate": 0.00028965517241379314,
675
+ "loss": 0.9804,
676
+ "step": 45
677
+ },
678
+ {
679
+ "epoch": 1.0465116279069768,
680
+ "eval_loss": 1.0297547578811646,
681
+ "eval_runtime": 10.5179,
682
+ "eval_samples_per_second": 11.124,
683
+ "eval_steps_per_second": 0.761,
684
+ "step": 45
685
+ },
686
+ {
687
+ "epoch": 1.069767441860465,
688
+ "grad_norm": 0.04108603671193123,
689
+ "learning_rate": 0.0002862068965517242,
690
+ "loss": 0.9993,
691
+ "step": 46
692
+ },
693
+ {
694
+ "epoch": 1.069767441860465,
695
+ "eval_loss": 1.029006838798523,
696
+ "eval_runtime": 10.536,
697
+ "eval_samples_per_second": 11.105,
698
+ "eval_steps_per_second": 0.759,
699
+ "step": 46
700
+ },
701
+ {
702
+ "epoch": 1.0930232558139534,
703
+ "grad_norm": 0.04773078113794327,
704
+ "learning_rate": 0.0002827586206896552,
705
+ "loss": 0.9748,
706
+ "step": 47
707
+ },
708
+ {
709
+ "epoch": 1.0930232558139534,
710
+ "eval_loss": 1.028093695640564,
711
+ "eval_runtime": 10.5568,
712
+ "eval_samples_per_second": 11.083,
713
+ "eval_steps_per_second": 0.758,
714
+ "step": 47
715
+ },
716
+ {
717
+ "epoch": 1.1162790697674418,
718
+ "grad_norm": 0.04372379183769226,
719
+ "learning_rate": 0.0002793103448275862,
720
+ "loss": 1.0025,
721
+ "step": 48
722
+ },
723
+ {
724
+ "epoch": 1.1162790697674418,
725
+ "eval_loss": 1.027159333229065,
726
+ "eval_runtime": 10.4931,
727
+ "eval_samples_per_second": 11.15,
728
+ "eval_steps_per_second": 0.762,
729
+ "step": 48
730
+ },
731
+ {
732
+ "epoch": 1.1395348837209303,
733
+ "grad_norm": 0.05692731961607933,
734
+ "learning_rate": 0.00027586206896551725,
735
+ "loss": 1.0038,
736
+ "step": 49
737
+ },
738
+ {
739
+ "epoch": 1.1395348837209303,
740
+ "eval_loss": 1.0262141227722168,
741
+ "eval_runtime": 10.5803,
742
+ "eval_samples_per_second": 11.058,
743
+ "eval_steps_per_second": 0.756,
744
+ "step": 49
745
+ },
746
+ {
747
+ "epoch": 1.1627906976744187,
748
+ "grad_norm": 0.04124248027801514,
749
+ "learning_rate": 0.00027241379310344825,
750
+ "loss": 0.9848,
751
+ "step": 50
752
+ },
753
+ {
754
+ "epoch": 1.1627906976744187,
755
+ "eval_loss": 1.0254372358322144,
756
+ "eval_runtime": 10.4687,
757
+ "eval_samples_per_second": 11.176,
758
+ "eval_steps_per_second": 0.764,
759
+ "step": 50
760
+ },
761
+ {
762
+ "epoch": 1.1860465116279069,
763
+ "grad_norm": 0.04371722787618637,
764
+ "learning_rate": 0.0002689655172413793,
765
+ "loss": 1.0188,
766
+ "step": 51
767
+ },
768
+ {
769
+ "epoch": 1.1860465116279069,
770
+ "eval_loss": 1.024755597114563,
771
+ "eval_runtime": 10.5207,
772
+ "eval_samples_per_second": 11.121,
773
+ "eval_steps_per_second": 0.76,
774
+ "step": 51
775
+ },
776
+ {
777
+ "epoch": 1.2093023255813953,
778
+ "grad_norm": 0.04607950523495674,
779
+ "learning_rate": 0.00026551724137931036,
780
+ "loss": 1.0058,
781
+ "step": 52
782
+ },
783
+ {
784
+ "epoch": 1.2093023255813953,
785
+ "eval_loss": 1.0241938829421997,
786
+ "eval_runtime": 10.5231,
787
+ "eval_samples_per_second": 11.118,
788
+ "eval_steps_per_second": 0.76,
789
+ "step": 52
790
+ },
791
+ {
792
+ "epoch": 1.2325581395348837,
793
+ "grad_norm": 0.05415044724941254,
794
+ "learning_rate": 0.00026206896551724137,
795
+ "loss": 0.9923,
796
+ "step": 53
797
+ },
798
+ {
799
+ "epoch": 1.2325581395348837,
800
+ "eval_loss": 1.0235437154769897,
801
+ "eval_runtime": 10.6072,
802
+ "eval_samples_per_second": 11.03,
803
+ "eval_steps_per_second": 0.754,
804
+ "step": 53
805
+ },
806
+ {
807
+ "epoch": 1.255813953488372,
808
+ "grad_norm": 0.06359652429819107,
809
+ "learning_rate": 0.0002586206896551724,
810
+ "loss": 0.9948,
811
+ "step": 54
812
+ },
813
+ {
814
+ "epoch": 1.255813953488372,
815
+ "eval_loss": 1.0229169130325317,
816
+ "eval_runtime": 10.5202,
817
+ "eval_samples_per_second": 11.122,
818
+ "eval_steps_per_second": 0.76,
819
+ "step": 54
820
+ },
821
+ {
822
+ "epoch": 1.2790697674418605,
823
+ "grad_norm": 0.058946821838617325,
824
+ "learning_rate": 0.0002551724137931035,
825
+ "loss": 1.0188,
826
+ "step": 55
827
+ },
828
+ {
829
+ "epoch": 1.2790697674418605,
830
+ "eval_loss": 1.0223369598388672,
831
+ "eval_runtime": 10.4661,
832
+ "eval_samples_per_second": 11.179,
833
+ "eval_steps_per_second": 0.764,
834
+ "step": 55
835
+ },
836
+ {
837
+ "epoch": 1.302325581395349,
838
+ "grad_norm": 0.05592051148414612,
839
+ "learning_rate": 0.0002517241379310345,
840
+ "loss": 0.9933,
841
+ "step": 56
842
+ },
843
+ {
844
+ "epoch": 1.302325581395349,
845
+ "eval_loss": 1.0219457149505615,
846
+ "eval_runtime": 10.5373,
847
+ "eval_samples_per_second": 11.103,
848
+ "eval_steps_per_second": 0.759,
849
+ "step": 56
850
+ },
851
+ {
852
+ "epoch": 1.3255813953488373,
853
+ "grad_norm": 0.05216487497091293,
854
+ "learning_rate": 0.00024827586206896553,
855
+ "loss": 0.9942,
856
+ "step": 57
857
+ },
858
+ {
859
+ "epoch": 1.3255813953488373,
860
+ "eval_loss": 1.0214743614196777,
861
+ "eval_runtime": 10.5663,
862
+ "eval_samples_per_second": 11.073,
863
+ "eval_steps_per_second": 0.757,
864
+ "step": 57
865
+ },
866
+ {
867
+ "epoch": 1.3488372093023255,
868
+ "grad_norm": 0.04820263013243675,
869
+ "learning_rate": 0.00024482758620689653,
870
+ "loss": 0.9758,
871
+ "step": 58
872
+ },
873
+ {
874
+ "epoch": 1.3488372093023255,
875
+ "eval_loss": 1.0210070610046387,
876
+ "eval_runtime": 10.4802,
877
+ "eval_samples_per_second": 11.164,
878
+ "eval_steps_per_second": 0.763,
879
+ "step": 58
880
+ },
881
+ {
882
+ "epoch": 1.372093023255814,
883
+ "grad_norm": 0.04773212969303131,
884
+ "learning_rate": 0.0002413793103448276,
885
+ "loss": 1.0005,
886
+ "step": 59
887
+ },
888
+ {
889
+ "epoch": 1.372093023255814,
890
+ "eval_loss": 1.0203553438186646,
891
+ "eval_runtime": 10.5585,
892
+ "eval_samples_per_second": 11.081,
893
+ "eval_steps_per_second": 0.758,
894
+ "step": 59
895
+ },
896
+ {
897
+ "epoch": 1.3953488372093024,
898
+ "grad_norm": 0.05814950540661812,
899
+ "learning_rate": 0.00023793103448275864,
900
+ "loss": 0.9629,
901
+ "step": 60
902
+ },
903
+ {
904
+ "epoch": 1.3953488372093024,
905
+ "eval_loss": 1.0196707248687744,
906
+ "eval_runtime": 10.4755,
907
+ "eval_samples_per_second": 11.169,
908
+ "eval_steps_per_second": 0.764,
909
+ "step": 60
910
+ },
911
+ {
912
+ "epoch": 1.4186046511627908,
913
+ "grad_norm": 0.052802279591560364,
914
+ "learning_rate": 0.00023448275862068965,
915
+ "loss": 0.9707,
916
+ "step": 61
917
+ },
918
+ {
919
+ "epoch": 1.4186046511627908,
920
+ "eval_loss": 1.0190174579620361,
921
+ "eval_runtime": 10.5033,
922
+ "eval_samples_per_second": 11.139,
923
+ "eval_steps_per_second": 0.762,
924
+ "step": 61
925
+ },
926
+ {
927
+ "epoch": 1.441860465116279,
928
+ "grad_norm": 0.04725794866681099,
929
+ "learning_rate": 0.0002310344827586207,
930
+ "loss": 1.0164,
931
+ "step": 62
932
+ },
933
+ {
934
+ "epoch": 1.441860465116279,
935
+ "eval_loss": 1.0183162689208984,
936
+ "eval_runtime": 10.4911,
937
+ "eval_samples_per_second": 11.152,
938
+ "eval_steps_per_second": 0.763,
939
+ "step": 62
940
+ },
941
+ {
942
+ "epoch": 1.4651162790697674,
943
+ "grad_norm": 0.050118036568164825,
944
+ "learning_rate": 0.00022758620689655176,
945
+ "loss": 0.9985,
946
+ "step": 63
947
+ },
948
+ {
949
+ "epoch": 1.4651162790697674,
950
+ "eval_loss": 1.0176469087600708,
951
+ "eval_runtime": 10.5936,
952
+ "eval_samples_per_second": 11.044,
953
+ "eval_steps_per_second": 0.755,
954
+ "step": 63
955
+ },
956
+ {
957
+ "epoch": 1.4883720930232558,
958
+ "grad_norm": 0.049090269953012466,
959
+ "learning_rate": 0.00022413793103448276,
960
+ "loss": 0.9873,
961
+ "step": 64
962
+ },
963
+ {
964
+ "epoch": 1.4883720930232558,
965
+ "eval_loss": 1.017175555229187,
966
+ "eval_runtime": 10.5197,
967
+ "eval_samples_per_second": 11.122,
968
+ "eval_steps_per_second": 0.76,
969
+ "step": 64
970
+ },
971
+ {
972
+ "epoch": 1.5116279069767442,
973
+ "grad_norm": 0.050224512815475464,
974
+ "learning_rate": 0.0002206896551724138,
975
+ "loss": 0.9896,
976
+ "step": 65
977
+ },
978
+ {
979
+ "epoch": 1.5116279069767442,
980
+ "eval_loss": 1.0169633626937866,
981
+ "eval_runtime": 10.5178,
982
+ "eval_samples_per_second": 11.124,
983
+ "eval_steps_per_second": 0.761,
984
+ "step": 65
985
+ },
986
+ {
987
+ "epoch": 1.5348837209302326,
988
+ "grad_norm": 0.0501309297978878,
989
+ "learning_rate": 0.00021724137931034484,
990
+ "loss": 0.9798,
991
+ "step": 66
992
+ },
993
+ {
994
+ "epoch": 1.5348837209302326,
995
+ "eval_loss": 1.0167036056518555,
996
+ "eval_runtime": 10.5339,
997
+ "eval_samples_per_second": 11.107,
998
+ "eval_steps_per_second": 0.759,
999
+ "step": 66
1000
+ },
1001
+ {
1002
+ "epoch": 1.558139534883721,
1003
+ "grad_norm": 0.051808878779411316,
1004
+ "learning_rate": 0.00021379310344827584,
1005
+ "loss": 1.0014,
1006
+ "step": 67
1007
+ },
1008
+ {
1009
+ "epoch": 1.558139534883721,
1010
+ "eval_loss": 1.0162760019302368,
1011
+ "eval_runtime": 10.5275,
1012
+ "eval_samples_per_second": 11.114,
1013
+ "eval_steps_per_second": 0.76,
1014
+ "step": 67
1015
+ },
1016
+ {
1017
+ "epoch": 1.5813953488372094,
1018
+ "grad_norm": 0.0500907227396965,
1019
+ "learning_rate": 0.0002103448275862069,
1020
+ "loss": 0.9933,
1021
+ "step": 68
1022
+ },
1023
+ {
1024
+ "epoch": 1.5813953488372094,
1025
+ "eval_loss": 1.0156968832015991,
1026
+ "eval_runtime": 10.5784,
1027
+ "eval_samples_per_second": 11.06,
1028
+ "eval_steps_per_second": 0.756,
1029
+ "step": 68
1030
+ },
1031
+ {
1032
+ "epoch": 1.6046511627906976,
1033
+ "grad_norm": 0.04874541610479355,
1034
+ "learning_rate": 0.00020689655172413795,
1035
+ "loss": 0.9668,
1036
+ "step": 69
1037
+ },
1038
+ {
1039
+ "epoch": 1.6046511627906976,
1040
+ "eval_loss": 1.0152900218963623,
1041
+ "eval_runtime": 10.5885,
1042
+ "eval_samples_per_second": 11.05,
1043
+ "eval_steps_per_second": 0.756,
1044
+ "step": 69
1045
+ },
1046
+ {
1047
+ "epoch": 1.627906976744186,
1048
+ "grad_norm": 0.05222810059785843,
1049
+ "learning_rate": 0.00020344827586206895,
1050
+ "loss": 0.9792,
1051
+ "step": 70
1052
+ },
1053
+ {
1054
+ "epoch": 1.627906976744186,
1055
+ "eval_loss": 1.0148221254348755,
1056
+ "eval_runtime": 10.4824,
1057
+ "eval_samples_per_second": 11.162,
1058
+ "eval_steps_per_second": 0.763,
1059
+ "step": 70
1060
+ },
1061
+ {
1062
+ "epoch": 1.6511627906976745,
1063
+ "grad_norm": 0.05117630958557129,
1064
+ "learning_rate": 0.0002,
1065
+ "loss": 0.9519,
1066
+ "step": 71
1067
+ },
1068
+ {
1069
+ "epoch": 1.6511627906976745,
1070
+ "eval_loss": 1.0145394802093506,
1071
+ "eval_runtime": 10.46,
1072
+ "eval_samples_per_second": 11.185,
1073
+ "eval_steps_per_second": 0.765,
1074
+ "step": 71
1075
+ },
1076
+ {
1077
+ "epoch": 1.6744186046511627,
1078
+ "grad_norm": 0.04879862442612648,
1079
+ "learning_rate": 0.00019655172413793104,
1080
+ "loss": 0.9833,
1081
+ "step": 72
1082
+ },
1083
+ {
1084
+ "epoch": 1.6744186046511627,
1085
+ "eval_loss": 1.01435124874115,
1086
+ "eval_runtime": 10.6034,
1087
+ "eval_samples_per_second": 11.034,
1088
+ "eval_steps_per_second": 0.754,
1089
+ "step": 72
1090
+ },
1091
+ {
1092
+ "epoch": 1.697674418604651,
1093
+ "grad_norm": 0.055211760103702545,
1094
+ "learning_rate": 0.0001931034482758621,
1095
+ "loss": 0.9782,
1096
+ "step": 73
1097
+ },
1098
+ {
1099
+ "epoch": 1.697674418604651,
1100
+ "eval_loss": 1.014180064201355,
1101
+ "eval_runtime": 10.5781,
1102
+ "eval_samples_per_second": 11.061,
1103
+ "eval_steps_per_second": 0.756,
1104
+ "step": 73
1105
+ },
1106
+ {
1107
+ "epoch": 1.7209302325581395,
1108
+ "grad_norm": 0.05374753847718239,
1109
+ "learning_rate": 0.00018965517241379312,
1110
+ "loss": 0.9824,
1111
+ "step": 74
1112
+ },
1113
+ {
1114
+ "epoch": 1.7209302325581395,
1115
+ "eval_loss": 1.0137970447540283,
1116
+ "eval_runtime": 10.5389,
1117
+ "eval_samples_per_second": 11.102,
1118
+ "eval_steps_per_second": 0.759,
1119
+ "step": 74
1120
+ },
1121
+ {
1122
+ "epoch": 1.744186046511628,
1123
+ "grad_norm": 0.053574249148368835,
1124
+ "learning_rate": 0.00018620689655172415,
1125
+ "loss": 0.9748,
1126
+ "step": 75
1127
+ },
1128
+ {
1129
+ "epoch": 1.744186046511628,
1130
+ "eval_loss": 1.013289213180542,
1131
+ "eval_runtime": 10.5759,
1132
+ "eval_samples_per_second": 11.063,
1133
+ "eval_steps_per_second": 0.756,
1134
+ "step": 75
1135
+ },
1136
+ {
1137
+ "epoch": 1.7674418604651163,
1138
+ "grad_norm": 0.05111626163125038,
1139
+ "learning_rate": 0.00018275862068965518,
1140
+ "loss": 0.9842,
1141
+ "step": 76
1142
+ },
1143
+ {
1144
+ "epoch": 1.7674418604651163,
1145
+ "eval_loss": 1.0127779245376587,
1146
+ "eval_runtime": 10.5155,
1147
+ "eval_samples_per_second": 11.126,
1148
+ "eval_steps_per_second": 0.761,
1149
+ "step": 76
1150
+ },
1151
+ {
1152
+ "epoch": 1.7906976744186047,
1153
+ "grad_norm": 0.05558936670422554,
1154
+ "learning_rate": 0.0001793103448275862,
1155
+ "loss": 0.9905,
1156
+ "step": 77
1157
+ },
1158
+ {
1159
+ "epoch": 1.7906976744186047,
1160
+ "eval_loss": 1.0122755765914917,
1161
+ "eval_runtime": 10.4815,
1162
+ "eval_samples_per_second": 11.162,
1163
+ "eval_steps_per_second": 0.763,
1164
+ "step": 77
1165
+ },
1166
+ {
1167
+ "epoch": 1.8139534883720931,
1168
+ "grad_norm": 0.053616516292095184,
1169
+ "learning_rate": 0.00017586206896551723,
1170
+ "loss": 0.9615,
1171
+ "step": 78
1172
+ },
1173
+ {
1174
+ "epoch": 1.8139534883720931,
1175
+ "eval_loss": 1.011860728263855,
1176
+ "eval_runtime": 10.5398,
1177
+ "eval_samples_per_second": 11.101,
1178
+ "eval_steps_per_second": 0.759,
1179
+ "step": 78
1180
+ },
1181
+ {
1182
+ "epoch": 1.8372093023255816,
1183
+ "grad_norm": 0.05707163363695145,
1184
+ "learning_rate": 0.00017241379310344826,
1185
+ "loss": 0.9583,
1186
+ "step": 79
1187
+ },
1188
+ {
1189
+ "epoch": 1.8372093023255816,
1190
+ "eval_loss": 1.0115560293197632,
1191
+ "eval_runtime": 10.5278,
1192
+ "eval_samples_per_second": 11.113,
1193
+ "eval_steps_per_second": 0.76,
1194
+ "step": 79
1195
+ },
1196
+ {
1197
+ "epoch": 1.8604651162790697,
1198
+ "grad_norm": 0.05251218378543854,
1199
+ "learning_rate": 0.00016896551724137932,
1200
+ "loss": 1.0081,
1201
+ "step": 80
1202
+ },
1203
+ {
1204
+ "epoch": 1.8604651162790697,
1205
+ "eval_loss": 1.01112699508667,
1206
+ "eval_runtime": 10.5774,
1207
+ "eval_samples_per_second": 11.061,
1208
+ "eval_steps_per_second": 0.756,
1209
+ "step": 80
1210
+ },
1211
+ {
1212
+ "epoch": 1.8837209302325582,
1213
+ "grad_norm": 0.05201508849859238,
1214
+ "learning_rate": 0.00016551724137931035,
1215
+ "loss": 0.9725,
1216
+ "step": 81
1217
+ },
1218
+ {
1219
+ "epoch": 1.8837209302325582,
1220
+ "eval_loss": 1.0107312202453613,
1221
+ "eval_runtime": 10.5289,
1222
+ "eval_samples_per_second": 11.112,
1223
+ "eval_steps_per_second": 0.76,
1224
+ "step": 81
1225
+ },
1226
+ {
1227
+ "epoch": 1.9069767441860463,
1228
+ "grad_norm": 0.05595965310931206,
1229
+ "learning_rate": 0.00016206896551724137,
1230
+ "loss": 0.9979,
1231
+ "step": 82
1232
+ },
1233
+ {
1234
+ "epoch": 1.9069767441860463,
1235
+ "eval_loss": 1.0103439092636108,
1236
+ "eval_runtime": 10.463,
1237
+ "eval_samples_per_second": 11.182,
1238
+ "eval_steps_per_second": 0.765,
1239
+ "step": 82
1240
+ },
1241
+ {
1242
+ "epoch": 1.9302325581395348,
1243
+ "grad_norm": 0.05475016310811043,
1244
+ "learning_rate": 0.00015862068965517243,
1245
+ "loss": 0.9965,
1246
+ "step": 83
1247
+ },
1248
+ {
1249
+ "epoch": 1.9302325581395348,
1250
+ "eval_loss": 1.0099685192108154,
1251
+ "eval_runtime": 10.5842,
1252
+ "eval_samples_per_second": 11.054,
1253
+ "eval_steps_per_second": 0.756,
1254
+ "step": 83
1255
+ },
1256
+ {
1257
+ "epoch": 1.9534883720930232,
1258
+ "grad_norm": 0.05614636465907097,
1259
+ "learning_rate": 0.00015517241379310346,
1260
+ "loss": 0.9761,
1261
+ "step": 84
1262
+ },
1263
+ {
1264
+ "epoch": 1.9534883720930232,
1265
+ "eval_loss": 1.009726643562317,
1266
+ "eval_runtime": 10.5108,
1267
+ "eval_samples_per_second": 11.131,
1268
+ "eval_steps_per_second": 0.761,
1269
+ "step": 84
1270
+ },
1271
+ {
1272
+ "epoch": 1.9767441860465116,
1273
+ "grad_norm": 0.05425894260406494,
1274
+ "learning_rate": 0.00015172413793103449,
1275
+ "loss": 0.9665,
1276
+ "step": 85
1277
+ },
1278
+ {
1279
+ "epoch": 1.9767441860465116,
1280
+ "eval_loss": 1.0094804763793945,
1281
+ "eval_runtime": 10.5158,
1282
+ "eval_samples_per_second": 11.126,
1283
+ "eval_steps_per_second": 0.761,
1284
+ "step": 85
1285
+ },
1286
+ {
1287
+ "epoch": 2.0,
1288
+ "grad_norm": 0.08465206623077393,
1289
+ "learning_rate": 0.00014827586206896554,
1290
+ "loss": 0.9676,
1291
+ "step": 86
1292
+ },
1293
+ {
1294
+ "epoch": 2.0,
1295
+ "eval_loss": 1.0091900825500488,
1296
+ "eval_runtime": 10.4594,
1297
+ "eval_samples_per_second": 11.186,
1298
+ "eval_steps_per_second": 0.765,
1299
+ "step": 86
1300
+ },
1301
+ {
1302
+ "epoch": 2.0232558139534884,
1303
+ "grad_norm": 0.05495611950755119,
1304
+ "learning_rate": 0.00014482758620689657,
1305
+ "loss": 0.9548,
1306
+ "step": 87
1307
+ },
1308
+ {
1309
+ "epoch": 2.0232558139534884,
1310
+ "eval_loss": 1.0090184211730957,
1311
+ "eval_runtime": 10.5403,
1312
+ "eval_samples_per_second": 11.1,
1313
+ "eval_steps_per_second": 0.759,
1314
+ "step": 87
1315
+ },
1316
+ {
1317
+ "epoch": 2.046511627906977,
1318
+ "grad_norm": 0.051615942269563675,
1319
+ "learning_rate": 0.0001413793103448276,
1320
+ "loss": 0.9413,
1321
+ "step": 88
1322
+ },
1323
+ {
1324
+ "epoch": 2.046511627906977,
1325
+ "eval_loss": 1.0091454982757568,
1326
+ "eval_runtime": 10.5362,
1327
+ "eval_samples_per_second": 11.105,
1328
+ "eval_steps_per_second": 0.759,
1329
+ "step": 88
1330
+ },
1331
+ {
1332
+ "epoch": 2.0697674418604652,
1333
+ "grad_norm": 0.06363032013177872,
1334
+ "learning_rate": 0.00013793103448275863,
1335
+ "loss": 0.9604,
1336
+ "step": 89
1337
+ },
1338
+ {
1339
+ "epoch": 2.0697674418604652,
1340
+ "eval_loss": 1.0093193054199219,
1341
+ "eval_runtime": 10.599,
1342
+ "eval_samples_per_second": 11.039,
1343
+ "eval_steps_per_second": 0.755,
1344
+ "step": 89
1345
+ },
1346
+ {
1347
+ "epoch": 2.0930232558139537,
1348
+ "grad_norm": 0.05618719756603241,
1349
+ "learning_rate": 0.00013448275862068965,
1350
+ "loss": 0.9627,
1351
+ "step": 90
1352
+ },
1353
+ {
1354
+ "epoch": 2.0930232558139537,
1355
+ "eval_loss": 1.0094503164291382,
1356
+ "eval_runtime": 10.5453,
1357
+ "eval_samples_per_second": 11.095,
1358
+ "eval_steps_per_second": 0.759,
1359
+ "step": 90
1360
+ },
1361
+ {
1362
+ "epoch": 2.116279069767442,
1363
+ "grad_norm": 0.061273831874132156,
1364
+ "learning_rate": 0.00013103448275862068,
1365
+ "loss": 0.9428,
1366
+ "step": 91
1367
+ },
1368
+ {
1369
+ "epoch": 2.116279069767442,
1370
+ "eval_loss": 1.0095981359481812,
1371
+ "eval_runtime": 10.5143,
1372
+ "eval_samples_per_second": 11.128,
1373
+ "eval_steps_per_second": 0.761,
1374
+ "step": 91
1375
+ },
1376
+ {
1377
+ "epoch": 2.13953488372093,
1378
+ "grad_norm": 0.05564780905842781,
1379
+ "learning_rate": 0.00012758620689655174,
1380
+ "loss": 0.9774,
1381
+ "step": 92
1382
+ },
1383
+ {
1384
+ "epoch": 2.13953488372093,
1385
+ "eval_loss": 1.009597897529602,
1386
+ "eval_runtime": 10.4849,
1387
+ "eval_samples_per_second": 11.159,
1388
+ "eval_steps_per_second": 0.763,
1389
+ "step": 92
1390
+ },
1391
+ {
1392
+ "epoch": 2.1627906976744184,
1393
+ "grad_norm": 0.05518029257655144,
1394
+ "learning_rate": 0.00012413793103448277,
1395
+ "loss": 0.9407,
1396
+ "step": 93
1397
+ },
1398
+ {
1399
+ "epoch": 2.1627906976744184,
1400
+ "eval_loss": 1.0096217393875122,
1401
+ "eval_runtime": 10.6044,
1402
+ "eval_samples_per_second": 11.033,
1403
+ "eval_steps_per_second": 0.754,
1404
+ "step": 93
1405
+ },
1406
+ {
1407
+ "epoch": 2.186046511627907,
1408
+ "grad_norm": 0.05853220447897911,
1409
+ "learning_rate": 0.0001206896551724138,
1410
+ "loss": 0.9396,
1411
+ "step": 94
1412
+ },
1413
+ {
1414
+ "epoch": 2.186046511627907,
1415
+ "eval_loss": 1.00962233543396,
1416
+ "eval_runtime": 10.5343,
1417
+ "eval_samples_per_second": 11.107,
1418
+ "eval_steps_per_second": 0.759,
1419
+ "step": 94
1420
+ },
1421
+ {
1422
+ "epoch": 2.2093023255813953,
1423
+ "grad_norm": 0.06098194420337677,
1424
+ "learning_rate": 0.00011724137931034482,
1425
+ "loss": 0.9705,
1426
+ "step": 95
1427
+ },
1428
+ {
1429
+ "epoch": 2.2093023255813953,
1430
+ "eval_loss": 1.0093858242034912,
1431
+ "eval_runtime": 10.4224,
1432
+ "eval_samples_per_second": 11.226,
1433
+ "eval_steps_per_second": 0.768,
1434
+ "step": 95
1435
+ },
1436
+ {
1437
+ "epoch": 2.2325581395348837,
1438
+ "grad_norm": 0.05779768154025078,
1439
+ "learning_rate": 0.00011379310344827588,
1440
+ "loss": 0.9427,
1441
+ "step": 96
1442
+ },
1443
+ {
1444
+ "epoch": 2.2325581395348837,
1445
+ "eval_loss": 1.0092523097991943,
1446
+ "eval_runtime": 10.6138,
1447
+ "eval_samples_per_second": 11.023,
1448
+ "eval_steps_per_second": 0.754,
1449
+ "step": 96
1450
+ },
1451
+ {
1452
+ "epoch": 2.255813953488372,
1453
+ "grad_norm": 0.059211816638708115,
1454
+ "learning_rate": 0.0001103448275862069,
1455
+ "loss": 0.9689,
1456
+ "step": 97
1457
+ },
1458
+ {
1459
+ "epoch": 2.255813953488372,
1460
+ "eval_loss": 1.0091252326965332,
1461
+ "eval_runtime": 10.504,
1462
+ "eval_samples_per_second": 11.139,
1463
+ "eval_steps_per_second": 0.762,
1464
+ "step": 97
1465
+ },
1466
+ {
1467
+ "epoch": 2.2790697674418605,
1468
+ "grad_norm": 0.058523572981357574,
1469
+ "learning_rate": 0.00010689655172413792,
1470
+ "loss": 0.9459,
1471
+ "step": 98
1472
+ },
1473
+ {
1474
+ "epoch": 2.2790697674418605,
1475
+ "eval_loss": 1.0091090202331543,
1476
+ "eval_runtime": 10.5529,
1477
+ "eval_samples_per_second": 11.087,
1478
+ "eval_steps_per_second": 0.758,
1479
+ "step": 98
1480
+ },
1481
+ {
1482
+ "epoch": 2.302325581395349,
1483
+ "grad_norm": 0.06452349573373795,
1484
+ "learning_rate": 0.00010344827586206898,
1485
+ "loss": 0.9434,
1486
+ "step": 99
1487
+ },
1488
+ {
1489
+ "epoch": 2.302325581395349,
1490
+ "eval_loss": 1.0091310739517212,
1491
+ "eval_runtime": 10.5481,
1492
+ "eval_samples_per_second": 11.092,
1493
+ "eval_steps_per_second": 0.758,
1494
+ "step": 99
1495
+ },
1496
+ {
1497
+ "epoch": 2.3255813953488373,
1498
+ "grad_norm": 0.06188109889626503,
1499
+ "learning_rate": 0.0001,
1500
+ "loss": 0.9453,
1501
+ "step": 100
1502
+ },
1503
+ {
1504
+ "epoch": 2.3255813953488373,
1505
+ "eval_loss": 1.0090867280960083,
1506
+ "eval_runtime": 10.5427,
1507
+ "eval_samples_per_second": 11.098,
1508
+ "eval_steps_per_second": 0.759,
1509
+ "step": 100
1510
+ },
1511
+ {
1512
+ "epoch": 2.3488372093023258,
1513
+ "grad_norm": 0.061615802347660065,
1514
+ "learning_rate": 9.655172413793105e-05,
1515
+ "loss": 0.9675,
1516
+ "step": 101
1517
+ },
1518
+ {
1519
+ "epoch": 2.3488372093023258,
1520
+ "eval_loss": 1.008958101272583,
1521
+ "eval_runtime": 10.62,
1522
+ "eval_samples_per_second": 11.017,
1523
+ "eval_steps_per_second": 0.753,
1524
+ "step": 101
1525
+ },
1526
+ {
1527
+ "epoch": 2.3720930232558137,
1528
+ "grad_norm": 0.06509216129779816,
1529
+ "learning_rate": 9.310344827586207e-05,
1530
+ "loss": 0.9571,
1531
+ "step": 102
1532
+ },
1533
+ {
1534
+ "epoch": 2.3720930232558137,
1535
+ "eval_loss": 1.008782982826233,
1536
+ "eval_runtime": 10.5067,
1537
+ "eval_samples_per_second": 11.136,
1538
+ "eval_steps_per_second": 0.761,
1539
+ "step": 102
1540
+ },
1541
+ {
1542
+ "epoch": 2.395348837209302,
1543
+ "grad_norm": 0.07000201940536499,
1544
+ "learning_rate": 8.96551724137931e-05,
1545
+ "loss": 0.9669,
1546
+ "step": 103
1547
+ },
1548
+ {
1549
+ "epoch": 2.395348837209302,
1550
+ "eval_loss": 1.0085687637329102,
1551
+ "eval_runtime": 10.5369,
1552
+ "eval_samples_per_second": 11.104,
1553
+ "eval_steps_per_second": 0.759,
1554
+ "step": 103
1555
+ },
1556
+ {
1557
+ "epoch": 2.4186046511627906,
1558
+ "grad_norm": 0.06144480034708977,
1559
+ "learning_rate": 8.620689655172413e-05,
1560
+ "loss": 0.9592,
1561
+ "step": 104
1562
+ },
1563
+ {
1564
+ "epoch": 2.4186046511627906,
1565
+ "eval_loss": 1.0085170269012451,
1566
+ "eval_runtime": 10.5319,
1567
+ "eval_samples_per_second": 11.109,
1568
+ "eval_steps_per_second": 0.76,
1569
+ "step": 104
1570
+ },
1571
+ {
1572
+ "epoch": 2.441860465116279,
1573
+ "grad_norm": 0.06189959496259689,
1574
+ "learning_rate": 8.275862068965517e-05,
1575
+ "loss": 0.9259,
1576
+ "step": 105
1577
+ },
1578
+ {
1579
+ "epoch": 2.441860465116279,
1580
+ "eval_loss": 1.0085593461990356,
1581
+ "eval_runtime": 10.501,
1582
+ "eval_samples_per_second": 11.142,
1583
+ "eval_steps_per_second": 0.762,
1584
+ "step": 105
1585
+ },
1586
+ {
1587
+ "epoch": 2.4651162790697674,
1588
+ "grad_norm": 0.06781314313411713,
1589
+ "learning_rate": 7.931034482758621e-05,
1590
+ "loss": 0.945,
1591
+ "step": 106
1592
+ },
1593
+ {
1594
+ "epoch": 2.4651162790697674,
1595
+ "eval_loss": 1.0086591243743896,
1596
+ "eval_runtime": 10.536,
1597
+ "eval_samples_per_second": 11.105,
1598
+ "eval_steps_per_second": 0.759,
1599
+ "step": 106
1600
+ },
1601
+ {
1602
+ "epoch": 2.488372093023256,
1603
+ "grad_norm": 0.06491964310407639,
1604
+ "learning_rate": 7.586206896551724e-05,
1605
+ "loss": 0.948,
1606
+ "step": 107
1607
+ },
1608
+ {
1609
+ "epoch": 2.488372093023256,
1610
+ "eval_loss": 1.0086618661880493,
1611
+ "eval_runtime": 10.5398,
1612
+ "eval_samples_per_second": 11.101,
1613
+ "eval_steps_per_second": 0.759,
1614
+ "step": 107
1615
+ },
1616
+ {
1617
+ "epoch": 2.511627906976744,
1618
+ "grad_norm": 0.13770791888237,
1619
+ "learning_rate": 7.241379310344828e-05,
1620
+ "loss": 0.9631,
1621
+ "step": 108
1622
+ },
1623
+ {
1624
+ "epoch": 2.511627906976744,
1625
+ "eval_loss": 1.008668303489685,
1626
+ "eval_runtime": 10.4916,
1627
+ "eval_samples_per_second": 11.152,
1628
+ "eval_steps_per_second": 0.763,
1629
+ "step": 108
1630
+ },
1631
+ {
1632
+ "epoch": 2.5348837209302326,
1633
+ "grad_norm": 0.06982796639204025,
1634
+ "learning_rate": 6.896551724137931e-05,
1635
+ "loss": 0.9315,
1636
+ "step": 109
1637
+ },
1638
+ {
1639
+ "epoch": 2.5348837209302326,
1640
+ "eval_loss": 1.008758783340454,
1641
+ "eval_runtime": 10.4002,
1642
+ "eval_samples_per_second": 11.25,
1643
+ "eval_steps_per_second": 0.769,
1644
+ "step": 109
1645
+ },
1646
+ {
1647
+ "epoch": 2.558139534883721,
1648
+ "grad_norm": 0.06349999457597733,
1649
+ "learning_rate": 6.551724137931034e-05,
1650
+ "loss": 0.9433,
1651
+ "step": 110
1652
+ },
1653
+ {
1654
+ "epoch": 2.558139534883721,
1655
+ "eval_loss": 1.0088914632797241,
1656
+ "eval_runtime": 10.5022,
1657
+ "eval_samples_per_second": 11.141,
1658
+ "eval_steps_per_second": 0.762,
1659
+ "step": 110
1660
+ },
1661
+ {
1662
+ "epoch": 2.5813953488372094,
1663
+ "grad_norm": 0.06294334679841995,
1664
+ "learning_rate": 6.206896551724138e-05,
1665
+ "loss": 0.9484,
1666
+ "step": 111
1667
+ },
1668
+ {
1669
+ "epoch": 2.5813953488372094,
1670
+ "eval_loss": 1.0089675188064575,
1671
+ "eval_runtime": 10.4932,
1672
+ "eval_samples_per_second": 11.15,
1673
+ "eval_steps_per_second": 0.762,
1674
+ "step": 111
1675
+ },
1676
+ {
1677
+ "epoch": 2.604651162790698,
1678
+ "grad_norm": 0.06499718874692917,
1679
+ "learning_rate": 5.862068965517241e-05,
1680
+ "loss": 0.9519,
1681
+ "step": 112
1682
+ },
1683
+ {
1684
+ "epoch": 2.604651162790698,
1685
+ "eval_loss": 1.0090423822402954,
1686
+ "eval_runtime": 10.5478,
1687
+ "eval_samples_per_second": 11.092,
1688
+ "eval_steps_per_second": 0.758,
1689
+ "step": 112
1690
+ },
1691
+ {
1692
+ "epoch": 2.6279069767441863,
1693
+ "grad_norm": 0.06928952783346176,
1694
+ "learning_rate": 5.517241379310345e-05,
1695
+ "loss": 0.9551,
1696
+ "step": 113
1697
+ },
1698
+ {
1699
+ "epoch": 2.6279069767441863,
1700
+ "eval_loss": 1.008984923362732,
1701
+ "eval_runtime": 10.493,
1702
+ "eval_samples_per_second": 11.15,
1703
+ "eval_steps_per_second": 0.762,
1704
+ "step": 113
1705
+ },
1706
+ {
1707
+ "epoch": 2.6511627906976747,
1708
+ "grad_norm": 0.06725452095270157,
1709
+ "learning_rate": 5.172413793103449e-05,
1710
+ "loss": 0.9404,
1711
+ "step": 114
1712
+ },
1713
+ {
1714
+ "epoch": 2.6511627906976747,
1715
+ "eval_loss": 1.0089396238327026,
1716
+ "eval_runtime": 10.479,
1717
+ "eval_samples_per_second": 11.165,
1718
+ "eval_steps_per_second": 0.763,
1719
+ "step": 114
1720
+ },
1721
+ {
1722
+ "epoch": 2.6744186046511627,
1723
+ "grad_norm": 0.06509212404489517,
1724
+ "learning_rate": 4.827586206896552e-05,
1725
+ "loss": 0.9377,
1726
+ "step": 115
1727
+ },
1728
+ {
1729
+ "epoch": 2.6744186046511627,
1730
+ "eval_loss": 1.0088380575180054,
1731
+ "eval_runtime": 10.4542,
1732
+ "eval_samples_per_second": 11.192,
1733
+ "eval_steps_per_second": 0.765,
1734
+ "step": 115
1735
+ },
1736
+ {
1737
+ "epoch": 2.697674418604651,
1738
+ "grad_norm": 0.06430088728666306,
1739
+ "learning_rate": 4.482758620689655e-05,
1740
+ "loss": 0.9447,
1741
+ "step": 116
1742
+ },
1743
+ {
1744
+ "epoch": 2.697674418604651,
1745
+ "eval_loss": 1.008760929107666,
1746
+ "eval_runtime": 10.529,
1747
+ "eval_samples_per_second": 11.112,
1748
+ "eval_steps_per_second": 0.76,
1749
+ "step": 116
1750
+ },
1751
+ {
1752
+ "epoch": 2.7209302325581395,
1753
+ "grad_norm": 0.07184050977230072,
1754
+ "learning_rate": 4.1379310344827587e-05,
1755
+ "loss": 0.9608,
1756
+ "step": 117
1757
+ },
1758
+ {
1759
+ "epoch": 2.7209302325581395,
1760
+ "eval_loss": 1.0085643529891968,
1761
+ "eval_runtime": 10.5269,
1762
+ "eval_samples_per_second": 11.114,
1763
+ "eval_steps_per_second": 0.76,
1764
+ "step": 117
1765
+ },
1766
+ {
1767
+ "epoch": 2.744186046511628,
1768
+ "grad_norm": 0.06626931577920914,
1769
+ "learning_rate": 3.793103448275862e-05,
1770
+ "loss": 0.9459,
1771
+ "step": 118
1772
+ },
1773
+ {
1774
+ "epoch": 2.744186046511628,
1775
+ "eval_loss": 1.008436918258667,
1776
+ "eval_runtime": 10.526,
1777
+ "eval_samples_per_second": 11.115,
1778
+ "eval_steps_per_second": 0.76,
1779
+ "step": 118
1780
+ },
1781
+ {
1782
+ "epoch": 2.7674418604651163,
1783
+ "grad_norm": 0.07003094255924225,
1784
+ "learning_rate": 3.4482758620689657e-05,
1785
+ "loss": 0.9601,
1786
+ "step": 119
1787
+ },
1788
+ {
1789
+ "epoch": 2.7674418604651163,
1790
+ "eval_loss": 1.0083117485046387,
1791
+ "eval_runtime": 10.6067,
1792
+ "eval_samples_per_second": 11.031,
1793
+ "eval_steps_per_second": 0.754,
1794
+ "step": 119
1795
+ },
1796
+ {
1797
+ "epoch": 2.7906976744186047,
1798
+ "grad_norm": 0.06969019770622253,
1799
+ "learning_rate": 3.103448275862069e-05,
1800
+ "loss": 0.9154,
1801
+ "step": 120
1802
+ },
1803
+ {
1804
+ "epoch": 2.7906976744186047,
1805
+ "eval_loss": 1.0082221031188965,
1806
+ "eval_runtime": 10.5347,
1807
+ "eval_samples_per_second": 11.106,
1808
+ "eval_steps_per_second": 0.759,
1809
+ "step": 120
1810
+ }
1811
+ ],
1812
+ "logging_steps": 1,
1813
+ "max_steps": 129,
1814
+ "num_input_tokens_seen": 0,
1815
+ "num_train_epochs": 3,
1816
+ "save_steps": 10,
1817
+ "stateful_callbacks": {
1818
+ "TrainerControl": {
1819
+ "args": {
1820
+ "should_epoch_stop": false,
1821
+ "should_evaluate": false,
1822
+ "should_log": false,
1823
+ "should_save": true,
1824
+ "should_training_stop": false
1825
+ },
1826
+ "attributes": {}
1827
+ }
1828
+ },
1829
+ "total_flos": 4.40434490867712e+16,
1830
+ "train_batch_size": 32,
1831
+ "trial_name": null,
1832
+ "trial_params": null
1833
+ }
checkpoint-120/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:175fa9b835782af01021c38d541ab432b2880714c5ad1998afed59d856d20031
3
+ size 5688
special_tokens_map.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<|endoftext|>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": "<unk>",
17
+ "unk_token": {
18
+ "content": "<unk>",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ }
24
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
tokenizer_config.json ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "0": {
6
+ "content": "<unk>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "1": {
14
+ "content": "<s>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "2": {
22
+ "content": "</s>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": true,
26
+ "single_word": false,
27
+ "special": false
28
+ },
29
+ "32000": {
30
+ "content": "<|endoftext|>",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ },
37
+ "32001": {
38
+ "content": "<|assistant|>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": true,
42
+ "single_word": false,
43
+ "special": true
44
+ },
45
+ "32002": {
46
+ "content": "<|placeholder1|>",
47
+ "lstrip": false,
48
+ "normalized": false,
49
+ "rstrip": true,
50
+ "single_word": false,
51
+ "special": true
52
+ },
53
+ "32003": {
54
+ "content": "<|placeholder2|>",
55
+ "lstrip": false,
56
+ "normalized": false,
57
+ "rstrip": true,
58
+ "single_word": false,
59
+ "special": true
60
+ },
61
+ "32004": {
62
+ "content": "<|placeholder3|>",
63
+ "lstrip": false,
64
+ "normalized": false,
65
+ "rstrip": true,
66
+ "single_word": false,
67
+ "special": true
68
+ },
69
+ "32005": {
70
+ "content": "<|placeholder4|>",
71
+ "lstrip": false,
72
+ "normalized": false,
73
+ "rstrip": true,
74
+ "single_word": false,
75
+ "special": true
76
+ },
77
+ "32006": {
78
+ "content": "<|system|>",
79
+ "lstrip": false,
80
+ "normalized": false,
81
+ "rstrip": true,
82
+ "single_word": false,
83
+ "special": true
84
+ },
85
+ "32007": {
86
+ "content": "<|end|>",
87
+ "lstrip": false,
88
+ "normalized": false,
89
+ "rstrip": true,
90
+ "single_word": false,
91
+ "special": true
92
+ },
93
+ "32008": {
94
+ "content": "<|placeholder5|>",
95
+ "lstrip": false,
96
+ "normalized": false,
97
+ "rstrip": true,
98
+ "single_word": false,
99
+ "special": true
100
+ },
101
+ "32009": {
102
+ "content": "<|placeholder6|>",
103
+ "lstrip": false,
104
+ "normalized": false,
105
+ "rstrip": true,
106
+ "single_word": false,
107
+ "special": true
108
+ },
109
+ "32010": {
110
+ "content": "<|user|>",
111
+ "lstrip": false,
112
+ "normalized": false,
113
+ "rstrip": true,
114
+ "single_word": false,
115
+ "special": true
116
+ },
117
+ "32011": {
118
+ "content": "eos_token",
119
+ "lstrip": false,
120
+ "normalized": false,
121
+ "rstrip": false,
122
+ "single_word": false,
123
+ "special": true
124
+ }
125
+ },
126
+ "bos_token": "<s>",
127
+ "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}",
128
+ "clean_up_tokenization_spaces": false,
129
+ "eos_token": "<|endoftext|>",
130
+ "legacy": false,
131
+ "model_max_length": 4096,
132
+ "pad_token": "<unk>",
133
+ "padding_side": "right",
134
+ "sp_model_kwargs": {},
135
+ "tokenizer_class": "LlamaTokenizer",
136
+ "truncation_side": "left",
137
+ "unk_token": "<unk>",
138
+ "use_default_system_prompt": false
139
+ }
train_results.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 3.0,
3
+ "train_loss": 1.1705420798109483,
4
+ "train_runtime": 2551.3345,
5
+ "train_samples_per_second": 1.594,
6
+ "train_steps_per_second": 0.051
7
+ }
trainer_state.json ADDED
@@ -0,0 +1,1977 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 1.0082221031188965,
3
+ "best_model_checkpoint": "./output/checkpoints/2024-06-11_15-07-46/checkpoint-120",
4
+ "epoch": 3.0,
5
+ "eval_steps": 1,
6
+ "global_step": 129,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.023255813953488372,
13
+ "grad_norm": 3.230428695678711,
14
+ "learning_rate": 3.0769230769230774e-05,
15
+ "loss": 5.1513,
16
+ "step": 1
17
+ },
18
+ {
19
+ "epoch": 0.023255813953488372,
20
+ "eval_loss": 5.098567485809326,
21
+ "eval_runtime": 10.4425,
22
+ "eval_samples_per_second": 11.204,
23
+ "eval_steps_per_second": 0.766,
24
+ "step": 1
25
+ },
26
+ {
27
+ "epoch": 0.046511627906976744,
28
+ "grad_norm": 3.204726219177246,
29
+ "learning_rate": 6.153846153846155e-05,
30
+ "loss": 5.1556,
31
+ "step": 2
32
+ },
33
+ {
34
+ "epoch": 0.046511627906976744,
35
+ "eval_loss": 5.000132083892822,
36
+ "eval_runtime": 10.4412,
37
+ "eval_samples_per_second": 11.206,
38
+ "eval_steps_per_second": 0.766,
39
+ "step": 2
40
+ },
41
+ {
42
+ "epoch": 0.06976744186046512,
43
+ "grad_norm": 3.1961917877197266,
44
+ "learning_rate": 9.230769230769232e-05,
45
+ "loss": 4.9354,
46
+ "step": 3
47
+ },
48
+ {
49
+ "epoch": 0.06976744186046512,
50
+ "eval_loss": 4.540091514587402,
51
+ "eval_runtime": 10.4665,
52
+ "eval_samples_per_second": 11.178,
53
+ "eval_steps_per_second": 0.764,
54
+ "step": 3
55
+ },
56
+ {
57
+ "epoch": 0.09302325581395349,
58
+ "grad_norm": 3.3630478382110596,
59
+ "learning_rate": 0.0001230769230769231,
60
+ "loss": 4.5394,
61
+ "step": 4
62
+ },
63
+ {
64
+ "epoch": 0.09302325581395349,
65
+ "eval_loss": 3.6481263637542725,
66
+ "eval_runtime": 10.5,
67
+ "eval_samples_per_second": 11.143,
68
+ "eval_steps_per_second": 0.762,
69
+ "step": 4
70
+ },
71
+ {
72
+ "epoch": 0.11627906976744186,
73
+ "grad_norm": 3.4904136657714844,
74
+ "learning_rate": 0.00015384615384615385,
75
+ "loss": 3.6545,
76
+ "step": 5
77
+ },
78
+ {
79
+ "epoch": 0.11627906976744186,
80
+ "eval_loss": 2.6286702156066895,
81
+ "eval_runtime": 10.4841,
82
+ "eval_samples_per_second": 11.16,
83
+ "eval_steps_per_second": 0.763,
84
+ "step": 5
85
+ },
86
+ {
87
+ "epoch": 0.13953488372093023,
88
+ "grad_norm": 2.7318434715270996,
89
+ "learning_rate": 0.00018461538461538463,
90
+ "loss": 2.5645,
91
+ "step": 6
92
+ },
93
+ {
94
+ "epoch": 0.13953488372093023,
95
+ "eval_loss": 2.047227144241333,
96
+ "eval_runtime": 10.5229,
97
+ "eval_samples_per_second": 11.119,
98
+ "eval_steps_per_second": 0.76,
99
+ "step": 6
100
+ },
101
+ {
102
+ "epoch": 0.16279069767441862,
103
+ "grad_norm": 2.1703038215637207,
104
+ "learning_rate": 0.0002153846153846154,
105
+ "loss": 2.0145,
106
+ "step": 7
107
+ },
108
+ {
109
+ "epoch": 0.16279069767441862,
110
+ "eval_loss": 1.6754586696624756,
111
+ "eval_runtime": 10.513,
112
+ "eval_samples_per_second": 11.129,
113
+ "eval_steps_per_second": 0.761,
114
+ "step": 7
115
+ },
116
+ {
117
+ "epoch": 0.18604651162790697,
118
+ "grad_norm": 1.1857593059539795,
119
+ "learning_rate": 0.0002461538461538462,
120
+ "loss": 1.6469,
121
+ "step": 8
122
+ },
123
+ {
124
+ "epoch": 0.18604651162790697,
125
+ "eval_loss": 1.4785383939743042,
126
+ "eval_runtime": 10.4789,
127
+ "eval_samples_per_second": 11.165,
128
+ "eval_steps_per_second": 0.763,
129
+ "step": 8
130
+ },
131
+ {
132
+ "epoch": 0.20930232558139536,
133
+ "grad_norm": 0.94368976354599,
134
+ "learning_rate": 0.00027692307692307695,
135
+ "loss": 1.4676,
136
+ "step": 9
137
+ },
138
+ {
139
+ "epoch": 0.20930232558139536,
140
+ "eval_loss": 1.3956246376037598,
141
+ "eval_runtime": 10.5914,
142
+ "eval_samples_per_second": 11.047,
143
+ "eval_steps_per_second": 0.755,
144
+ "step": 9
145
+ },
146
+ {
147
+ "epoch": 0.23255813953488372,
148
+ "grad_norm": 1.3516250848770142,
149
+ "learning_rate": 0.0003076923076923077,
150
+ "loss": 1.377,
151
+ "step": 10
152
+ },
153
+ {
154
+ "epoch": 0.23255813953488372,
155
+ "eval_loss": 1.2937995195388794,
156
+ "eval_runtime": 10.5422,
157
+ "eval_samples_per_second": 11.098,
158
+ "eval_steps_per_second": 0.759,
159
+ "step": 10
160
+ },
161
+ {
162
+ "epoch": 0.2558139534883721,
163
+ "grad_norm": 0.21453160047531128,
164
+ "learning_rate": 0.00033846153846153846,
165
+ "loss": 1.2814,
166
+ "step": 11
167
+ },
168
+ {
169
+ "epoch": 0.2558139534883721,
170
+ "eval_loss": 1.258020281791687,
171
+ "eval_runtime": 10.4428,
172
+ "eval_samples_per_second": 11.204,
173
+ "eval_steps_per_second": 0.766,
174
+ "step": 11
175
+ },
176
+ {
177
+ "epoch": 0.27906976744186046,
178
+ "grad_norm": 0.1224498599767685,
179
+ "learning_rate": 0.00036923076923076927,
180
+ "loss": 1.2261,
181
+ "step": 12
182
+ },
183
+ {
184
+ "epoch": 0.27906976744186046,
185
+ "eval_loss": 1.228270411491394,
186
+ "eval_runtime": 10.5707,
187
+ "eval_samples_per_second": 11.068,
188
+ "eval_steps_per_second": 0.757,
189
+ "step": 12
190
+ },
191
+ {
192
+ "epoch": 0.3023255813953488,
193
+ "grad_norm": 0.11027181148529053,
194
+ "learning_rate": 0.0004,
195
+ "loss": 1.1848,
196
+ "step": 13
197
+ },
198
+ {
199
+ "epoch": 0.3023255813953488,
200
+ "eval_loss": 1.2002977132797241,
201
+ "eval_runtime": 10.5152,
202
+ "eval_samples_per_second": 11.127,
203
+ "eval_steps_per_second": 0.761,
204
+ "step": 13
205
+ },
206
+ {
207
+ "epoch": 0.32558139534883723,
208
+ "grad_norm": 0.14012101292610168,
209
+ "learning_rate": 0.0003965517241379311,
210
+ "loss": 1.1961,
211
+ "step": 14
212
+ },
213
+ {
214
+ "epoch": 0.32558139534883723,
215
+ "eval_loss": 1.1754467487335205,
216
+ "eval_runtime": 10.5013,
217
+ "eval_samples_per_second": 11.141,
218
+ "eval_steps_per_second": 0.762,
219
+ "step": 14
220
+ },
221
+ {
222
+ "epoch": 0.3488372093023256,
223
+ "grad_norm": 0.22193074226379395,
224
+ "learning_rate": 0.0003931034482758621,
225
+ "loss": 1.1407,
226
+ "step": 15
227
+ },
228
+ {
229
+ "epoch": 0.3488372093023256,
230
+ "eval_loss": 1.1523548364639282,
231
+ "eval_runtime": 10.4708,
232
+ "eval_samples_per_second": 11.174,
233
+ "eval_steps_per_second": 0.764,
234
+ "step": 15
235
+ },
236
+ {
237
+ "epoch": 0.37209302325581395,
238
+ "grad_norm": 0.15213920176029205,
239
+ "learning_rate": 0.00038965517241379313,
240
+ "loss": 1.1364,
241
+ "step": 16
242
+ },
243
+ {
244
+ "epoch": 0.37209302325581395,
245
+ "eval_loss": 1.1305698156356812,
246
+ "eval_runtime": 10.5494,
247
+ "eval_samples_per_second": 11.091,
248
+ "eval_steps_per_second": 0.758,
249
+ "step": 16
250
+ },
251
+ {
252
+ "epoch": 0.3953488372093023,
253
+ "grad_norm": 0.08833792805671692,
254
+ "learning_rate": 0.0003862068965517242,
255
+ "loss": 1.1057,
256
+ "step": 17
257
+ },
258
+ {
259
+ "epoch": 0.3953488372093023,
260
+ "eval_loss": 1.1136623620986938,
261
+ "eval_runtime": 10.6102,
262
+ "eval_samples_per_second": 11.027,
263
+ "eval_steps_per_second": 0.754,
264
+ "step": 17
265
+ },
266
+ {
267
+ "epoch": 0.4186046511627907,
268
+ "grad_norm": 0.07902093976736069,
269
+ "learning_rate": 0.0003827586206896552,
270
+ "loss": 1.089,
271
+ "step": 18
272
+ },
273
+ {
274
+ "epoch": 0.4186046511627907,
275
+ "eval_loss": 1.10086989402771,
276
+ "eval_runtime": 10.5681,
277
+ "eval_samples_per_second": 11.071,
278
+ "eval_steps_per_second": 0.757,
279
+ "step": 18
280
+ },
281
+ {
282
+ "epoch": 0.4418604651162791,
283
+ "grad_norm": 0.07194280624389648,
284
+ "learning_rate": 0.00037931034482758624,
285
+ "loss": 1.0849,
286
+ "step": 19
287
+ },
288
+ {
289
+ "epoch": 0.4418604651162791,
290
+ "eval_loss": 1.091064691543579,
291
+ "eval_runtime": 10.5436,
292
+ "eval_samples_per_second": 11.097,
293
+ "eval_steps_per_second": 0.759,
294
+ "step": 19
295
+ },
296
+ {
297
+ "epoch": 0.46511627906976744,
298
+ "grad_norm": 0.05419577285647392,
299
+ "learning_rate": 0.0003758620689655173,
300
+ "loss": 1.0729,
301
+ "step": 20
302
+ },
303
+ {
304
+ "epoch": 0.46511627906976744,
305
+ "eval_loss": 1.0844314098358154,
306
+ "eval_runtime": 10.46,
307
+ "eval_samples_per_second": 11.185,
308
+ "eval_steps_per_second": 0.765,
309
+ "step": 20
310
+ },
311
+ {
312
+ "epoch": 0.4883720930232558,
313
+ "grad_norm": 0.05308014526963234,
314
+ "learning_rate": 0.0003724137931034483,
315
+ "loss": 1.0576,
316
+ "step": 21
317
+ },
318
+ {
319
+ "epoch": 0.4883720930232558,
320
+ "eval_loss": 1.0792510509490967,
321
+ "eval_runtime": 10.5659,
322
+ "eval_samples_per_second": 11.073,
323
+ "eval_steps_per_second": 0.757,
324
+ "step": 21
325
+ },
326
+ {
327
+ "epoch": 0.5116279069767442,
328
+ "grad_norm": 0.05068132281303406,
329
+ "learning_rate": 0.00036896551724137935,
330
+ "loss": 1.0591,
331
+ "step": 22
332
+ },
333
+ {
334
+ "epoch": 0.5116279069767442,
335
+ "eval_loss": 1.074510931968689,
336
+ "eval_runtime": 10.5969,
337
+ "eval_samples_per_second": 11.041,
338
+ "eval_steps_per_second": 0.755,
339
+ "step": 22
340
+ },
341
+ {
342
+ "epoch": 0.5348837209302325,
343
+ "grad_norm": 0.05279520899057388,
344
+ "learning_rate": 0.00036551724137931036,
345
+ "loss": 1.0216,
346
+ "step": 23
347
+ },
348
+ {
349
+ "epoch": 0.5348837209302325,
350
+ "eval_loss": 1.0699939727783203,
351
+ "eval_runtime": 10.5288,
352
+ "eval_samples_per_second": 11.112,
353
+ "eval_steps_per_second": 0.76,
354
+ "step": 23
355
+ },
356
+ {
357
+ "epoch": 0.5581395348837209,
358
+ "grad_norm": 0.051589012145996094,
359
+ "learning_rate": 0.0003620689655172414,
360
+ "loss": 1.0384,
361
+ "step": 24
362
+ },
363
+ {
364
+ "epoch": 0.5581395348837209,
365
+ "eval_loss": 1.0660674571990967,
366
+ "eval_runtime": 10.4984,
367
+ "eval_samples_per_second": 11.145,
368
+ "eval_steps_per_second": 0.762,
369
+ "step": 24
370
+ },
371
+ {
372
+ "epoch": 0.5813953488372093,
373
+ "grad_norm": 0.04975143447518349,
374
+ "learning_rate": 0.0003586206896551724,
375
+ "loss": 1.0754,
376
+ "step": 25
377
+ },
378
+ {
379
+ "epoch": 0.5813953488372093,
380
+ "eval_loss": 1.0627647638320923,
381
+ "eval_runtime": 10.5768,
382
+ "eval_samples_per_second": 11.062,
383
+ "eval_steps_per_second": 0.756,
384
+ "step": 25
385
+ },
386
+ {
387
+ "epoch": 0.6046511627906976,
388
+ "grad_norm": 0.048871301114559174,
389
+ "learning_rate": 0.00035517241379310347,
390
+ "loss": 1.037,
391
+ "step": 26
392
+ },
393
+ {
394
+ "epoch": 0.6046511627906976,
395
+ "eval_loss": 1.0597318410873413,
396
+ "eval_runtime": 10.4738,
397
+ "eval_samples_per_second": 11.171,
398
+ "eval_steps_per_second": 0.764,
399
+ "step": 26
400
+ },
401
+ {
402
+ "epoch": 0.627906976744186,
403
+ "grad_norm": 0.05112334340810776,
404
+ "learning_rate": 0.00035172413793103447,
405
+ "loss": 1.0579,
406
+ "step": 27
407
+ },
408
+ {
409
+ "epoch": 0.627906976744186,
410
+ "eval_loss": 1.0566964149475098,
411
+ "eval_runtime": 10.5953,
412
+ "eval_samples_per_second": 11.043,
413
+ "eval_steps_per_second": 0.755,
414
+ "step": 27
415
+ },
416
+ {
417
+ "epoch": 0.6511627906976745,
418
+ "grad_norm": 0.08299978822469711,
419
+ "learning_rate": 0.0003482758620689655,
420
+ "loss": 1.039,
421
+ "step": 28
422
+ },
423
+ {
424
+ "epoch": 0.6511627906976745,
425
+ "eval_loss": 1.053879976272583,
426
+ "eval_runtime": 10.5064,
427
+ "eval_samples_per_second": 11.136,
428
+ "eval_steps_per_second": 0.761,
429
+ "step": 28
430
+ },
431
+ {
432
+ "epoch": 0.6744186046511628,
433
+ "grad_norm": 0.04733407124876976,
434
+ "learning_rate": 0.0003448275862068965,
435
+ "loss": 1.054,
436
+ "step": 29
437
+ },
438
+ {
439
+ "epoch": 0.6744186046511628,
440
+ "eval_loss": 1.0513161420822144,
441
+ "eval_runtime": 10.5869,
442
+ "eval_samples_per_second": 11.051,
443
+ "eval_steps_per_second": 0.756,
444
+ "step": 29
445
+ },
446
+ {
447
+ "epoch": 0.6976744186046512,
448
+ "grad_norm": 0.045666251331567764,
449
+ "learning_rate": 0.0003413793103448276,
450
+ "loss": 1.0237,
451
+ "step": 30
452
+ },
453
+ {
454
+ "epoch": 0.6976744186046512,
455
+ "eval_loss": 1.0490825176239014,
456
+ "eval_runtime": 10.5294,
457
+ "eval_samples_per_second": 11.112,
458
+ "eval_steps_per_second": 0.76,
459
+ "step": 30
460
+ },
461
+ {
462
+ "epoch": 0.7209302325581395,
463
+ "grad_norm": 0.04347428306937218,
464
+ "learning_rate": 0.00033793103448275864,
465
+ "loss": 1.0238,
466
+ "step": 31
467
+ },
468
+ {
469
+ "epoch": 0.7209302325581395,
470
+ "eval_loss": 1.0473451614379883,
471
+ "eval_runtime": 10.4778,
472
+ "eval_samples_per_second": 11.166,
473
+ "eval_steps_per_second": 0.764,
474
+ "step": 31
475
+ },
476
+ {
477
+ "epoch": 0.7441860465116279,
478
+ "grad_norm": 0.04594381898641586,
479
+ "learning_rate": 0.00033448275862068964,
480
+ "loss": 1.0339,
481
+ "step": 32
482
+ },
483
+ {
484
+ "epoch": 0.7441860465116279,
485
+ "eval_loss": 1.0460575819015503,
486
+ "eval_runtime": 10.4914,
487
+ "eval_samples_per_second": 11.152,
488
+ "eval_steps_per_second": 0.763,
489
+ "step": 32
490
+ },
491
+ {
492
+ "epoch": 0.7674418604651163,
493
+ "grad_norm": 0.049526289105415344,
494
+ "learning_rate": 0.0003310344827586207,
495
+ "loss": 1.0418,
496
+ "step": 33
497
+ },
498
+ {
499
+ "epoch": 0.7674418604651163,
500
+ "eval_loss": 1.04463791847229,
501
+ "eval_runtime": 10.4747,
502
+ "eval_samples_per_second": 11.17,
503
+ "eval_steps_per_second": 0.764,
504
+ "step": 33
505
+ },
506
+ {
507
+ "epoch": 0.7906976744186046,
508
+ "grad_norm": 0.04762812703847885,
509
+ "learning_rate": 0.00032758620689655175,
510
+ "loss": 1.0302,
511
+ "step": 34
512
+ },
513
+ {
514
+ "epoch": 0.7906976744186046,
515
+ "eval_loss": 1.042962670326233,
516
+ "eval_runtime": 10.6089,
517
+ "eval_samples_per_second": 11.028,
518
+ "eval_steps_per_second": 0.754,
519
+ "step": 34
520
+ },
521
+ {
522
+ "epoch": 0.813953488372093,
523
+ "grad_norm": 0.05331471934914589,
524
+ "learning_rate": 0.00032413793103448275,
525
+ "loss": 1.0401,
526
+ "step": 35
527
+ },
528
+ {
529
+ "epoch": 0.813953488372093,
530
+ "eval_loss": 1.0412102937698364,
531
+ "eval_runtime": 10.4972,
532
+ "eval_samples_per_second": 11.146,
533
+ "eval_steps_per_second": 0.762,
534
+ "step": 35
535
+ },
536
+ {
537
+ "epoch": 0.8372093023255814,
538
+ "grad_norm": 0.041780710220336914,
539
+ "learning_rate": 0.0003206896551724138,
540
+ "loss": 1.0038,
541
+ "step": 36
542
+ },
543
+ {
544
+ "epoch": 0.8372093023255814,
545
+ "eval_loss": 1.0397316217422485,
546
+ "eval_runtime": 10.4649,
547
+ "eval_samples_per_second": 11.18,
548
+ "eval_steps_per_second": 0.764,
549
+ "step": 36
550
+ },
551
+ {
552
+ "epoch": 0.8604651162790697,
553
+ "grad_norm": 0.04302731156349182,
554
+ "learning_rate": 0.00031724137931034486,
555
+ "loss": 1.0249,
556
+ "step": 37
557
+ },
558
+ {
559
+ "epoch": 0.8604651162790697,
560
+ "eval_loss": 1.0385717153549194,
561
+ "eval_runtime": 10.5101,
562
+ "eval_samples_per_second": 11.132,
563
+ "eval_steps_per_second": 0.761,
564
+ "step": 37
565
+ },
566
+ {
567
+ "epoch": 0.8837209302325582,
568
+ "grad_norm": 0.0549028106033802,
569
+ "learning_rate": 0.00031379310344827586,
570
+ "loss": 1.0128,
571
+ "step": 38
572
+ },
573
+ {
574
+ "epoch": 0.8837209302325582,
575
+ "eval_loss": 1.0375303030014038,
576
+ "eval_runtime": 10.479,
577
+ "eval_samples_per_second": 11.165,
578
+ "eval_steps_per_second": 0.763,
579
+ "step": 38
580
+ },
581
+ {
582
+ "epoch": 0.9069767441860465,
583
+ "grad_norm": 0.04620625823736191,
584
+ "learning_rate": 0.0003103448275862069,
585
+ "loss": 1.0278,
586
+ "step": 39
587
+ },
588
+ {
589
+ "epoch": 0.9069767441860465,
590
+ "eval_loss": 1.03622567653656,
591
+ "eval_runtime": 10.496,
592
+ "eval_samples_per_second": 11.147,
593
+ "eval_steps_per_second": 0.762,
594
+ "step": 39
595
+ },
596
+ {
597
+ "epoch": 0.9302325581395349,
598
+ "grad_norm": 0.04755247011780739,
599
+ "learning_rate": 0.00030689655172413797,
600
+ "loss": 1.0059,
601
+ "step": 40
602
+ },
603
+ {
604
+ "epoch": 0.9302325581395349,
605
+ "eval_loss": 1.0347106456756592,
606
+ "eval_runtime": 10.522,
607
+ "eval_samples_per_second": 11.12,
608
+ "eval_steps_per_second": 0.76,
609
+ "step": 40
610
+ },
611
+ {
612
+ "epoch": 0.9534883720930233,
613
+ "grad_norm": 0.04391615465283394,
614
+ "learning_rate": 0.00030344827586206897,
615
+ "loss": 1.0234,
616
+ "step": 41
617
+ },
618
+ {
619
+ "epoch": 0.9534883720930233,
620
+ "eval_loss": 1.033333420753479,
621
+ "eval_runtime": 10.5498,
622
+ "eval_samples_per_second": 11.09,
623
+ "eval_steps_per_second": 0.758,
624
+ "step": 41
625
+ },
626
+ {
627
+ "epoch": 0.9767441860465116,
628
+ "grad_norm": 0.04336484521627426,
629
+ "learning_rate": 0.00030000000000000003,
630
+ "loss": 0.9913,
631
+ "step": 42
632
+ },
633
+ {
634
+ "epoch": 0.9767441860465116,
635
+ "eval_loss": 1.0323785543441772,
636
+ "eval_runtime": 10.5082,
637
+ "eval_samples_per_second": 11.134,
638
+ "eval_steps_per_second": 0.761,
639
+ "step": 42
640
+ },
641
+ {
642
+ "epoch": 1.0,
643
+ "grad_norm": 0.07296782732009888,
644
+ "learning_rate": 0.0002965517241379311,
645
+ "loss": 0.9975,
646
+ "step": 43
647
+ },
648
+ {
649
+ "epoch": 1.0,
650
+ "eval_loss": 1.0313116312026978,
651
+ "eval_runtime": 10.5008,
652
+ "eval_samples_per_second": 11.142,
653
+ "eval_steps_per_second": 0.762,
654
+ "step": 43
655
+ },
656
+ {
657
+ "epoch": 1.0232558139534884,
658
+ "grad_norm": 0.041788019239902496,
659
+ "learning_rate": 0.0002931034482758621,
660
+ "loss": 0.9841,
661
+ "step": 44
662
+ },
663
+ {
664
+ "epoch": 1.0232558139534884,
665
+ "eval_loss": 1.0304553508758545,
666
+ "eval_runtime": 10.4812,
667
+ "eval_samples_per_second": 11.163,
668
+ "eval_steps_per_second": 0.763,
669
+ "step": 44
670
+ },
671
+ {
672
+ "epoch": 1.0465116279069768,
673
+ "grad_norm": 0.044039350003004074,
674
+ "learning_rate": 0.00028965517241379314,
675
+ "loss": 0.9804,
676
+ "step": 45
677
+ },
678
+ {
679
+ "epoch": 1.0465116279069768,
680
+ "eval_loss": 1.0297547578811646,
681
+ "eval_runtime": 10.5179,
682
+ "eval_samples_per_second": 11.124,
683
+ "eval_steps_per_second": 0.761,
684
+ "step": 45
685
+ },
686
+ {
687
+ "epoch": 1.069767441860465,
688
+ "grad_norm": 0.04108603671193123,
689
+ "learning_rate": 0.0002862068965517242,
690
+ "loss": 0.9993,
691
+ "step": 46
692
+ },
693
+ {
694
+ "epoch": 1.069767441860465,
695
+ "eval_loss": 1.029006838798523,
696
+ "eval_runtime": 10.536,
697
+ "eval_samples_per_second": 11.105,
698
+ "eval_steps_per_second": 0.759,
699
+ "step": 46
700
+ },
701
+ {
702
+ "epoch": 1.0930232558139534,
703
+ "grad_norm": 0.04773078113794327,
704
+ "learning_rate": 0.0002827586206896552,
705
+ "loss": 0.9748,
706
+ "step": 47
707
+ },
708
+ {
709
+ "epoch": 1.0930232558139534,
710
+ "eval_loss": 1.028093695640564,
711
+ "eval_runtime": 10.5568,
712
+ "eval_samples_per_second": 11.083,
713
+ "eval_steps_per_second": 0.758,
714
+ "step": 47
715
+ },
716
+ {
717
+ "epoch": 1.1162790697674418,
718
+ "grad_norm": 0.04372379183769226,
719
+ "learning_rate": 0.0002793103448275862,
720
+ "loss": 1.0025,
721
+ "step": 48
722
+ },
723
+ {
724
+ "epoch": 1.1162790697674418,
725
+ "eval_loss": 1.027159333229065,
726
+ "eval_runtime": 10.4931,
727
+ "eval_samples_per_second": 11.15,
728
+ "eval_steps_per_second": 0.762,
729
+ "step": 48
730
+ },
731
+ {
732
+ "epoch": 1.1395348837209303,
733
+ "grad_norm": 0.05692731961607933,
734
+ "learning_rate": 0.00027586206896551725,
735
+ "loss": 1.0038,
736
+ "step": 49
737
+ },
738
+ {
739
+ "epoch": 1.1395348837209303,
740
+ "eval_loss": 1.0262141227722168,
741
+ "eval_runtime": 10.5803,
742
+ "eval_samples_per_second": 11.058,
743
+ "eval_steps_per_second": 0.756,
744
+ "step": 49
745
+ },
746
+ {
747
+ "epoch": 1.1627906976744187,
748
+ "grad_norm": 0.04124248027801514,
749
+ "learning_rate": 0.00027241379310344825,
750
+ "loss": 0.9848,
751
+ "step": 50
752
+ },
753
+ {
754
+ "epoch": 1.1627906976744187,
755
+ "eval_loss": 1.0254372358322144,
756
+ "eval_runtime": 10.4687,
757
+ "eval_samples_per_second": 11.176,
758
+ "eval_steps_per_second": 0.764,
759
+ "step": 50
760
+ },
761
+ {
762
+ "epoch": 1.1860465116279069,
763
+ "grad_norm": 0.04371722787618637,
764
+ "learning_rate": 0.0002689655172413793,
765
+ "loss": 1.0188,
766
+ "step": 51
767
+ },
768
+ {
769
+ "epoch": 1.1860465116279069,
770
+ "eval_loss": 1.024755597114563,
771
+ "eval_runtime": 10.5207,
772
+ "eval_samples_per_second": 11.121,
773
+ "eval_steps_per_second": 0.76,
774
+ "step": 51
775
+ },
776
+ {
777
+ "epoch": 1.2093023255813953,
778
+ "grad_norm": 0.04607950523495674,
779
+ "learning_rate": 0.00026551724137931036,
780
+ "loss": 1.0058,
781
+ "step": 52
782
+ },
783
+ {
784
+ "epoch": 1.2093023255813953,
785
+ "eval_loss": 1.0241938829421997,
786
+ "eval_runtime": 10.5231,
787
+ "eval_samples_per_second": 11.118,
788
+ "eval_steps_per_second": 0.76,
789
+ "step": 52
790
+ },
791
+ {
792
+ "epoch": 1.2325581395348837,
793
+ "grad_norm": 0.05415044724941254,
794
+ "learning_rate": 0.00026206896551724137,
795
+ "loss": 0.9923,
796
+ "step": 53
797
+ },
798
+ {
799
+ "epoch": 1.2325581395348837,
800
+ "eval_loss": 1.0235437154769897,
801
+ "eval_runtime": 10.6072,
802
+ "eval_samples_per_second": 11.03,
803
+ "eval_steps_per_second": 0.754,
804
+ "step": 53
805
+ },
806
+ {
807
+ "epoch": 1.255813953488372,
808
+ "grad_norm": 0.06359652429819107,
809
+ "learning_rate": 0.0002586206896551724,
810
+ "loss": 0.9948,
811
+ "step": 54
812
+ },
813
+ {
814
+ "epoch": 1.255813953488372,
815
+ "eval_loss": 1.0229169130325317,
816
+ "eval_runtime": 10.5202,
817
+ "eval_samples_per_second": 11.122,
818
+ "eval_steps_per_second": 0.76,
819
+ "step": 54
820
+ },
821
+ {
822
+ "epoch": 1.2790697674418605,
823
+ "grad_norm": 0.058946821838617325,
824
+ "learning_rate": 0.0002551724137931035,
825
+ "loss": 1.0188,
826
+ "step": 55
827
+ },
828
+ {
829
+ "epoch": 1.2790697674418605,
830
+ "eval_loss": 1.0223369598388672,
831
+ "eval_runtime": 10.4661,
832
+ "eval_samples_per_second": 11.179,
833
+ "eval_steps_per_second": 0.764,
834
+ "step": 55
835
+ },
836
+ {
837
+ "epoch": 1.302325581395349,
838
+ "grad_norm": 0.05592051148414612,
839
+ "learning_rate": 0.0002517241379310345,
840
+ "loss": 0.9933,
841
+ "step": 56
842
+ },
843
+ {
844
+ "epoch": 1.302325581395349,
845
+ "eval_loss": 1.0219457149505615,
846
+ "eval_runtime": 10.5373,
847
+ "eval_samples_per_second": 11.103,
848
+ "eval_steps_per_second": 0.759,
849
+ "step": 56
850
+ },
851
+ {
852
+ "epoch": 1.3255813953488373,
853
+ "grad_norm": 0.05216487497091293,
854
+ "learning_rate": 0.00024827586206896553,
855
+ "loss": 0.9942,
856
+ "step": 57
857
+ },
858
+ {
859
+ "epoch": 1.3255813953488373,
860
+ "eval_loss": 1.0214743614196777,
861
+ "eval_runtime": 10.5663,
862
+ "eval_samples_per_second": 11.073,
863
+ "eval_steps_per_second": 0.757,
864
+ "step": 57
865
+ },
866
+ {
867
+ "epoch": 1.3488372093023255,
868
+ "grad_norm": 0.04820263013243675,
869
+ "learning_rate": 0.00024482758620689653,
870
+ "loss": 0.9758,
871
+ "step": 58
872
+ },
873
+ {
874
+ "epoch": 1.3488372093023255,
875
+ "eval_loss": 1.0210070610046387,
876
+ "eval_runtime": 10.4802,
877
+ "eval_samples_per_second": 11.164,
878
+ "eval_steps_per_second": 0.763,
879
+ "step": 58
880
+ },
881
+ {
882
+ "epoch": 1.372093023255814,
883
+ "grad_norm": 0.04773212969303131,
884
+ "learning_rate": 0.0002413793103448276,
885
+ "loss": 1.0005,
886
+ "step": 59
887
+ },
888
+ {
889
+ "epoch": 1.372093023255814,
890
+ "eval_loss": 1.0203553438186646,
891
+ "eval_runtime": 10.5585,
892
+ "eval_samples_per_second": 11.081,
893
+ "eval_steps_per_second": 0.758,
894
+ "step": 59
895
+ },
896
+ {
897
+ "epoch": 1.3953488372093024,
898
+ "grad_norm": 0.05814950540661812,
899
+ "learning_rate": 0.00023793103448275864,
900
+ "loss": 0.9629,
901
+ "step": 60
902
+ },
903
+ {
904
+ "epoch": 1.3953488372093024,
905
+ "eval_loss": 1.0196707248687744,
906
+ "eval_runtime": 10.4755,
907
+ "eval_samples_per_second": 11.169,
908
+ "eval_steps_per_second": 0.764,
909
+ "step": 60
910
+ },
911
+ {
912
+ "epoch": 1.4186046511627908,
913
+ "grad_norm": 0.052802279591560364,
914
+ "learning_rate": 0.00023448275862068965,
915
+ "loss": 0.9707,
916
+ "step": 61
917
+ },
918
+ {
919
+ "epoch": 1.4186046511627908,
920
+ "eval_loss": 1.0190174579620361,
921
+ "eval_runtime": 10.5033,
922
+ "eval_samples_per_second": 11.139,
923
+ "eval_steps_per_second": 0.762,
924
+ "step": 61
925
+ },
926
+ {
927
+ "epoch": 1.441860465116279,
928
+ "grad_norm": 0.04725794866681099,
929
+ "learning_rate": 0.0002310344827586207,
930
+ "loss": 1.0164,
931
+ "step": 62
932
+ },
933
+ {
934
+ "epoch": 1.441860465116279,
935
+ "eval_loss": 1.0183162689208984,
936
+ "eval_runtime": 10.4911,
937
+ "eval_samples_per_second": 11.152,
938
+ "eval_steps_per_second": 0.763,
939
+ "step": 62
940
+ },
941
+ {
942
+ "epoch": 1.4651162790697674,
943
+ "grad_norm": 0.050118036568164825,
944
+ "learning_rate": 0.00022758620689655176,
945
+ "loss": 0.9985,
946
+ "step": 63
947
+ },
948
+ {
949
+ "epoch": 1.4651162790697674,
950
+ "eval_loss": 1.0176469087600708,
951
+ "eval_runtime": 10.5936,
952
+ "eval_samples_per_second": 11.044,
953
+ "eval_steps_per_second": 0.755,
954
+ "step": 63
955
+ },
956
+ {
957
+ "epoch": 1.4883720930232558,
958
+ "grad_norm": 0.049090269953012466,
959
+ "learning_rate": 0.00022413793103448276,
960
+ "loss": 0.9873,
961
+ "step": 64
962
+ },
963
+ {
964
+ "epoch": 1.4883720930232558,
965
+ "eval_loss": 1.017175555229187,
966
+ "eval_runtime": 10.5197,
967
+ "eval_samples_per_second": 11.122,
968
+ "eval_steps_per_second": 0.76,
969
+ "step": 64
970
+ },
971
+ {
972
+ "epoch": 1.5116279069767442,
973
+ "grad_norm": 0.050224512815475464,
974
+ "learning_rate": 0.0002206896551724138,
975
+ "loss": 0.9896,
976
+ "step": 65
977
+ },
978
+ {
979
+ "epoch": 1.5116279069767442,
980
+ "eval_loss": 1.0169633626937866,
981
+ "eval_runtime": 10.5178,
982
+ "eval_samples_per_second": 11.124,
983
+ "eval_steps_per_second": 0.761,
984
+ "step": 65
985
+ },
986
+ {
987
+ "epoch": 1.5348837209302326,
988
+ "grad_norm": 0.0501309297978878,
989
+ "learning_rate": 0.00021724137931034484,
990
+ "loss": 0.9798,
991
+ "step": 66
992
+ },
993
+ {
994
+ "epoch": 1.5348837209302326,
995
+ "eval_loss": 1.0167036056518555,
996
+ "eval_runtime": 10.5339,
997
+ "eval_samples_per_second": 11.107,
998
+ "eval_steps_per_second": 0.759,
999
+ "step": 66
1000
+ },
1001
+ {
1002
+ "epoch": 1.558139534883721,
1003
+ "grad_norm": 0.051808878779411316,
1004
+ "learning_rate": 0.00021379310344827584,
1005
+ "loss": 1.0014,
1006
+ "step": 67
1007
+ },
1008
+ {
1009
+ "epoch": 1.558139534883721,
1010
+ "eval_loss": 1.0162760019302368,
1011
+ "eval_runtime": 10.5275,
1012
+ "eval_samples_per_second": 11.114,
1013
+ "eval_steps_per_second": 0.76,
1014
+ "step": 67
1015
+ },
1016
+ {
1017
+ "epoch": 1.5813953488372094,
1018
+ "grad_norm": 0.0500907227396965,
1019
+ "learning_rate": 0.0002103448275862069,
1020
+ "loss": 0.9933,
1021
+ "step": 68
1022
+ },
1023
+ {
1024
+ "epoch": 1.5813953488372094,
1025
+ "eval_loss": 1.0156968832015991,
1026
+ "eval_runtime": 10.5784,
1027
+ "eval_samples_per_second": 11.06,
1028
+ "eval_steps_per_second": 0.756,
1029
+ "step": 68
1030
+ },
1031
+ {
1032
+ "epoch": 1.6046511627906976,
1033
+ "grad_norm": 0.04874541610479355,
1034
+ "learning_rate": 0.00020689655172413795,
1035
+ "loss": 0.9668,
1036
+ "step": 69
1037
+ },
1038
+ {
1039
+ "epoch": 1.6046511627906976,
1040
+ "eval_loss": 1.0152900218963623,
1041
+ "eval_runtime": 10.5885,
1042
+ "eval_samples_per_second": 11.05,
1043
+ "eval_steps_per_second": 0.756,
1044
+ "step": 69
1045
+ },
1046
+ {
1047
+ "epoch": 1.627906976744186,
1048
+ "grad_norm": 0.05222810059785843,
1049
+ "learning_rate": 0.00020344827586206895,
1050
+ "loss": 0.9792,
1051
+ "step": 70
1052
+ },
1053
+ {
1054
+ "epoch": 1.627906976744186,
1055
+ "eval_loss": 1.0148221254348755,
1056
+ "eval_runtime": 10.4824,
1057
+ "eval_samples_per_second": 11.162,
1058
+ "eval_steps_per_second": 0.763,
1059
+ "step": 70
1060
+ },
1061
+ {
1062
+ "epoch": 1.6511627906976745,
1063
+ "grad_norm": 0.05117630958557129,
1064
+ "learning_rate": 0.0002,
1065
+ "loss": 0.9519,
1066
+ "step": 71
1067
+ },
1068
+ {
1069
+ "epoch": 1.6511627906976745,
1070
+ "eval_loss": 1.0145394802093506,
1071
+ "eval_runtime": 10.46,
1072
+ "eval_samples_per_second": 11.185,
1073
+ "eval_steps_per_second": 0.765,
1074
+ "step": 71
1075
+ },
1076
+ {
1077
+ "epoch": 1.6744186046511627,
1078
+ "grad_norm": 0.04879862442612648,
1079
+ "learning_rate": 0.00019655172413793104,
1080
+ "loss": 0.9833,
1081
+ "step": 72
1082
+ },
1083
+ {
1084
+ "epoch": 1.6744186046511627,
1085
+ "eval_loss": 1.01435124874115,
1086
+ "eval_runtime": 10.6034,
1087
+ "eval_samples_per_second": 11.034,
1088
+ "eval_steps_per_second": 0.754,
1089
+ "step": 72
1090
+ },
1091
+ {
1092
+ "epoch": 1.697674418604651,
1093
+ "grad_norm": 0.055211760103702545,
1094
+ "learning_rate": 0.0001931034482758621,
1095
+ "loss": 0.9782,
1096
+ "step": 73
1097
+ },
1098
+ {
1099
+ "epoch": 1.697674418604651,
1100
+ "eval_loss": 1.014180064201355,
1101
+ "eval_runtime": 10.5781,
1102
+ "eval_samples_per_second": 11.061,
1103
+ "eval_steps_per_second": 0.756,
1104
+ "step": 73
1105
+ },
1106
+ {
1107
+ "epoch": 1.7209302325581395,
1108
+ "grad_norm": 0.05374753847718239,
1109
+ "learning_rate": 0.00018965517241379312,
1110
+ "loss": 0.9824,
1111
+ "step": 74
1112
+ },
1113
+ {
1114
+ "epoch": 1.7209302325581395,
1115
+ "eval_loss": 1.0137970447540283,
1116
+ "eval_runtime": 10.5389,
1117
+ "eval_samples_per_second": 11.102,
1118
+ "eval_steps_per_second": 0.759,
1119
+ "step": 74
1120
+ },
1121
+ {
1122
+ "epoch": 1.744186046511628,
1123
+ "grad_norm": 0.053574249148368835,
1124
+ "learning_rate": 0.00018620689655172415,
1125
+ "loss": 0.9748,
1126
+ "step": 75
1127
+ },
1128
+ {
1129
+ "epoch": 1.744186046511628,
1130
+ "eval_loss": 1.013289213180542,
1131
+ "eval_runtime": 10.5759,
1132
+ "eval_samples_per_second": 11.063,
1133
+ "eval_steps_per_second": 0.756,
1134
+ "step": 75
1135
+ },
1136
+ {
1137
+ "epoch": 1.7674418604651163,
1138
+ "grad_norm": 0.05111626163125038,
1139
+ "learning_rate": 0.00018275862068965518,
1140
+ "loss": 0.9842,
1141
+ "step": 76
1142
+ },
1143
+ {
1144
+ "epoch": 1.7674418604651163,
1145
+ "eval_loss": 1.0127779245376587,
1146
+ "eval_runtime": 10.5155,
1147
+ "eval_samples_per_second": 11.126,
1148
+ "eval_steps_per_second": 0.761,
1149
+ "step": 76
1150
+ },
1151
+ {
1152
+ "epoch": 1.7906976744186047,
1153
+ "grad_norm": 0.05558936670422554,
1154
+ "learning_rate": 0.0001793103448275862,
1155
+ "loss": 0.9905,
1156
+ "step": 77
1157
+ },
1158
+ {
1159
+ "epoch": 1.7906976744186047,
1160
+ "eval_loss": 1.0122755765914917,
1161
+ "eval_runtime": 10.4815,
1162
+ "eval_samples_per_second": 11.162,
1163
+ "eval_steps_per_second": 0.763,
1164
+ "step": 77
1165
+ },
1166
+ {
1167
+ "epoch": 1.8139534883720931,
1168
+ "grad_norm": 0.053616516292095184,
1169
+ "learning_rate": 0.00017586206896551723,
1170
+ "loss": 0.9615,
1171
+ "step": 78
1172
+ },
1173
+ {
1174
+ "epoch": 1.8139534883720931,
1175
+ "eval_loss": 1.011860728263855,
1176
+ "eval_runtime": 10.5398,
1177
+ "eval_samples_per_second": 11.101,
1178
+ "eval_steps_per_second": 0.759,
1179
+ "step": 78
1180
+ },
1181
+ {
1182
+ "epoch": 1.8372093023255816,
1183
+ "grad_norm": 0.05707163363695145,
1184
+ "learning_rate": 0.00017241379310344826,
1185
+ "loss": 0.9583,
1186
+ "step": 79
1187
+ },
1188
+ {
1189
+ "epoch": 1.8372093023255816,
1190
+ "eval_loss": 1.0115560293197632,
1191
+ "eval_runtime": 10.5278,
1192
+ "eval_samples_per_second": 11.113,
1193
+ "eval_steps_per_second": 0.76,
1194
+ "step": 79
1195
+ },
1196
+ {
1197
+ "epoch": 1.8604651162790697,
1198
+ "grad_norm": 0.05251218378543854,
1199
+ "learning_rate": 0.00016896551724137932,
1200
+ "loss": 1.0081,
1201
+ "step": 80
1202
+ },
1203
+ {
1204
+ "epoch": 1.8604651162790697,
1205
+ "eval_loss": 1.01112699508667,
1206
+ "eval_runtime": 10.5774,
1207
+ "eval_samples_per_second": 11.061,
1208
+ "eval_steps_per_second": 0.756,
1209
+ "step": 80
1210
+ },
1211
+ {
1212
+ "epoch": 1.8837209302325582,
1213
+ "grad_norm": 0.05201508849859238,
1214
+ "learning_rate": 0.00016551724137931035,
1215
+ "loss": 0.9725,
1216
+ "step": 81
1217
+ },
1218
+ {
1219
+ "epoch": 1.8837209302325582,
1220
+ "eval_loss": 1.0107312202453613,
1221
+ "eval_runtime": 10.5289,
1222
+ "eval_samples_per_second": 11.112,
1223
+ "eval_steps_per_second": 0.76,
1224
+ "step": 81
1225
+ },
1226
+ {
1227
+ "epoch": 1.9069767441860463,
1228
+ "grad_norm": 0.05595965310931206,
1229
+ "learning_rate": 0.00016206896551724137,
1230
+ "loss": 0.9979,
1231
+ "step": 82
1232
+ },
1233
+ {
1234
+ "epoch": 1.9069767441860463,
1235
+ "eval_loss": 1.0103439092636108,
1236
+ "eval_runtime": 10.463,
1237
+ "eval_samples_per_second": 11.182,
1238
+ "eval_steps_per_second": 0.765,
1239
+ "step": 82
1240
+ },
1241
+ {
1242
+ "epoch": 1.9302325581395348,
1243
+ "grad_norm": 0.05475016310811043,
1244
+ "learning_rate": 0.00015862068965517243,
1245
+ "loss": 0.9965,
1246
+ "step": 83
1247
+ },
1248
+ {
1249
+ "epoch": 1.9302325581395348,
1250
+ "eval_loss": 1.0099685192108154,
1251
+ "eval_runtime": 10.5842,
1252
+ "eval_samples_per_second": 11.054,
1253
+ "eval_steps_per_second": 0.756,
1254
+ "step": 83
1255
+ },
1256
+ {
1257
+ "epoch": 1.9534883720930232,
1258
+ "grad_norm": 0.05614636465907097,
1259
+ "learning_rate": 0.00015517241379310346,
1260
+ "loss": 0.9761,
1261
+ "step": 84
1262
+ },
1263
+ {
1264
+ "epoch": 1.9534883720930232,
1265
+ "eval_loss": 1.009726643562317,
1266
+ "eval_runtime": 10.5108,
1267
+ "eval_samples_per_second": 11.131,
1268
+ "eval_steps_per_second": 0.761,
1269
+ "step": 84
1270
+ },
1271
+ {
1272
+ "epoch": 1.9767441860465116,
1273
+ "grad_norm": 0.05425894260406494,
1274
+ "learning_rate": 0.00015172413793103449,
1275
+ "loss": 0.9665,
1276
+ "step": 85
1277
+ },
1278
+ {
1279
+ "epoch": 1.9767441860465116,
1280
+ "eval_loss": 1.0094804763793945,
1281
+ "eval_runtime": 10.5158,
1282
+ "eval_samples_per_second": 11.126,
1283
+ "eval_steps_per_second": 0.761,
1284
+ "step": 85
1285
+ },
1286
+ {
1287
+ "epoch": 2.0,
1288
+ "grad_norm": 0.08465206623077393,
1289
+ "learning_rate": 0.00014827586206896554,
1290
+ "loss": 0.9676,
1291
+ "step": 86
1292
+ },
1293
+ {
1294
+ "epoch": 2.0,
1295
+ "eval_loss": 1.0091900825500488,
1296
+ "eval_runtime": 10.4594,
1297
+ "eval_samples_per_second": 11.186,
1298
+ "eval_steps_per_second": 0.765,
1299
+ "step": 86
1300
+ },
1301
+ {
1302
+ "epoch": 2.0232558139534884,
1303
+ "grad_norm": 0.05495611950755119,
1304
+ "learning_rate": 0.00014482758620689657,
1305
+ "loss": 0.9548,
1306
+ "step": 87
1307
+ },
1308
+ {
1309
+ "epoch": 2.0232558139534884,
1310
+ "eval_loss": 1.0090184211730957,
1311
+ "eval_runtime": 10.5403,
1312
+ "eval_samples_per_second": 11.1,
1313
+ "eval_steps_per_second": 0.759,
1314
+ "step": 87
1315
+ },
1316
+ {
1317
+ "epoch": 2.046511627906977,
1318
+ "grad_norm": 0.051615942269563675,
1319
+ "learning_rate": 0.0001413793103448276,
1320
+ "loss": 0.9413,
1321
+ "step": 88
1322
+ },
1323
+ {
1324
+ "epoch": 2.046511627906977,
1325
+ "eval_loss": 1.0091454982757568,
1326
+ "eval_runtime": 10.5362,
1327
+ "eval_samples_per_second": 11.105,
1328
+ "eval_steps_per_second": 0.759,
1329
+ "step": 88
1330
+ },
1331
+ {
1332
+ "epoch": 2.0697674418604652,
1333
+ "grad_norm": 0.06363032013177872,
1334
+ "learning_rate": 0.00013793103448275863,
1335
+ "loss": 0.9604,
1336
+ "step": 89
1337
+ },
1338
+ {
1339
+ "epoch": 2.0697674418604652,
1340
+ "eval_loss": 1.0093193054199219,
1341
+ "eval_runtime": 10.599,
1342
+ "eval_samples_per_second": 11.039,
1343
+ "eval_steps_per_second": 0.755,
1344
+ "step": 89
1345
+ },
1346
+ {
1347
+ "epoch": 2.0930232558139537,
1348
+ "grad_norm": 0.05618719756603241,
1349
+ "learning_rate": 0.00013448275862068965,
1350
+ "loss": 0.9627,
1351
+ "step": 90
1352
+ },
1353
+ {
1354
+ "epoch": 2.0930232558139537,
1355
+ "eval_loss": 1.0094503164291382,
1356
+ "eval_runtime": 10.5453,
1357
+ "eval_samples_per_second": 11.095,
1358
+ "eval_steps_per_second": 0.759,
1359
+ "step": 90
1360
+ },
1361
+ {
1362
+ "epoch": 2.116279069767442,
1363
+ "grad_norm": 0.061273831874132156,
1364
+ "learning_rate": 0.00013103448275862068,
1365
+ "loss": 0.9428,
1366
+ "step": 91
1367
+ },
1368
+ {
1369
+ "epoch": 2.116279069767442,
1370
+ "eval_loss": 1.0095981359481812,
1371
+ "eval_runtime": 10.5143,
1372
+ "eval_samples_per_second": 11.128,
1373
+ "eval_steps_per_second": 0.761,
1374
+ "step": 91
1375
+ },
1376
+ {
1377
+ "epoch": 2.13953488372093,
1378
+ "grad_norm": 0.05564780905842781,
1379
+ "learning_rate": 0.00012758620689655174,
1380
+ "loss": 0.9774,
1381
+ "step": 92
1382
+ },
1383
+ {
1384
+ "epoch": 2.13953488372093,
1385
+ "eval_loss": 1.009597897529602,
1386
+ "eval_runtime": 10.4849,
1387
+ "eval_samples_per_second": 11.159,
1388
+ "eval_steps_per_second": 0.763,
1389
+ "step": 92
1390
+ },
1391
+ {
1392
+ "epoch": 2.1627906976744184,
1393
+ "grad_norm": 0.05518029257655144,
1394
+ "learning_rate": 0.00012413793103448277,
1395
+ "loss": 0.9407,
1396
+ "step": 93
1397
+ },
1398
+ {
1399
+ "epoch": 2.1627906976744184,
1400
+ "eval_loss": 1.0096217393875122,
1401
+ "eval_runtime": 10.6044,
1402
+ "eval_samples_per_second": 11.033,
1403
+ "eval_steps_per_second": 0.754,
1404
+ "step": 93
1405
+ },
1406
+ {
1407
+ "epoch": 2.186046511627907,
1408
+ "grad_norm": 0.05853220447897911,
1409
+ "learning_rate": 0.0001206896551724138,
1410
+ "loss": 0.9396,
1411
+ "step": 94
1412
+ },
1413
+ {
1414
+ "epoch": 2.186046511627907,
1415
+ "eval_loss": 1.00962233543396,
1416
+ "eval_runtime": 10.5343,
1417
+ "eval_samples_per_second": 11.107,
1418
+ "eval_steps_per_second": 0.759,
1419
+ "step": 94
1420
+ },
1421
+ {
1422
+ "epoch": 2.2093023255813953,
1423
+ "grad_norm": 0.06098194420337677,
1424
+ "learning_rate": 0.00011724137931034482,
1425
+ "loss": 0.9705,
1426
+ "step": 95
1427
+ },
1428
+ {
1429
+ "epoch": 2.2093023255813953,
1430
+ "eval_loss": 1.0093858242034912,
1431
+ "eval_runtime": 10.4224,
1432
+ "eval_samples_per_second": 11.226,
1433
+ "eval_steps_per_second": 0.768,
1434
+ "step": 95
1435
+ },
1436
+ {
1437
+ "epoch": 2.2325581395348837,
1438
+ "grad_norm": 0.05779768154025078,
1439
+ "learning_rate": 0.00011379310344827588,
1440
+ "loss": 0.9427,
1441
+ "step": 96
1442
+ },
1443
+ {
1444
+ "epoch": 2.2325581395348837,
1445
+ "eval_loss": 1.0092523097991943,
1446
+ "eval_runtime": 10.6138,
1447
+ "eval_samples_per_second": 11.023,
1448
+ "eval_steps_per_second": 0.754,
1449
+ "step": 96
1450
+ },
1451
+ {
1452
+ "epoch": 2.255813953488372,
1453
+ "grad_norm": 0.059211816638708115,
1454
+ "learning_rate": 0.0001103448275862069,
1455
+ "loss": 0.9689,
1456
+ "step": 97
1457
+ },
1458
+ {
1459
+ "epoch": 2.255813953488372,
1460
+ "eval_loss": 1.0091252326965332,
1461
+ "eval_runtime": 10.504,
1462
+ "eval_samples_per_second": 11.139,
1463
+ "eval_steps_per_second": 0.762,
1464
+ "step": 97
1465
+ },
1466
+ {
1467
+ "epoch": 2.2790697674418605,
1468
+ "grad_norm": 0.058523572981357574,
1469
+ "learning_rate": 0.00010689655172413792,
1470
+ "loss": 0.9459,
1471
+ "step": 98
1472
+ },
1473
+ {
1474
+ "epoch": 2.2790697674418605,
1475
+ "eval_loss": 1.0091090202331543,
1476
+ "eval_runtime": 10.5529,
1477
+ "eval_samples_per_second": 11.087,
1478
+ "eval_steps_per_second": 0.758,
1479
+ "step": 98
1480
+ },
1481
+ {
1482
+ "epoch": 2.302325581395349,
1483
+ "grad_norm": 0.06452349573373795,
1484
+ "learning_rate": 0.00010344827586206898,
1485
+ "loss": 0.9434,
1486
+ "step": 99
1487
+ },
1488
+ {
1489
+ "epoch": 2.302325581395349,
1490
+ "eval_loss": 1.0091310739517212,
1491
+ "eval_runtime": 10.5481,
1492
+ "eval_samples_per_second": 11.092,
1493
+ "eval_steps_per_second": 0.758,
1494
+ "step": 99
1495
+ },
1496
+ {
1497
+ "epoch": 2.3255813953488373,
1498
+ "grad_norm": 0.06188109889626503,
1499
+ "learning_rate": 0.0001,
1500
+ "loss": 0.9453,
1501
+ "step": 100
1502
+ },
1503
+ {
1504
+ "epoch": 2.3255813953488373,
1505
+ "eval_loss": 1.0090867280960083,
1506
+ "eval_runtime": 10.5427,
1507
+ "eval_samples_per_second": 11.098,
1508
+ "eval_steps_per_second": 0.759,
1509
+ "step": 100
1510
+ },
1511
+ {
1512
+ "epoch": 2.3488372093023258,
1513
+ "grad_norm": 0.061615802347660065,
1514
+ "learning_rate": 9.655172413793105e-05,
1515
+ "loss": 0.9675,
1516
+ "step": 101
1517
+ },
1518
+ {
1519
+ "epoch": 2.3488372093023258,
1520
+ "eval_loss": 1.008958101272583,
1521
+ "eval_runtime": 10.62,
1522
+ "eval_samples_per_second": 11.017,
1523
+ "eval_steps_per_second": 0.753,
1524
+ "step": 101
1525
+ },
1526
+ {
1527
+ "epoch": 2.3720930232558137,
1528
+ "grad_norm": 0.06509216129779816,
1529
+ "learning_rate": 9.310344827586207e-05,
1530
+ "loss": 0.9571,
1531
+ "step": 102
1532
+ },
1533
+ {
1534
+ "epoch": 2.3720930232558137,
1535
+ "eval_loss": 1.008782982826233,
1536
+ "eval_runtime": 10.5067,
1537
+ "eval_samples_per_second": 11.136,
1538
+ "eval_steps_per_second": 0.761,
1539
+ "step": 102
1540
+ },
1541
+ {
1542
+ "epoch": 2.395348837209302,
1543
+ "grad_norm": 0.07000201940536499,
1544
+ "learning_rate": 8.96551724137931e-05,
1545
+ "loss": 0.9669,
1546
+ "step": 103
1547
+ },
1548
+ {
1549
+ "epoch": 2.395348837209302,
1550
+ "eval_loss": 1.0085687637329102,
1551
+ "eval_runtime": 10.5369,
1552
+ "eval_samples_per_second": 11.104,
1553
+ "eval_steps_per_second": 0.759,
1554
+ "step": 103
1555
+ },
1556
+ {
1557
+ "epoch": 2.4186046511627906,
1558
+ "grad_norm": 0.06144480034708977,
1559
+ "learning_rate": 8.620689655172413e-05,
1560
+ "loss": 0.9592,
1561
+ "step": 104
1562
+ },
1563
+ {
1564
+ "epoch": 2.4186046511627906,
1565
+ "eval_loss": 1.0085170269012451,
1566
+ "eval_runtime": 10.5319,
1567
+ "eval_samples_per_second": 11.109,
1568
+ "eval_steps_per_second": 0.76,
1569
+ "step": 104
1570
+ },
1571
+ {
1572
+ "epoch": 2.441860465116279,
1573
+ "grad_norm": 0.06189959496259689,
1574
+ "learning_rate": 8.275862068965517e-05,
1575
+ "loss": 0.9259,
1576
+ "step": 105
1577
+ },
1578
+ {
1579
+ "epoch": 2.441860465116279,
1580
+ "eval_loss": 1.0085593461990356,
1581
+ "eval_runtime": 10.501,
1582
+ "eval_samples_per_second": 11.142,
1583
+ "eval_steps_per_second": 0.762,
1584
+ "step": 105
1585
+ },
1586
+ {
1587
+ "epoch": 2.4651162790697674,
1588
+ "grad_norm": 0.06781314313411713,
1589
+ "learning_rate": 7.931034482758621e-05,
1590
+ "loss": 0.945,
1591
+ "step": 106
1592
+ },
1593
+ {
1594
+ "epoch": 2.4651162790697674,
1595
+ "eval_loss": 1.0086591243743896,
1596
+ "eval_runtime": 10.536,
1597
+ "eval_samples_per_second": 11.105,
1598
+ "eval_steps_per_second": 0.759,
1599
+ "step": 106
1600
+ },
1601
+ {
1602
+ "epoch": 2.488372093023256,
1603
+ "grad_norm": 0.06491964310407639,
1604
+ "learning_rate": 7.586206896551724e-05,
1605
+ "loss": 0.948,
1606
+ "step": 107
1607
+ },
1608
+ {
1609
+ "epoch": 2.488372093023256,
1610
+ "eval_loss": 1.0086618661880493,
1611
+ "eval_runtime": 10.5398,
1612
+ "eval_samples_per_second": 11.101,
1613
+ "eval_steps_per_second": 0.759,
1614
+ "step": 107
1615
+ },
1616
+ {
1617
+ "epoch": 2.511627906976744,
1618
+ "grad_norm": 0.13770791888237,
1619
+ "learning_rate": 7.241379310344828e-05,
1620
+ "loss": 0.9631,
1621
+ "step": 108
1622
+ },
1623
+ {
1624
+ "epoch": 2.511627906976744,
1625
+ "eval_loss": 1.008668303489685,
1626
+ "eval_runtime": 10.4916,
1627
+ "eval_samples_per_second": 11.152,
1628
+ "eval_steps_per_second": 0.763,
1629
+ "step": 108
1630
+ },
1631
+ {
1632
+ "epoch": 2.5348837209302326,
1633
+ "grad_norm": 0.06982796639204025,
1634
+ "learning_rate": 6.896551724137931e-05,
1635
+ "loss": 0.9315,
1636
+ "step": 109
1637
+ },
1638
+ {
1639
+ "epoch": 2.5348837209302326,
1640
+ "eval_loss": 1.008758783340454,
1641
+ "eval_runtime": 10.4002,
1642
+ "eval_samples_per_second": 11.25,
1643
+ "eval_steps_per_second": 0.769,
1644
+ "step": 109
1645
+ },
1646
+ {
1647
+ "epoch": 2.558139534883721,
1648
+ "grad_norm": 0.06349999457597733,
1649
+ "learning_rate": 6.551724137931034e-05,
1650
+ "loss": 0.9433,
1651
+ "step": 110
1652
+ },
1653
+ {
1654
+ "epoch": 2.558139534883721,
1655
+ "eval_loss": 1.0088914632797241,
1656
+ "eval_runtime": 10.5022,
1657
+ "eval_samples_per_second": 11.141,
1658
+ "eval_steps_per_second": 0.762,
1659
+ "step": 110
1660
+ },
1661
+ {
1662
+ "epoch": 2.5813953488372094,
1663
+ "grad_norm": 0.06294334679841995,
1664
+ "learning_rate": 6.206896551724138e-05,
1665
+ "loss": 0.9484,
1666
+ "step": 111
1667
+ },
1668
+ {
1669
+ "epoch": 2.5813953488372094,
1670
+ "eval_loss": 1.0089675188064575,
1671
+ "eval_runtime": 10.4932,
1672
+ "eval_samples_per_second": 11.15,
1673
+ "eval_steps_per_second": 0.762,
1674
+ "step": 111
1675
+ },
1676
+ {
1677
+ "epoch": 2.604651162790698,
1678
+ "grad_norm": 0.06499718874692917,
1679
+ "learning_rate": 5.862068965517241e-05,
1680
+ "loss": 0.9519,
1681
+ "step": 112
1682
+ },
1683
+ {
1684
+ "epoch": 2.604651162790698,
1685
+ "eval_loss": 1.0090423822402954,
1686
+ "eval_runtime": 10.5478,
1687
+ "eval_samples_per_second": 11.092,
1688
+ "eval_steps_per_second": 0.758,
1689
+ "step": 112
1690
+ },
1691
+ {
1692
+ "epoch": 2.6279069767441863,
1693
+ "grad_norm": 0.06928952783346176,
1694
+ "learning_rate": 5.517241379310345e-05,
1695
+ "loss": 0.9551,
1696
+ "step": 113
1697
+ },
1698
+ {
1699
+ "epoch": 2.6279069767441863,
1700
+ "eval_loss": 1.008984923362732,
1701
+ "eval_runtime": 10.493,
1702
+ "eval_samples_per_second": 11.15,
1703
+ "eval_steps_per_second": 0.762,
1704
+ "step": 113
1705
+ },
1706
+ {
1707
+ "epoch": 2.6511627906976747,
1708
+ "grad_norm": 0.06725452095270157,
1709
+ "learning_rate": 5.172413793103449e-05,
1710
+ "loss": 0.9404,
1711
+ "step": 114
1712
+ },
1713
+ {
1714
+ "epoch": 2.6511627906976747,
1715
+ "eval_loss": 1.0089396238327026,
1716
+ "eval_runtime": 10.479,
1717
+ "eval_samples_per_second": 11.165,
1718
+ "eval_steps_per_second": 0.763,
1719
+ "step": 114
1720
+ },
1721
+ {
1722
+ "epoch": 2.6744186046511627,
1723
+ "grad_norm": 0.06509212404489517,
1724
+ "learning_rate": 4.827586206896552e-05,
1725
+ "loss": 0.9377,
1726
+ "step": 115
1727
+ },
1728
+ {
1729
+ "epoch": 2.6744186046511627,
1730
+ "eval_loss": 1.0088380575180054,
1731
+ "eval_runtime": 10.4542,
1732
+ "eval_samples_per_second": 11.192,
1733
+ "eval_steps_per_second": 0.765,
1734
+ "step": 115
1735
+ },
1736
+ {
1737
+ "epoch": 2.697674418604651,
1738
+ "grad_norm": 0.06430088728666306,
1739
+ "learning_rate": 4.482758620689655e-05,
1740
+ "loss": 0.9447,
1741
+ "step": 116
1742
+ },
1743
+ {
1744
+ "epoch": 2.697674418604651,
1745
+ "eval_loss": 1.008760929107666,
1746
+ "eval_runtime": 10.529,
1747
+ "eval_samples_per_second": 11.112,
1748
+ "eval_steps_per_second": 0.76,
1749
+ "step": 116
1750
+ },
1751
+ {
1752
+ "epoch": 2.7209302325581395,
1753
+ "grad_norm": 0.07184050977230072,
1754
+ "learning_rate": 4.1379310344827587e-05,
1755
+ "loss": 0.9608,
1756
+ "step": 117
1757
+ },
1758
+ {
1759
+ "epoch": 2.7209302325581395,
1760
+ "eval_loss": 1.0085643529891968,
1761
+ "eval_runtime": 10.5269,
1762
+ "eval_samples_per_second": 11.114,
1763
+ "eval_steps_per_second": 0.76,
1764
+ "step": 117
1765
+ },
1766
+ {
1767
+ "epoch": 2.744186046511628,
1768
+ "grad_norm": 0.06626931577920914,
1769
+ "learning_rate": 3.793103448275862e-05,
1770
+ "loss": 0.9459,
1771
+ "step": 118
1772
+ },
1773
+ {
1774
+ "epoch": 2.744186046511628,
1775
+ "eval_loss": 1.008436918258667,
1776
+ "eval_runtime": 10.526,
1777
+ "eval_samples_per_second": 11.115,
1778
+ "eval_steps_per_second": 0.76,
1779
+ "step": 118
1780
+ },
1781
+ {
1782
+ "epoch": 2.7674418604651163,
1783
+ "grad_norm": 0.07003094255924225,
1784
+ "learning_rate": 3.4482758620689657e-05,
1785
+ "loss": 0.9601,
1786
+ "step": 119
1787
+ },
1788
+ {
1789
+ "epoch": 2.7674418604651163,
1790
+ "eval_loss": 1.0083117485046387,
1791
+ "eval_runtime": 10.6067,
1792
+ "eval_samples_per_second": 11.031,
1793
+ "eval_steps_per_second": 0.754,
1794
+ "step": 119
1795
+ },
1796
+ {
1797
+ "epoch": 2.7906976744186047,
1798
+ "grad_norm": 0.06969019770622253,
1799
+ "learning_rate": 3.103448275862069e-05,
1800
+ "loss": 0.9154,
1801
+ "step": 120
1802
+ },
1803
+ {
1804
+ "epoch": 2.7906976744186047,
1805
+ "eval_loss": 1.0082221031188965,
1806
+ "eval_runtime": 10.5347,
1807
+ "eval_samples_per_second": 11.106,
1808
+ "eval_steps_per_second": 0.759,
1809
+ "step": 120
1810
+ },
1811
+ {
1812
+ "epoch": 2.813953488372093,
1813
+ "grad_norm": 0.06678508222103119,
1814
+ "learning_rate": 2.7586206896551727e-05,
1815
+ "loss": 0.964,
1816
+ "step": 121
1817
+ },
1818
+ {
1819
+ "epoch": 2.813953488372093,
1820
+ "eval_loss": 1.00809907913208,
1821
+ "eval_runtime": 10.5791,
1822
+ "eval_samples_per_second": 11.06,
1823
+ "eval_steps_per_second": 0.756,
1824
+ "step": 121
1825
+ },
1826
+ {
1827
+ "epoch": 2.8372093023255816,
1828
+ "grad_norm": 0.06645270437002182,
1829
+ "learning_rate": 2.413793103448276e-05,
1830
+ "loss": 0.9593,
1831
+ "step": 122
1832
+ },
1833
+ {
1834
+ "epoch": 2.8372093023255816,
1835
+ "eval_loss": 1.0079501867294312,
1836
+ "eval_runtime": 10.5303,
1837
+ "eval_samples_per_second": 11.111,
1838
+ "eval_steps_per_second": 0.76,
1839
+ "step": 122
1840
+ },
1841
+ {
1842
+ "epoch": 2.8604651162790695,
1843
+ "grad_norm": 0.06668780744075775,
1844
+ "learning_rate": 2.0689655172413793e-05,
1845
+ "loss": 0.9513,
1846
+ "step": 123
1847
+ },
1848
+ {
1849
+ "epoch": 2.8604651162790695,
1850
+ "eval_loss": 1.0078498125076294,
1851
+ "eval_runtime": 10.537,
1852
+ "eval_samples_per_second": 11.104,
1853
+ "eval_steps_per_second": 0.759,
1854
+ "step": 123
1855
+ },
1856
+ {
1857
+ "epoch": 2.883720930232558,
1858
+ "grad_norm": 0.07333076000213623,
1859
+ "learning_rate": 1.7241379310344828e-05,
1860
+ "loss": 0.9319,
1861
+ "step": 124
1862
+ },
1863
+ {
1864
+ "epoch": 2.883720930232558,
1865
+ "eval_loss": 1.0077686309814453,
1866
+ "eval_runtime": 10.5258,
1867
+ "eval_samples_per_second": 11.116,
1868
+ "eval_steps_per_second": 0.76,
1869
+ "step": 124
1870
+ },
1871
+ {
1872
+ "epoch": 2.9069767441860463,
1873
+ "grad_norm": 0.0677880272269249,
1874
+ "learning_rate": 1.3793103448275863e-05,
1875
+ "loss": 0.9456,
1876
+ "step": 125
1877
+ },
1878
+ {
1879
+ "epoch": 2.9069767441860463,
1880
+ "eval_loss": 1.0076959133148193,
1881
+ "eval_runtime": 10.5226,
1882
+ "eval_samples_per_second": 11.119,
1883
+ "eval_steps_per_second": 0.76,
1884
+ "step": 125
1885
+ },
1886
+ {
1887
+ "epoch": 2.9302325581395348,
1888
+ "grad_norm": 0.06794837862253189,
1889
+ "learning_rate": 1.0344827586206897e-05,
1890
+ "loss": 0.9446,
1891
+ "step": 126
1892
+ },
1893
+ {
1894
+ "epoch": 2.9302325581395348,
1895
+ "eval_loss": 1.0076078176498413,
1896
+ "eval_runtime": 10.5921,
1897
+ "eval_samples_per_second": 11.046,
1898
+ "eval_steps_per_second": 0.755,
1899
+ "step": 126
1900
+ },
1901
+ {
1902
+ "epoch": 2.953488372093023,
1903
+ "grad_norm": 0.0673353523015976,
1904
+ "learning_rate": 6.896551724137932e-06,
1905
+ "loss": 0.9215,
1906
+ "step": 127
1907
+ },
1908
+ {
1909
+ "epoch": 2.953488372093023,
1910
+ "eval_loss": 1.007556676864624,
1911
+ "eval_runtime": 10.5629,
1912
+ "eval_samples_per_second": 11.077,
1913
+ "eval_steps_per_second": 0.757,
1914
+ "step": 127
1915
+ },
1916
+ {
1917
+ "epoch": 2.9767441860465116,
1918
+ "grad_norm": 0.06916272640228271,
1919
+ "learning_rate": 3.448275862068966e-06,
1920
+ "loss": 0.9492,
1921
+ "step": 128
1922
+ },
1923
+ {
1924
+ "epoch": 2.9767441860465116,
1925
+ "eval_loss": 1.007544755935669,
1926
+ "eval_runtime": 10.5461,
1927
+ "eval_samples_per_second": 11.094,
1928
+ "eval_steps_per_second": 0.759,
1929
+ "step": 128
1930
+ },
1931
+ {
1932
+ "epoch": 3.0,
1933
+ "grad_norm": 0.10813567787408829,
1934
+ "learning_rate": 0.0,
1935
+ "loss": 0.962,
1936
+ "step": 129
1937
+ },
1938
+ {
1939
+ "epoch": 3.0,
1940
+ "eval_loss": 1.0075572729110718,
1941
+ "eval_runtime": 10.4988,
1942
+ "eval_samples_per_second": 11.144,
1943
+ "eval_steps_per_second": 0.762,
1944
+ "step": 129
1945
+ },
1946
+ {
1947
+ "epoch": 3.0,
1948
+ "step": 129,
1949
+ "total_flos": 4.714967128552243e+16,
1950
+ "train_loss": 1.1705420798109483,
1951
+ "train_runtime": 2551.3345,
1952
+ "train_samples_per_second": 1.594,
1953
+ "train_steps_per_second": 0.051
1954
+ }
1955
+ ],
1956
+ "logging_steps": 1,
1957
+ "max_steps": 129,
1958
+ "num_input_tokens_seen": 0,
1959
+ "num_train_epochs": 3,
1960
+ "save_steps": 10,
1961
+ "stateful_callbacks": {
1962
+ "TrainerControl": {
1963
+ "args": {
1964
+ "should_epoch_stop": false,
1965
+ "should_evaluate": false,
1966
+ "should_log": false,
1967
+ "should_save": true,
1968
+ "should_training_stop": false
1969
+ },
1970
+ "attributes": {}
1971
+ }
1972
+ },
1973
+ "total_flos": 4.714967128552243e+16,
1974
+ "train_batch_size": 32,
1975
+ "trial_name": null,
1976
+ "trial_params": null
1977
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:175fa9b835782af01021c38d541ab432b2880714c5ad1998afed59d856d20031
3
+ size 5688