kanishka commited on
Commit
7ed8f63
1 Parent(s): dae8d3f

End of training

Browse files
README.md CHANGED
@@ -1,11 +1,23 @@
1
  ---
2
  tags:
3
  - generated_from_trainer
 
 
4
  metrics:
5
  - accuracy
6
  model-index:
7
  - name: smolm-autoreg-bpe-counterfactual_babylm_aann_low_variability_numeral_1024-1e-3
8
- results: []
 
 
 
 
 
 
 
 
 
 
9
  ---
10
 
11
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -13,7 +25,7 @@ should probably proofread and complete it, then remove this comment. -->
13
 
14
  # smolm-autoreg-bpe-counterfactual_babylm_aann_low_variability_numeral_1024-1e-3
15
 
16
- This model was trained from scratch on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
  - Loss: 3.3975
19
  - Accuracy: 0.4113
 
1
  ---
2
  tags:
3
  - generated_from_trainer
4
+ datasets:
5
+ - kanishka/counterfactual_babylm_aann_low_variability_numeral
6
  metrics:
7
  - accuracy
8
  model-index:
9
  - name: smolm-autoreg-bpe-counterfactual_babylm_aann_low_variability_numeral_1024-1e-3
10
+ results:
11
+ - task:
12
+ name: Causal Language Modeling
13
+ type: text-generation
14
+ dataset:
15
+ name: kanishka/counterfactual_babylm_aann_low_variability_numeral
16
+ type: kanishka/counterfactual_babylm_aann_low_variability_numeral
17
+ metrics:
18
+ - name: Accuracy
19
+ type: accuracy
20
+ value: 0.41125847180686265
21
  ---
22
 
23
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
25
 
26
  # smolm-autoreg-bpe-counterfactual_babylm_aann_low_variability_numeral_1024-1e-3
27
 
28
+ This model was trained from scratch on the kanishka/counterfactual_babylm_aann_low_variability_numeral dataset.
29
  It achieves the following results on the evaluation set:
30
  - Loss: 3.3975
31
  - Accuracy: 0.4113
all_results.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 20.0,
3
+ "eval_accuracy": 0.41125847180686265,
4
+ "eval_loss": 3.397514581680298,
5
+ "eval_runtime": 153.7992,
6
+ "eval_samples": 57921,
7
+ "eval_samples_per_second": 376.601,
8
+ "eval_steps_per_second": 5.891,
9
+ "perplexity": 29.889719196260273,
10
+ "train_loss": 3.027371880730273,
11
+ "train_runtime": 81125.0441,
12
+ "train_samples": 594966,
13
+ "train_samples_per_second": 146.679,
14
+ "train_steps_per_second": 4.584
15
+ }
eval_results.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 20.0,
3
+ "eval_accuracy": 0.41125847180686265,
4
+ "eval_loss": 3.397514581680298,
5
+ "eval_runtime": 153.7992,
6
+ "eval_samples": 57921,
7
+ "eval_samples_per_second": 376.601,
8
+ "eval_steps_per_second": 5.891,
9
+ "perplexity": 29.889719196260273
10
+ }
runs/Jul15_13-15-43_phyl-ling-p01.la.utexas.edu/events.out.tfevents.1721148997.phyl-ling-p01.la.utexas.edu.2792777.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97cd137b88f3e1a0c27a21f3b82a0c3a1821d1f6cd71a50438d9d98093e5a4eb
3
+ size 417
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 20.0,
3
+ "train_loss": 3.027371880730273,
4
+ "train_runtime": 81125.0441,
5
+ "train_samples": 594966,
6
+ "train_samples_per_second": 146.679,
7
+ "train_steps_per_second": 4.584
8
+ }
trainer_state.json ADDED
@@ -0,0 +1,2807 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 20.0,
5
+ "eval_steps": 500,
6
+ "global_step": 371860,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.05,
13
+ "grad_norm": 0.8706823587417603,
14
+ "learning_rate": 3.125e-05,
15
+ "loss": 6.2283,
16
+ "step": 1000
17
+ },
18
+ {
19
+ "epoch": 0.11,
20
+ "grad_norm": 0.9266456365585327,
21
+ "learning_rate": 6.25e-05,
22
+ "loss": 5.0066,
23
+ "step": 2000
24
+ },
25
+ {
26
+ "epoch": 0.16,
27
+ "grad_norm": 0.9616362452507019,
28
+ "learning_rate": 9.375e-05,
29
+ "loss": 4.6763,
30
+ "step": 3000
31
+ },
32
+ {
33
+ "epoch": 0.22,
34
+ "grad_norm": 0.7885107398033142,
35
+ "learning_rate": 0.000125,
36
+ "loss": 4.4613,
37
+ "step": 4000
38
+ },
39
+ {
40
+ "epoch": 0.27,
41
+ "grad_norm": 0.7654786705970764,
42
+ "learning_rate": 0.00015625,
43
+ "loss": 4.3065,
44
+ "step": 5000
45
+ },
46
+ {
47
+ "epoch": 0.32,
48
+ "grad_norm": 0.7073574662208557,
49
+ "learning_rate": 0.0001875,
50
+ "loss": 4.1746,
51
+ "step": 6000
52
+ },
53
+ {
54
+ "epoch": 0.38,
55
+ "grad_norm": 0.6433243751525879,
56
+ "learning_rate": 0.00021875,
57
+ "loss": 4.075,
58
+ "step": 7000
59
+ },
60
+ {
61
+ "epoch": 0.43,
62
+ "grad_norm": 0.6497963070869446,
63
+ "learning_rate": 0.00025,
64
+ "loss": 3.9794,
65
+ "step": 8000
66
+ },
67
+ {
68
+ "epoch": 0.48,
69
+ "grad_norm": 0.6403814554214478,
70
+ "learning_rate": 0.00028121875,
71
+ "loss": 3.9048,
72
+ "step": 9000
73
+ },
74
+ {
75
+ "epoch": 0.54,
76
+ "grad_norm": 0.6297944188117981,
77
+ "learning_rate": 0.00031246875000000003,
78
+ "loss": 3.8448,
79
+ "step": 10000
80
+ },
81
+ {
82
+ "epoch": 0.59,
83
+ "grad_norm": 0.5447595715522766,
84
+ "learning_rate": 0.00034371875,
85
+ "loss": 3.7899,
86
+ "step": 11000
87
+ },
88
+ {
89
+ "epoch": 0.65,
90
+ "grad_norm": 0.5269156098365784,
91
+ "learning_rate": 0.0003749375,
92
+ "loss": 3.7537,
93
+ "step": 12000
94
+ },
95
+ {
96
+ "epoch": 0.7,
97
+ "grad_norm": 0.5088472366333008,
98
+ "learning_rate": 0.0004061875,
99
+ "loss": 3.7246,
100
+ "step": 13000
101
+ },
102
+ {
103
+ "epoch": 0.75,
104
+ "grad_norm": 0.4877682030200958,
105
+ "learning_rate": 0.00043737500000000005,
106
+ "loss": 3.6885,
107
+ "step": 14000
108
+ },
109
+ {
110
+ "epoch": 0.81,
111
+ "grad_norm": 0.45420414209365845,
112
+ "learning_rate": 0.000468625,
113
+ "loss": 3.6676,
114
+ "step": 15000
115
+ },
116
+ {
117
+ "epoch": 0.86,
118
+ "grad_norm": 0.44210752844810486,
119
+ "learning_rate": 0.000499875,
120
+ "loss": 3.6451,
121
+ "step": 16000
122
+ },
123
+ {
124
+ "epoch": 0.91,
125
+ "grad_norm": 0.41903868317604065,
126
+ "learning_rate": 0.000531125,
127
+ "loss": 3.6174,
128
+ "step": 17000
129
+ },
130
+ {
131
+ "epoch": 0.97,
132
+ "grad_norm": 0.4097944498062134,
133
+ "learning_rate": 0.0005623749999999999,
134
+ "loss": 3.598,
135
+ "step": 18000
136
+ },
137
+ {
138
+ "epoch": 1.0,
139
+ "eval_accuracy": 0.35856486935076,
140
+ "eval_loss": 3.8055434226989746,
141
+ "eval_runtime": 153.4601,
142
+ "eval_samples_per_second": 377.434,
143
+ "eval_steps_per_second": 5.904,
144
+ "step": 18593
145
+ },
146
+ {
147
+ "epoch": 1.02,
148
+ "grad_norm": 0.36287716031074524,
149
+ "learning_rate": 0.0005935625,
150
+ "loss": 3.5686,
151
+ "step": 19000
152
+ },
153
+ {
154
+ "epoch": 1.08,
155
+ "grad_norm": 0.34093010425567627,
156
+ "learning_rate": 0.0006248125,
157
+ "loss": 3.5426,
158
+ "step": 20000
159
+ },
160
+ {
161
+ "epoch": 1.13,
162
+ "grad_norm": 0.3330960273742676,
163
+ "learning_rate": 0.0006560625,
164
+ "loss": 3.5376,
165
+ "step": 21000
166
+ },
167
+ {
168
+ "epoch": 1.18,
169
+ "grad_norm": 0.30257412791252136,
170
+ "learning_rate": 0.00068728125,
171
+ "loss": 3.5228,
172
+ "step": 22000
173
+ },
174
+ {
175
+ "epoch": 1.24,
176
+ "grad_norm": 0.2785021960735321,
177
+ "learning_rate": 0.00071853125,
178
+ "loss": 3.5095,
179
+ "step": 23000
180
+ },
181
+ {
182
+ "epoch": 1.29,
183
+ "grad_norm": 0.2798495292663574,
184
+ "learning_rate": 0.00074978125,
185
+ "loss": 3.502,
186
+ "step": 24000
187
+ },
188
+ {
189
+ "epoch": 1.34,
190
+ "grad_norm": 0.27469247579574585,
191
+ "learning_rate": 0.0007810312499999999,
192
+ "loss": 3.4931,
193
+ "step": 25000
194
+ },
195
+ {
196
+ "epoch": 1.4,
197
+ "grad_norm": 0.28253281116485596,
198
+ "learning_rate": 0.00081225,
199
+ "loss": 3.4818,
200
+ "step": 26000
201
+ },
202
+ {
203
+ "epoch": 1.45,
204
+ "grad_norm": 0.2425285279750824,
205
+ "learning_rate": 0.0008435000000000001,
206
+ "loss": 3.4757,
207
+ "step": 27000
208
+ },
209
+ {
210
+ "epoch": 1.51,
211
+ "grad_norm": 0.26154839992523193,
212
+ "learning_rate": 0.00087471875,
213
+ "loss": 3.465,
214
+ "step": 28000
215
+ },
216
+ {
217
+ "epoch": 1.56,
218
+ "grad_norm": 0.24984851479530334,
219
+ "learning_rate": 0.00090596875,
220
+ "loss": 3.4591,
221
+ "step": 29000
222
+ },
223
+ {
224
+ "epoch": 1.61,
225
+ "grad_norm": 0.24280162155628204,
226
+ "learning_rate": 0.00093721875,
227
+ "loss": 3.4481,
228
+ "step": 30000
229
+ },
230
+ {
231
+ "epoch": 1.67,
232
+ "grad_norm": 0.2471531629562378,
233
+ "learning_rate": 0.00096846875,
234
+ "loss": 3.4382,
235
+ "step": 31000
236
+ },
237
+ {
238
+ "epoch": 1.72,
239
+ "grad_norm": 0.21386803686618805,
240
+ "learning_rate": 0.0009996875,
241
+ "loss": 3.4374,
242
+ "step": 32000
243
+ },
244
+ {
245
+ "epoch": 1.77,
246
+ "grad_norm": 0.23056912422180176,
247
+ "learning_rate": 0.0009970870358382863,
248
+ "loss": 3.4262,
249
+ "step": 33000
250
+ },
251
+ {
252
+ "epoch": 1.83,
253
+ "grad_norm": 0.22734513878822327,
254
+ "learning_rate": 0.0009941446477961513,
255
+ "loss": 3.4206,
256
+ "step": 34000
257
+ },
258
+ {
259
+ "epoch": 1.88,
260
+ "grad_norm": 0.27341413497924805,
261
+ "learning_rate": 0.0009912052021420585,
262
+ "loss": 3.4028,
263
+ "step": 35000
264
+ },
265
+ {
266
+ "epoch": 1.94,
267
+ "grad_norm": 0.2197876125574112,
268
+ "learning_rate": 0.0009882628140999235,
269
+ "loss": 3.3941,
270
+ "step": 36000
271
+ },
272
+ {
273
+ "epoch": 1.99,
274
+ "grad_norm": 0.2100793868303299,
275
+ "learning_rate": 0.0009853204260577885,
276
+ "loss": 3.3819,
277
+ "step": 37000
278
+ },
279
+ {
280
+ "epoch": 2.0,
281
+ "eval_accuracy": 0.3806616512481747,
282
+ "eval_loss": 3.5619659423828125,
283
+ "eval_runtime": 155.2921,
284
+ "eval_samples_per_second": 372.981,
285
+ "eval_steps_per_second": 5.834,
286
+ "step": 37186
287
+ },
288
+ {
289
+ "epoch": 2.04,
290
+ "grad_norm": 0.2338385134935379,
291
+ "learning_rate": 0.0009823839227917379,
292
+ "loss": 3.3373,
293
+ "step": 38000
294
+ },
295
+ {
296
+ "epoch": 2.1,
297
+ "grad_norm": 0.20092932879924774,
298
+ "learning_rate": 0.0009794415347496028,
299
+ "loss": 3.3246,
300
+ "step": 39000
301
+ },
302
+ {
303
+ "epoch": 2.15,
304
+ "grad_norm": 0.2200673222541809,
305
+ "learning_rate": 0.0009764991467074677,
306
+ "loss": 3.3266,
307
+ "step": 40000
308
+ },
309
+ {
310
+ "epoch": 2.21,
311
+ "grad_norm": 0.2211606204509735,
312
+ "learning_rate": 0.0009735567586653328,
313
+ "loss": 3.3149,
314
+ "step": 41000
315
+ },
316
+ {
317
+ "epoch": 2.26,
318
+ "grad_norm": 0.19502712786197662,
319
+ "learning_rate": 0.0009706143706231978,
320
+ "loss": 3.3163,
321
+ "step": 42000
322
+ },
323
+ {
324
+ "epoch": 2.31,
325
+ "grad_norm": 0.2075120508670807,
326
+ "learning_rate": 0.000967674924969105,
327
+ "loss": 3.3074,
328
+ "step": 43000
329
+ },
330
+ {
331
+ "epoch": 2.37,
332
+ "grad_norm": 0.22068411111831665,
333
+ "learning_rate": 0.0009647325369269699,
334
+ "loss": 3.3013,
335
+ "step": 44000
336
+ },
337
+ {
338
+ "epoch": 2.42,
339
+ "grad_norm": 0.19794422388076782,
340
+ "learning_rate": 0.000961793091272877,
341
+ "loss": 3.3064,
342
+ "step": 45000
343
+ },
344
+ {
345
+ "epoch": 2.47,
346
+ "grad_norm": 0.2203867882490158,
347
+ "learning_rate": 0.0009588507032307421,
348
+ "loss": 3.2953,
349
+ "step": 46000
350
+ },
351
+ {
352
+ "epoch": 2.53,
353
+ "grad_norm": 0.20799222588539124,
354
+ "learning_rate": 0.0009559112575766492,
355
+ "loss": 3.2912,
356
+ "step": 47000
357
+ },
358
+ {
359
+ "epoch": 2.58,
360
+ "grad_norm": 0.22151567041873932,
361
+ "learning_rate": 0.0009529688695345143,
362
+ "loss": 3.2828,
363
+ "step": 48000
364
+ },
365
+ {
366
+ "epoch": 2.64,
367
+ "grad_norm": 0.21492493152618408,
368
+ "learning_rate": 0.0009500264814923793,
369
+ "loss": 3.2807,
370
+ "step": 49000
371
+ },
372
+ {
373
+ "epoch": 2.69,
374
+ "grad_norm": 0.22774606943130493,
375
+ "learning_rate": 0.0009470840934502442,
376
+ "loss": 3.277,
377
+ "step": 50000
378
+ },
379
+ {
380
+ "epoch": 2.74,
381
+ "grad_norm": 0.20704419910907745,
382
+ "learning_rate": 0.0009441417054081093,
383
+ "loss": 3.2756,
384
+ "step": 51000
385
+ },
386
+ {
387
+ "epoch": 2.8,
388
+ "grad_norm": 0.18878625333309174,
389
+ "learning_rate": 0.0009412052021420585,
390
+ "loss": 3.2698,
391
+ "step": 52000
392
+ },
393
+ {
394
+ "epoch": 2.85,
395
+ "grad_norm": 0.21784694492816925,
396
+ "learning_rate": 0.0009382628140999236,
397
+ "loss": 3.2633,
398
+ "step": 53000
399
+ },
400
+ {
401
+ "epoch": 2.9,
402
+ "grad_norm": 0.19541580975055695,
403
+ "learning_rate": 0.0009353204260577886,
404
+ "loss": 3.2603,
405
+ "step": 54000
406
+ },
407
+ {
408
+ "epoch": 2.96,
409
+ "grad_norm": 0.1930837631225586,
410
+ "learning_rate": 0.0009323780380156535,
411
+ "loss": 3.2622,
412
+ "step": 55000
413
+ },
414
+ {
415
+ "epoch": 3.0,
416
+ "eval_accuracy": 0.3929298036900791,
417
+ "eval_loss": 3.4625182151794434,
418
+ "eval_runtime": 154.3214,
419
+ "eval_samples_per_second": 375.327,
420
+ "eval_steps_per_second": 5.871,
421
+ "step": 55779
422
+ },
423
+ {
424
+ "epoch": 3.01,
425
+ "grad_norm": 0.20553229749202728,
426
+ "learning_rate": 0.0009294356499735185,
427
+ "loss": 3.2366,
428
+ "step": 56000
429
+ },
430
+ {
431
+ "epoch": 3.07,
432
+ "grad_norm": 0.20729704201221466,
433
+ "learning_rate": 0.0009264962043194257,
434
+ "loss": 3.1896,
435
+ "step": 57000
436
+ },
437
+ {
438
+ "epoch": 3.12,
439
+ "grad_norm": 0.20280729234218597,
440
+ "learning_rate": 0.0009235538162772906,
441
+ "loss": 3.1933,
442
+ "step": 58000
443
+ },
444
+ {
445
+ "epoch": 3.17,
446
+ "grad_norm": 0.1880486160516739,
447
+ "learning_rate": 0.0009206114282351557,
448
+ "loss": 3.1987,
449
+ "step": 59000
450
+ },
451
+ {
452
+ "epoch": 3.23,
453
+ "grad_norm": 0.2021629512310028,
454
+ "learning_rate": 0.0009176719825810627,
455
+ "loss": 3.2011,
456
+ "step": 60000
457
+ },
458
+ {
459
+ "epoch": 3.28,
460
+ "grad_norm": 0.24562139809131622,
461
+ "learning_rate": 0.0009147295945389278,
462
+ "loss": 3.1928,
463
+ "step": 61000
464
+ },
465
+ {
466
+ "epoch": 3.33,
467
+ "grad_norm": 0.18961690366268158,
468
+ "learning_rate": 0.0009117901488848349,
469
+ "loss": 3.1966,
470
+ "step": 62000
471
+ },
472
+ {
473
+ "epoch": 3.39,
474
+ "grad_norm": 0.2056853473186493,
475
+ "learning_rate": 0.0009088477608426999,
476
+ "loss": 3.1888,
477
+ "step": 63000
478
+ },
479
+ {
480
+ "epoch": 3.44,
481
+ "grad_norm": 0.18212585151195526,
482
+ "learning_rate": 0.000905905372800565,
483
+ "loss": 3.1906,
484
+ "step": 64000
485
+ },
486
+ {
487
+ "epoch": 3.5,
488
+ "grad_norm": 0.21010248363018036,
489
+ "learning_rate": 0.000902965927146472,
490
+ "loss": 3.1925,
491
+ "step": 65000
492
+ },
493
+ {
494
+ "epoch": 3.55,
495
+ "grad_norm": 0.21237117052078247,
496
+ "learning_rate": 0.0009000235391043371,
497
+ "loss": 3.1918,
498
+ "step": 66000
499
+ },
500
+ {
501
+ "epoch": 3.6,
502
+ "grad_norm": 0.24303843080997467,
503
+ "learning_rate": 0.0008970840934502442,
504
+ "loss": 3.1918,
505
+ "step": 67000
506
+ },
507
+ {
508
+ "epoch": 3.66,
509
+ "grad_norm": 0.19125622510910034,
510
+ "learning_rate": 0.0008941417054081092,
511
+ "loss": 3.1866,
512
+ "step": 68000
513
+ },
514
+ {
515
+ "epoch": 3.71,
516
+ "grad_norm": 0.23382623493671417,
517
+ "learning_rate": 0.0008912022597540164,
518
+ "loss": 3.1844,
519
+ "step": 69000
520
+ },
521
+ {
522
+ "epoch": 3.76,
523
+ "grad_norm": 0.21234822273254395,
524
+ "learning_rate": 0.0008882598717118814,
525
+ "loss": 3.1852,
526
+ "step": 70000
527
+ },
528
+ {
529
+ "epoch": 3.82,
530
+ "grad_norm": 0.2681824564933777,
531
+ "learning_rate": 0.0008853204260577885,
532
+ "loss": 3.1855,
533
+ "step": 71000
534
+ },
535
+ {
536
+ "epoch": 3.87,
537
+ "grad_norm": 0.19967304170131683,
538
+ "learning_rate": 0.0008823780380156535,
539
+ "loss": 3.1781,
540
+ "step": 72000
541
+ },
542
+ {
543
+ "epoch": 3.93,
544
+ "grad_norm": 0.19710466265678406,
545
+ "learning_rate": 0.0008794385923615607,
546
+ "loss": 3.1804,
547
+ "step": 73000
548
+ },
549
+ {
550
+ "epoch": 3.98,
551
+ "grad_norm": 0.2174214869737625,
552
+ "learning_rate": 0.0008764962043194257,
553
+ "loss": 3.1788,
554
+ "step": 74000
555
+ },
556
+ {
557
+ "epoch": 4.0,
558
+ "eval_accuracy": 0.39836750674153854,
559
+ "eval_loss": 3.4131386280059814,
560
+ "eval_runtime": 154.2916,
561
+ "eval_samples_per_second": 375.4,
562
+ "eval_steps_per_second": 5.872,
563
+ "step": 74372
564
+ },
565
+ {
566
+ "epoch": 4.03,
567
+ "grad_norm": 0.23124143481254578,
568
+ "learning_rate": 0.0008735567586653328,
569
+ "loss": 3.1339,
570
+ "step": 75000
571
+ },
572
+ {
573
+ "epoch": 4.09,
574
+ "grad_norm": 0.2329677790403366,
575
+ "learning_rate": 0.0008706143706231978,
576
+ "loss": 3.1142,
577
+ "step": 76000
578
+ },
579
+ {
580
+ "epoch": 4.14,
581
+ "grad_norm": 0.21693040430545807,
582
+ "learning_rate": 0.0008676749249691049,
583
+ "loss": 3.1174,
584
+ "step": 77000
585
+ },
586
+ {
587
+ "epoch": 4.2,
588
+ "grad_norm": 0.2263847142457962,
589
+ "learning_rate": 0.00086473253692697,
590
+ "loss": 3.1182,
591
+ "step": 78000
592
+ },
593
+ {
594
+ "epoch": 4.25,
595
+ "grad_norm": 0.23874357342720032,
596
+ "learning_rate": 0.0008617930912728771,
597
+ "loss": 3.1247,
598
+ "step": 79000
599
+ },
600
+ {
601
+ "epoch": 4.3,
602
+ "grad_norm": 0.21914631128311157,
603
+ "learning_rate": 0.0008588507032307421,
604
+ "loss": 3.1239,
605
+ "step": 80000
606
+ },
607
+ {
608
+ "epoch": 4.36,
609
+ "grad_norm": 0.24384135007858276,
610
+ "learning_rate": 0.0008559083151886072,
611
+ "loss": 3.1273,
612
+ "step": 81000
613
+ },
614
+ {
615
+ "epoch": 4.41,
616
+ "grad_norm": 0.20350953936576843,
617
+ "learning_rate": 0.0008529688695345142,
618
+ "loss": 3.1251,
619
+ "step": 82000
620
+ },
621
+ {
622
+ "epoch": 4.46,
623
+ "grad_norm": 0.19013893604278564,
624
+ "learning_rate": 0.0008500264814923793,
625
+ "loss": 3.1249,
626
+ "step": 83000
627
+ },
628
+ {
629
+ "epoch": 4.52,
630
+ "grad_norm": 0.20296384394168854,
631
+ "learning_rate": 0.0008470840934502443,
632
+ "loss": 3.1283,
633
+ "step": 84000
634
+ },
635
+ {
636
+ "epoch": 4.57,
637
+ "grad_norm": 0.19276568293571472,
638
+ "learning_rate": 0.0008441417054081092,
639
+ "loss": 3.1257,
640
+ "step": 85000
641
+ },
642
+ {
643
+ "epoch": 4.63,
644
+ "grad_norm": 0.196748286485672,
645
+ "learning_rate": 0.0008412022597540165,
646
+ "loss": 3.1242,
647
+ "step": 86000
648
+ },
649
+ {
650
+ "epoch": 4.68,
651
+ "grad_norm": 0.20028959214687347,
652
+ "learning_rate": 0.0008382598717118813,
653
+ "loss": 3.1246,
654
+ "step": 87000
655
+ },
656
+ {
657
+ "epoch": 4.73,
658
+ "grad_norm": 0.20650836825370789,
659
+ "learning_rate": 0.0008353174836697463,
660
+ "loss": 3.1273,
661
+ "step": 88000
662
+ },
663
+ {
664
+ "epoch": 4.79,
665
+ "grad_norm": 0.2691803574562073,
666
+ "learning_rate": 0.0008323780380156535,
667
+ "loss": 3.1219,
668
+ "step": 89000
669
+ },
670
+ {
671
+ "epoch": 4.84,
672
+ "grad_norm": 0.20094037055969238,
673
+ "learning_rate": 0.0008294385923615606,
674
+ "loss": 3.1236,
675
+ "step": 90000
676
+ },
677
+ {
678
+ "epoch": 4.89,
679
+ "grad_norm": 0.23795652389526367,
680
+ "learning_rate": 0.0008264962043194257,
681
+ "loss": 3.1256,
682
+ "step": 91000
683
+ },
684
+ {
685
+ "epoch": 4.95,
686
+ "grad_norm": 0.20850218832492828,
687
+ "learning_rate": 0.0008235538162772906,
688
+ "loss": 3.1251,
689
+ "step": 92000
690
+ },
691
+ {
692
+ "epoch": 5.0,
693
+ "eval_accuracy": 0.40224626364489346,
694
+ "eval_loss": 3.385948419570923,
695
+ "eval_runtime": 154.3892,
696
+ "eval_samples_per_second": 375.162,
697
+ "eval_steps_per_second": 5.868,
698
+ "step": 92965
699
+ },
700
+ {
701
+ "epoch": 5.0,
702
+ "grad_norm": 0.19185079634189606,
703
+ "learning_rate": 0.0008206114282351556,
704
+ "loss": 3.1207,
705
+ "step": 93000
706
+ },
707
+ {
708
+ "epoch": 5.06,
709
+ "grad_norm": 0.2044549584388733,
710
+ "learning_rate": 0.0008176719825810628,
711
+ "loss": 3.0552,
712
+ "step": 94000
713
+ },
714
+ {
715
+ "epoch": 5.11,
716
+ "grad_norm": 0.2207447588443756,
717
+ "learning_rate": 0.0008147295945389278,
718
+ "loss": 3.0595,
719
+ "step": 95000
720
+ },
721
+ {
722
+ "epoch": 5.16,
723
+ "grad_norm": 0.2116185575723648,
724
+ "learning_rate": 0.0008117872064967929,
725
+ "loss": 3.0654,
726
+ "step": 96000
727
+ },
728
+ {
729
+ "epoch": 5.22,
730
+ "grad_norm": 0.20827573537826538,
731
+ "learning_rate": 0.0008088477608427,
732
+ "loss": 3.0653,
733
+ "step": 97000
734
+ },
735
+ {
736
+ "epoch": 5.27,
737
+ "grad_norm": 0.21885184943675995,
738
+ "learning_rate": 0.0008059053728005649,
739
+ "loss": 3.0689,
740
+ "step": 98000
741
+ },
742
+ {
743
+ "epoch": 5.32,
744
+ "grad_norm": 0.22063520550727844,
745
+ "learning_rate": 0.0008029659271464721,
746
+ "loss": 3.0734,
747
+ "step": 99000
748
+ },
749
+ {
750
+ "epoch": 5.38,
751
+ "grad_norm": 0.21756470203399658,
752
+ "learning_rate": 0.0008000235391043371,
753
+ "loss": 3.0761,
754
+ "step": 100000
755
+ },
756
+ {
757
+ "epoch": 5.43,
758
+ "grad_norm": 0.2308843582868576,
759
+ "learning_rate": 0.0007970811510622022,
760
+ "loss": 3.0764,
761
+ "step": 101000
762
+ },
763
+ {
764
+ "epoch": 5.49,
765
+ "grad_norm": 0.2255343496799469,
766
+ "learning_rate": 0.0007941387630200672,
767
+ "loss": 3.0786,
768
+ "step": 102000
769
+ },
770
+ {
771
+ "epoch": 5.54,
772
+ "grad_norm": 0.21067161858081818,
773
+ "learning_rate": 0.0007912022597540164,
774
+ "loss": 3.0786,
775
+ "step": 103000
776
+ },
777
+ {
778
+ "epoch": 5.59,
779
+ "grad_norm": 0.21768967807292938,
780
+ "learning_rate": 0.0007882598717118814,
781
+ "loss": 3.0773,
782
+ "step": 104000
783
+ },
784
+ {
785
+ "epoch": 5.65,
786
+ "grad_norm": 0.20667089521884918,
787
+ "learning_rate": 0.0007853174836697464,
788
+ "loss": 3.0795,
789
+ "step": 105000
790
+ },
791
+ {
792
+ "epoch": 5.7,
793
+ "grad_norm": 0.21367168426513672,
794
+ "learning_rate": 0.0007823750956276114,
795
+ "loss": 3.082,
796
+ "step": 106000
797
+ },
798
+ {
799
+ "epoch": 5.75,
800
+ "grad_norm": 0.2038014829158783,
801
+ "learning_rate": 0.0007794327075854764,
802
+ "loss": 3.0785,
803
+ "step": 107000
804
+ },
805
+ {
806
+ "epoch": 5.81,
807
+ "grad_norm": 0.21575333178043365,
808
+ "learning_rate": 0.0007764932619313834,
809
+ "loss": 3.0808,
810
+ "step": 108000
811
+ },
812
+ {
813
+ "epoch": 5.86,
814
+ "grad_norm": 0.20344999432563782,
815
+ "learning_rate": 0.0007735508738892485,
816
+ "loss": 3.0829,
817
+ "step": 109000
818
+ },
819
+ {
820
+ "epoch": 5.92,
821
+ "grad_norm": 0.2332640290260315,
822
+ "learning_rate": 0.0007706114282351556,
823
+ "loss": 3.0769,
824
+ "step": 110000
825
+ },
826
+ {
827
+ "epoch": 5.97,
828
+ "grad_norm": 0.23659543693065643,
829
+ "learning_rate": 0.0007676690401930207,
830
+ "loss": 3.0765,
831
+ "step": 111000
832
+ },
833
+ {
834
+ "epoch": 6.0,
835
+ "eval_accuracy": 0.4050011228899796,
836
+ "eval_loss": 3.3707284927368164,
837
+ "eval_runtime": 153.9554,
838
+ "eval_samples_per_second": 376.219,
839
+ "eval_steps_per_second": 5.885,
840
+ "step": 111558
841
+ },
842
+ {
843
+ "epoch": 6.02,
844
+ "grad_norm": 0.21739870309829712,
845
+ "learning_rate": 0.0007647266521508857,
846
+ "loss": 3.0492,
847
+ "step": 112000
848
+ },
849
+ {
850
+ "epoch": 6.08,
851
+ "grad_norm": 0.23045282065868378,
852
+ "learning_rate": 0.0007617872064967927,
853
+ "loss": 3.0123,
854
+ "step": 113000
855
+ },
856
+ {
857
+ "epoch": 6.13,
858
+ "grad_norm": 0.2114897221326828,
859
+ "learning_rate": 0.0007588448184546578,
860
+ "loss": 3.0196,
861
+ "step": 114000
862
+ },
863
+ {
864
+ "epoch": 6.19,
865
+ "grad_norm": 0.21402007341384888,
866
+ "learning_rate": 0.0007559053728005649,
867
+ "loss": 3.0229,
868
+ "step": 115000
869
+ },
870
+ {
871
+ "epoch": 6.24,
872
+ "grad_norm": 0.22187209129333496,
873
+ "learning_rate": 0.00075296298475843,
874
+ "loss": 3.0269,
875
+ "step": 116000
876
+ },
877
+ {
878
+ "epoch": 6.29,
879
+ "grad_norm": 0.2102159708738327,
880
+ "learning_rate": 0.000750020596716295,
881
+ "loss": 3.0273,
882
+ "step": 117000
883
+ },
884
+ {
885
+ "epoch": 6.35,
886
+ "grad_norm": 0.2769564092159271,
887
+ "learning_rate": 0.000747081151062202,
888
+ "loss": 3.0318,
889
+ "step": 118000
890
+ },
891
+ {
892
+ "epoch": 6.4,
893
+ "grad_norm": 0.2173321396112442,
894
+ "learning_rate": 0.0007441387630200671,
895
+ "loss": 3.0331,
896
+ "step": 119000
897
+ },
898
+ {
899
+ "epoch": 6.45,
900
+ "grad_norm": 0.24418728053569794,
901
+ "learning_rate": 0.0007411993173659742,
902
+ "loss": 3.0376,
903
+ "step": 120000
904
+ },
905
+ {
906
+ "epoch": 6.51,
907
+ "grad_norm": 0.23827865719795227,
908
+ "learning_rate": 0.0007382569293238393,
909
+ "loss": 3.0404,
910
+ "step": 121000
911
+ },
912
+ {
913
+ "epoch": 6.56,
914
+ "grad_norm": 0.2125864326953888,
915
+ "learning_rate": 0.0007353174836697464,
916
+ "loss": 3.0366,
917
+ "step": 122000
918
+ },
919
+ {
920
+ "epoch": 6.62,
921
+ "grad_norm": 0.21209298074245453,
922
+ "learning_rate": 0.0007323750956276114,
923
+ "loss": 3.0408,
924
+ "step": 123000
925
+ },
926
+ {
927
+ "epoch": 6.67,
928
+ "grad_norm": 0.21926939487457275,
929
+ "learning_rate": 0.0007294327075854764,
930
+ "loss": 3.0435,
931
+ "step": 124000
932
+ },
933
+ {
934
+ "epoch": 6.72,
935
+ "grad_norm": 0.2240251749753952,
936
+ "learning_rate": 0.0007264932619313835,
937
+ "loss": 3.0439,
938
+ "step": 125000
939
+ },
940
+ {
941
+ "epoch": 6.78,
942
+ "grad_norm": 0.21099534630775452,
943
+ "learning_rate": 0.0007235508738892486,
944
+ "loss": 3.0415,
945
+ "step": 126000
946
+ },
947
+ {
948
+ "epoch": 6.83,
949
+ "grad_norm": 0.20424066483974457,
950
+ "learning_rate": 0.0007206084858471136,
951
+ "loss": 3.0444,
952
+ "step": 127000
953
+ },
954
+ {
955
+ "epoch": 6.88,
956
+ "grad_norm": 0.22034712135791779,
957
+ "learning_rate": 0.0007176660978049785,
958
+ "loss": 3.0448,
959
+ "step": 128000
960
+ },
961
+ {
962
+ "epoch": 6.94,
963
+ "grad_norm": 0.23952622711658478,
964
+ "learning_rate": 0.0007147237097628436,
965
+ "loss": 3.0427,
966
+ "step": 129000
967
+ },
968
+ {
969
+ "epoch": 6.99,
970
+ "grad_norm": 0.21370893716812134,
971
+ "learning_rate": 0.0007117842641087506,
972
+ "loss": 3.0458,
973
+ "step": 130000
974
+ },
975
+ {
976
+ "epoch": 7.0,
977
+ "eval_accuracy": 0.4076835636248676,
978
+ "eval_loss": 3.3522255420684814,
979
+ "eval_runtime": 154.4925,
980
+ "eval_samples_per_second": 374.911,
981
+ "eval_steps_per_second": 5.864,
982
+ "step": 130151
983
+ },
984
+ {
985
+ "epoch": 7.05,
986
+ "grad_norm": 0.22439521551132202,
987
+ "learning_rate": 0.0007088448184546579,
988
+ "loss": 2.9871,
989
+ "step": 131000
990
+ },
991
+ {
992
+ "epoch": 7.1,
993
+ "grad_norm": 0.24310387670993805,
994
+ "learning_rate": 0.0007059024304125228,
995
+ "loss": 2.98,
996
+ "step": 132000
997
+ },
998
+ {
999
+ "epoch": 7.15,
1000
+ "grad_norm": 0.22332601249217987,
1001
+ "learning_rate": 0.0007029600423703878,
1002
+ "loss": 2.9863,
1003
+ "step": 133000
1004
+ },
1005
+ {
1006
+ "epoch": 7.21,
1007
+ "grad_norm": 0.24013490974903107,
1008
+ "learning_rate": 0.0007000176543282529,
1009
+ "loss": 2.9879,
1010
+ "step": 134000
1011
+ },
1012
+ {
1013
+ "epoch": 7.26,
1014
+ "grad_norm": 0.23142094910144806,
1015
+ "learning_rate": 0.0006970782086741599,
1016
+ "loss": 2.9914,
1017
+ "step": 135000
1018
+ },
1019
+ {
1020
+ "epoch": 7.31,
1021
+ "grad_norm": 0.236104354262352,
1022
+ "learning_rate": 0.0006941358206320249,
1023
+ "loss": 2.9964,
1024
+ "step": 136000
1025
+ },
1026
+ {
1027
+ "epoch": 7.37,
1028
+ "grad_norm": 0.33047351241111755,
1029
+ "learning_rate": 0.0006911963749779321,
1030
+ "loss": 2.9986,
1031
+ "step": 137000
1032
+ },
1033
+ {
1034
+ "epoch": 7.42,
1035
+ "grad_norm": 0.26023513078689575,
1036
+ "learning_rate": 0.0006882539869357971,
1037
+ "loss": 2.9995,
1038
+ "step": 138000
1039
+ },
1040
+ {
1041
+ "epoch": 7.48,
1042
+ "grad_norm": 0.26877066493034363,
1043
+ "learning_rate": 0.0006853115988936622,
1044
+ "loss": 3.0005,
1045
+ "step": 139000
1046
+ },
1047
+ {
1048
+ "epoch": 7.53,
1049
+ "grad_norm": 0.2331291288137436,
1050
+ "learning_rate": 0.0006823721532395692,
1051
+ "loss": 3.0057,
1052
+ "step": 140000
1053
+ },
1054
+ {
1055
+ "epoch": 7.58,
1056
+ "grad_norm": 0.21710553765296936,
1057
+ "learning_rate": 0.0006794297651974342,
1058
+ "loss": 3.0107,
1059
+ "step": 141000
1060
+ },
1061
+ {
1062
+ "epoch": 7.64,
1063
+ "grad_norm": 0.24514536559581757,
1064
+ "learning_rate": 0.0006764903195433414,
1065
+ "loss": 3.0055,
1066
+ "step": 142000
1067
+ },
1068
+ {
1069
+ "epoch": 7.69,
1070
+ "grad_norm": 0.21572381258010864,
1071
+ "learning_rate": 0.0006735508738892485,
1072
+ "loss": 3.0102,
1073
+ "step": 143000
1074
+ },
1075
+ {
1076
+ "epoch": 7.74,
1077
+ "grad_norm": 0.23894068598747253,
1078
+ "learning_rate": 0.0006706084858471136,
1079
+ "loss": 3.0101,
1080
+ "step": 144000
1081
+ },
1082
+ {
1083
+ "epoch": 7.8,
1084
+ "grad_norm": 0.20779648423194885,
1085
+ "learning_rate": 0.0006676660978049786,
1086
+ "loss": 3.0136,
1087
+ "step": 145000
1088
+ },
1089
+ {
1090
+ "epoch": 7.85,
1091
+ "grad_norm": 0.21832165122032166,
1092
+ "learning_rate": 0.0006647237097628435,
1093
+ "loss": 3.0144,
1094
+ "step": 146000
1095
+ },
1096
+ {
1097
+ "epoch": 7.91,
1098
+ "grad_norm": 0.2295515537261963,
1099
+ "learning_rate": 0.0006617813217207086,
1100
+ "loss": 3.0162,
1101
+ "step": 147000
1102
+ },
1103
+ {
1104
+ "epoch": 7.96,
1105
+ "grad_norm": 0.21139173209667206,
1106
+ "learning_rate": 0.0006588418760666157,
1107
+ "loss": 3.0116,
1108
+ "step": 148000
1109
+ },
1110
+ {
1111
+ "epoch": 8.0,
1112
+ "eval_accuracy": 0.4079684679864168,
1113
+ "eval_loss": 3.3560173511505127,
1114
+ "eval_runtime": 154.3168,
1115
+ "eval_samples_per_second": 375.338,
1116
+ "eval_steps_per_second": 5.871,
1117
+ "step": 148744
1118
+ },
1119
+ {
1120
+ "epoch": 8.01,
1121
+ "grad_norm": 0.2352878898382187,
1122
+ "learning_rate": 0.0006558994880244807,
1123
+ "loss": 2.9921,
1124
+ "step": 149000
1125
+ },
1126
+ {
1127
+ "epoch": 8.07,
1128
+ "grad_norm": 0.26401156187057495,
1129
+ "learning_rate": 0.0006529600423703879,
1130
+ "loss": 2.9483,
1131
+ "step": 150000
1132
+ },
1133
+ {
1134
+ "epoch": 8.12,
1135
+ "grad_norm": 0.2650126814842224,
1136
+ "learning_rate": 0.0006500176543282527,
1137
+ "loss": 2.9538,
1138
+ "step": 151000
1139
+ },
1140
+ {
1141
+ "epoch": 8.18,
1142
+ "grad_norm": 0.2173539251089096,
1143
+ "learning_rate": 0.00064707820867416,
1144
+ "loss": 2.9592,
1145
+ "step": 152000
1146
+ },
1147
+ {
1148
+ "epoch": 8.23,
1149
+ "grad_norm": 0.23582415282726288,
1150
+ "learning_rate": 0.0006441358206320249,
1151
+ "loss": 2.9674,
1152
+ "step": 153000
1153
+ },
1154
+ {
1155
+ "epoch": 8.28,
1156
+ "grad_norm": 0.24998264014720917,
1157
+ "learning_rate": 0.0006411963749779322,
1158
+ "loss": 2.9634,
1159
+ "step": 154000
1160
+ },
1161
+ {
1162
+ "epoch": 8.34,
1163
+ "grad_norm": 0.2233651876449585,
1164
+ "learning_rate": 0.0006382539869357971,
1165
+ "loss": 2.9725,
1166
+ "step": 155000
1167
+ },
1168
+ {
1169
+ "epoch": 8.39,
1170
+ "grad_norm": 0.2446517050266266,
1171
+ "learning_rate": 0.000635311598893662,
1172
+ "loss": 2.9668,
1173
+ "step": 156000
1174
+ },
1175
+ {
1176
+ "epoch": 8.44,
1177
+ "grad_norm": 0.27410104870796204,
1178
+ "learning_rate": 0.0006323692108515271,
1179
+ "loss": 2.9716,
1180
+ "step": 157000
1181
+ },
1182
+ {
1183
+ "epoch": 8.5,
1184
+ "grad_norm": 0.19913798570632935,
1185
+ "learning_rate": 0.0006294268228093921,
1186
+ "loss": 2.9744,
1187
+ "step": 158000
1188
+ },
1189
+ {
1190
+ "epoch": 8.55,
1191
+ "grad_norm": 0.24324330687522888,
1192
+ "learning_rate": 0.0006264844347672571,
1193
+ "loss": 2.975,
1194
+ "step": 159000
1195
+ },
1196
+ {
1197
+ "epoch": 8.61,
1198
+ "grad_norm": 0.2302367389202118,
1199
+ "learning_rate": 0.0006235479315012064,
1200
+ "loss": 2.9765,
1201
+ "step": 160000
1202
+ },
1203
+ {
1204
+ "epoch": 8.66,
1205
+ "grad_norm": 0.25450703501701355,
1206
+ "learning_rate": 0.0006206055434590713,
1207
+ "loss": 2.9762,
1208
+ "step": 161000
1209
+ },
1210
+ {
1211
+ "epoch": 8.71,
1212
+ "grad_norm": 0.2539065480232239,
1213
+ "learning_rate": 0.0006176631554169364,
1214
+ "loss": 2.9819,
1215
+ "step": 162000
1216
+ },
1217
+ {
1218
+ "epoch": 8.77,
1219
+ "grad_norm": 0.2278948873281479,
1220
+ "learning_rate": 0.0006147207673748014,
1221
+ "loss": 2.9809,
1222
+ "step": 163000
1223
+ },
1224
+ {
1225
+ "epoch": 8.82,
1226
+ "grad_norm": 0.23980122804641724,
1227
+ "learning_rate": 0.0006117813217207086,
1228
+ "loss": 2.9829,
1229
+ "step": 164000
1230
+ },
1231
+ {
1232
+ "epoch": 8.87,
1233
+ "grad_norm": 0.25505828857421875,
1234
+ "learning_rate": 0.0006088389336785736,
1235
+ "loss": 2.9845,
1236
+ "step": 165000
1237
+ },
1238
+ {
1239
+ "epoch": 8.93,
1240
+ "grad_norm": 0.26612409949302673,
1241
+ "learning_rate": 0.0006058965456364386,
1242
+ "loss": 2.9874,
1243
+ "step": 166000
1244
+ },
1245
+ {
1246
+ "epoch": 8.98,
1247
+ "grad_norm": 0.2189650535583496,
1248
+ "learning_rate": 0.0006029541575943035,
1249
+ "loss": 2.9832,
1250
+ "step": 167000
1251
+ },
1252
+ {
1253
+ "epoch": 9.0,
1254
+ "eval_accuracy": 0.40796309369994566,
1255
+ "eval_loss": 3.3773229122161865,
1256
+ "eval_runtime": 154.2001,
1257
+ "eval_samples_per_second": 375.622,
1258
+ "eval_steps_per_second": 5.875,
1259
+ "step": 167337
1260
+ },
1261
+ {
1262
+ "epoch": 9.04,
1263
+ "grad_norm": 0.27406546473503113,
1264
+ "learning_rate": 0.0006000147119402107,
1265
+ "loss": 2.9377,
1266
+ "step": 168000
1267
+ },
1268
+ {
1269
+ "epoch": 9.09,
1270
+ "grad_norm": 0.2886415719985962,
1271
+ "learning_rate": 0.0005970723238980756,
1272
+ "loss": 2.9216,
1273
+ "step": 169000
1274
+ },
1275
+ {
1276
+ "epoch": 9.14,
1277
+ "grad_norm": 0.30712640285491943,
1278
+ "learning_rate": 0.0005941299358559407,
1279
+ "loss": 2.9295,
1280
+ "step": 170000
1281
+ },
1282
+ {
1283
+ "epoch": 9.2,
1284
+ "grad_norm": 0.24219301342964172,
1285
+ "learning_rate": 0.0005911875478138057,
1286
+ "loss": 2.9333,
1287
+ "step": 171000
1288
+ },
1289
+ {
1290
+ "epoch": 9.25,
1291
+ "grad_norm": 0.24113836884498596,
1292
+ "learning_rate": 0.0005882481021597129,
1293
+ "loss": 2.9348,
1294
+ "step": 172000
1295
+ },
1296
+ {
1297
+ "epoch": 9.3,
1298
+ "grad_norm": 0.26300373673439026,
1299
+ "learning_rate": 0.0005853057141175778,
1300
+ "loss": 2.94,
1301
+ "step": 173000
1302
+ },
1303
+ {
1304
+ "epoch": 9.36,
1305
+ "grad_norm": 0.2541556656360626,
1306
+ "learning_rate": 0.0005823662684634849,
1307
+ "loss": 2.9424,
1308
+ "step": 174000
1309
+ },
1310
+ {
1311
+ "epoch": 9.41,
1312
+ "grad_norm": 0.2480422556400299,
1313
+ "learning_rate": 0.00057942388042135,
1314
+ "loss": 2.9424,
1315
+ "step": 175000
1316
+ },
1317
+ {
1318
+ "epoch": 9.47,
1319
+ "grad_norm": 0.22237752377986908,
1320
+ "learning_rate": 0.0005764844347672571,
1321
+ "loss": 2.9463,
1322
+ "step": 176000
1323
+ },
1324
+ {
1325
+ "epoch": 9.52,
1326
+ "grad_norm": 0.23950238525867462,
1327
+ "learning_rate": 0.0005735420467251222,
1328
+ "loss": 2.9531,
1329
+ "step": 177000
1330
+ },
1331
+ {
1332
+ "epoch": 9.57,
1333
+ "grad_norm": 0.2510799467563629,
1334
+ "learning_rate": 0.0005705996586829871,
1335
+ "loss": 2.952,
1336
+ "step": 178000
1337
+ },
1338
+ {
1339
+ "epoch": 9.63,
1340
+ "grad_norm": 0.23875580728054047,
1341
+ "learning_rate": 0.0005676572706408521,
1342
+ "loss": 2.955,
1343
+ "step": 179000
1344
+ },
1345
+ {
1346
+ "epoch": 9.68,
1347
+ "grad_norm": 0.2786506414413452,
1348
+ "learning_rate": 0.0005647178249867593,
1349
+ "loss": 2.9576,
1350
+ "step": 180000
1351
+ },
1352
+ {
1353
+ "epoch": 9.73,
1354
+ "grad_norm": 0.22881832718849182,
1355
+ "learning_rate": 0.0005617754369446243,
1356
+ "loss": 2.9578,
1357
+ "step": 181000
1358
+ },
1359
+ {
1360
+ "epoch": 9.79,
1361
+ "grad_norm": 0.24316935241222382,
1362
+ "learning_rate": 0.0005588359912905315,
1363
+ "loss": 2.9569,
1364
+ "step": 182000
1365
+ },
1366
+ {
1367
+ "epoch": 9.84,
1368
+ "grad_norm": 0.2705481946468353,
1369
+ "learning_rate": 0.0005558936032483965,
1370
+ "loss": 2.9565,
1371
+ "step": 183000
1372
+ },
1373
+ {
1374
+ "epoch": 9.9,
1375
+ "grad_norm": 0.2366609275341034,
1376
+ "learning_rate": 0.0005529512152062614,
1377
+ "loss": 2.9613,
1378
+ "step": 184000
1379
+ },
1380
+ {
1381
+ "epoch": 9.95,
1382
+ "grad_norm": 0.2798861265182495,
1383
+ "learning_rate": 0.0005500088271641264,
1384
+ "loss": 2.9565,
1385
+ "step": 185000
1386
+ },
1387
+ {
1388
+ "epoch": 10.0,
1389
+ "eval_accuracy": 0.40983898839268323,
1390
+ "eval_loss": 3.351311683654785,
1391
+ "eval_runtime": 153.7734,
1392
+ "eval_samples_per_second": 376.665,
1393
+ "eval_steps_per_second": 5.892,
1394
+ "step": 185930
1395
+ },
1396
+ {
1397
+ "epoch": 10.0,
1398
+ "grad_norm": 0.26953232288360596,
1399
+ "learning_rate": 0.0005470693815100336,
1400
+ "loss": 2.951,
1401
+ "step": 186000
1402
+ },
1403
+ {
1404
+ "epoch": 10.06,
1405
+ "grad_norm": 0.2257799357175827,
1406
+ "learning_rate": 0.0005441269934678985,
1407
+ "loss": 2.8908,
1408
+ "step": 187000
1409
+ },
1410
+ {
1411
+ "epoch": 10.11,
1412
+ "grad_norm": 0.23467403650283813,
1413
+ "learning_rate": 0.0005411875478138058,
1414
+ "loss": 2.9006,
1415
+ "step": 188000
1416
+ },
1417
+ {
1418
+ "epoch": 10.17,
1419
+ "grad_norm": 0.2749410569667816,
1420
+ "learning_rate": 0.0005382451597716706,
1421
+ "loss": 2.9069,
1422
+ "step": 189000
1423
+ },
1424
+ {
1425
+ "epoch": 10.22,
1426
+ "grad_norm": 0.2599303722381592,
1427
+ "learning_rate": 0.0005353057141175779,
1428
+ "loss": 2.9104,
1429
+ "step": 190000
1430
+ },
1431
+ {
1432
+ "epoch": 10.27,
1433
+ "grad_norm": 0.2843225598335266,
1434
+ "learning_rate": 0.0005323633260754428,
1435
+ "loss": 2.9137,
1436
+ "step": 191000
1437
+ },
1438
+ {
1439
+ "epoch": 10.33,
1440
+ "grad_norm": 0.24611413478851318,
1441
+ "learning_rate": 0.00052942388042135,
1442
+ "loss": 2.9161,
1443
+ "step": 192000
1444
+ },
1445
+ {
1446
+ "epoch": 10.38,
1447
+ "grad_norm": 0.2826565206050873,
1448
+ "learning_rate": 0.000526481492379215,
1449
+ "loss": 2.9175,
1450
+ "step": 193000
1451
+ },
1452
+ {
1453
+ "epoch": 10.43,
1454
+ "grad_norm": 0.28213128447532654,
1455
+ "learning_rate": 0.000523542046725122,
1456
+ "loss": 2.9164,
1457
+ "step": 194000
1458
+ },
1459
+ {
1460
+ "epoch": 10.49,
1461
+ "grad_norm": 0.2640922963619232,
1462
+ "learning_rate": 0.0005205996586829871,
1463
+ "loss": 2.9258,
1464
+ "step": 195000
1465
+ },
1466
+ {
1467
+ "epoch": 10.54,
1468
+ "grad_norm": 0.2534860670566559,
1469
+ "learning_rate": 0.0005176572706408521,
1470
+ "loss": 2.9278,
1471
+ "step": 196000
1472
+ },
1473
+ {
1474
+ "epoch": 10.6,
1475
+ "grad_norm": 0.24981430172920227,
1476
+ "learning_rate": 0.0005147148825987171,
1477
+ "loss": 2.9292,
1478
+ "step": 197000
1479
+ },
1480
+ {
1481
+ "epoch": 10.65,
1482
+ "grad_norm": 0.2640092372894287,
1483
+ "learning_rate": 0.0005117724945565822,
1484
+ "loss": 2.9365,
1485
+ "step": 198000
1486
+ },
1487
+ {
1488
+ "epoch": 10.7,
1489
+ "grad_norm": 0.27058008313179016,
1490
+ "learning_rate": 0.0005088330489024892,
1491
+ "loss": 2.9324,
1492
+ "step": 199000
1493
+ },
1494
+ {
1495
+ "epoch": 10.76,
1496
+ "grad_norm": 0.25028932094573975,
1497
+ "learning_rate": 0.0005058906608603543,
1498
+ "loss": 2.9324,
1499
+ "step": 200000
1500
+ },
1501
+ {
1502
+ "epoch": 10.81,
1503
+ "grad_norm": 0.2733946144580841,
1504
+ "learning_rate": 0.0005029512152062614,
1505
+ "loss": 2.9309,
1506
+ "step": 201000
1507
+ },
1508
+ {
1509
+ "epoch": 10.86,
1510
+ "grad_norm": 0.2877325117588043,
1511
+ "learning_rate": 0.0005000088271641264,
1512
+ "loss": 2.9353,
1513
+ "step": 202000
1514
+ },
1515
+ {
1516
+ "epoch": 10.92,
1517
+ "grad_norm": 0.2410973757505417,
1518
+ "learning_rate": 0.0004970664391219914,
1519
+ "loss": 2.9363,
1520
+ "step": 203000
1521
+ },
1522
+ {
1523
+ "epoch": 10.97,
1524
+ "grad_norm": 0.2605952024459839,
1525
+ "learning_rate": 0.0004941240510798565,
1526
+ "loss": 2.9381,
1527
+ "step": 204000
1528
+ },
1529
+ {
1530
+ "epoch": 11.0,
1531
+ "eval_accuracy": 0.40963281732793566,
1532
+ "eval_loss": 3.3482396602630615,
1533
+ "eval_runtime": 154.276,
1534
+ "eval_samples_per_second": 375.437,
1535
+ "eval_steps_per_second": 5.873,
1536
+ "step": 204523
1537
+ },
1538
+ {
1539
+ "epoch": 11.03,
1540
+ "grad_norm": 0.2494332194328308,
1541
+ "learning_rate": 0.0004911846054257635,
1542
+ "loss": 2.9059,
1543
+ "step": 205000
1544
+ },
1545
+ {
1546
+ "epoch": 11.08,
1547
+ "grad_norm": 0.27014607191085815,
1548
+ "learning_rate": 0.0004882451597716707,
1549
+ "loss": 2.8743,
1550
+ "step": 206000
1551
+ },
1552
+ {
1553
+ "epoch": 11.13,
1554
+ "grad_norm": 0.24216848611831665,
1555
+ "learning_rate": 0.0004853027717295357,
1556
+ "loss": 2.8821,
1557
+ "step": 207000
1558
+ },
1559
+ {
1560
+ "epoch": 11.19,
1561
+ "grad_norm": 0.2651168704032898,
1562
+ "learning_rate": 0.0004823633260754429,
1563
+ "loss": 2.8867,
1564
+ "step": 208000
1565
+ },
1566
+ {
1567
+ "epoch": 11.24,
1568
+ "grad_norm": 0.2583751082420349,
1569
+ "learning_rate": 0.0004794209380333078,
1570
+ "loss": 2.8907,
1571
+ "step": 209000
1572
+ },
1573
+ {
1574
+ "epoch": 11.29,
1575
+ "grad_norm": 0.26034820079803467,
1576
+ "learning_rate": 0.00047647854999117284,
1577
+ "loss": 2.8953,
1578
+ "step": 210000
1579
+ },
1580
+ {
1581
+ "epoch": 11.35,
1582
+ "grad_norm": 0.3137526214122772,
1583
+ "learning_rate": 0.0004735361619490379,
1584
+ "loss": 2.8943,
1585
+ "step": 211000
1586
+ },
1587
+ {
1588
+ "epoch": 11.4,
1589
+ "grad_norm": 0.29816457629203796,
1590
+ "learning_rate": 0.000470596716294945,
1591
+ "loss": 2.8985,
1592
+ "step": 212000
1593
+ },
1594
+ {
1595
+ "epoch": 11.46,
1596
+ "grad_norm": 0.2567712664604187,
1597
+ "learning_rate": 0.00046765727064085213,
1598
+ "loss": 2.9024,
1599
+ "step": 213000
1600
+ },
1601
+ {
1602
+ "epoch": 11.51,
1603
+ "grad_norm": 0.2891099154949188,
1604
+ "learning_rate": 0.0004647148825987171,
1605
+ "loss": 2.9014,
1606
+ "step": 214000
1607
+ },
1608
+ {
1609
+ "epoch": 11.56,
1610
+ "grad_norm": 0.26076436042785645,
1611
+ "learning_rate": 0.00046177249455658215,
1612
+ "loss": 2.9061,
1613
+ "step": 215000
1614
+ },
1615
+ {
1616
+ "epoch": 11.62,
1617
+ "grad_norm": 0.2649165689945221,
1618
+ "learning_rate": 0.00045883010651444713,
1619
+ "loss": 2.9061,
1620
+ "step": 216000
1621
+ },
1622
+ {
1623
+ "epoch": 11.67,
1624
+ "grad_norm": 0.30298641324043274,
1625
+ "learning_rate": 0.0004558877184723121,
1626
+ "loss": 2.9054,
1627
+ "step": 217000
1628
+ },
1629
+ {
1630
+ "epoch": 11.72,
1631
+ "grad_norm": 0.2831864058971405,
1632
+ "learning_rate": 0.00045294827281821924,
1633
+ "loss": 2.9075,
1634
+ "step": 218000
1635
+ },
1636
+ {
1637
+ "epoch": 11.78,
1638
+ "grad_norm": 0.2540026307106018,
1639
+ "learning_rate": 0.00045000882716412637,
1640
+ "loss": 2.9112,
1641
+ "step": 219000
1642
+ },
1643
+ {
1644
+ "epoch": 11.83,
1645
+ "grad_norm": 0.2612316906452179,
1646
+ "learning_rate": 0.0004470664391219914,
1647
+ "loss": 2.9124,
1648
+ "step": 220000
1649
+ },
1650
+ {
1651
+ "epoch": 11.89,
1652
+ "grad_norm": 0.26292499899864197,
1653
+ "learning_rate": 0.00044412405107985644,
1654
+ "loss": 2.9111,
1655
+ "step": 221000
1656
+ },
1657
+ {
1658
+ "epoch": 11.94,
1659
+ "grad_norm": 0.2696852684020996,
1660
+ "learning_rate": 0.0004411816630377214,
1661
+ "loss": 2.9151,
1662
+ "step": 222000
1663
+ },
1664
+ {
1665
+ "epoch": 11.99,
1666
+ "grad_norm": 0.2645638883113861,
1667
+ "learning_rate": 0.00043824221738362855,
1668
+ "loss": 2.9171,
1669
+ "step": 223000
1670
+ },
1671
+ {
1672
+ "epoch": 12.0,
1673
+ "eval_accuracy": 0.41176466241385945,
1674
+ "eval_loss": 3.3354427814483643,
1675
+ "eval_runtime": 153.951,
1676
+ "eval_samples_per_second": 376.23,
1677
+ "eval_steps_per_second": 5.885,
1678
+ "step": 223116
1679
+ },
1680
+ {
1681
+ "epoch": 12.05,
1682
+ "grad_norm": 0.289524108171463,
1683
+ "learning_rate": 0.0004352998293414936,
1684
+ "loss": 2.8568,
1685
+ "step": 224000
1686
+ },
1687
+ {
1688
+ "epoch": 12.1,
1689
+ "grad_norm": 0.28729820251464844,
1690
+ "learning_rate": 0.00043235744129935857,
1691
+ "loss": 2.858,
1692
+ "step": 225000
1693
+ },
1694
+ {
1695
+ "epoch": 12.16,
1696
+ "grad_norm": 0.2749260663986206,
1697
+ "learning_rate": 0.00042941505325722355,
1698
+ "loss": 2.8627,
1699
+ "step": 226000
1700
+ },
1701
+ {
1702
+ "epoch": 12.21,
1703
+ "grad_norm": 0.2578941583633423,
1704
+ "learning_rate": 0.0004264785499911729,
1705
+ "loss": 2.8695,
1706
+ "step": 227000
1707
+ },
1708
+ {
1709
+ "epoch": 12.26,
1710
+ "grad_norm": 0.2701773941516876,
1711
+ "learning_rate": 0.0004235361619490378,
1712
+ "loss": 2.8702,
1713
+ "step": 228000
1714
+ },
1715
+ {
1716
+ "epoch": 12.32,
1717
+ "grad_norm": 0.2578844428062439,
1718
+ "learning_rate": 0.00042059377390690284,
1719
+ "loss": 2.8706,
1720
+ "step": 229000
1721
+ },
1722
+ {
1723
+ "epoch": 12.37,
1724
+ "grad_norm": 0.2564055621623993,
1725
+ "learning_rate": 0.0004176513858647679,
1726
+ "loss": 2.8719,
1727
+ "step": 230000
1728
+ },
1729
+ {
1730
+ "epoch": 12.42,
1731
+ "grad_norm": 0.2670447528362274,
1732
+ "learning_rate": 0.000414711940210675,
1733
+ "loss": 2.8779,
1734
+ "step": 231000
1735
+ },
1736
+ {
1737
+ "epoch": 12.48,
1738
+ "grad_norm": 0.25996309518814087,
1739
+ "learning_rate": 0.00041176955216854,
1740
+ "loss": 2.8834,
1741
+ "step": 232000
1742
+ },
1743
+ {
1744
+ "epoch": 12.53,
1745
+ "grad_norm": 0.2569678723812103,
1746
+ "learning_rate": 0.00040882716412640497,
1747
+ "loss": 2.8813,
1748
+ "step": 233000
1749
+ },
1750
+ {
1751
+ "epoch": 12.59,
1752
+ "grad_norm": 0.25648847222328186,
1753
+ "learning_rate": 0.00040588771847231215,
1754
+ "loss": 2.8887,
1755
+ "step": 234000
1756
+ },
1757
+ {
1758
+ "epoch": 12.64,
1759
+ "grad_norm": 0.27697890996932983,
1760
+ "learning_rate": 0.00040294533043017713,
1761
+ "loss": 2.8892,
1762
+ "step": 235000
1763
+ },
1764
+ {
1765
+ "epoch": 12.69,
1766
+ "grad_norm": 0.2665017247200012,
1767
+ "learning_rate": 0.0004000058847760843,
1768
+ "loss": 2.8903,
1769
+ "step": 236000
1770
+ },
1771
+ {
1772
+ "epoch": 12.75,
1773
+ "grad_norm": 0.27752918004989624,
1774
+ "learning_rate": 0.00039706349673394924,
1775
+ "loss": 2.8866,
1776
+ "step": 237000
1777
+ },
1778
+ {
1779
+ "epoch": 12.8,
1780
+ "grad_norm": 0.28366342186927795,
1781
+ "learning_rate": 0.0003941240510798564,
1782
+ "loss": 2.8901,
1783
+ "step": 238000
1784
+ },
1785
+ {
1786
+ "epoch": 12.85,
1787
+ "grad_norm": 0.268888920545578,
1788
+ "learning_rate": 0.0003911816630377214,
1789
+ "loss": 2.8909,
1790
+ "step": 239000
1791
+ },
1792
+ {
1793
+ "epoch": 12.91,
1794
+ "grad_norm": 0.2524598240852356,
1795
+ "learning_rate": 0.00038823927499558644,
1796
+ "loss": 2.8904,
1797
+ "step": 240000
1798
+ },
1799
+ {
1800
+ "epoch": 12.96,
1801
+ "grad_norm": 0.2472527027130127,
1802
+ "learning_rate": 0.00038529982934149356,
1803
+ "loss": 2.893,
1804
+ "step": 241000
1805
+ },
1806
+ {
1807
+ "epoch": 13.0,
1808
+ "eval_accuracy": 0.4111454102552269,
1809
+ "eval_loss": 3.3519082069396973,
1810
+ "eval_runtime": 153.4319,
1811
+ "eval_samples_per_second": 377.503,
1812
+ "eval_steps_per_second": 5.905,
1813
+ "step": 241709
1814
+ },
1815
+ {
1816
+ "epoch": 13.02,
1817
+ "grad_norm": 0.26184141635894775,
1818
+ "learning_rate": 0.00038235744129935855,
1819
+ "loss": 2.8762,
1820
+ "step": 242000
1821
+ },
1822
+ {
1823
+ "epoch": 13.07,
1824
+ "grad_norm": 0.26451557874679565,
1825
+ "learning_rate": 0.00037941799564526567,
1826
+ "loss": 2.8369,
1827
+ "step": 243000
1828
+ },
1829
+ {
1830
+ "epoch": 13.12,
1831
+ "grad_norm": 0.26714783906936646,
1832
+ "learning_rate": 0.0003764756076031307,
1833
+ "loss": 2.8415,
1834
+ "step": 244000
1835
+ },
1836
+ {
1837
+ "epoch": 13.18,
1838
+ "grad_norm": 0.25869232416152954,
1839
+ "learning_rate": 0.00037353321956099574,
1840
+ "loss": 2.8465,
1841
+ "step": 245000
1842
+ },
1843
+ {
1844
+ "epoch": 13.23,
1845
+ "grad_norm": 0.30912360548973083,
1846
+ "learning_rate": 0.00037059377390690287,
1847
+ "loss": 2.8481,
1848
+ "step": 246000
1849
+ },
1850
+ {
1851
+ "epoch": 13.28,
1852
+ "grad_norm": 0.30928054451942444,
1853
+ "learning_rate": 0.00036765138586476785,
1854
+ "loss": 2.8496,
1855
+ "step": 247000
1856
+ },
1857
+ {
1858
+ "epoch": 13.34,
1859
+ "grad_norm": 0.32625919580459595,
1860
+ "learning_rate": 0.00036470899782263283,
1861
+ "loss": 2.8551,
1862
+ "step": 248000
1863
+ },
1864
+ {
1865
+ "epoch": 13.39,
1866
+ "grad_norm": 0.2953076958656311,
1867
+ "learning_rate": 0.00036176955216854,
1868
+ "loss": 2.8563,
1869
+ "step": 249000
1870
+ },
1871
+ {
1872
+ "epoch": 13.45,
1873
+ "grad_norm": 0.2896451950073242,
1874
+ "learning_rate": 0.000358827164126405,
1875
+ "loss": 2.8619,
1876
+ "step": 250000
1877
+ },
1878
+ {
1879
+ "epoch": 13.5,
1880
+ "grad_norm": 0.271090567111969,
1881
+ "learning_rate": 0.0003558877184723122,
1882
+ "loss": 2.8616,
1883
+ "step": 251000
1884
+ },
1885
+ {
1886
+ "epoch": 13.55,
1887
+ "grad_norm": 0.2772102952003479,
1888
+ "learning_rate": 0.0003529453304301771,
1889
+ "loss": 2.8644,
1890
+ "step": 252000
1891
+ },
1892
+ {
1893
+ "epoch": 13.61,
1894
+ "grad_norm": 0.25150129199028015,
1895
+ "learning_rate": 0.00035000294238804214,
1896
+ "loss": 2.8627,
1897
+ "step": 253000
1898
+ },
1899
+ {
1900
+ "epoch": 13.66,
1901
+ "grad_norm": 0.27617818117141724,
1902
+ "learning_rate": 0.00034706349673394927,
1903
+ "loss": 2.861,
1904
+ "step": 254000
1905
+ },
1906
+ {
1907
+ "epoch": 13.71,
1908
+ "grad_norm": 0.2900349795818329,
1909
+ "learning_rate": 0.0003441211086918143,
1910
+ "loss": 2.8657,
1911
+ "step": 255000
1912
+ },
1913
+ {
1914
+ "epoch": 13.77,
1915
+ "grad_norm": 0.2805399000644684,
1916
+ "learning_rate": 0.0003411787206496793,
1917
+ "loss": 2.871,
1918
+ "step": 256000
1919
+ },
1920
+ {
1921
+ "epoch": 13.82,
1922
+ "grad_norm": 0.2589282691478729,
1923
+ "learning_rate": 0.00033823633260754427,
1924
+ "loss": 2.8672,
1925
+ "step": 257000
1926
+ },
1927
+ {
1928
+ "epoch": 13.88,
1929
+ "grad_norm": 0.291664183139801,
1930
+ "learning_rate": 0.00033529688695345145,
1931
+ "loss": 2.8715,
1932
+ "step": 258000
1933
+ },
1934
+ {
1935
+ "epoch": 13.93,
1936
+ "grad_norm": 0.2693411707878113,
1937
+ "learning_rate": 0.00033235449891131643,
1938
+ "loss": 2.8763,
1939
+ "step": 259000
1940
+ },
1941
+ {
1942
+ "epoch": 13.98,
1943
+ "grad_norm": 0.27113044261932373,
1944
+ "learning_rate": 0.0003294150532572236,
1945
+ "loss": 2.876,
1946
+ "step": 260000
1947
+ },
1948
+ {
1949
+ "epoch": 14.0,
1950
+ "eval_accuracy": 0.41093769408311887,
1951
+ "eval_loss": 3.3631937503814697,
1952
+ "eval_runtime": 153.9063,
1953
+ "eval_samples_per_second": 376.339,
1954
+ "eval_steps_per_second": 5.887,
1955
+ "step": 260302
1956
+ },
1957
+ {
1958
+ "epoch": 14.04,
1959
+ "grad_norm": 0.2805856168270111,
1960
+ "learning_rate": 0.00032647266521508854,
1961
+ "loss": 2.8345,
1962
+ "step": 261000
1963
+ },
1964
+ {
1965
+ "epoch": 14.09,
1966
+ "grad_norm": 0.2932230234146118,
1967
+ "learning_rate": 0.00032353321956099567,
1968
+ "loss": 2.8245,
1969
+ "step": 262000
1970
+ },
1971
+ {
1972
+ "epoch": 14.15,
1973
+ "grad_norm": 0.2711317837238312,
1974
+ "learning_rate": 0.0003205908315188607,
1975
+ "loss": 2.8265,
1976
+ "step": 263000
1977
+ },
1978
+ {
1979
+ "epoch": 14.2,
1980
+ "grad_norm": 0.30451709032058716,
1981
+ "learning_rate": 0.00031765138586476783,
1982
+ "loss": 2.8294,
1983
+ "step": 264000
1984
+ },
1985
+ {
1986
+ "epoch": 14.25,
1987
+ "grad_norm": 0.2669803500175476,
1988
+ "learning_rate": 0.00031470899782263287,
1989
+ "loss": 2.8349,
1990
+ "step": 265000
1991
+ },
1992
+ {
1993
+ "epoch": 14.31,
1994
+ "grad_norm": 0.3362666070461273,
1995
+ "learning_rate": 0.00031176660978049785,
1996
+ "loss": 2.8385,
1997
+ "step": 266000
1998
+ },
1999
+ {
2000
+ "epoch": 14.36,
2001
+ "grad_norm": 0.2826295495033264,
2002
+ "learning_rate": 0.000308827164126405,
2003
+ "loss": 2.8351,
2004
+ "step": 267000
2005
+ },
2006
+ {
2007
+ "epoch": 14.41,
2008
+ "grad_norm": 0.29114797711372375,
2009
+ "learning_rate": 0.00030588477608427,
2010
+ "loss": 2.8372,
2011
+ "step": 268000
2012
+ },
2013
+ {
2014
+ "epoch": 14.47,
2015
+ "grad_norm": 0.29871678352355957,
2016
+ "learning_rate": 0.00030294533043017714,
2017
+ "loss": 2.8397,
2018
+ "step": 269000
2019
+ },
2020
+ {
2021
+ "epoch": 14.52,
2022
+ "grad_norm": 0.2672843933105469,
2023
+ "learning_rate": 0.0003000029423880422,
2024
+ "loss": 2.8448,
2025
+ "step": 270000
2026
+ },
2027
+ {
2028
+ "epoch": 14.58,
2029
+ "grad_norm": 0.25932183861732483,
2030
+ "learning_rate": 0.0002970605543459071,
2031
+ "loss": 2.8443,
2032
+ "step": 271000
2033
+ },
2034
+ {
2035
+ "epoch": 14.63,
2036
+ "grad_norm": 0.2911001443862915,
2037
+ "learning_rate": 0.00029411816630377214,
2038
+ "loss": 2.8493,
2039
+ "step": 272000
2040
+ },
2041
+ {
2042
+ "epoch": 14.68,
2043
+ "grad_norm": 0.2664991617202759,
2044
+ "learning_rate": 0.00029117577826163717,
2045
+ "loss": 2.8422,
2046
+ "step": 273000
2047
+ },
2048
+ {
2049
+ "epoch": 14.74,
2050
+ "grad_norm": 0.30084219574928284,
2051
+ "learning_rate": 0.0002882363326075443,
2052
+ "loss": 2.8505,
2053
+ "step": 274000
2054
+ },
2055
+ {
2056
+ "epoch": 14.79,
2057
+ "grad_norm": 0.2992892861366272,
2058
+ "learning_rate": 0.0002852939445654093,
2059
+ "loss": 2.8497,
2060
+ "step": 275000
2061
+ },
2062
+ {
2063
+ "epoch": 14.84,
2064
+ "grad_norm": 0.335448682308197,
2065
+ "learning_rate": 0.0002823544989113164,
2066
+ "loss": 2.8529,
2067
+ "step": 276000
2068
+ },
2069
+ {
2070
+ "epoch": 14.9,
2071
+ "grad_norm": 0.28675752878189087,
2072
+ "learning_rate": 0.00027941211086918144,
2073
+ "loss": 2.8497,
2074
+ "step": 277000
2075
+ },
2076
+ {
2077
+ "epoch": 14.95,
2078
+ "grad_norm": 0.3506409525871277,
2079
+ "learning_rate": 0.00027647266521508857,
2080
+ "loss": 2.8472,
2081
+ "step": 278000
2082
+ },
2083
+ {
2084
+ "epoch": 15.0,
2085
+ "eval_accuracy": 0.41203740745226775,
2086
+ "eval_loss": 3.3530044555664062,
2087
+ "eval_runtime": 154.3507,
2088
+ "eval_samples_per_second": 375.256,
2089
+ "eval_steps_per_second": 5.87,
2090
+ "step": 278895
2091
+ },
2092
+ {
2093
+ "epoch": 15.01,
2094
+ "grad_norm": 0.2708049416542053,
2095
+ "learning_rate": 0.0002735302771729536,
2096
+ "loss": 2.8479,
2097
+ "step": 279000
2098
+ },
2099
+ {
2100
+ "epoch": 15.06,
2101
+ "grad_norm": 0.2830442190170288,
2102
+ "learning_rate": 0.00027059083151886073,
2103
+ "loss": 2.8018,
2104
+ "step": 280000
2105
+ },
2106
+ {
2107
+ "epoch": 15.11,
2108
+ "grad_norm": 0.2702392041683197,
2109
+ "learning_rate": 0.0002676484434767257,
2110
+ "loss": 2.805,
2111
+ "step": 281000
2112
+ },
2113
+ {
2114
+ "epoch": 15.17,
2115
+ "grad_norm": 0.3452757000923157,
2116
+ "learning_rate": 0.00026470899782263284,
2117
+ "loss": 2.8103,
2118
+ "step": 282000
2119
+ },
2120
+ {
2121
+ "epoch": 15.22,
2122
+ "grad_norm": 0.28475674986839294,
2123
+ "learning_rate": 0.0002617666097804979,
2124
+ "loss": 2.8141,
2125
+ "step": 283000
2126
+ },
2127
+ {
2128
+ "epoch": 15.27,
2129
+ "grad_norm": 0.2740528881549835,
2130
+ "learning_rate": 0.00025882422173836286,
2131
+ "loss": 2.8162,
2132
+ "step": 284000
2133
+ },
2134
+ {
2135
+ "epoch": 15.33,
2136
+ "grad_norm": 0.3067547380924225,
2137
+ "learning_rate": 0.00025588183369622784,
2138
+ "loss": 2.8182,
2139
+ "step": 285000
2140
+ },
2141
+ {
2142
+ "epoch": 15.38,
2143
+ "grad_norm": 0.3173828721046448,
2144
+ "learning_rate": 0.00025294238804213497,
2145
+ "loss": 2.8179,
2146
+ "step": 286000
2147
+ },
2148
+ {
2149
+ "epoch": 15.44,
2150
+ "grad_norm": 0.32790786027908325,
2151
+ "learning_rate": 0.00025,
2152
+ "loss": 2.8222,
2153
+ "step": 287000
2154
+ },
2155
+ {
2156
+ "epoch": 15.49,
2157
+ "grad_norm": 0.3280278444290161,
2158
+ "learning_rate": 0.00024706055434590713,
2159
+ "loss": 2.8263,
2160
+ "step": 288000
2161
+ },
2162
+ {
2163
+ "epoch": 15.54,
2164
+ "grad_norm": 0.30356574058532715,
2165
+ "learning_rate": 0.00024411816630377214,
2166
+ "loss": 2.826,
2167
+ "step": 289000
2168
+ },
2169
+ {
2170
+ "epoch": 15.6,
2171
+ "grad_norm": 0.2901437282562256,
2172
+ "learning_rate": 0.00024117577826163715,
2173
+ "loss": 2.8254,
2174
+ "step": 290000
2175
+ },
2176
+ {
2177
+ "epoch": 15.65,
2178
+ "grad_norm": 0.34759074449539185,
2179
+ "learning_rate": 0.0002382363326075443,
2180
+ "loss": 2.8291,
2181
+ "step": 291000
2182
+ },
2183
+ {
2184
+ "epoch": 15.7,
2185
+ "grad_norm": 0.38021716475486755,
2186
+ "learning_rate": 0.00023529394456540929,
2187
+ "loss": 2.8286,
2188
+ "step": 292000
2189
+ },
2190
+ {
2191
+ "epoch": 15.76,
2192
+ "grad_norm": 0.29485049843788147,
2193
+ "learning_rate": 0.0002323515565232743,
2194
+ "loss": 2.8312,
2195
+ "step": 293000
2196
+ },
2197
+ {
2198
+ "epoch": 15.81,
2199
+ "grad_norm": 0.27850472927093506,
2200
+ "learning_rate": 0.00022941211086918142,
2201
+ "loss": 2.8346,
2202
+ "step": 294000
2203
+ },
2204
+ {
2205
+ "epoch": 15.87,
2206
+ "grad_norm": 0.2848072052001953,
2207
+ "learning_rate": 0.00022646972282704646,
2208
+ "loss": 2.8291,
2209
+ "step": 295000
2210
+ },
2211
+ {
2212
+ "epoch": 15.92,
2213
+ "grad_norm": 0.3154236674308777,
2214
+ "learning_rate": 0.00022353027717295358,
2215
+ "loss": 2.8338,
2216
+ "step": 296000
2217
+ },
2218
+ {
2219
+ "epoch": 15.97,
2220
+ "grad_norm": 0.2986476719379425,
2221
+ "learning_rate": 0.00022058788913081857,
2222
+ "loss": 2.8335,
2223
+ "step": 297000
2224
+ },
2225
+ {
2226
+ "epoch": 16.0,
2227
+ "eval_accuracy": 0.41131288645738256,
2228
+ "eval_loss": 3.3726773262023926,
2229
+ "eval_runtime": 153.5545,
2230
+ "eval_samples_per_second": 377.201,
2231
+ "eval_steps_per_second": 5.9,
2232
+ "step": 297488
2233
+ },
2234
+ {
2235
+ "epoch": 16.03,
2236
+ "grad_norm": 0.31234875321388245,
2237
+ "learning_rate": 0.00021764844347672572,
2238
+ "loss": 2.8076,
2239
+ "step": 298000
2240
+ },
2241
+ {
2242
+ "epoch": 16.08,
2243
+ "grad_norm": 0.30626383423805237,
2244
+ "learning_rate": 0.0002147060554345907,
2245
+ "loss": 2.7883,
2246
+ "step": 299000
2247
+ },
2248
+ {
2249
+ "epoch": 16.14,
2250
+ "grad_norm": 0.2825140655040741,
2251
+ "learning_rate": 0.00021176366739245574,
2252
+ "loss": 2.7947,
2253
+ "step": 300000
2254
+ },
2255
+ {
2256
+ "epoch": 16.19,
2257
+ "grad_norm": 0.28852617740631104,
2258
+ "learning_rate": 0.00020882422173836286,
2259
+ "loss": 2.795,
2260
+ "step": 301000
2261
+ },
2262
+ {
2263
+ "epoch": 16.24,
2264
+ "grad_norm": 0.31543779373168945,
2265
+ "learning_rate": 0.00020588477608427,
2266
+ "loss": 2.798,
2267
+ "step": 302000
2268
+ },
2269
+ {
2270
+ "epoch": 16.3,
2271
+ "grad_norm": 0.2947680652141571,
2272
+ "learning_rate": 0.000202942388042135,
2273
+ "loss": 2.8025,
2274
+ "step": 303000
2275
+ },
2276
+ {
2277
+ "epoch": 16.35,
2278
+ "grad_norm": 0.291638046503067,
2279
+ "learning_rate": 0.0002,
2280
+ "loss": 2.8015,
2281
+ "step": 304000
2282
+ },
2283
+ {
2284
+ "epoch": 16.4,
2285
+ "grad_norm": 0.29190531373023987,
2286
+ "learning_rate": 0.00019705761195786502,
2287
+ "loss": 2.8063,
2288
+ "step": 305000
2289
+ },
2290
+ {
2291
+ "epoch": 16.46,
2292
+ "grad_norm": 0.3504582643508911,
2293
+ "learning_rate": 0.00019411522391573,
2294
+ "loss": 2.8084,
2295
+ "step": 306000
2296
+ },
2297
+ {
2298
+ "epoch": 16.51,
2299
+ "grad_norm": 0.2807118892669678,
2300
+ "learning_rate": 0.000191172835873595,
2301
+ "loss": 2.8028,
2302
+ "step": 307000
2303
+ },
2304
+ {
2305
+ "epoch": 16.57,
2306
+ "grad_norm": 0.29211023449897766,
2307
+ "learning_rate": 0.00018823339021950214,
2308
+ "loss": 2.8073,
2309
+ "step": 308000
2310
+ },
2311
+ {
2312
+ "epoch": 16.62,
2313
+ "grad_norm": 0.28356799483299255,
2314
+ "learning_rate": 0.0001852939445654093,
2315
+ "loss": 2.8114,
2316
+ "step": 309000
2317
+ },
2318
+ {
2319
+ "epoch": 16.67,
2320
+ "grad_norm": 0.32647237181663513,
2321
+ "learning_rate": 0.0001823515565232743,
2322
+ "loss": 2.809,
2323
+ "step": 310000
2324
+ },
2325
+ {
2326
+ "epoch": 16.73,
2327
+ "grad_norm": 0.3036649227142334,
2328
+ "learning_rate": 0.0001794091684811393,
2329
+ "loss": 2.8088,
2330
+ "step": 311000
2331
+ },
2332
+ {
2333
+ "epoch": 16.78,
2334
+ "grad_norm": 0.3010052740573883,
2335
+ "learning_rate": 0.0001764667804390043,
2336
+ "loss": 2.8145,
2337
+ "step": 312000
2338
+ },
2339
+ {
2340
+ "epoch": 16.83,
2341
+ "grad_norm": 0.2830561101436615,
2342
+ "learning_rate": 0.00017352733478491144,
2343
+ "loss": 2.8093,
2344
+ "step": 313000
2345
+ },
2346
+ {
2347
+ "epoch": 16.89,
2348
+ "grad_norm": 0.3039667308330536,
2349
+ "learning_rate": 0.00017058494674277645,
2350
+ "loss": 2.8143,
2351
+ "step": 314000
2352
+ },
2353
+ {
2354
+ "epoch": 16.94,
2355
+ "grad_norm": 0.3185649514198303,
2356
+ "learning_rate": 0.00016764255870064143,
2357
+ "loss": 2.8144,
2358
+ "step": 315000
2359
+ },
2360
+ {
2361
+ "epoch": 17.0,
2362
+ "grad_norm": 0.2941153049468994,
2363
+ "learning_rate": 0.00016470017065850644,
2364
+ "loss": 2.812,
2365
+ "step": 316000
2366
+ },
2367
+ {
2368
+ "epoch": 17.0,
2369
+ "eval_accuracy": 0.41104343316943776,
2370
+ "eval_loss": 3.3814456462860107,
2371
+ "eval_runtime": 153.7991,
2372
+ "eval_samples_per_second": 376.602,
2373
+ "eval_steps_per_second": 5.891,
2374
+ "step": 316081
2375
+ },
2376
+ {
2377
+ "epoch": 17.05,
2378
+ "grad_norm": 0.30029386281967163,
2379
+ "learning_rate": 0.00016176366739245572,
2380
+ "loss": 2.7825,
2381
+ "step": 317000
2382
+ },
2383
+ {
2384
+ "epoch": 17.1,
2385
+ "grad_norm": 0.2988363206386566,
2386
+ "learning_rate": 0.00015882127935032072,
2387
+ "loss": 2.7782,
2388
+ "step": 318000
2389
+ },
2390
+ {
2391
+ "epoch": 17.16,
2392
+ "grad_norm": 0.31658461689949036,
2393
+ "learning_rate": 0.00015588183369622785,
2394
+ "loss": 2.7813,
2395
+ "step": 319000
2396
+ },
2397
+ {
2398
+ "epoch": 17.21,
2399
+ "grad_norm": 0.3059552311897278,
2400
+ "learning_rate": 0.0001529394456540929,
2401
+ "loss": 2.7835,
2402
+ "step": 320000
2403
+ },
2404
+ {
2405
+ "epoch": 17.26,
2406
+ "grad_norm": 0.30100756883621216,
2407
+ "learning_rate": 0.00014999705761195787,
2408
+ "loss": 2.7827,
2409
+ "step": 321000
2410
+ },
2411
+ {
2412
+ "epoch": 17.32,
2413
+ "grad_norm": 0.30339285731315613,
2414
+ "learning_rate": 0.00014705466956982285,
2415
+ "loss": 2.7887,
2416
+ "step": 322000
2417
+ },
2418
+ {
2419
+ "epoch": 17.37,
2420
+ "grad_norm": 0.3006446957588196,
2421
+ "learning_rate": 0.00014411522391573,
2422
+ "loss": 2.7872,
2423
+ "step": 323000
2424
+ },
2425
+ {
2426
+ "epoch": 17.43,
2427
+ "grad_norm": 0.28081661462783813,
2428
+ "learning_rate": 0.000141172835873595,
2429
+ "loss": 2.7869,
2430
+ "step": 324000
2431
+ },
2432
+ {
2433
+ "epoch": 17.48,
2434
+ "grad_norm": 0.28526371717453003,
2435
+ "learning_rate": 0.00013823339021950214,
2436
+ "loss": 2.7881,
2437
+ "step": 325000
2438
+ },
2439
+ {
2440
+ "epoch": 17.53,
2441
+ "grad_norm": 0.31043946743011475,
2442
+ "learning_rate": 0.00013529100217736715,
2443
+ "loss": 2.7894,
2444
+ "step": 326000
2445
+ },
2446
+ {
2447
+ "epoch": 17.59,
2448
+ "grad_norm": 0.3015703558921814,
2449
+ "learning_rate": 0.00013234861413523216,
2450
+ "loss": 2.79,
2451
+ "step": 327000
2452
+ },
2453
+ {
2454
+ "epoch": 17.64,
2455
+ "grad_norm": 0.29077813029289246,
2456
+ "learning_rate": 0.00012940916848113928,
2457
+ "loss": 2.7921,
2458
+ "step": 328000
2459
+ },
2460
+ {
2461
+ "epoch": 17.69,
2462
+ "grad_norm": 0.29008862376213074,
2463
+ "learning_rate": 0.00012646972282704644,
2464
+ "loss": 2.7921,
2465
+ "step": 329000
2466
+ },
2467
+ {
2468
+ "epoch": 17.75,
2469
+ "grad_norm": 0.29531145095825195,
2470
+ "learning_rate": 0.00012352733478491142,
2471
+ "loss": 2.7908,
2472
+ "step": 330000
2473
+ },
2474
+ {
2475
+ "epoch": 17.8,
2476
+ "grad_norm": 0.3173753619194031,
2477
+ "learning_rate": 0.00012058494674277644,
2478
+ "loss": 2.7954,
2479
+ "step": 331000
2480
+ },
2481
+ {
2482
+ "epoch": 17.86,
2483
+ "grad_norm": 0.28667834401130676,
2484
+ "learning_rate": 0.00011764550108868357,
2485
+ "loss": 2.791,
2486
+ "step": 332000
2487
+ },
2488
+ {
2489
+ "epoch": 17.91,
2490
+ "grad_norm": 0.3025910556316376,
2491
+ "learning_rate": 0.00011470311304654858,
2492
+ "loss": 2.7956,
2493
+ "step": 333000
2494
+ },
2495
+ {
2496
+ "epoch": 17.96,
2497
+ "grad_norm": 0.32427656650543213,
2498
+ "learning_rate": 0.00011176366739245572,
2499
+ "loss": 2.7944,
2500
+ "step": 334000
2501
+ },
2502
+ {
2503
+ "epoch": 18.0,
2504
+ "eval_accuracy": 0.4119177623997049,
2505
+ "eval_loss": 3.3788328170776367,
2506
+ "eval_runtime": 153.909,
2507
+ "eval_samples_per_second": 376.333,
2508
+ "eval_steps_per_second": 5.887,
2509
+ "step": 334674
2510
+ },
2511
+ {
2512
+ "epoch": 18.02,
2513
+ "grad_norm": 0.28937357664108276,
2514
+ "learning_rate": 0.00010882127935032073,
2515
+ "loss": 2.785,
2516
+ "step": 335000
2517
+ },
2518
+ {
2519
+ "epoch": 18.07,
2520
+ "grad_norm": 0.3173358738422394,
2521
+ "learning_rate": 0.00010588183369622787,
2522
+ "loss": 2.7681,
2523
+ "step": 336000
2524
+ },
2525
+ {
2526
+ "epoch": 18.13,
2527
+ "grad_norm": 0.3219173550605774,
2528
+ "learning_rate": 0.00010293944565409286,
2529
+ "loss": 2.7649,
2530
+ "step": 337000
2531
+ },
2532
+ {
2533
+ "epoch": 18.18,
2534
+ "grad_norm": 0.30825838446617126,
2535
+ "learning_rate": 0.0001,
2536
+ "loss": 2.7678,
2537
+ "step": 338000
2538
+ },
2539
+ {
2540
+ "epoch": 18.23,
2541
+ "grad_norm": 0.2984776198863983,
2542
+ "learning_rate": 9.7057611957865e-05,
2543
+ "loss": 2.7691,
2544
+ "step": 339000
2545
+ },
2546
+ {
2547
+ "epoch": 18.29,
2548
+ "grad_norm": 0.3033863604068756,
2549
+ "learning_rate": 9.411522391573001e-05,
2550
+ "loss": 2.7718,
2551
+ "step": 340000
2552
+ },
2553
+ {
2554
+ "epoch": 18.34,
2555
+ "grad_norm": 0.298531711101532,
2556
+ "learning_rate": 9.117283587359502e-05,
2557
+ "loss": 2.7688,
2558
+ "step": 341000
2559
+ },
2560
+ {
2561
+ "epoch": 18.39,
2562
+ "grad_norm": 0.37657690048217773,
2563
+ "learning_rate": 8.823339021950214e-05,
2564
+ "loss": 2.7737,
2565
+ "step": 342000
2566
+ },
2567
+ {
2568
+ "epoch": 18.45,
2569
+ "grad_norm": 0.30152538418769836,
2570
+ "learning_rate": 8.529394456540929e-05,
2571
+ "loss": 2.7705,
2572
+ "step": 343000
2573
+ },
2574
+ {
2575
+ "epoch": 18.5,
2576
+ "grad_norm": 0.2887534499168396,
2577
+ "learning_rate": 8.23515565232743e-05,
2578
+ "loss": 2.7716,
2579
+ "step": 344000
2580
+ },
2581
+ {
2582
+ "epoch": 18.56,
2583
+ "grad_norm": 0.31988152861595154,
2584
+ "learning_rate": 7.94091684811393e-05,
2585
+ "loss": 2.7738,
2586
+ "step": 345000
2587
+ },
2588
+ {
2589
+ "epoch": 18.61,
2590
+ "grad_norm": 0.2952527105808258,
2591
+ "learning_rate": 7.646972282704644e-05,
2592
+ "loss": 2.7732,
2593
+ "step": 346000
2594
+ },
2595
+ {
2596
+ "epoch": 18.66,
2597
+ "grad_norm": 0.318419873714447,
2598
+ "learning_rate": 7.352733478491143e-05,
2599
+ "loss": 2.7787,
2600
+ "step": 347000
2601
+ },
2602
+ {
2603
+ "epoch": 18.72,
2604
+ "grad_norm": 0.290344774723053,
2605
+ "learning_rate": 7.058494674277643e-05,
2606
+ "loss": 2.7724,
2607
+ "step": 348000
2608
+ },
2609
+ {
2610
+ "epoch": 18.77,
2611
+ "grad_norm": 0.34806349873542786,
2612
+ "learning_rate": 6.764550108868357e-05,
2613
+ "loss": 2.7785,
2614
+ "step": 349000
2615
+ },
2616
+ {
2617
+ "epoch": 18.82,
2618
+ "grad_norm": 0.27911612391471863,
2619
+ "learning_rate": 6.470311304654858e-05,
2620
+ "loss": 2.7754,
2621
+ "step": 350000
2622
+ },
2623
+ {
2624
+ "epoch": 18.88,
2625
+ "grad_norm": 0.36661508679389954,
2626
+ "learning_rate": 6.176072500441358e-05,
2627
+ "loss": 2.7737,
2628
+ "step": 351000
2629
+ },
2630
+ {
2631
+ "epoch": 18.93,
2632
+ "grad_norm": 0.2794508635997772,
2633
+ "learning_rate": 5.881833696227858e-05,
2634
+ "loss": 2.7796,
2635
+ "step": 352000
2636
+ },
2637
+ {
2638
+ "epoch": 18.99,
2639
+ "grad_norm": 0.3130718171596527,
2640
+ "learning_rate": 5.587889130818573e-05,
2641
+ "loss": 2.7761,
2642
+ "step": 353000
2643
+ },
2644
+ {
2645
+ "epoch": 19.0,
2646
+ "eval_accuracy": 0.4113871859678455,
2647
+ "eval_loss": 3.3925299644470215,
2648
+ "eval_runtime": 155.9595,
2649
+ "eval_samples_per_second": 371.385,
2650
+ "eval_steps_per_second": 5.809,
2651
+ "step": 353267
2652
+ },
2653
+ {
2654
+ "epoch": 19.04,
2655
+ "grad_norm": 0.2997041940689087,
2656
+ "learning_rate": 5.293650326605073e-05,
2657
+ "loss": 2.7625,
2658
+ "step": 354000
2659
+ },
2660
+ {
2661
+ "epoch": 19.09,
2662
+ "grad_norm": 0.31620392203330994,
2663
+ "learning_rate": 4.999411522391573e-05,
2664
+ "loss": 2.7545,
2665
+ "step": 355000
2666
+ },
2667
+ {
2668
+ "epoch": 19.15,
2669
+ "grad_norm": 0.3096230626106262,
2670
+ "learning_rate": 4.7051727181780734e-05,
2671
+ "loss": 2.7559,
2672
+ "step": 356000
2673
+ },
2674
+ {
2675
+ "epoch": 19.2,
2676
+ "grad_norm": 0.29621490836143494,
2677
+ "learning_rate": 4.4115223915730006e-05,
2678
+ "loss": 2.7603,
2679
+ "step": 357000
2680
+ },
2681
+ {
2682
+ "epoch": 19.25,
2683
+ "grad_norm": 0.3432357609272003,
2684
+ "learning_rate": 4.117283587359501e-05,
2685
+ "loss": 2.7593,
2686
+ "step": 358000
2687
+ },
2688
+ {
2689
+ "epoch": 19.31,
2690
+ "grad_norm": 0.32921677827835083,
2691
+ "learning_rate": 3.823044783146001e-05,
2692
+ "loss": 2.7558,
2693
+ "step": 359000
2694
+ },
2695
+ {
2696
+ "epoch": 19.36,
2697
+ "grad_norm": 0.3113718032836914,
2698
+ "learning_rate": 3.528805978932502e-05,
2699
+ "loss": 2.7584,
2700
+ "step": 360000
2701
+ },
2702
+ {
2703
+ "epoch": 19.42,
2704
+ "grad_norm": 0.29590970277786255,
2705
+ "learning_rate": 3.234861413523216e-05,
2706
+ "loss": 2.7579,
2707
+ "step": 361000
2708
+ },
2709
+ {
2710
+ "epoch": 19.47,
2711
+ "grad_norm": 0.3046259582042694,
2712
+ "learning_rate": 2.940622609309716e-05,
2713
+ "loss": 2.757,
2714
+ "step": 362000
2715
+ },
2716
+ {
2717
+ "epoch": 19.52,
2718
+ "grad_norm": 0.31513723731040955,
2719
+ "learning_rate": 2.64667804390043e-05,
2720
+ "loss": 2.7601,
2721
+ "step": 363000
2722
+ },
2723
+ {
2724
+ "epoch": 19.58,
2725
+ "grad_norm": 0.31137752532958984,
2726
+ "learning_rate": 2.35243923968693e-05,
2727
+ "loss": 2.7618,
2728
+ "step": 364000
2729
+ },
2730
+ {
2731
+ "epoch": 19.63,
2732
+ "grad_norm": 0.3485892117023468,
2733
+ "learning_rate": 2.0582004354734303e-05,
2734
+ "loss": 2.7603,
2735
+ "step": 365000
2736
+ },
2737
+ {
2738
+ "epoch": 19.68,
2739
+ "grad_norm": 0.3177970051765442,
2740
+ "learning_rate": 1.7639616312599305e-05,
2741
+ "loss": 2.7573,
2742
+ "step": 366000
2743
+ },
2744
+ {
2745
+ "epoch": 19.74,
2746
+ "grad_norm": 0.3040596842765808,
2747
+ "learning_rate": 1.4700170658506444e-05,
2748
+ "loss": 2.7569,
2749
+ "step": 367000
2750
+ },
2751
+ {
2752
+ "epoch": 19.79,
2753
+ "grad_norm": 0.31309446692466736,
2754
+ "learning_rate": 1.1757782616371448e-05,
2755
+ "loss": 2.7592,
2756
+ "step": 368000
2757
+ },
2758
+ {
2759
+ "epoch": 19.85,
2760
+ "grad_norm": 0.3122311234474182,
2761
+ "learning_rate": 8.815394574236451e-06,
2762
+ "loss": 2.761,
2763
+ "step": 369000
2764
+ },
2765
+ {
2766
+ "epoch": 19.9,
2767
+ "grad_norm": 0.30037108063697815,
2768
+ "learning_rate": 5.8759489201435885e-06,
2769
+ "loss": 2.7553,
2770
+ "step": 370000
2771
+ },
2772
+ {
2773
+ "epoch": 19.95,
2774
+ "grad_norm": 0.33223336935043335,
2775
+ "learning_rate": 2.933560878008592e-06,
2776
+ "loss": 2.7606,
2777
+ "step": 371000
2778
+ },
2779
+ {
2780
+ "epoch": 20.0,
2781
+ "eval_accuracy": 0.41125847180686265,
2782
+ "eval_loss": 3.397514581680298,
2783
+ "eval_runtime": 153.9424,
2784
+ "eval_samples_per_second": 376.251,
2785
+ "eval_steps_per_second": 5.885,
2786
+ "step": 371860
2787
+ },
2788
+ {
2789
+ "epoch": 20.0,
2790
+ "step": 371860,
2791
+ "total_flos": 1.56674405385216e+18,
2792
+ "train_loss": 3.027371880730273,
2793
+ "train_runtime": 81125.0441,
2794
+ "train_samples_per_second": 146.679,
2795
+ "train_steps_per_second": 4.584
2796
+ }
2797
+ ],
2798
+ "logging_steps": 1000,
2799
+ "max_steps": 371860,
2800
+ "num_input_tokens_seen": 0,
2801
+ "num_train_epochs": 20,
2802
+ "save_steps": 5000,
2803
+ "total_flos": 1.56674405385216e+18,
2804
+ "train_batch_size": 32,
2805
+ "trial_name": null,
2806
+ "trial_params": null
2807
+ }