Hugging Face's logo

SeyedAli
/
Melanoma-Classification

Image Classification
Transformers
TensorBoard
Safetensors
vit
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
SeyedAli commited on
Commit
f99dc78
•
1 Parent(s): 7a96dce

SeyedAli/Melanoma-Classification

Browse files
Files changed (12) hide show
  1. README.md +110 -0
  2. all_results.json +13 -0
  3. config.json +44 -0
  4. eval_results.json +8 -0
  5. model.safetensors +3 -0
  6. preprocessor_config.json +22 -0
  7. runs/Mar02_08-08-31_953686467e09/events.out.tfevents.1709366926.953686467e09.362.0 +3 -0
  8. runs/Mar02_08-32-26_953686467e09/events.out.tfevents.1709368351.953686467e09.362.1 +3 -0
  9. runs/Mar02_08-32-26_953686467e09/events.out.tfevents.1709377020.953686467e09.362.2 +3 -0
  10. train_results.json +8 -0
  11. trainer_state.json +4022 -0
  12. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: google/vit-base-patch16-224-in21k
4
+ tags:
5
+ - image-classification
6
+ - generated_from_trainer
7
+ metrics:
8
+ - accuracy
9
+ model-index:
10
+ - name: Melanoma-Classification
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # Melanoma-Classification
18
+
19
+ This model is a fine-tuned version of [google/vit-base-patch16-224-in21k](https://huggingface.co/google/vit-base-patch16-224-in21k) on the SeyedAli/Skin-Lesion-Dataset dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.5750
22
+ - Accuracy: 0.8167
23
+
24
+ ## Model description
25
+
26
+ More information needed
27
+
28
+ ## Intended uses & limitations
29
+
30
+ More information needed
31
+
32
+ ## Training and evaluation data
33
+
34
+ More information needed
35
+
36
+ ## Training procedure
37
+
38
+ ### Training hyperparameters
39
+
40
+ The following hyperparameters were used during training:
41
+ - learning_rate: 0.0002
42
+ - train_batch_size: 16
43
+ - eval_batch_size: 8
44
+ - seed: 42
45
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
+ - lr_scheduler_type: linear
47
+ - num_epochs: 4
48
+
49
+ ### Training results
50
+
51
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
52
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|
53
+ | 0.9779 | 0.08 | 100 | 1.1158 | 0.6041 |
54
+ | 0.9934 | 0.16 | 200 | 1.0227 | 0.6501 |
55
+ | 0.9562 | 0.24 | 300 | 0.9276 | 0.6748 |
56
+ | 1.0995 | 0.32 | 400 | 0.9088 | 0.6836 |
57
+ | 0.8198 | 0.39 | 500 | 0.8581 | 0.6949 |
58
+ | 0.8034 | 0.47 | 600 | 0.8444 | 0.6967 |
59
+ | 0.8319 | 0.55 | 700 | 0.8196 | 0.7148 |
60
+ | 0.787 | 0.63 | 800 | 0.8360 | 0.6975 |
61
+ | 0.8642 | 0.71 | 900 | 0.8250 | 0.7008 |
62
+ | 0.8329 | 0.79 | 1000 | 0.7939 | 0.7172 |
63
+ | 0.9678 | 0.87 | 1100 | 0.7661 | 0.7332 |
64
+ | 0.8226 | 0.95 | 1200 | 0.7284 | 0.7373 |
65
+ | 0.7967 | 1.03 | 1300 | 0.7355 | 0.7411 |
66
+ | 0.6531 | 1.1 | 1400 | 0.7561 | 0.7247 |
67
+ | 0.5719 | 1.18 | 1500 | 0.6839 | 0.7638 |
68
+ | 0.6123 | 1.26 | 1600 | 0.6857 | 0.7584 |
69
+ | 0.6504 | 1.34 | 1700 | 0.6970 | 0.7531 |
70
+ | 0.6214 | 1.42 | 1800 | 0.6841 | 0.7576 |
71
+ | 0.4925 | 1.5 | 1900 | 0.6624 | 0.7642 |
72
+ | 0.5797 | 1.58 | 2000 | 0.6287 | 0.7709 |
73
+ | 0.6018 | 1.66 | 2100 | 0.6537 | 0.7622 |
74
+ | 0.6334 | 1.74 | 2200 | 0.6413 | 0.7713 |
75
+ | 0.4111 | 1.82 | 2300 | 0.6242 | 0.7786 |
76
+ | 0.4779 | 1.89 | 2400 | 0.6260 | 0.7790 |
77
+ | 0.5488 | 1.97 | 2500 | 0.6146 | 0.7807 |
78
+ | 0.3212 | 2.05 | 2600 | 0.6975 | 0.7707 |
79
+ | 0.4282 | 2.13 | 2700 | 0.6344 | 0.7790 |
80
+ | 0.2822 | 2.21 | 2800 | 0.6985 | 0.7845 |
81
+ | 0.3003 | 2.29 | 2900 | 0.5954 | 0.7993 |
82
+ | 0.2982 | 2.37 | 3000 | 0.6156 | 0.7940 |
83
+ | 0.2628 | 2.45 | 3100 | 0.6318 | 0.7963 |
84
+ | 0.2987 | 2.53 | 3200 | 0.6495 | 0.8030 |
85
+ | 0.2714 | 2.6 | 3300 | 0.6018 | 0.8052 |
86
+ | 0.3059 | 2.68 | 3400 | 0.5944 | 0.8078 |
87
+ | 0.2762 | 2.76 | 3500 | 0.6296 | 0.7936 |
88
+ | 0.3685 | 2.84 | 3600 | 0.6277 | 0.8017 |
89
+ | 0.2299 | 2.92 | 3700 | 0.5834 | 0.8125 |
90
+ | 0.3414 | 3.0 | 3800 | 0.5750 | 0.8167 |
91
+ | 0.1082 | 3.08 | 3900 | 0.6201 | 0.8196 |
92
+ | 0.049 | 3.16 | 4000 | 0.6475 | 0.8161 |
93
+ | 0.102 | 3.24 | 4100 | 0.6791 | 0.8097 |
94
+ | 0.0483 | 3.31 | 4200 | 0.6582 | 0.8216 |
95
+ | 0.1204 | 3.39 | 4300 | 0.6603 | 0.8222 |
96
+ | 0.0611 | 3.47 | 4400 | 0.7174 | 0.8190 |
97
+ | 0.0555 | 3.55 | 4500 | 0.6841 | 0.8236 |
98
+ | 0.0188 | 3.63 | 4600 | 0.7009 | 0.8240 |
99
+ | 0.1292 | 3.71 | 4700 | 0.7040 | 0.8204 |
100
+ | 0.0661 | 3.79 | 4800 | 0.7074 | 0.8238 |
101
+ | 0.1061 | 3.87 | 4900 | 0.6984 | 0.8210 |
102
+ | 0.0861 | 3.95 | 5000 | 0.6913 | 0.8230 |
103
+
104
+
105
+ ### Framework versions
106
+
107
+ - Transformers 4.38.1
108
+ - Pytorch 2.1.0+cu121
109
+ - Datasets 2.18.0
110
+ - Tokenizers 0.15.2
all_results.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 4.0,
3
+ "eval_accuracy": 0.8166567988948096,
4
+ "eval_loss": 0.5749832987785339,
5
+ "eval_runtime": 91.2537,
6
+ "eval_samples_per_second": 55.527,
7
+ "eval_steps_per_second": 6.948,
8
+ "total_flos": 6.281528488153842e+18,
9
+ "train_loss": 0.4664627312864129,
10
+ "train_runtime": 8576.6293,
11
+ "train_samples_per_second": 9.451,
12
+ "train_steps_per_second": 0.591
13
+ }
config.json ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "google/vit-base-patch16-224-in21k",
3
+ "architectures": [
4
+ "ViTForImageClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.0,
7
+ "encoder_stride": 16,
8
+ "hidden_act": "gelu",
9
+ "hidden_dropout_prob": 0.0,
10
+ "hidden_size": 768,
11
+ "id2label": {
12
+ "0": "Melanoma",
13
+ "1": "Melanocytic nevus",
14
+ "2": "Basal cell carcinoma",
15
+ "3": "Actinic keratosis",
16
+ "4": "Benign keratosis",
17
+ "5": "Dermatofibroma",
18
+ "6": "Vascular lesion",
19
+ "7": "Squamous cell carcinoma"
20
+ },
21
+ "image_size": 224,
22
+ "initializer_range": 0.02,
23
+ "intermediate_size": 3072,
24
+ "label2id": {
25
+ "Actinic keratosis": "3",
26
+ "Basal cell carcinoma": "2",
27
+ "Benign keratosis": "4",
28
+ "Dermatofibroma": "5",
29
+ "Melanocytic nevus": "1",
30
+ "Melanoma": "0",
31
+ "Squamous cell carcinoma": "7",
32
+ "Vascular lesion": "6"
33
+ },
34
+ "layer_norm_eps": 1e-12,
35
+ "model_type": "vit",
36
+ "num_attention_heads": 12,
37
+ "num_channels": 3,
38
+ "num_hidden_layers": 12,
39
+ "patch_size": 16,
40
+ "problem_type": "single_label_classification",
41
+ "qkv_bias": true,
42
+ "torch_dtype": "float32",
43
+ "transformers_version": "4.38.1"
44
+ }
eval_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 4.0,
3
+ "eval_accuracy": 0.8166567988948096,
4
+ "eval_loss": 0.5749832987785339,
5
+ "eval_runtime": 91.2537,
6
+ "eval_samples_per_second": 55.527,
7
+ "eval_steps_per_second": 6.948
8
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99c65ff5a90b014b9a231f553d92e19054c7d9902fef90b630e378a85af44724
3
+ size 343242432
preprocessor_config.json ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_normalize": true,
3
+ "do_rescale": true,
4
+ "do_resize": true,
5
+ "image_mean": [
6
+ 0.5,
7
+ 0.5,
8
+ 0.5
9
+ ],
10
+ "image_processor_type": "ViTImageProcessor",
11
+ "image_std": [
12
+ 0.5,
13
+ 0.5,
14
+ 0.5
15
+ ],
16
+ "resample": 2,
17
+ "rescale_factor": 0.00392156862745098,
18
+ "size": {
19
+ "height": 224,
20
+ "width": 224
21
+ }
22
+ }
runs/Mar02_08-08-31_953686467e09/events.out.tfevents.1709366926.953686467e09.362.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e64892e5fe1bce7c7c0814197b167771bc03ec0a0fa9c7816932696b97232e0
3
+ size 9858
runs/Mar02_08-32-26_953686467e09/events.out.tfevents.1709368351.953686467e09.362.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44b4c7ee327d3d880e3c5fe96eeb54f624fd030e2d59dfc370020ec6c7676c90
3
+ size 128189
runs/Mar02_08-32-26_953686467e09/events.out.tfevents.1709377020.953686467e09.362.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f53fec4c510bed316d741c889d5d5370267cd89d974ca046647b52bdb8c7b90
3
+ size 411
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 4.0,
3
+ "total_flos": 6.281528488153842e+18,
4
+ "train_loss": 0.4664627312864129,
5
+ "train_runtime": 8576.6293,
6
+ "train_samples_per_second": 9.451,
7
+ "train_steps_per_second": 0.591
8
+ }
trainer_state.json ADDED
@@ -0,0 +1,4022 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.5749832987785339,
3
+ "best_model_checkpoint": "./Melanoma-Classification/checkpoint-3800",
4
+ "epoch": 4.0,
5
+ "eval_steps": 100,
6
+ "global_step": 5068,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.01,
13
+ "grad_norm": 1.4924156665802002,
14
+ "learning_rate": 0.00019960536700868193,
15
+ "loss": 1.5595,
16
+ "step": 10
17
+ },
18
+ {
19
+ "epoch": 0.02,
20
+ "grad_norm": 1.687912106513977,
21
+ "learning_rate": 0.00019921073401736386,
22
+ "loss": 1.3491,
23
+ "step": 20
24
+ },
25
+ {
26
+ "epoch": 0.02,
27
+ "grad_norm": 2.1836044788360596,
28
+ "learning_rate": 0.0001988161010260458,
29
+ "loss": 1.1583,
30
+ "step": 30
31
+ },
32
+ {
33
+ "epoch": 0.03,
34
+ "grad_norm": 2.7617909908294678,
35
+ "learning_rate": 0.00019842146803472773,
36
+ "loss": 1.4836,
37
+ "step": 40
38
+ },
39
+ {
40
+ "epoch": 0.04,
41
+ "grad_norm": 1.7186870574951172,
42
+ "learning_rate": 0.00019802683504340965,
43
+ "loss": 1.1296,
44
+ "step": 50
45
+ },
46
+ {
47
+ "epoch": 0.05,
48
+ "grad_norm": 1.4756615161895752,
49
+ "learning_rate": 0.00019763220205209155,
50
+ "loss": 1.0864,
51
+ "step": 60
52
+ },
53
+ {
54
+ "epoch": 0.06,
55
+ "grad_norm": 2.723090648651123,
56
+ "learning_rate": 0.00019723756906077347,
57
+ "loss": 1.1339,
58
+ "step": 70
59
+ },
60
+ {
61
+ "epoch": 0.06,
62
+ "grad_norm": 1.304826021194458,
63
+ "learning_rate": 0.0001968429360694554,
64
+ "loss": 1.0937,
65
+ "step": 80
66
+ },
67
+ {
68
+ "epoch": 0.07,
69
+ "grad_norm": 1.5404345989227295,
70
+ "learning_rate": 0.00019644830307813735,
71
+ "loss": 1.0765,
72
+ "step": 90
73
+ },
74
+ {
75
+ "epoch": 0.08,
76
+ "grad_norm": 2.1473004817962646,
77
+ "learning_rate": 0.00019605367008681927,
78
+ "loss": 0.9779,
79
+ "step": 100
80
+ },
81
+ {
82
+ "epoch": 0.08,
83
+ "eval_accuracy": 0.6041049930925597,
84
+ "eval_loss": 1.1158318519592285,
85
+ "eval_runtime": 91.4624,
86
+ "eval_samples_per_second": 55.4,
87
+ "eval_steps_per_second": 6.932,
88
+ "step": 100
89
+ },
90
+ {
91
+ "epoch": 0.09,
92
+ "grad_norm": 2.886522054672241,
93
+ "learning_rate": 0.0001956590370955012,
94
+ "loss": 1.0835,
95
+ "step": 110
96
+ },
97
+ {
98
+ "epoch": 0.09,
99
+ "grad_norm": 2.163769006729126,
100
+ "learning_rate": 0.00019526440410418312,
101
+ "loss": 1.1449,
102
+ "step": 120
103
+ },
104
+ {
105
+ "epoch": 0.1,
106
+ "grad_norm": 1.7149088382720947,
107
+ "learning_rate": 0.00019486977111286504,
108
+ "loss": 1.115,
109
+ "step": 130
110
+ },
111
+ {
112
+ "epoch": 0.11,
113
+ "grad_norm": 2.6274876594543457,
114
+ "learning_rate": 0.00019447513812154697,
115
+ "loss": 1.1516,
116
+ "step": 140
117
+ },
118
+ {
119
+ "epoch": 0.12,
120
+ "grad_norm": 1.5108281373977661,
121
+ "learning_rate": 0.00019408050513022892,
122
+ "loss": 1.0328,
123
+ "step": 150
124
+ },
125
+ {
126
+ "epoch": 0.13,
127
+ "grad_norm": 2.575366497039795,
128
+ "learning_rate": 0.00019368587213891084,
129
+ "loss": 1.0288,
130
+ "step": 160
131
+ },
132
+ {
133
+ "epoch": 0.13,
134
+ "grad_norm": 1.9967373609542847,
135
+ "learning_rate": 0.00019329123914759276,
136
+ "loss": 0.89,
137
+ "step": 170
138
+ },
139
+ {
140
+ "epoch": 0.14,
141
+ "grad_norm": 2.47580885887146,
142
+ "learning_rate": 0.00019289660615627466,
143
+ "loss": 1.0238,
144
+ "step": 180
145
+ },
146
+ {
147
+ "epoch": 0.15,
148
+ "grad_norm": 2.0836246013641357,
149
+ "learning_rate": 0.00019250197316495658,
150
+ "loss": 1.034,
151
+ "step": 190
152
+ },
153
+ {
154
+ "epoch": 0.16,
155
+ "grad_norm": 1.9168826341629028,
156
+ "learning_rate": 0.0001921073401736385,
157
+ "loss": 0.9934,
158
+ "step": 200
159
+ },
160
+ {
161
+ "epoch": 0.16,
162
+ "eval_accuracy": 0.650088809946714,
163
+ "eval_loss": 1.0227293968200684,
164
+ "eval_runtime": 91.1657,
165
+ "eval_samples_per_second": 55.58,
166
+ "eval_steps_per_second": 6.954,
167
+ "step": 200
168
+ },
169
+ {
170
+ "epoch": 0.17,
171
+ "grad_norm": 2.0433690547943115,
172
+ "learning_rate": 0.00019171270718232046,
173
+ "loss": 0.9561,
174
+ "step": 210
175
+ },
176
+ {
177
+ "epoch": 0.17,
178
+ "grad_norm": 1.9006619453430176,
179
+ "learning_rate": 0.00019131807419100238,
180
+ "loss": 1.1087,
181
+ "step": 220
182
+ },
183
+ {
184
+ "epoch": 0.18,
185
+ "grad_norm": 1.693882942199707,
186
+ "learning_rate": 0.0001909234411996843,
187
+ "loss": 1.0212,
188
+ "step": 230
189
+ },
190
+ {
191
+ "epoch": 0.19,
192
+ "grad_norm": 1.44376802444458,
193
+ "learning_rate": 0.00019052880820836623,
194
+ "loss": 0.9192,
195
+ "step": 240
196
+ },
197
+ {
198
+ "epoch": 0.2,
199
+ "grad_norm": 1.168645977973938,
200
+ "learning_rate": 0.00019013417521704815,
201
+ "loss": 0.9394,
202
+ "step": 250
203
+ },
204
+ {
205
+ "epoch": 0.21,
206
+ "grad_norm": 2.2017979621887207,
207
+ "learning_rate": 0.00018973954222573008,
208
+ "loss": 1.0321,
209
+ "step": 260
210
+ },
211
+ {
212
+ "epoch": 0.21,
213
+ "grad_norm": 2.930981397628784,
214
+ "learning_rate": 0.00018934490923441203,
215
+ "loss": 1.019,
216
+ "step": 270
217
+ },
218
+ {
219
+ "epoch": 0.22,
220
+ "grad_norm": 2.8777360916137695,
221
+ "learning_rate": 0.00018895027624309395,
222
+ "loss": 1.1093,
223
+ "step": 280
224
+ },
225
+ {
226
+ "epoch": 0.23,
227
+ "grad_norm": 2.2270009517669678,
228
+ "learning_rate": 0.00018855564325177585,
229
+ "loss": 0.9979,
230
+ "step": 290
231
+ },
232
+ {
233
+ "epoch": 0.24,
234
+ "grad_norm": 1.8015016317367554,
235
+ "learning_rate": 0.00018816101026045777,
236
+ "loss": 0.9562,
237
+ "step": 300
238
+ },
239
+ {
240
+ "epoch": 0.24,
241
+ "eval_accuracy": 0.6747582395895007,
242
+ "eval_loss": 0.9276195168495178,
243
+ "eval_runtime": 91.6369,
244
+ "eval_samples_per_second": 55.294,
245
+ "eval_steps_per_second": 6.919,
246
+ "step": 300
247
+ },
248
+ {
249
+ "epoch": 0.24,
250
+ "grad_norm": 2.6960203647613525,
251
+ "learning_rate": 0.0001877663772691397,
252
+ "loss": 0.95,
253
+ "step": 310
254
+ },
255
+ {
256
+ "epoch": 0.25,
257
+ "grad_norm": 1.5449939966201782,
258
+ "learning_rate": 0.00018737174427782162,
259
+ "loss": 0.9869,
260
+ "step": 320
261
+ },
262
+ {
263
+ "epoch": 0.26,
264
+ "grad_norm": 1.7589311599731445,
265
+ "learning_rate": 0.00018697711128650357,
266
+ "loss": 0.8735,
267
+ "step": 330
268
+ },
269
+ {
270
+ "epoch": 0.27,
271
+ "grad_norm": 1.1408846378326416,
272
+ "learning_rate": 0.0001865824782951855,
273
+ "loss": 0.8768,
274
+ "step": 340
275
+ },
276
+ {
277
+ "epoch": 0.28,
278
+ "grad_norm": 1.8114129304885864,
279
+ "learning_rate": 0.00018618784530386741,
280
+ "loss": 0.8912,
281
+ "step": 350
282
+ },
283
+ {
284
+ "epoch": 0.28,
285
+ "grad_norm": 2.1425697803497314,
286
+ "learning_rate": 0.00018579321231254934,
287
+ "loss": 0.9495,
288
+ "step": 360
289
+ },
290
+ {
291
+ "epoch": 0.29,
292
+ "grad_norm": 2.0084307193756104,
293
+ "learning_rate": 0.00018539857932123126,
294
+ "loss": 1.0388,
295
+ "step": 370
296
+ },
297
+ {
298
+ "epoch": 0.3,
299
+ "grad_norm": 1.8898825645446777,
300
+ "learning_rate": 0.00018500394632991319,
301
+ "loss": 0.9202,
302
+ "step": 380
303
+ },
304
+ {
305
+ "epoch": 0.31,
306
+ "grad_norm": 3.13079833984375,
307
+ "learning_rate": 0.00018460931333859514,
308
+ "loss": 0.9567,
309
+ "step": 390
310
+ },
311
+ {
312
+ "epoch": 0.32,
313
+ "grad_norm": 2.111664056777954,
314
+ "learning_rate": 0.00018421468034727706,
315
+ "loss": 1.0995,
316
+ "step": 400
317
+ },
318
+ {
319
+ "epoch": 0.32,
320
+ "eval_accuracy": 0.6836392342609039,
321
+ "eval_loss": 0.9088240265846252,
322
+ "eval_runtime": 91.7358,
323
+ "eval_samples_per_second": 55.235,
324
+ "eval_steps_per_second": 6.911,
325
+ "step": 400
326
+ },
327
+ {
328
+ "epoch": 0.32,
329
+ "grad_norm": 1.5040955543518066,
330
+ "learning_rate": 0.00018382004735595896,
331
+ "loss": 0.7407,
332
+ "step": 410
333
+ },
334
+ {
335
+ "epoch": 0.33,
336
+ "grad_norm": 1.1850649118423462,
337
+ "learning_rate": 0.00018342541436464088,
338
+ "loss": 0.8685,
339
+ "step": 420
340
+ },
341
+ {
342
+ "epoch": 0.34,
343
+ "grad_norm": 1.618028163909912,
344
+ "learning_rate": 0.0001830307813733228,
345
+ "loss": 0.9087,
346
+ "step": 430
347
+ },
348
+ {
349
+ "epoch": 0.35,
350
+ "grad_norm": 3.015228509902954,
351
+ "learning_rate": 0.00018263614838200473,
352
+ "loss": 1.0173,
353
+ "step": 440
354
+ },
355
+ {
356
+ "epoch": 0.36,
357
+ "grad_norm": 1.2800592184066772,
358
+ "learning_rate": 0.00018224151539068668,
359
+ "loss": 1.1207,
360
+ "step": 450
361
+ },
362
+ {
363
+ "epoch": 0.36,
364
+ "grad_norm": 1.40296471118927,
365
+ "learning_rate": 0.0001818468823993686,
366
+ "loss": 0.7813,
367
+ "step": 460
368
+ },
369
+ {
370
+ "epoch": 0.37,
371
+ "grad_norm": 2.0328774452209473,
372
+ "learning_rate": 0.00018145224940805052,
373
+ "loss": 0.8776,
374
+ "step": 470
375
+ },
376
+ {
377
+ "epoch": 0.38,
378
+ "grad_norm": 2.16869854927063,
379
+ "learning_rate": 0.00018105761641673245,
380
+ "loss": 1.1089,
381
+ "step": 480
382
+ },
383
+ {
384
+ "epoch": 0.39,
385
+ "grad_norm": 1.867370367050171,
386
+ "learning_rate": 0.00018066298342541437,
387
+ "loss": 0.9493,
388
+ "step": 490
389
+ },
390
+ {
391
+ "epoch": 0.39,
392
+ "grad_norm": 2.4438893795013428,
393
+ "learning_rate": 0.0001802683504340963,
394
+ "loss": 0.8198,
395
+ "step": 500
396
+ },
397
+ {
398
+ "epoch": 0.39,
399
+ "eval_accuracy": 0.6948884941780146,
400
+ "eval_loss": 0.8580543398857117,
401
+ "eval_runtime": 92.2035,
402
+ "eval_samples_per_second": 54.955,
403
+ "eval_steps_per_second": 6.876,
404
+ "step": 500
405
+ },
406
+ {
407
+ "epoch": 0.4,
408
+ "grad_norm": 1.3336217403411865,
409
+ "learning_rate": 0.00017987371744277825,
410
+ "loss": 0.7372,
411
+ "step": 510
412
+ },
413
+ {
414
+ "epoch": 0.41,
415
+ "grad_norm": 2.482779026031494,
416
+ "learning_rate": 0.00017947908445146017,
417
+ "loss": 0.8516,
418
+ "step": 520
419
+ },
420
+ {
421
+ "epoch": 0.42,
422
+ "grad_norm": 1.5005723237991333,
423
+ "learning_rate": 0.00017908445146014207,
424
+ "loss": 0.7964,
425
+ "step": 530
426
+ },
427
+ {
428
+ "epoch": 0.43,
429
+ "grad_norm": 1.8133363723754883,
430
+ "learning_rate": 0.000178689818468824,
431
+ "loss": 0.8267,
432
+ "step": 540
433
+ },
434
+ {
435
+ "epoch": 0.43,
436
+ "grad_norm": 1.8624382019042969,
437
+ "learning_rate": 0.0001782951854775059,
438
+ "loss": 0.8205,
439
+ "step": 550
440
+ },
441
+ {
442
+ "epoch": 0.44,
443
+ "grad_norm": 2.319140911102295,
444
+ "learning_rate": 0.00017790055248618784,
445
+ "loss": 0.8772,
446
+ "step": 560
447
+ },
448
+ {
449
+ "epoch": 0.45,
450
+ "grad_norm": 2.922405958175659,
451
+ "learning_rate": 0.0001775059194948698,
452
+ "loss": 0.9475,
453
+ "step": 570
454
+ },
455
+ {
456
+ "epoch": 0.46,
457
+ "grad_norm": 0.8725391030311584,
458
+ "learning_rate": 0.0001771112865035517,
459
+ "loss": 0.8155,
460
+ "step": 580
461
+ },
462
+ {
463
+ "epoch": 0.47,
464
+ "grad_norm": 2.170085906982422,
465
+ "learning_rate": 0.00017671665351223363,
466
+ "loss": 1.0075,
467
+ "step": 590
468
+ },
469
+ {
470
+ "epoch": 0.47,
471
+ "grad_norm": 2.645709276199341,
472
+ "learning_rate": 0.00017632202052091556,
473
+ "loss": 0.8034,
474
+ "step": 600
475
+ },
476
+ {
477
+ "epoch": 0.47,
478
+ "eval_accuracy": 0.6966646931122953,
479
+ "eval_loss": 0.8444026112556458,
480
+ "eval_runtime": 91.3762,
481
+ "eval_samples_per_second": 55.452,
482
+ "eval_steps_per_second": 6.938,
483
+ "step": 600
484
+ },
485
+ {
486
+ "epoch": 0.48,
487
+ "grad_norm": 1.4042195081710815,
488
+ "learning_rate": 0.00017592738752959748,
489
+ "loss": 0.8112,
490
+ "step": 610
491
+ },
492
+ {
493
+ "epoch": 0.49,
494
+ "grad_norm": 1.9543468952178955,
495
+ "learning_rate": 0.0001755327545382794,
496
+ "loss": 0.8895,
497
+ "step": 620
498
+ },
499
+ {
500
+ "epoch": 0.5,
501
+ "grad_norm": 1.4769212007522583,
502
+ "learning_rate": 0.00017513812154696135,
503
+ "loss": 0.8816,
504
+ "step": 630
505
+ },
506
+ {
507
+ "epoch": 0.51,
508
+ "grad_norm": 1.843929648399353,
509
+ "learning_rate": 0.00017474348855564325,
510
+ "loss": 0.834,
511
+ "step": 640
512
+ },
513
+ {
514
+ "epoch": 0.51,
515
+ "grad_norm": 2.1005053520202637,
516
+ "learning_rate": 0.00017434885556432517,
517
+ "loss": 0.8459,
518
+ "step": 650
519
+ },
520
+ {
521
+ "epoch": 0.52,
522
+ "grad_norm": 1.8301466703414917,
523
+ "learning_rate": 0.0001739542225730071,
524
+ "loss": 0.745,
525
+ "step": 660
526
+ },
527
+ {
528
+ "epoch": 0.53,
529
+ "grad_norm": 1.7254234552383423,
530
+ "learning_rate": 0.00017355958958168902,
531
+ "loss": 0.7985,
532
+ "step": 670
533
+ },
534
+ {
535
+ "epoch": 0.54,
536
+ "grad_norm": 2.2055907249450684,
537
+ "learning_rate": 0.00017316495659037095,
538
+ "loss": 0.768,
539
+ "step": 680
540
+ },
541
+ {
542
+ "epoch": 0.54,
543
+ "grad_norm": 2.17266845703125,
544
+ "learning_rate": 0.0001727703235990529,
545
+ "loss": 0.8608,
546
+ "step": 690
547
+ },
548
+ {
549
+ "epoch": 0.55,
550
+ "grad_norm": 1.5750426054000854,
551
+ "learning_rate": 0.00017237569060773482,
552
+ "loss": 0.8319,
553
+ "step": 700
554
+ },
555
+ {
556
+ "epoch": 0.55,
557
+ "eval_accuracy": 0.7148213933293862,
558
+ "eval_loss": 0.819585919380188,
559
+ "eval_runtime": 91.97,
560
+ "eval_samples_per_second": 55.094,
561
+ "eval_steps_per_second": 6.894,
562
+ "step": 700
563
+ },
564
+ {
565
+ "epoch": 0.56,
566
+ "grad_norm": 1.9275529384613037,
567
+ "learning_rate": 0.00017198105761641674,
568
+ "loss": 0.8694,
569
+ "step": 710
570
+ },
571
+ {
572
+ "epoch": 0.57,
573
+ "grad_norm": 2.029745101928711,
574
+ "learning_rate": 0.00017158642462509867,
575
+ "loss": 0.814,
576
+ "step": 720
577
+ },
578
+ {
579
+ "epoch": 0.58,
580
+ "grad_norm": 2.5681614875793457,
581
+ "learning_rate": 0.0001711917916337806,
582
+ "loss": 0.8117,
583
+ "step": 730
584
+ },
585
+ {
586
+ "epoch": 0.58,
587
+ "grad_norm": 1.8536593914031982,
588
+ "learning_rate": 0.00017079715864246251,
589
+ "loss": 0.8844,
590
+ "step": 740
591
+ },
592
+ {
593
+ "epoch": 0.59,
594
+ "grad_norm": 1.9550455808639526,
595
+ "learning_rate": 0.00017040252565114446,
596
+ "loss": 0.8574,
597
+ "step": 750
598
+ },
599
+ {
600
+ "epoch": 0.6,
601
+ "grad_norm": 2.0112359523773193,
602
+ "learning_rate": 0.00017000789265982636,
603
+ "loss": 0.7197,
604
+ "step": 760
605
+ },
606
+ {
607
+ "epoch": 0.61,
608
+ "grad_norm": 2.2003233432769775,
609
+ "learning_rate": 0.00016961325966850828,
610
+ "loss": 0.8009,
611
+ "step": 770
612
+ },
613
+ {
614
+ "epoch": 0.62,
615
+ "grad_norm": 2.5492188930511475,
616
+ "learning_rate": 0.0001692186266771902,
617
+ "loss": 0.7504,
618
+ "step": 780
619
+ },
620
+ {
621
+ "epoch": 0.62,
622
+ "grad_norm": 1.8917009830474854,
623
+ "learning_rate": 0.00016882399368587213,
624
+ "loss": 0.7638,
625
+ "step": 790
626
+ },
627
+ {
628
+ "epoch": 0.63,
629
+ "grad_norm": 1.2614909410476685,
630
+ "learning_rate": 0.00016842936069455408,
631
+ "loss": 0.787,
632
+ "step": 800
633
+ },
634
+ {
635
+ "epoch": 0.63,
636
+ "eval_accuracy": 0.6974541148608644,
637
+ "eval_loss": 0.8360146880149841,
638
+ "eval_runtime": 91.0831,
639
+ "eval_samples_per_second": 55.63,
640
+ "eval_steps_per_second": 6.961,
641
+ "step": 800
642
+ },
643
+ {
644
+ "epoch": 0.64,
645
+ "grad_norm": 1.555303692817688,
646
+ "learning_rate": 0.000168034727703236,
647
+ "loss": 0.9073,
648
+ "step": 810
649
+ },
650
+ {
651
+ "epoch": 0.65,
652
+ "grad_norm": 1.3937816619873047,
653
+ "learning_rate": 0.00016764009471191793,
654
+ "loss": 0.7977,
655
+ "step": 820
656
+ },
657
+ {
658
+ "epoch": 0.66,
659
+ "grad_norm": 1.7314428091049194,
660
+ "learning_rate": 0.00016724546172059985,
661
+ "loss": 0.8166,
662
+ "step": 830
663
+ },
664
+ {
665
+ "epoch": 0.66,
666
+ "grad_norm": 2.7047677040100098,
667
+ "learning_rate": 0.00016685082872928178,
668
+ "loss": 0.9118,
669
+ "step": 840
670
+ },
671
+ {
672
+ "epoch": 0.67,
673
+ "grad_norm": 2.1908257007598877,
674
+ "learning_rate": 0.0001664561957379637,
675
+ "loss": 0.7528,
676
+ "step": 850
677
+ },
678
+ {
679
+ "epoch": 0.68,
680
+ "grad_norm": 2.966177463531494,
681
+ "learning_rate": 0.00016606156274664565,
682
+ "loss": 0.8306,
683
+ "step": 860
684
+ },
685
+ {
686
+ "epoch": 0.69,
687
+ "grad_norm": 2.368105411529541,
688
+ "learning_rate": 0.00016566692975532755,
689
+ "loss": 0.8289,
690
+ "step": 870
691
+ },
692
+ {
693
+ "epoch": 0.69,
694
+ "grad_norm": 2.197537660598755,
695
+ "learning_rate": 0.00016527229676400947,
696
+ "loss": 0.9994,
697
+ "step": 880
698
+ },
699
+ {
700
+ "epoch": 0.7,
701
+ "grad_norm": 2.863797187805176,
702
+ "learning_rate": 0.0001648776637726914,
703
+ "loss": 0.852,
704
+ "step": 890
705
+ },
706
+ {
707
+ "epoch": 0.71,
708
+ "grad_norm": 2.704500198364258,
709
+ "learning_rate": 0.00016448303078137332,
710
+ "loss": 0.8642,
711
+ "step": 900
712
+ },
713
+ {
714
+ "epoch": 0.71,
715
+ "eval_accuracy": 0.7008091572922834,
716
+ "eval_loss": 0.8250275254249573,
717
+ "eval_runtime": 90.6591,
718
+ "eval_samples_per_second": 55.891,
719
+ "eval_steps_per_second": 6.993,
720
+ "step": 900
721
+ },
722
+ {
723
+ "epoch": 0.72,
724
+ "grad_norm": 2.438966751098633,
725
+ "learning_rate": 0.00016408839779005524,
726
+ "loss": 0.7703,
727
+ "step": 910
728
+ },
729
+ {
730
+ "epoch": 0.73,
731
+ "grad_norm": 2.4163198471069336,
732
+ "learning_rate": 0.0001636937647987372,
733
+ "loss": 0.7739,
734
+ "step": 920
735
+ },
736
+ {
737
+ "epoch": 0.73,
738
+ "grad_norm": 3.480591297149658,
739
+ "learning_rate": 0.00016329913180741912,
740
+ "loss": 0.8631,
741
+ "step": 930
742
+ },
743
+ {
744
+ "epoch": 0.74,
745
+ "grad_norm": 2.24957537651062,
746
+ "learning_rate": 0.00016290449881610104,
747
+ "loss": 0.8008,
748
+ "step": 940
749
+ },
750
+ {
751
+ "epoch": 0.75,
752
+ "grad_norm": 2.1198952198028564,
753
+ "learning_rate": 0.00016250986582478296,
754
+ "loss": 0.8948,
755
+ "step": 950
756
+ },
757
+ {
758
+ "epoch": 0.76,
759
+ "grad_norm": 1.7699896097183228,
760
+ "learning_rate": 0.00016211523283346489,
761
+ "loss": 0.8871,
762
+ "step": 960
763
+ },
764
+ {
765
+ "epoch": 0.77,
766
+ "grad_norm": 1.3637826442718506,
767
+ "learning_rate": 0.0001617205998421468,
768
+ "loss": 0.7904,
769
+ "step": 970
770
+ },
771
+ {
772
+ "epoch": 0.77,
773
+ "grad_norm": 2.0125231742858887,
774
+ "learning_rate": 0.00016132596685082876,
775
+ "loss": 0.6222,
776
+ "step": 980
777
+ },
778
+ {
779
+ "epoch": 0.78,
780
+ "grad_norm": 1.767691731452942,
781
+ "learning_rate": 0.00016093133385951066,
782
+ "loss": 0.8987,
783
+ "step": 990
784
+ },
785
+ {
786
+ "epoch": 0.79,
787
+ "grad_norm": 1.2865104675292969,
788
+ "learning_rate": 0.00016053670086819258,
789
+ "loss": 0.8329,
790
+ "step": 1000
791
+ },
792
+ {
793
+ "epoch": 0.79,
794
+ "eval_accuracy": 0.7171896585750938,
795
+ "eval_loss": 0.79388028383255,
796
+ "eval_runtime": 90.2373,
797
+ "eval_samples_per_second": 56.152,
798
+ "eval_steps_per_second": 7.026,
799
+ "step": 1000
800
+ },
801
+ {
802
+ "epoch": 0.8,
803
+ "grad_norm": 2.2386913299560547,
804
+ "learning_rate": 0.0001601420678768745,
805
+ "loss": 0.7724,
806
+ "step": 1010
807
+ },
808
+ {
809
+ "epoch": 0.81,
810
+ "grad_norm": 1.5864025354385376,
811
+ "learning_rate": 0.00015974743488555643,
812
+ "loss": 0.7434,
813
+ "step": 1020
814
+ },
815
+ {
816
+ "epoch": 0.81,
817
+ "grad_norm": 2.7106010913848877,
818
+ "learning_rate": 0.00015935280189423835,
819
+ "loss": 0.6527,
820
+ "step": 1030
821
+ },
822
+ {
823
+ "epoch": 0.82,
824
+ "grad_norm": 2.2001113891601562,
825
+ "learning_rate": 0.0001589581689029203,
826
+ "loss": 0.7301,
827
+ "step": 1040
828
+ },
829
+ {
830
+ "epoch": 0.83,
831
+ "grad_norm": 1.8353298902511597,
832
+ "learning_rate": 0.00015856353591160222,
833
+ "loss": 0.7831,
834
+ "step": 1050
835
+ },
836
+ {
837
+ "epoch": 0.84,
838
+ "grad_norm": 2.228792190551758,
839
+ "learning_rate": 0.00015816890292028415,
840
+ "loss": 0.7633,
841
+ "step": 1060
842
+ },
843
+ {
844
+ "epoch": 0.84,
845
+ "grad_norm": 3.594202756881714,
846
+ "learning_rate": 0.00015777426992896607,
847
+ "loss": 0.8804,
848
+ "step": 1070
849
+ },
850
+ {
851
+ "epoch": 0.85,
852
+ "grad_norm": 2.274918794631958,
853
+ "learning_rate": 0.000157379636937648,
854
+ "loss": 0.9008,
855
+ "step": 1080
856
+ },
857
+ {
858
+ "epoch": 0.86,
859
+ "grad_norm": 2.0492002964019775,
860
+ "learning_rate": 0.00015698500394632992,
861
+ "loss": 0.6768,
862
+ "step": 1090
863
+ },
864
+ {
865
+ "epoch": 0.87,
866
+ "grad_norm": 2.415807008743286,
867
+ "learning_rate": 0.00015659037095501187,
868
+ "loss": 0.9678,
869
+ "step": 1100
870
+ },
871
+ {
872
+ "epoch": 0.87,
873
+ "eval_accuracy": 0.7331754489836195,
874
+ "eval_loss": 0.7660978436470032,
875
+ "eval_runtime": 92.2043,
876
+ "eval_samples_per_second": 54.954,
877
+ "eval_steps_per_second": 6.876,
878
+ "step": 1100
879
+ },
880
+ {
881
+ "epoch": 0.88,
882
+ "grad_norm": 1.9822345972061157,
883
+ "learning_rate": 0.00015619573796369377,
884
+ "loss": 0.7861,
885
+ "step": 1110
886
+ },
887
+ {
888
+ "epoch": 0.88,
889
+ "grad_norm": 1.869279384613037,
890
+ "learning_rate": 0.0001558011049723757,
891
+ "loss": 0.7472,
892
+ "step": 1120
893
+ },
894
+ {
895
+ "epoch": 0.89,
896
+ "grad_norm": 2.2551605701446533,
897
+ "learning_rate": 0.0001554064719810576,
898
+ "loss": 0.6913,
899
+ "step": 1130
900
+ },
901
+ {
902
+ "epoch": 0.9,
903
+ "grad_norm": 2.4671690464019775,
904
+ "learning_rate": 0.00015501183898973954,
905
+ "loss": 0.6751,
906
+ "step": 1140
907
+ },
908
+ {
909
+ "epoch": 0.91,
910
+ "grad_norm": 1.8815134763717651,
911
+ "learning_rate": 0.00015461720599842146,
912
+ "loss": 0.6414,
913
+ "step": 1150
914
+ },
915
+ {
916
+ "epoch": 0.92,
917
+ "grad_norm": 2.746532917022705,
918
+ "learning_rate": 0.0001542225730071034,
919
+ "loss": 0.849,
920
+ "step": 1160
921
+ },
922
+ {
923
+ "epoch": 0.92,
924
+ "grad_norm": 1.844710111618042,
925
+ "learning_rate": 0.00015382794001578533,
926
+ "loss": 0.8739,
927
+ "step": 1170
928
+ },
929
+ {
930
+ "epoch": 0.93,
931
+ "grad_norm": 2.393015146255493,
932
+ "learning_rate": 0.00015343330702446726,
933
+ "loss": 0.7106,
934
+ "step": 1180
935
+ },
936
+ {
937
+ "epoch": 0.94,
938
+ "grad_norm": 1.9429343938827515,
939
+ "learning_rate": 0.00015303867403314918,
940
+ "loss": 0.7832,
941
+ "step": 1190
942
+ },
943
+ {
944
+ "epoch": 0.95,
945
+ "grad_norm": 2.263422727584839,
946
+ "learning_rate": 0.0001526440410418311,
947
+ "loss": 0.8226,
948
+ "step": 1200
949
+ },
950
+ {
951
+ "epoch": 0.95,
952
+ "eval_accuracy": 0.7373199131636077,
953
+ "eval_loss": 0.7284455895423889,
954
+ "eval_runtime": 91.1355,
955
+ "eval_samples_per_second": 55.599,
956
+ "eval_steps_per_second": 6.957,
957
+ "step": 1200
958
+ },
959
+ {
960
+ "epoch": 0.96,
961
+ "grad_norm": 2.1170480251312256,
962
+ "learning_rate": 0.00015224940805051303,
963
+ "loss": 0.8428,
964
+ "step": 1210
965
+ },
966
+ {
967
+ "epoch": 0.96,
968
+ "grad_norm": 2.143359422683716,
969
+ "learning_rate": 0.00015185477505919495,
970
+ "loss": 0.8429,
971
+ "step": 1220
972
+ },
973
+ {
974
+ "epoch": 0.97,
975
+ "grad_norm": 1.9562945365905762,
976
+ "learning_rate": 0.00015146014206787688,
977
+ "loss": 0.7666,
978
+ "step": 1230
979
+ },
980
+ {
981
+ "epoch": 0.98,
982
+ "grad_norm": 1.7012232542037964,
983
+ "learning_rate": 0.0001510655090765588,
984
+ "loss": 0.7828,
985
+ "step": 1240
986
+ },
987
+ {
988
+ "epoch": 0.99,
989
+ "grad_norm": 1.563348412513733,
990
+ "learning_rate": 0.00015067087608524072,
991
+ "loss": 0.8311,
992
+ "step": 1250
993
+ },
994
+ {
995
+ "epoch": 0.99,
996
+ "grad_norm": 2.0692856311798096,
997
+ "learning_rate": 0.00015027624309392265,
998
+ "loss": 0.7133,
999
+ "step": 1260
1000
+ },
1001
+ {
1002
+ "epoch": 1.0,
1003
+ "grad_norm": 2.52514910697937,
1004
+ "learning_rate": 0.00014988161010260457,
1005
+ "loss": 0.8335,
1006
+ "step": 1270
1007
+ },
1008
+ {
1009
+ "epoch": 1.01,
1010
+ "grad_norm": 1.8508520126342773,
1011
+ "learning_rate": 0.00014948697711128652,
1012
+ "loss": 0.5996,
1013
+ "step": 1280
1014
+ },
1015
+ {
1016
+ "epoch": 1.02,
1017
+ "grad_norm": 1.858475685119629,
1018
+ "learning_rate": 0.00014909234411996844,
1019
+ "loss": 0.6434,
1020
+ "step": 1290
1021
+ },
1022
+ {
1023
+ "epoch": 1.03,
1024
+ "grad_norm": 4.348726272583008,
1025
+ "learning_rate": 0.00014869771112865037,
1026
+ "loss": 0.7967,
1027
+ "step": 1300
1028
+ },
1029
+ {
1030
+ "epoch": 1.03,
1031
+ "eval_accuracy": 0.7410696664693113,
1032
+ "eval_loss": 0.7354866862297058,
1033
+ "eval_runtime": 91.4806,
1034
+ "eval_samples_per_second": 55.389,
1035
+ "eval_steps_per_second": 6.93,
1036
+ "step": 1300
1037
+ },
1038
+ {
1039
+ "epoch": 1.03,
1040
+ "grad_norm": 1.194589614868164,
1041
+ "learning_rate": 0.0001483030781373323,
1042
+ "loss": 0.5958,
1043
+ "step": 1310
1044
+ },
1045
+ {
1046
+ "epoch": 1.04,
1047
+ "grad_norm": 1.5000673532485962,
1048
+ "learning_rate": 0.00014790844514601421,
1049
+ "loss": 0.7041,
1050
+ "step": 1320
1051
+ },
1052
+ {
1053
+ "epoch": 1.05,
1054
+ "grad_norm": 1.6893486976623535,
1055
+ "learning_rate": 0.00014751381215469614,
1056
+ "loss": 0.6521,
1057
+ "step": 1330
1058
+ },
1059
+ {
1060
+ "epoch": 1.06,
1061
+ "grad_norm": 2.065706968307495,
1062
+ "learning_rate": 0.00014711917916337806,
1063
+ "loss": 0.6359,
1064
+ "step": 1340
1065
+ },
1066
+ {
1067
+ "epoch": 1.07,
1068
+ "grad_norm": 2.6649951934814453,
1069
+ "learning_rate": 0.00014672454617205998,
1070
+ "loss": 0.7392,
1071
+ "step": 1350
1072
+ },
1073
+ {
1074
+ "epoch": 1.07,
1075
+ "grad_norm": 2.844940185546875,
1076
+ "learning_rate": 0.0001463299131807419,
1077
+ "loss": 0.6207,
1078
+ "step": 1360
1079
+ },
1080
+ {
1081
+ "epoch": 1.08,
1082
+ "grad_norm": 2.581749677658081,
1083
+ "learning_rate": 0.00014593528018942383,
1084
+ "loss": 0.551,
1085
+ "step": 1370
1086
+ },
1087
+ {
1088
+ "epoch": 1.09,
1089
+ "grad_norm": 1.7631639242172241,
1090
+ "learning_rate": 0.00014554064719810576,
1091
+ "loss": 0.6514,
1092
+ "step": 1380
1093
+ },
1094
+ {
1095
+ "epoch": 1.1,
1096
+ "grad_norm": 4.243679523468018,
1097
+ "learning_rate": 0.00014514601420678768,
1098
+ "loss": 0.5341,
1099
+ "step": 1390
1100
+ },
1101
+ {
1102
+ "epoch": 1.1,
1103
+ "grad_norm": 3.421487808227539,
1104
+ "learning_rate": 0.00014475138121546963,
1105
+ "loss": 0.6531,
1106
+ "step": 1400
1107
+ },
1108
+ {
1109
+ "epoch": 1.1,
1110
+ "eval_accuracy": 0.7246891651865008,
1111
+ "eval_loss": 0.7560569047927856,
1112
+ "eval_runtime": 91.2074,
1113
+ "eval_samples_per_second": 55.555,
1114
+ "eval_steps_per_second": 6.951,
1115
+ "step": 1400
1116
+ },
1117
+ {
1118
+ "epoch": 1.11,
1119
+ "grad_norm": 4.670535564422607,
1120
+ "learning_rate": 0.00014435674822415155,
1121
+ "loss": 0.6982,
1122
+ "step": 1410
1123
+ },
1124
+ {
1125
+ "epoch": 1.12,
1126
+ "grad_norm": 3.2248194217681885,
1127
+ "learning_rate": 0.00014396211523283348,
1128
+ "loss": 0.5345,
1129
+ "step": 1420
1130
+ },
1131
+ {
1132
+ "epoch": 1.13,
1133
+ "grad_norm": 3.1267943382263184,
1134
+ "learning_rate": 0.0001435674822415154,
1135
+ "loss": 0.7239,
1136
+ "step": 1430
1137
+ },
1138
+ {
1139
+ "epoch": 1.14,
1140
+ "grad_norm": 3.158299684524536,
1141
+ "learning_rate": 0.00014317284925019732,
1142
+ "loss": 0.6854,
1143
+ "step": 1440
1144
+ },
1145
+ {
1146
+ "epoch": 1.14,
1147
+ "grad_norm": 2.1028189659118652,
1148
+ "learning_rate": 0.00014277821625887925,
1149
+ "loss": 0.6367,
1150
+ "step": 1450
1151
+ },
1152
+ {
1153
+ "epoch": 1.15,
1154
+ "grad_norm": 2.811735153198242,
1155
+ "learning_rate": 0.00014238358326756117,
1156
+ "loss": 0.6858,
1157
+ "step": 1460
1158
+ },
1159
+ {
1160
+ "epoch": 1.16,
1161
+ "grad_norm": 2.150059461593628,
1162
+ "learning_rate": 0.0001419889502762431,
1163
+ "loss": 0.7487,
1164
+ "step": 1470
1165
+ },
1166
+ {
1167
+ "epoch": 1.17,
1168
+ "grad_norm": 2.536576509475708,
1169
+ "learning_rate": 0.00014159431728492502,
1170
+ "loss": 0.5976,
1171
+ "step": 1480
1172
+ },
1173
+ {
1174
+ "epoch": 1.18,
1175
+ "grad_norm": 2.9393413066864014,
1176
+ "learning_rate": 0.00014119968429360694,
1177
+ "loss": 0.5959,
1178
+ "step": 1490
1179
+ },
1180
+ {
1181
+ "epoch": 1.18,
1182
+ "grad_norm": 1.6796705722808838,
1183
+ "learning_rate": 0.00014080505130228886,
1184
+ "loss": 0.5719,
1185
+ "step": 1500
1186
+ },
1187
+ {
1188
+ "epoch": 1.18,
1189
+ "eval_accuracy": 0.7637655417406749,
1190
+ "eval_loss": 0.6839306354522705,
1191
+ "eval_runtime": 91.2672,
1192
+ "eval_samples_per_second": 55.518,
1193
+ "eval_steps_per_second": 6.947,
1194
+ "step": 1500
1195
+ },
1196
+ {
1197
+ "epoch": 1.19,
1198
+ "grad_norm": 2.542813301086426,
1199
+ "learning_rate": 0.00014041041831097082,
1200
+ "loss": 0.5247,
1201
+ "step": 1510
1202
+ },
1203
+ {
1204
+ "epoch": 1.2,
1205
+ "grad_norm": 0.9640020132064819,
1206
+ "learning_rate": 0.00014001578531965274,
1207
+ "loss": 0.4843,
1208
+ "step": 1520
1209
+ },
1210
+ {
1211
+ "epoch": 1.21,
1212
+ "grad_norm": 2.0161819458007812,
1213
+ "learning_rate": 0.00013962115232833466,
1214
+ "loss": 0.676,
1215
+ "step": 1530
1216
+ },
1217
+ {
1218
+ "epoch": 1.22,
1219
+ "grad_norm": 2.028648853302002,
1220
+ "learning_rate": 0.00013922651933701659,
1221
+ "loss": 0.5935,
1222
+ "step": 1540
1223
+ },
1224
+ {
1225
+ "epoch": 1.22,
1226
+ "grad_norm": 1.2919349670410156,
1227
+ "learning_rate": 0.0001388318863456985,
1228
+ "loss": 0.5941,
1229
+ "step": 1550
1230
+ },
1231
+ {
1232
+ "epoch": 1.23,
1233
+ "grad_norm": 1.8597910404205322,
1234
+ "learning_rate": 0.00013843725335438043,
1235
+ "loss": 0.6447,
1236
+ "step": 1560
1237
+ },
1238
+ {
1239
+ "epoch": 1.24,
1240
+ "grad_norm": 1.6460645198822021,
1241
+ "learning_rate": 0.00013804262036306236,
1242
+ "loss": 0.5576,
1243
+ "step": 1570
1244
+ },
1245
+ {
1246
+ "epoch": 1.25,
1247
+ "grad_norm": 3.1643693447113037,
1248
+ "learning_rate": 0.00013764798737174428,
1249
+ "loss": 0.6632,
1250
+ "step": 1580
1251
+ },
1252
+ {
1253
+ "epoch": 1.25,
1254
+ "grad_norm": 3.991893768310547,
1255
+ "learning_rate": 0.0001372533543804262,
1256
+ "loss": 0.6406,
1257
+ "step": 1590
1258
+ },
1259
+ {
1260
+ "epoch": 1.26,
1261
+ "grad_norm": 3.314627170562744,
1262
+ "learning_rate": 0.00013685872138910813,
1263
+ "loss": 0.6123,
1264
+ "step": 1600
1265
+ },
1266
+ {
1267
+ "epoch": 1.26,
1268
+ "eval_accuracy": 0.7584369449378331,
1269
+ "eval_loss": 0.6857106685638428,
1270
+ "eval_runtime": 91.5334,
1271
+ "eval_samples_per_second": 55.357,
1272
+ "eval_steps_per_second": 6.926,
1273
+ "step": 1600
1274
+ },
1275
+ {
1276
+ "epoch": 1.27,
1277
+ "grad_norm": 1.986220121383667,
1278
+ "learning_rate": 0.00013646408839779005,
1279
+ "loss": 0.7161,
1280
+ "step": 1610
1281
+ },
1282
+ {
1283
+ "epoch": 1.28,
1284
+ "grad_norm": 4.052389144897461,
1285
+ "learning_rate": 0.00013606945540647197,
1286
+ "loss": 0.7073,
1287
+ "step": 1620
1288
+ },
1289
+ {
1290
+ "epoch": 1.29,
1291
+ "grad_norm": 2.59963321685791,
1292
+ "learning_rate": 0.00013567482241515392,
1293
+ "loss": 0.6871,
1294
+ "step": 1630
1295
+ },
1296
+ {
1297
+ "epoch": 1.29,
1298
+ "grad_norm": 3.5070436000823975,
1299
+ "learning_rate": 0.00013528018942383585,
1300
+ "loss": 0.6279,
1301
+ "step": 1640
1302
+ },
1303
+ {
1304
+ "epoch": 1.3,
1305
+ "grad_norm": 2.9049060344696045,
1306
+ "learning_rate": 0.00013488555643251777,
1307
+ "loss": 0.5972,
1308
+ "step": 1650
1309
+ },
1310
+ {
1311
+ "epoch": 1.31,
1312
+ "grad_norm": 1.6428894996643066,
1313
+ "learning_rate": 0.0001344909234411997,
1314
+ "loss": 0.5128,
1315
+ "step": 1660
1316
+ },
1317
+ {
1318
+ "epoch": 1.32,
1319
+ "grad_norm": 2.659148693084717,
1320
+ "learning_rate": 0.00013409629044988162,
1321
+ "loss": 0.7127,
1322
+ "step": 1670
1323
+ },
1324
+ {
1325
+ "epoch": 1.33,
1326
+ "grad_norm": 1.5462294816970825,
1327
+ "learning_rate": 0.00013370165745856354,
1328
+ "loss": 0.6444,
1329
+ "step": 1680
1330
+ },
1331
+ {
1332
+ "epoch": 1.33,
1333
+ "grad_norm": 2.8382019996643066,
1334
+ "learning_rate": 0.00013330702446724547,
1335
+ "loss": 0.5236,
1336
+ "step": 1690
1337
+ },
1338
+ {
1339
+ "epoch": 1.34,
1340
+ "grad_norm": 2.1057918071746826,
1341
+ "learning_rate": 0.0001329123914759274,
1342
+ "loss": 0.6504,
1343
+ "step": 1700
1344
+ },
1345
+ {
1346
+ "epoch": 1.34,
1347
+ "eval_accuracy": 0.7531083481349912,
1348
+ "eval_loss": 0.6970384120941162,
1349
+ "eval_runtime": 92.1039,
1350
+ "eval_samples_per_second": 55.014,
1351
+ "eval_steps_per_second": 6.884,
1352
+ "step": 1700
1353
+ },
1354
+ {
1355
+ "epoch": 1.35,
1356
+ "grad_norm": 1.670453429222107,
1357
+ "learning_rate": 0.0001325177584846093,
1358
+ "loss": 0.5204,
1359
+ "step": 1710
1360
+ },
1361
+ {
1362
+ "epoch": 1.36,
1363
+ "grad_norm": 3.400521755218506,
1364
+ "learning_rate": 0.00013212312549329124,
1365
+ "loss": 0.5707,
1366
+ "step": 1720
1367
+ },
1368
+ {
1369
+ "epoch": 1.37,
1370
+ "grad_norm": 2.378593683242798,
1371
+ "learning_rate": 0.00013172849250197316,
1372
+ "loss": 0.6149,
1373
+ "step": 1730
1374
+ },
1375
+ {
1376
+ "epoch": 1.37,
1377
+ "grad_norm": 4.090423583984375,
1378
+ "learning_rate": 0.00013133385951065508,
1379
+ "loss": 0.7014,
1380
+ "step": 1740
1381
+ },
1382
+ {
1383
+ "epoch": 1.38,
1384
+ "grad_norm": 1.9423013925552368,
1385
+ "learning_rate": 0.00013093922651933703,
1386
+ "loss": 0.4657,
1387
+ "step": 1750
1388
+ },
1389
+ {
1390
+ "epoch": 1.39,
1391
+ "grad_norm": 2.23858380317688,
1392
+ "learning_rate": 0.00013054459352801896,
1393
+ "loss": 0.5976,
1394
+ "step": 1760
1395
+ },
1396
+ {
1397
+ "epoch": 1.4,
1398
+ "grad_norm": 3.8164286613464355,
1399
+ "learning_rate": 0.00013014996053670088,
1400
+ "loss": 0.5735,
1401
+ "step": 1770
1402
+ },
1403
+ {
1404
+ "epoch": 1.4,
1405
+ "grad_norm": 3.1099419593811035,
1406
+ "learning_rate": 0.0001297553275453828,
1407
+ "loss": 0.5255,
1408
+ "step": 1780
1409
+ },
1410
+ {
1411
+ "epoch": 1.41,
1412
+ "grad_norm": 3.818204879760742,
1413
+ "learning_rate": 0.00012936069455406473,
1414
+ "loss": 0.6007,
1415
+ "step": 1790
1416
+ },
1417
+ {
1418
+ "epoch": 1.42,
1419
+ "grad_norm": 3.2938928604125977,
1420
+ "learning_rate": 0.00012896606156274665,
1421
+ "loss": 0.6214,
1422
+ "step": 1800
1423
+ },
1424
+ {
1425
+ "epoch": 1.42,
1426
+ "eval_accuracy": 0.7576475231892639,
1427
+ "eval_loss": 0.6841119527816772,
1428
+ "eval_runtime": 91.5596,
1429
+ "eval_samples_per_second": 55.341,
1430
+ "eval_steps_per_second": 6.924,
1431
+ "step": 1800
1432
+ },
1433
+ {
1434
+ "epoch": 1.43,
1435
+ "grad_norm": 3.2717528343200684,
1436
+ "learning_rate": 0.00012857142857142858,
1437
+ "loss": 0.6118,
1438
+ "step": 1810
1439
+ },
1440
+ {
1441
+ "epoch": 1.44,
1442
+ "grad_norm": 4.299696445465088,
1443
+ "learning_rate": 0.0001281767955801105,
1444
+ "loss": 0.5825,
1445
+ "step": 1820
1446
+ },
1447
+ {
1448
+ "epoch": 1.44,
1449
+ "grad_norm": 3.3908324241638184,
1450
+ "learning_rate": 0.00012778216258879242,
1451
+ "loss": 0.6013,
1452
+ "step": 1830
1453
+ },
1454
+ {
1455
+ "epoch": 1.45,
1456
+ "grad_norm": 1.867518424987793,
1457
+ "learning_rate": 0.00012738752959747435,
1458
+ "loss": 0.4792,
1459
+ "step": 1840
1460
+ },
1461
+ {
1462
+ "epoch": 1.46,
1463
+ "grad_norm": 4.130500793457031,
1464
+ "learning_rate": 0.00012699289660615627,
1465
+ "loss": 0.624,
1466
+ "step": 1850
1467
+ },
1468
+ {
1469
+ "epoch": 1.47,
1470
+ "grad_norm": 1.728039026260376,
1471
+ "learning_rate": 0.0001265982636148382,
1472
+ "loss": 0.534,
1473
+ "step": 1860
1474
+ },
1475
+ {
1476
+ "epoch": 1.48,
1477
+ "grad_norm": 3.357947826385498,
1478
+ "learning_rate": 0.00012620363062352014,
1479
+ "loss": 0.6773,
1480
+ "step": 1870
1481
+ },
1482
+ {
1483
+ "epoch": 1.48,
1484
+ "grad_norm": 1.5584005117416382,
1485
+ "learning_rate": 0.00012580899763220207,
1486
+ "loss": 0.634,
1487
+ "step": 1880
1488
+ },
1489
+ {
1490
+ "epoch": 1.49,
1491
+ "grad_norm": 1.4084281921386719,
1492
+ "learning_rate": 0.000125414364640884,
1493
+ "loss": 0.5695,
1494
+ "step": 1890
1495
+ },
1496
+ {
1497
+ "epoch": 1.5,
1498
+ "grad_norm": 3.2529728412628174,
1499
+ "learning_rate": 0.00012501973164956591,
1500
+ "loss": 0.4925,
1501
+ "step": 1900
1502
+ },
1503
+ {
1504
+ "epoch": 1.5,
1505
+ "eval_accuracy": 0.7641602526149596,
1506
+ "eval_loss": 0.6623676419258118,
1507
+ "eval_runtime": 91.2699,
1508
+ "eval_samples_per_second": 55.517,
1509
+ "eval_steps_per_second": 6.946,
1510
+ "step": 1900
1511
+ },
1512
+ {
1513
+ "epoch": 1.51,
1514
+ "grad_norm": 2.2992420196533203,
1515
+ "learning_rate": 0.00012462509865824784,
1516
+ "loss": 0.554,
1517
+ "step": 1910
1518
+ },
1519
+ {
1520
+ "epoch": 1.52,
1521
+ "grad_norm": 2.778937339782715,
1522
+ "learning_rate": 0.00012423046566692976,
1523
+ "loss": 0.6068,
1524
+ "step": 1920
1525
+ },
1526
+ {
1527
+ "epoch": 1.52,
1528
+ "grad_norm": 1.7722057104110718,
1529
+ "learning_rate": 0.00012383583267561169,
1530
+ "loss": 0.6889,
1531
+ "step": 1930
1532
+ },
1533
+ {
1534
+ "epoch": 1.53,
1535
+ "grad_norm": 1.0860401391983032,
1536
+ "learning_rate": 0.0001234411996842936,
1537
+ "loss": 0.5685,
1538
+ "step": 1940
1539
+ },
1540
+ {
1541
+ "epoch": 1.54,
1542
+ "grad_norm": 1.8181560039520264,
1543
+ "learning_rate": 0.00012304656669297553,
1544
+ "loss": 0.4731,
1545
+ "step": 1950
1546
+ },
1547
+ {
1548
+ "epoch": 1.55,
1549
+ "grad_norm": 2.5842840671539307,
1550
+ "learning_rate": 0.00012265193370165746,
1551
+ "loss": 0.6794,
1552
+ "step": 1960
1553
+ },
1554
+ {
1555
+ "epoch": 1.55,
1556
+ "grad_norm": 2.0809130668640137,
1557
+ "learning_rate": 0.00012225730071033938,
1558
+ "loss": 0.602,
1559
+ "step": 1970
1560
+ },
1561
+ {
1562
+ "epoch": 1.56,
1563
+ "grad_norm": 4.095306396484375,
1564
+ "learning_rate": 0.0001218626677190213,
1565
+ "loss": 0.5275,
1566
+ "step": 1980
1567
+ },
1568
+ {
1569
+ "epoch": 1.57,
1570
+ "grad_norm": 2.1503665447235107,
1571
+ "learning_rate": 0.00012146803472770325,
1572
+ "loss": 0.5502,
1573
+ "step": 1990
1574
+ },
1575
+ {
1576
+ "epoch": 1.58,
1577
+ "grad_norm": 1.8762742280960083,
1578
+ "learning_rate": 0.00012107340173638518,
1579
+ "loss": 0.5797,
1580
+ "step": 2000
1581
+ },
1582
+ {
1583
+ "epoch": 1.58,
1584
+ "eval_accuracy": 0.7708703374777975,
1585
+ "eval_loss": 0.6286503076553345,
1586
+ "eval_runtime": 91.5869,
1587
+ "eval_samples_per_second": 55.325,
1588
+ "eval_steps_per_second": 6.922,
1589
+ "step": 2000
1590
+ },
1591
+ {
1592
+ "epoch": 1.59,
1593
+ "grad_norm": 1.8651785850524902,
1594
+ "learning_rate": 0.0001206787687450671,
1595
+ "loss": 0.4879,
1596
+ "step": 2010
1597
+ },
1598
+ {
1599
+ "epoch": 1.59,
1600
+ "grad_norm": 1.7065809965133667,
1601
+ "learning_rate": 0.00012028413575374901,
1602
+ "loss": 0.4858,
1603
+ "step": 2020
1604
+ },
1605
+ {
1606
+ "epoch": 1.6,
1607
+ "grad_norm": 2.897512435913086,
1608
+ "learning_rate": 0.00011988950276243093,
1609
+ "loss": 0.5983,
1610
+ "step": 2030
1611
+ },
1612
+ {
1613
+ "epoch": 1.61,
1614
+ "grad_norm": 2.6783411502838135,
1615
+ "learning_rate": 0.00011949486977111286,
1616
+ "loss": 0.475,
1617
+ "step": 2040
1618
+ },
1619
+ {
1620
+ "epoch": 1.62,
1621
+ "grad_norm": 3.3606057167053223,
1622
+ "learning_rate": 0.00011910023677979481,
1623
+ "loss": 0.6417,
1624
+ "step": 2050
1625
+ },
1626
+ {
1627
+ "epoch": 1.63,
1628
+ "grad_norm": 2.4237632751464844,
1629
+ "learning_rate": 0.00011870560378847673,
1630
+ "loss": 0.543,
1631
+ "step": 2060
1632
+ },
1633
+ {
1634
+ "epoch": 1.63,
1635
+ "grad_norm": 2.7784054279327393,
1636
+ "learning_rate": 0.00011831097079715866,
1637
+ "loss": 0.6355,
1638
+ "step": 2070
1639
+ },
1640
+ {
1641
+ "epoch": 1.64,
1642
+ "grad_norm": 2.962653398513794,
1643
+ "learning_rate": 0.00011791633780584057,
1644
+ "loss": 0.6289,
1645
+ "step": 2080
1646
+ },
1647
+ {
1648
+ "epoch": 1.65,
1649
+ "grad_norm": 2.9475271701812744,
1650
+ "learning_rate": 0.00011752170481452249,
1651
+ "loss": 0.4379,
1652
+ "step": 2090
1653
+ },
1654
+ {
1655
+ "epoch": 1.66,
1656
+ "grad_norm": 2.1250455379486084,
1657
+ "learning_rate": 0.00011712707182320441,
1658
+ "loss": 0.6018,
1659
+ "step": 2100
1660
+ },
1661
+ {
1662
+ "epoch": 1.66,
1663
+ "eval_accuracy": 0.7621866982435366,
1664
+ "eval_loss": 0.6536552309989929,
1665
+ "eval_runtime": 91.6935,
1666
+ "eval_samples_per_second": 55.26,
1667
+ "eval_steps_per_second": 6.914,
1668
+ "step": 2100
1669
+ },
1670
+ {
1671
+ "epoch": 1.67,
1672
+ "grad_norm": 3.269709348678589,
1673
+ "learning_rate": 0.00011673243883188636,
1674
+ "loss": 0.6077,
1675
+ "step": 2110
1676
+ },
1677
+ {
1678
+ "epoch": 1.67,
1679
+ "grad_norm": 2.2141342163085938,
1680
+ "learning_rate": 0.00011633780584056829,
1681
+ "loss": 0.543,
1682
+ "step": 2120
1683
+ },
1684
+ {
1685
+ "epoch": 1.68,
1686
+ "grad_norm": 3.65494966506958,
1687
+ "learning_rate": 0.00011594317284925021,
1688
+ "loss": 0.5,
1689
+ "step": 2130
1690
+ },
1691
+ {
1692
+ "epoch": 1.69,
1693
+ "grad_norm": 2.4164280891418457,
1694
+ "learning_rate": 0.00011554853985793212,
1695
+ "loss": 0.5186,
1696
+ "step": 2140
1697
+ },
1698
+ {
1699
+ "epoch": 1.7,
1700
+ "grad_norm": 1.6607398986816406,
1701
+ "learning_rate": 0.00011515390686661404,
1702
+ "loss": 0.5087,
1703
+ "step": 2150
1704
+ },
1705
+ {
1706
+ "epoch": 1.7,
1707
+ "grad_norm": 1.4734584093093872,
1708
+ "learning_rate": 0.00011475927387529597,
1709
+ "loss": 0.5642,
1710
+ "step": 2160
1711
+ },
1712
+ {
1713
+ "epoch": 1.71,
1714
+ "grad_norm": 2.652514934539795,
1715
+ "learning_rate": 0.00011436464088397792,
1716
+ "loss": 0.5276,
1717
+ "step": 2170
1718
+ },
1719
+ {
1720
+ "epoch": 1.72,
1721
+ "grad_norm": 1.7451727390289307,
1722
+ "learning_rate": 0.00011397000789265984,
1723
+ "loss": 0.4694,
1724
+ "step": 2180
1725
+ },
1726
+ {
1727
+ "epoch": 1.73,
1728
+ "grad_norm": 2.4469006061553955,
1729
+ "learning_rate": 0.00011357537490134175,
1730
+ "loss": 0.611,
1731
+ "step": 2190
1732
+ },
1733
+ {
1734
+ "epoch": 1.74,
1735
+ "grad_norm": 2.6196885108947754,
1736
+ "learning_rate": 0.00011318074191002367,
1737
+ "loss": 0.6334,
1738
+ "step": 2200
1739
+ },
1740
+ {
1741
+ "epoch": 1.74,
1742
+ "eval_accuracy": 0.7712650483520821,
1743
+ "eval_loss": 0.6413267254829407,
1744
+ "eval_runtime": 91.8747,
1745
+ "eval_samples_per_second": 55.151,
1746
+ "eval_steps_per_second": 6.901,
1747
+ "step": 2200
1748
+ },
1749
+ {
1750
+ "epoch": 1.74,
1751
+ "grad_norm": 3.294318437576294,
1752
+ "learning_rate": 0.0001127861089187056,
1753
+ "loss": 0.5822,
1754
+ "step": 2210
1755
+ },
1756
+ {
1757
+ "epoch": 1.75,
1758
+ "grad_norm": 3.1869616508483887,
1759
+ "learning_rate": 0.00011239147592738752,
1760
+ "loss": 0.5093,
1761
+ "step": 2220
1762
+ },
1763
+ {
1764
+ "epoch": 1.76,
1765
+ "grad_norm": 1.102219820022583,
1766
+ "learning_rate": 0.00011199684293606947,
1767
+ "loss": 0.6136,
1768
+ "step": 2230
1769
+ },
1770
+ {
1771
+ "epoch": 1.77,
1772
+ "grad_norm": 2.6941049098968506,
1773
+ "learning_rate": 0.0001116022099447514,
1774
+ "loss": 0.5639,
1775
+ "step": 2240
1776
+ },
1777
+ {
1778
+ "epoch": 1.78,
1779
+ "grad_norm": 2.5528244972229004,
1780
+ "learning_rate": 0.0001112075769534333,
1781
+ "loss": 0.5713,
1782
+ "step": 2250
1783
+ },
1784
+ {
1785
+ "epoch": 1.78,
1786
+ "grad_norm": 2.0887537002563477,
1787
+ "learning_rate": 0.00011081294396211523,
1788
+ "loss": 0.3819,
1789
+ "step": 2260
1790
+ },
1791
+ {
1792
+ "epoch": 1.79,
1793
+ "grad_norm": 0.9082605242729187,
1794
+ "learning_rate": 0.00011041831097079715,
1795
+ "loss": 0.6017,
1796
+ "step": 2270
1797
+ },
1798
+ {
1799
+ "epoch": 1.8,
1800
+ "grad_norm": 3.7689263820648193,
1801
+ "learning_rate": 0.0001100236779794791,
1802
+ "loss": 0.6816,
1803
+ "step": 2280
1804
+ },
1805
+ {
1806
+ "epoch": 1.81,
1807
+ "grad_norm": 1.2326387166976929,
1808
+ "learning_rate": 0.00010962904498816103,
1809
+ "loss": 0.4845,
1810
+ "step": 2290
1811
+ },
1812
+ {
1813
+ "epoch": 1.82,
1814
+ "grad_norm": 2.5216434001922607,
1815
+ "learning_rate": 0.00010923441199684295,
1816
+ "loss": 0.4111,
1817
+ "step": 2300
1818
+ },
1819
+ {
1820
+ "epoch": 1.82,
1821
+ "eval_accuracy": 0.7785671995263469,
1822
+ "eval_loss": 0.6241939663887024,
1823
+ "eval_runtime": 92.1094,
1824
+ "eval_samples_per_second": 55.011,
1825
+ "eval_steps_per_second": 6.883,
1826
+ "step": 2300
1827
+ },
1828
+ {
1829
+ "epoch": 1.82,
1830
+ "grad_norm": 3.051318407058716,
1831
+ "learning_rate": 0.00010883977900552486,
1832
+ "loss": 0.4867,
1833
+ "step": 2310
1834
+ },
1835
+ {
1836
+ "epoch": 1.83,
1837
+ "grad_norm": 2.2874298095703125,
1838
+ "learning_rate": 0.00010844514601420678,
1839
+ "loss": 0.594,
1840
+ "step": 2320
1841
+ },
1842
+ {
1843
+ "epoch": 1.84,
1844
+ "grad_norm": 1.954172134399414,
1845
+ "learning_rate": 0.00010805051302288871,
1846
+ "loss": 0.4688,
1847
+ "step": 2330
1848
+ },
1849
+ {
1850
+ "epoch": 1.85,
1851
+ "grad_norm": 2.589738607406616,
1852
+ "learning_rate": 0.00010765588003157066,
1853
+ "loss": 0.5692,
1854
+ "step": 2340
1855
+ },
1856
+ {
1857
+ "epoch": 1.85,
1858
+ "grad_norm": 2.662523031234741,
1859
+ "learning_rate": 0.00010726124704025258,
1860
+ "loss": 0.6033,
1861
+ "step": 2350
1862
+ },
1863
+ {
1864
+ "epoch": 1.86,
1865
+ "grad_norm": 1.6577619314193726,
1866
+ "learning_rate": 0.0001068666140489345,
1867
+ "loss": 0.4772,
1868
+ "step": 2360
1869
+ },
1870
+ {
1871
+ "epoch": 1.87,
1872
+ "grad_norm": 2.9956750869750977,
1873
+ "learning_rate": 0.00010647198105761642,
1874
+ "loss": 0.4724,
1875
+ "step": 2370
1876
+ },
1877
+ {
1878
+ "epoch": 1.88,
1879
+ "grad_norm": 2.4675168991088867,
1880
+ "learning_rate": 0.00010607734806629834,
1881
+ "loss": 0.5699,
1882
+ "step": 2380
1883
+ },
1884
+ {
1885
+ "epoch": 1.89,
1886
+ "grad_norm": 3.3173062801361084,
1887
+ "learning_rate": 0.00010568271507498026,
1888
+ "loss": 0.7105,
1889
+ "step": 2390
1890
+ },
1891
+ {
1892
+ "epoch": 1.89,
1893
+ "grad_norm": 2.4665205478668213,
1894
+ "learning_rate": 0.00010528808208366221,
1895
+ "loss": 0.4779,
1896
+ "step": 2400
1897
+ },
1898
+ {
1899
+ "epoch": 1.89,
1900
+ "eval_accuracy": 0.7789619104006316,
1901
+ "eval_loss": 0.6259533166885376,
1902
+ "eval_runtime": 92.0715,
1903
+ "eval_samples_per_second": 55.033,
1904
+ "eval_steps_per_second": 6.886,
1905
+ "step": 2400
1906
+ },
1907
+ {
1908
+ "epoch": 1.9,
1909
+ "grad_norm": 2.5439658164978027,
1910
+ "learning_rate": 0.00010489344909234414,
1911
+ "loss": 0.5352,
1912
+ "step": 2410
1913
+ },
1914
+ {
1915
+ "epoch": 1.91,
1916
+ "grad_norm": 2.340139627456665,
1917
+ "learning_rate": 0.00010449881610102606,
1918
+ "loss": 0.5646,
1919
+ "step": 2420
1920
+ },
1921
+ {
1922
+ "epoch": 1.92,
1923
+ "grad_norm": 2.9097132682800293,
1924
+ "learning_rate": 0.00010410418310970797,
1925
+ "loss": 0.6391,
1926
+ "step": 2430
1927
+ },
1928
+ {
1929
+ "epoch": 1.93,
1930
+ "grad_norm": 4.203121662139893,
1931
+ "learning_rate": 0.0001037095501183899,
1932
+ "loss": 0.4706,
1933
+ "step": 2440
1934
+ },
1935
+ {
1936
+ "epoch": 1.93,
1937
+ "grad_norm": 4.017784595489502,
1938
+ "learning_rate": 0.00010331491712707182,
1939
+ "loss": 0.514,
1940
+ "step": 2450
1941
+ },
1942
+ {
1943
+ "epoch": 1.94,
1944
+ "grad_norm": 2.4876112937927246,
1945
+ "learning_rate": 0.00010292028413575377,
1946
+ "loss": 0.4246,
1947
+ "step": 2460
1948
+ },
1949
+ {
1950
+ "epoch": 1.95,
1951
+ "grad_norm": 1.8711539506912231,
1952
+ "learning_rate": 0.00010252565114443569,
1953
+ "loss": 0.4967,
1954
+ "step": 2470
1955
+ },
1956
+ {
1957
+ "epoch": 1.96,
1958
+ "grad_norm": 2.180922508239746,
1959
+ "learning_rate": 0.0001021310181531176,
1960
+ "loss": 0.5203,
1961
+ "step": 2480
1962
+ },
1963
+ {
1964
+ "epoch": 1.97,
1965
+ "grad_norm": 2.5203304290771484,
1966
+ "learning_rate": 0.00010173638516179952,
1967
+ "loss": 0.5806,
1968
+ "step": 2490
1969
+ },
1970
+ {
1971
+ "epoch": 1.97,
1972
+ "grad_norm": 2.462113857269287,
1973
+ "learning_rate": 0.00010134175217048145,
1974
+ "loss": 0.5488,
1975
+ "step": 2500
1976
+ },
1977
+ {
1978
+ "epoch": 1.97,
1979
+ "eval_accuracy": 0.7807381093349122,
1980
+ "eval_loss": 0.6145808696746826,
1981
+ "eval_runtime": 92.3214,
1982
+ "eval_samples_per_second": 54.884,
1983
+ "eval_steps_per_second": 6.867,
1984
+ "step": 2500
1985
+ },
1986
+ {
1987
+ "epoch": 1.98,
1988
+ "grad_norm": 2.9260380268096924,
1989
+ "learning_rate": 0.00010094711917916337,
1990
+ "loss": 0.5797,
1991
+ "step": 2510
1992
+ },
1993
+ {
1994
+ "epoch": 1.99,
1995
+ "grad_norm": 2.5453708171844482,
1996
+ "learning_rate": 0.00010055248618784532,
1997
+ "loss": 0.5223,
1998
+ "step": 2520
1999
+ },
2000
+ {
2001
+ "epoch": 2.0,
2002
+ "grad_norm": 1.0558793544769287,
2003
+ "learning_rate": 0.00010015785319652725,
2004
+ "loss": 0.3824,
2005
+ "step": 2530
2006
+ },
2007
+ {
2008
+ "epoch": 2.0,
2009
+ "grad_norm": 2.0377509593963623,
2010
+ "learning_rate": 9.976322020520916e-05,
2011
+ "loss": 0.4497,
2012
+ "step": 2540
2013
+ },
2014
+ {
2015
+ "epoch": 2.01,
2016
+ "grad_norm": 2.023796319961548,
2017
+ "learning_rate": 9.936858721389108e-05,
2018
+ "loss": 0.3679,
2019
+ "step": 2550
2020
+ },
2021
+ {
2022
+ "epoch": 2.02,
2023
+ "grad_norm": 1.737573504447937,
2024
+ "learning_rate": 9.8973954222573e-05,
2025
+ "loss": 0.2755,
2026
+ "step": 2560
2027
+ },
2028
+ {
2029
+ "epoch": 2.03,
2030
+ "grad_norm": 1.711060881614685,
2031
+ "learning_rate": 9.857932123125494e-05,
2032
+ "loss": 0.2844,
2033
+ "step": 2570
2034
+ },
2035
+ {
2036
+ "epoch": 2.04,
2037
+ "grad_norm": 1.0967369079589844,
2038
+ "learning_rate": 9.818468823993686e-05,
2039
+ "loss": 0.3734,
2040
+ "step": 2580
2041
+ },
2042
+ {
2043
+ "epoch": 2.04,
2044
+ "grad_norm": 4.402666091918945,
2045
+ "learning_rate": 9.779005524861879e-05,
2046
+ "loss": 0.3492,
2047
+ "step": 2590
2048
+ },
2049
+ {
2050
+ "epoch": 2.05,
2051
+ "grad_norm": 0.23064565658569336,
2052
+ "learning_rate": 9.739542225730071e-05,
2053
+ "loss": 0.3212,
2054
+ "step": 2600
2055
+ },
2056
+ {
2057
+ "epoch": 2.05,
2058
+ "eval_accuracy": 0.7706729820406553,
2059
+ "eval_loss": 0.697548508644104,
2060
+ "eval_runtime": 93.0724,
2061
+ "eval_samples_per_second": 54.441,
2062
+ "eval_steps_per_second": 6.812,
2063
+ "step": 2600
2064
+ },
2065
+ {
2066
+ "epoch": 2.06,
2067
+ "grad_norm": 2.1419265270233154,
2068
+ "learning_rate": 9.700078926598263e-05,
2069
+ "loss": 0.3762,
2070
+ "step": 2610
2071
+ },
2072
+ {
2073
+ "epoch": 2.07,
2074
+ "grad_norm": 2.5868382453918457,
2075
+ "learning_rate": 9.660615627466457e-05,
2076
+ "loss": 0.3973,
2077
+ "step": 2620
2078
+ },
2079
+ {
2080
+ "epoch": 2.08,
2081
+ "grad_norm": 1.9295076131820679,
2082
+ "learning_rate": 9.62115232833465e-05,
2083
+ "loss": 0.3264,
2084
+ "step": 2630
2085
+ },
2086
+ {
2087
+ "epoch": 2.08,
2088
+ "grad_norm": 2.8431601524353027,
2089
+ "learning_rate": 9.581689029202842e-05,
2090
+ "loss": 0.2892,
2091
+ "step": 2640
2092
+ },
2093
+ {
2094
+ "epoch": 2.09,
2095
+ "grad_norm": 2.848076105117798,
2096
+ "learning_rate": 9.542225730071036e-05,
2097
+ "loss": 0.3393,
2098
+ "step": 2650
2099
+ },
2100
+ {
2101
+ "epoch": 2.1,
2102
+ "grad_norm": 1.3948113918304443,
2103
+ "learning_rate": 9.502762430939227e-05,
2104
+ "loss": 0.3074,
2105
+ "step": 2660
2106
+ },
2107
+ {
2108
+ "epoch": 2.11,
2109
+ "grad_norm": 7.114987850189209,
2110
+ "learning_rate": 9.463299131807419e-05,
2111
+ "loss": 0.3326,
2112
+ "step": 2670
2113
+ },
2114
+ {
2115
+ "epoch": 2.12,
2116
+ "grad_norm": 3.4275946617126465,
2117
+ "learning_rate": 9.423835832675613e-05,
2118
+ "loss": 0.3915,
2119
+ "step": 2680
2120
+ },
2121
+ {
2122
+ "epoch": 2.12,
2123
+ "grad_norm": 2.183617353439331,
2124
+ "learning_rate": 9.384372533543805e-05,
2125
+ "loss": 0.3841,
2126
+ "step": 2690
2127
+ },
2128
+ {
2129
+ "epoch": 2.13,
2130
+ "grad_norm": 2.8311221599578857,
2131
+ "learning_rate": 9.344909234411997e-05,
2132
+ "loss": 0.4282,
2133
+ "step": 2700
2134
+ },
2135
+ {
2136
+ "epoch": 2.13,
2137
+ "eval_accuracy": 0.7789619104006316,
2138
+ "eval_loss": 0.634354293346405,
2139
+ "eval_runtime": 93.2533,
2140
+ "eval_samples_per_second": 54.336,
2141
+ "eval_steps_per_second": 6.799,
2142
+ "step": 2700
2143
+ },
2144
+ {
2145
+ "epoch": 2.14,
2146
+ "grad_norm": 2.839000701904297,
2147
+ "learning_rate": 9.305445935280191e-05,
2148
+ "loss": 0.2869,
2149
+ "step": 2710
2150
+ },
2151
+ {
2152
+ "epoch": 2.15,
2153
+ "grad_norm": 1.4117017984390259,
2154
+ "learning_rate": 9.265982636148382e-05,
2155
+ "loss": 0.35,
2156
+ "step": 2720
2157
+ },
2158
+ {
2159
+ "epoch": 2.15,
2160
+ "grad_norm": 6.646859645843506,
2161
+ "learning_rate": 9.226519337016574e-05,
2162
+ "loss": 0.1956,
2163
+ "step": 2730
2164
+ },
2165
+ {
2166
+ "epoch": 2.16,
2167
+ "grad_norm": 2.4397947788238525,
2168
+ "learning_rate": 9.187056037884768e-05,
2169
+ "loss": 0.3768,
2170
+ "step": 2740
2171
+ },
2172
+ {
2173
+ "epoch": 2.17,
2174
+ "grad_norm": 1.0888396501541138,
2175
+ "learning_rate": 9.14759273875296e-05,
2176
+ "loss": 0.2825,
2177
+ "step": 2750
2178
+ },
2179
+ {
2180
+ "epoch": 2.18,
2181
+ "grad_norm": 1.1045578718185425,
2182
+ "learning_rate": 9.108129439621153e-05,
2183
+ "loss": 0.1746,
2184
+ "step": 2760
2185
+ },
2186
+ {
2187
+ "epoch": 2.19,
2188
+ "grad_norm": 1.6195358037948608,
2189
+ "learning_rate": 9.068666140489345e-05,
2190
+ "loss": 0.2362,
2191
+ "step": 2770
2192
+ },
2193
+ {
2194
+ "epoch": 2.19,
2195
+ "grad_norm": 2.6458139419555664,
2196
+ "learning_rate": 9.029202841357538e-05,
2197
+ "loss": 0.2583,
2198
+ "step": 2780
2199
+ },
2200
+ {
2201
+ "epoch": 2.2,
2202
+ "grad_norm": 2.6736512184143066,
2203
+ "learning_rate": 8.98973954222573e-05,
2204
+ "loss": 0.4217,
2205
+ "step": 2790
2206
+ },
2207
+ {
2208
+ "epoch": 2.21,
2209
+ "grad_norm": 2.508667230606079,
2210
+ "learning_rate": 8.950276243093924e-05,
2211
+ "loss": 0.2822,
2212
+ "step": 2800
2213
+ },
2214
+ {
2215
+ "epoch": 2.21,
2216
+ "eval_accuracy": 0.7844878626406158,
2217
+ "eval_loss": 0.6984525322914124,
2218
+ "eval_runtime": 93.5792,
2219
+ "eval_samples_per_second": 54.147,
2220
+ "eval_steps_per_second": 6.775,
2221
+ "step": 2800
2222
+ },
2223
+ {
2224
+ "epoch": 2.22,
2225
+ "grad_norm": 1.4989869594573975,
2226
+ "learning_rate": 8.910812943962116e-05,
2227
+ "loss": 0.3101,
2228
+ "step": 2810
2229
+ },
2230
+ {
2231
+ "epoch": 2.23,
2232
+ "grad_norm": 0.30821582674980164,
2233
+ "learning_rate": 8.871349644830308e-05,
2234
+ "loss": 0.2419,
2235
+ "step": 2820
2236
+ },
2237
+ {
2238
+ "epoch": 2.23,
2239
+ "grad_norm": 5.179461479187012,
2240
+ "learning_rate": 8.8318863456985e-05,
2241
+ "loss": 0.4989,
2242
+ "step": 2830
2243
+ },
2244
+ {
2245
+ "epoch": 2.24,
2246
+ "grad_norm": 3.597341775894165,
2247
+ "learning_rate": 8.792423046566693e-05,
2248
+ "loss": 0.3628,
2249
+ "step": 2840
2250
+ },
2251
+ {
2252
+ "epoch": 2.25,
2253
+ "grad_norm": 1.0267224311828613,
2254
+ "learning_rate": 8.752959747434885e-05,
2255
+ "loss": 0.2395,
2256
+ "step": 2850
2257
+ },
2258
+ {
2259
+ "epoch": 2.26,
2260
+ "grad_norm": 4.339228630065918,
2261
+ "learning_rate": 8.713496448303079e-05,
2262
+ "loss": 0.385,
2263
+ "step": 2860
2264
+ },
2265
+ {
2266
+ "epoch": 2.27,
2267
+ "grad_norm": 2.7694778442382812,
2268
+ "learning_rate": 8.674033149171271e-05,
2269
+ "loss": 0.3129,
2270
+ "step": 2870
2271
+ },
2272
+ {
2273
+ "epoch": 2.27,
2274
+ "grad_norm": 3.7710163593292236,
2275
+ "learning_rate": 8.634569850039464e-05,
2276
+ "loss": 0.3818,
2277
+ "step": 2880
2278
+ },
2279
+ {
2280
+ "epoch": 2.28,
2281
+ "grad_norm": 3.384129047393799,
2282
+ "learning_rate": 8.595106550907656e-05,
2283
+ "loss": 0.2772,
2284
+ "step": 2890
2285
+ },
2286
+ {
2287
+ "epoch": 2.29,
2288
+ "grad_norm": 1.9485032558441162,
2289
+ "learning_rate": 8.555643251775848e-05,
2290
+ "loss": 0.3003,
2291
+ "step": 2900
2292
+ },
2293
+ {
2294
+ "epoch": 2.29,
2295
+ "eval_accuracy": 0.7992895204262878,
2296
+ "eval_loss": 0.5954017639160156,
2297
+ "eval_runtime": 90.8299,
2298
+ "eval_samples_per_second": 55.786,
2299
+ "eval_steps_per_second": 6.98,
2300
+ "step": 2900
2301
+ },
2302
+ {
2303
+ "epoch": 2.3,
2304
+ "grad_norm": 1.648350715637207,
2305
+ "learning_rate": 8.516179952644041e-05,
2306
+ "loss": 0.3255,
2307
+ "step": 2910
2308
+ },
2309
+ {
2310
+ "epoch": 2.3,
2311
+ "grad_norm": 2.1421117782592773,
2312
+ "learning_rate": 8.476716653512235e-05,
2313
+ "loss": 0.3721,
2314
+ "step": 2920
2315
+ },
2316
+ {
2317
+ "epoch": 2.31,
2318
+ "grad_norm": 3.3461079597473145,
2319
+ "learning_rate": 8.437253354380427e-05,
2320
+ "loss": 0.3533,
2321
+ "step": 2930
2322
+ },
2323
+ {
2324
+ "epoch": 2.32,
2325
+ "grad_norm": 2.0906074047088623,
2326
+ "learning_rate": 8.397790055248619e-05,
2327
+ "loss": 0.2805,
2328
+ "step": 2940
2329
+ },
2330
+ {
2331
+ "epoch": 2.33,
2332
+ "grad_norm": 2.1087684631347656,
2333
+ "learning_rate": 8.358326756116812e-05,
2334
+ "loss": 0.2588,
2335
+ "step": 2950
2336
+ },
2337
+ {
2338
+ "epoch": 2.34,
2339
+ "grad_norm": 4.184020042419434,
2340
+ "learning_rate": 8.318863456985004e-05,
2341
+ "loss": 0.3646,
2342
+ "step": 2960
2343
+ },
2344
+ {
2345
+ "epoch": 2.34,
2346
+ "grad_norm": 0.7979826927185059,
2347
+ "learning_rate": 8.279400157853196e-05,
2348
+ "loss": 0.2358,
2349
+ "step": 2970
2350
+ },
2351
+ {
2352
+ "epoch": 2.35,
2353
+ "grad_norm": 3.317765235900879,
2354
+ "learning_rate": 8.23993685872139e-05,
2355
+ "loss": 0.3134,
2356
+ "step": 2980
2357
+ },
2358
+ {
2359
+ "epoch": 2.36,
2360
+ "grad_norm": 4.421618938446045,
2361
+ "learning_rate": 8.200473559589582e-05,
2362
+ "loss": 0.3644,
2363
+ "step": 2990
2364
+ },
2365
+ {
2366
+ "epoch": 2.37,
2367
+ "grad_norm": 4.3369879722595215,
2368
+ "learning_rate": 8.161010260457775e-05,
2369
+ "loss": 0.2982,
2370
+ "step": 3000
2371
+ },
2372
+ {
2373
+ "epoch": 2.37,
2374
+ "eval_accuracy": 0.7939609236234458,
2375
+ "eval_loss": 0.6156049966812134,
2376
+ "eval_runtime": 90.2416,
2377
+ "eval_samples_per_second": 56.149,
2378
+ "eval_steps_per_second": 7.026,
2379
+ "step": 3000
2380
+ },
2381
+ {
2382
+ "epoch": 2.38,
2383
+ "grad_norm": 3.2008187770843506,
2384
+ "learning_rate": 8.121546961325967e-05,
2385
+ "loss": 0.3314,
2386
+ "step": 3010
2387
+ },
2388
+ {
2389
+ "epoch": 2.38,
2390
+ "grad_norm": 2.550990343093872,
2391
+ "learning_rate": 8.08208366219416e-05,
2392
+ "loss": 0.2522,
2393
+ "step": 3020
2394
+ },
2395
+ {
2396
+ "epoch": 2.39,
2397
+ "grad_norm": 0.7245291471481323,
2398
+ "learning_rate": 8.042620363062352e-05,
2399
+ "loss": 0.325,
2400
+ "step": 3030
2401
+ },
2402
+ {
2403
+ "epoch": 2.4,
2404
+ "grad_norm": 3.1475908756256104,
2405
+ "learning_rate": 8.003157063930545e-05,
2406
+ "loss": 0.3212,
2407
+ "step": 3040
2408
+ },
2409
+ {
2410
+ "epoch": 2.41,
2411
+ "grad_norm": 1.6802873611450195,
2412
+ "learning_rate": 7.963693764798738e-05,
2413
+ "loss": 0.2484,
2414
+ "step": 3050
2415
+ },
2416
+ {
2417
+ "epoch": 2.42,
2418
+ "grad_norm": 3.3150057792663574,
2419
+ "learning_rate": 7.92423046566693e-05,
2420
+ "loss": 0.3014,
2421
+ "step": 3060
2422
+ },
2423
+ {
2424
+ "epoch": 2.42,
2425
+ "grad_norm": 1.040041208267212,
2426
+ "learning_rate": 7.884767166535123e-05,
2427
+ "loss": 0.324,
2428
+ "step": 3070
2429
+ },
2430
+ {
2431
+ "epoch": 2.43,
2432
+ "grad_norm": 1.5080398321151733,
2433
+ "learning_rate": 7.845303867403315e-05,
2434
+ "loss": 0.2253,
2435
+ "step": 3080
2436
+ },
2437
+ {
2438
+ "epoch": 2.44,
2439
+ "grad_norm": 1.1280837059020996,
2440
+ "learning_rate": 7.805840568271507e-05,
2441
+ "loss": 0.2373,
2442
+ "step": 3090
2443
+ },
2444
+ {
2445
+ "epoch": 2.45,
2446
+ "grad_norm": 4.508062839508057,
2447
+ "learning_rate": 7.766377269139701e-05,
2448
+ "loss": 0.2628,
2449
+ "step": 3100
2450
+ },
2451
+ {
2452
+ "epoch": 2.45,
2453
+ "eval_accuracy": 0.7963291888691534,
2454
+ "eval_loss": 0.6317601203918457,
2455
+ "eval_runtime": 91.5531,
2456
+ "eval_samples_per_second": 55.345,
2457
+ "eval_steps_per_second": 6.925,
2458
+ "step": 3100
2459
+ },
2460
+ {
2461
+ "epoch": 2.45,
2462
+ "grad_norm": 1.481145977973938,
2463
+ "learning_rate": 7.726913970007893e-05,
2464
+ "loss": 0.3059,
2465
+ "step": 3110
2466
+ },
2467
+ {
2468
+ "epoch": 2.46,
2469
+ "grad_norm": 1.0478160381317139,
2470
+ "learning_rate": 7.687450670876086e-05,
2471
+ "loss": 0.2453,
2472
+ "step": 3120
2473
+ },
2474
+ {
2475
+ "epoch": 2.47,
2476
+ "grad_norm": 3.6211369037628174,
2477
+ "learning_rate": 7.647987371744278e-05,
2478
+ "loss": 0.293,
2479
+ "step": 3130
2480
+ },
2481
+ {
2482
+ "epoch": 2.48,
2483
+ "grad_norm": 2.617844343185425,
2484
+ "learning_rate": 7.60852407261247e-05,
2485
+ "loss": 0.2081,
2486
+ "step": 3140
2487
+ },
2488
+ {
2489
+ "epoch": 2.49,
2490
+ "grad_norm": 2.3399300575256348,
2491
+ "learning_rate": 7.569060773480663e-05,
2492
+ "loss": 0.239,
2493
+ "step": 3150
2494
+ },
2495
+ {
2496
+ "epoch": 2.49,
2497
+ "grad_norm": 5.697124481201172,
2498
+ "learning_rate": 7.529597474348856e-05,
2499
+ "loss": 0.2893,
2500
+ "step": 3160
2501
+ },
2502
+ {
2503
+ "epoch": 2.5,
2504
+ "grad_norm": 3.405291795730591,
2505
+ "learning_rate": 7.490134175217049e-05,
2506
+ "loss": 0.3245,
2507
+ "step": 3170
2508
+ },
2509
+ {
2510
+ "epoch": 2.51,
2511
+ "grad_norm": 3.2485592365264893,
2512
+ "learning_rate": 7.450670876085241e-05,
2513
+ "loss": 0.4137,
2514
+ "step": 3180
2515
+ },
2516
+ {
2517
+ "epoch": 2.52,
2518
+ "grad_norm": 1.0946143865585327,
2519
+ "learning_rate": 7.411207576953433e-05,
2520
+ "loss": 0.1969,
2521
+ "step": 3190
2522
+ },
2523
+ {
2524
+ "epoch": 2.53,
2525
+ "grad_norm": 0.3789944052696228,
2526
+ "learning_rate": 7.371744277821626e-05,
2527
+ "loss": 0.2987,
2528
+ "step": 3200
2529
+ },
2530
+ {
2531
+ "epoch": 2.53,
2532
+ "eval_accuracy": 0.8030392737319914,
2533
+ "eval_loss": 0.6494620442390442,
2534
+ "eval_runtime": 92.4486,
2535
+ "eval_samples_per_second": 54.809,
2536
+ "eval_steps_per_second": 6.858,
2537
+ "step": 3200
2538
+ },
2539
+ {
2540
+ "epoch": 2.53,
2541
+ "grad_norm": 3.4358811378479004,
2542
+ "learning_rate": 7.332280978689818e-05,
2543
+ "loss": 0.2757,
2544
+ "step": 3210
2545
+ },
2546
+ {
2547
+ "epoch": 2.54,
2548
+ "grad_norm": 3.544767141342163,
2549
+ "learning_rate": 7.292817679558012e-05,
2550
+ "loss": 0.2126,
2551
+ "step": 3220
2552
+ },
2553
+ {
2554
+ "epoch": 2.55,
2555
+ "grad_norm": 2.8036537170410156,
2556
+ "learning_rate": 7.253354380426204e-05,
2557
+ "loss": 0.2455,
2558
+ "step": 3230
2559
+ },
2560
+ {
2561
+ "epoch": 2.56,
2562
+ "grad_norm": 2.1251418590545654,
2563
+ "learning_rate": 7.213891081294397e-05,
2564
+ "loss": 0.2136,
2565
+ "step": 3240
2566
+ },
2567
+ {
2568
+ "epoch": 2.57,
2569
+ "grad_norm": 4.44752311706543,
2570
+ "learning_rate": 7.174427782162589e-05,
2571
+ "loss": 0.3819,
2572
+ "step": 3250
2573
+ },
2574
+ {
2575
+ "epoch": 2.57,
2576
+ "grad_norm": 0.8341973423957825,
2577
+ "learning_rate": 7.134964483030781e-05,
2578
+ "loss": 0.2532,
2579
+ "step": 3260
2580
+ },
2581
+ {
2582
+ "epoch": 2.58,
2583
+ "grad_norm": 3.4972732067108154,
2584
+ "learning_rate": 7.095501183898974e-05,
2585
+ "loss": 0.2948,
2586
+ "step": 3270
2587
+ },
2588
+ {
2589
+ "epoch": 2.59,
2590
+ "grad_norm": 3.1307532787323,
2591
+ "learning_rate": 7.056037884767167e-05,
2592
+ "loss": 0.2987,
2593
+ "step": 3280
2594
+ },
2595
+ {
2596
+ "epoch": 2.6,
2597
+ "grad_norm": 2.040160894393921,
2598
+ "learning_rate": 7.01657458563536e-05,
2599
+ "loss": 0.2939,
2600
+ "step": 3290
2601
+ },
2602
+ {
2603
+ "epoch": 2.6,
2604
+ "grad_norm": 0.5896205902099609,
2605
+ "learning_rate": 6.977111286503552e-05,
2606
+ "loss": 0.2714,
2607
+ "step": 3300
2608
+ },
2609
+ {
2610
+ "epoch": 2.6,
2611
+ "eval_accuracy": 0.8052101835405565,
2612
+ "eval_loss": 0.6017640829086304,
2613
+ "eval_runtime": 92.2458,
2614
+ "eval_samples_per_second": 54.929,
2615
+ "eval_steps_per_second": 6.873,
2616
+ "step": 3300
2617
+ },
2618
+ {
2619
+ "epoch": 2.61,
2620
+ "grad_norm": 4.136446475982666,
2621
+ "learning_rate": 6.937647987371744e-05,
2622
+ "loss": 0.2848,
2623
+ "step": 3310
2624
+ },
2625
+ {
2626
+ "epoch": 2.62,
2627
+ "grad_norm": 1.1288126707077026,
2628
+ "learning_rate": 6.898184688239937e-05,
2629
+ "loss": 0.3115,
2630
+ "step": 3320
2631
+ },
2632
+ {
2633
+ "epoch": 2.63,
2634
+ "grad_norm": 1.7702901363372803,
2635
+ "learning_rate": 6.858721389108129e-05,
2636
+ "loss": 0.2547,
2637
+ "step": 3330
2638
+ },
2639
+ {
2640
+ "epoch": 2.64,
2641
+ "grad_norm": 2.859607696533203,
2642
+ "learning_rate": 6.819258089976323e-05,
2643
+ "loss": 0.2793,
2644
+ "step": 3340
2645
+ },
2646
+ {
2647
+ "epoch": 2.64,
2648
+ "grad_norm": 5.98909854888916,
2649
+ "learning_rate": 6.779794790844515e-05,
2650
+ "loss": 0.2718,
2651
+ "step": 3350
2652
+ },
2653
+ {
2654
+ "epoch": 2.65,
2655
+ "grad_norm": 4.067326545715332,
2656
+ "learning_rate": 6.740331491712708e-05,
2657
+ "loss": 0.3174,
2658
+ "step": 3360
2659
+ },
2660
+ {
2661
+ "epoch": 2.66,
2662
+ "grad_norm": 5.376368522644043,
2663
+ "learning_rate": 6.7008681925809e-05,
2664
+ "loss": 0.3494,
2665
+ "step": 3370
2666
+ },
2667
+ {
2668
+ "epoch": 2.67,
2669
+ "grad_norm": 2.6737453937530518,
2670
+ "learning_rate": 6.661404893449092e-05,
2671
+ "loss": 0.293,
2672
+ "step": 3380
2673
+ },
2674
+ {
2675
+ "epoch": 2.68,
2676
+ "grad_norm": 2.012742042541504,
2677
+ "learning_rate": 6.621941594317286e-05,
2678
+ "loss": 0.2291,
2679
+ "step": 3390
2680
+ },
2681
+ {
2682
+ "epoch": 2.68,
2683
+ "grad_norm": 4.339968204498291,
2684
+ "learning_rate": 6.582478295185478e-05,
2685
+ "loss": 0.3059,
2686
+ "step": 3400
2687
+ },
2688
+ {
2689
+ "epoch": 2.68,
2690
+ "eval_accuracy": 0.8077758042234063,
2691
+ "eval_loss": 0.5943508148193359,
2692
+ "eval_runtime": 92.7623,
2693
+ "eval_samples_per_second": 54.624,
2694
+ "eval_steps_per_second": 6.835,
2695
+ "step": 3400
2696
+ },
2697
+ {
2698
+ "epoch": 2.69,
2699
+ "grad_norm": 1.6194040775299072,
2700
+ "learning_rate": 6.543014996053671e-05,
2701
+ "loss": 0.3704,
2702
+ "step": 3410
2703
+ },
2704
+ {
2705
+ "epoch": 2.7,
2706
+ "grad_norm": 4.174539089202881,
2707
+ "learning_rate": 6.503551696921863e-05,
2708
+ "loss": 0.3716,
2709
+ "step": 3420
2710
+ },
2711
+ {
2712
+ "epoch": 2.71,
2713
+ "grad_norm": 4.487131118774414,
2714
+ "learning_rate": 6.464088397790055e-05,
2715
+ "loss": 0.3054,
2716
+ "step": 3430
2717
+ },
2718
+ {
2719
+ "epoch": 2.72,
2720
+ "grad_norm": 3.870863437652588,
2721
+ "learning_rate": 6.424625098658248e-05,
2722
+ "loss": 0.2811,
2723
+ "step": 3440
2724
+ },
2725
+ {
2726
+ "epoch": 2.72,
2727
+ "grad_norm": 4.740366458892822,
2728
+ "learning_rate": 6.385161799526441e-05,
2729
+ "loss": 0.2143,
2730
+ "step": 3450
2731
+ },
2732
+ {
2733
+ "epoch": 2.73,
2734
+ "grad_norm": 2.337819814682007,
2735
+ "learning_rate": 6.345698500394634e-05,
2736
+ "loss": 0.4391,
2737
+ "step": 3460
2738
+ },
2739
+ {
2740
+ "epoch": 2.74,
2741
+ "grad_norm": 6.155264854431152,
2742
+ "learning_rate": 6.306235201262826e-05,
2743
+ "loss": 0.3491,
2744
+ "step": 3470
2745
+ },
2746
+ {
2747
+ "epoch": 2.75,
2748
+ "grad_norm": 2.4382271766662598,
2749
+ "learning_rate": 6.266771902131019e-05,
2750
+ "loss": 0.2899,
2751
+ "step": 3480
2752
+ },
2753
+ {
2754
+ "epoch": 2.75,
2755
+ "grad_norm": 2.5207793712615967,
2756
+ "learning_rate": 6.227308602999211e-05,
2757
+ "loss": 0.2522,
2758
+ "step": 3490
2759
+ },
2760
+ {
2761
+ "epoch": 2.76,
2762
+ "grad_norm": 2.243215322494507,
2763
+ "learning_rate": 6.187845303867403e-05,
2764
+ "loss": 0.2762,
2765
+ "step": 3500
2766
+ },
2767
+ {
2768
+ "epoch": 2.76,
2769
+ "eval_accuracy": 0.7935662127491613,
2770
+ "eval_loss": 0.6296067237854004,
2771
+ "eval_runtime": 92.1328,
2772
+ "eval_samples_per_second": 54.997,
2773
+ "eval_steps_per_second": 6.881,
2774
+ "step": 3500
2775
+ },
2776
+ {
2777
+ "epoch": 2.77,
2778
+ "grad_norm": 2.194077968597412,
2779
+ "learning_rate": 6.148382004735597e-05,
2780
+ "loss": 0.2279,
2781
+ "step": 3510
2782
+ },
2783
+ {
2784
+ "epoch": 2.78,
2785
+ "grad_norm": 2.3025989532470703,
2786
+ "learning_rate": 6.108918705603789e-05,
2787
+ "loss": 0.3366,
2788
+ "step": 3520
2789
+ },
2790
+ {
2791
+ "epoch": 2.79,
2792
+ "grad_norm": 3.9970479011535645,
2793
+ "learning_rate": 6.069455406471981e-05,
2794
+ "loss": 0.2698,
2795
+ "step": 3530
2796
+ },
2797
+ {
2798
+ "epoch": 2.79,
2799
+ "grad_norm": 3.0827438831329346,
2800
+ "learning_rate": 6.029992107340175e-05,
2801
+ "loss": 0.2195,
2802
+ "step": 3540
2803
+ },
2804
+ {
2805
+ "epoch": 2.8,
2806
+ "grad_norm": 2.7207252979278564,
2807
+ "learning_rate": 5.9905288082083663e-05,
2808
+ "loss": 0.2549,
2809
+ "step": 3550
2810
+ },
2811
+ {
2812
+ "epoch": 2.81,
2813
+ "grad_norm": 2.6517016887664795,
2814
+ "learning_rate": 5.951065509076559e-05,
2815
+ "loss": 0.2294,
2816
+ "step": 3560
2817
+ },
2818
+ {
2819
+ "epoch": 2.82,
2820
+ "grad_norm": 1.5923064947128296,
2821
+ "learning_rate": 5.9116022099447524e-05,
2822
+ "loss": 0.2465,
2823
+ "step": 3570
2824
+ },
2825
+ {
2826
+ "epoch": 2.83,
2827
+ "grad_norm": 4.049023151397705,
2828
+ "learning_rate": 5.872138910812944e-05,
2829
+ "loss": 0.2712,
2830
+ "step": 3580
2831
+ },
2832
+ {
2833
+ "epoch": 2.83,
2834
+ "grad_norm": 0.7295699119567871,
2835
+ "learning_rate": 5.8326756116811364e-05,
2836
+ "loss": 0.3309,
2837
+ "step": 3590
2838
+ },
2839
+ {
2840
+ "epoch": 2.84,
2841
+ "grad_norm": 3.531484365463257,
2842
+ "learning_rate": 5.79321231254933e-05,
2843
+ "loss": 0.3685,
2844
+ "step": 3600
2845
+ },
2846
+ {
2847
+ "epoch": 2.84,
2848
+ "eval_accuracy": 0.8016577856719953,
2849
+ "eval_loss": 0.6276537179946899,
2850
+ "eval_runtime": 92.0744,
2851
+ "eval_samples_per_second": 55.032,
2852
+ "eval_steps_per_second": 6.886,
2853
+ "step": 3600
2854
+ },
2855
+ {
2856
+ "epoch": 2.85,
2857
+ "grad_norm": 2.7010326385498047,
2858
+ "learning_rate": 5.753749013417522e-05,
2859
+ "loss": 0.1898,
2860
+ "step": 3610
2861
+ },
2862
+ {
2863
+ "epoch": 2.86,
2864
+ "grad_norm": 2.8486061096191406,
2865
+ "learning_rate": 5.714285714285714e-05,
2866
+ "loss": 0.2965,
2867
+ "step": 3620
2868
+ },
2869
+ {
2870
+ "epoch": 2.87,
2871
+ "grad_norm": 5.980003833770752,
2872
+ "learning_rate": 5.674822415153908e-05,
2873
+ "loss": 0.297,
2874
+ "step": 3630
2875
+ },
2876
+ {
2877
+ "epoch": 2.87,
2878
+ "grad_norm": 0.3644309639930725,
2879
+ "learning_rate": 5.6353591160220996e-05,
2880
+ "loss": 0.2627,
2881
+ "step": 3640
2882
+ },
2883
+ {
2884
+ "epoch": 2.88,
2885
+ "grad_norm": 4.6136064529418945,
2886
+ "learning_rate": 5.595895816890292e-05,
2887
+ "loss": 0.3327,
2888
+ "step": 3650
2889
+ },
2890
+ {
2891
+ "epoch": 2.89,
2892
+ "grad_norm": 1.8576825857162476,
2893
+ "learning_rate": 5.556432517758485e-05,
2894
+ "loss": 0.2064,
2895
+ "step": 3660
2896
+ },
2897
+ {
2898
+ "epoch": 2.9,
2899
+ "grad_norm": 3.2825279235839844,
2900
+ "learning_rate": 5.516969218626677e-05,
2901
+ "loss": 0.2582,
2902
+ "step": 3670
2903
+ },
2904
+ {
2905
+ "epoch": 2.9,
2906
+ "grad_norm": 4.109126567840576,
2907
+ "learning_rate": 5.4775059194948696e-05,
2908
+ "loss": 0.2412,
2909
+ "step": 3680
2910
+ },
2911
+ {
2912
+ "epoch": 2.91,
2913
+ "grad_norm": 0.9957514405250549,
2914
+ "learning_rate": 5.438042620363063e-05,
2915
+ "loss": 0.1818,
2916
+ "step": 3690
2917
+ },
2918
+ {
2919
+ "epoch": 2.92,
2920
+ "grad_norm": 1.8618041276931763,
2921
+ "learning_rate": 5.398579321231255e-05,
2922
+ "loss": 0.2299,
2923
+ "step": 3700
2924
+ },
2925
+ {
2926
+ "epoch": 2.92,
2927
+ "eval_accuracy": 0.8125123347148214,
2928
+ "eval_loss": 0.5834479928016663,
2929
+ "eval_runtime": 91.926,
2930
+ "eval_samples_per_second": 55.12,
2931
+ "eval_steps_per_second": 6.897,
2932
+ "step": 3700
2933
+ },
2934
+ {
2935
+ "epoch": 2.93,
2936
+ "grad_norm": 3.655374526977539,
2937
+ "learning_rate": 5.3591160220994474e-05,
2938
+ "loss": 0.3183,
2939
+ "step": 3710
2940
+ },
2941
+ {
2942
+ "epoch": 2.94,
2943
+ "grad_norm": 2.711798906326294,
2944
+ "learning_rate": 5.3196527229676404e-05,
2945
+ "loss": 0.3336,
2946
+ "step": 3720
2947
+ },
2948
+ {
2949
+ "epoch": 2.94,
2950
+ "grad_norm": 4.209230422973633,
2951
+ "learning_rate": 5.280189423835833e-05,
2952
+ "loss": 0.2944,
2953
+ "step": 3730
2954
+ },
2955
+ {
2956
+ "epoch": 2.95,
2957
+ "grad_norm": 1.0983171463012695,
2958
+ "learning_rate": 5.240726124704025e-05,
2959
+ "loss": 0.2846,
2960
+ "step": 3740
2961
+ },
2962
+ {
2963
+ "epoch": 2.96,
2964
+ "grad_norm": 2.4771692752838135,
2965
+ "learning_rate": 5.201262825572218e-05,
2966
+ "loss": 0.3192,
2967
+ "step": 3750
2968
+ },
2969
+ {
2970
+ "epoch": 2.97,
2971
+ "grad_norm": 2.7510647773742676,
2972
+ "learning_rate": 5.1617995264404105e-05,
2973
+ "loss": 0.1813,
2974
+ "step": 3760
2975
+ },
2976
+ {
2977
+ "epoch": 2.98,
2978
+ "grad_norm": 3.2778220176696777,
2979
+ "learning_rate": 5.122336227308603e-05,
2980
+ "loss": 0.1935,
2981
+ "step": 3770
2982
+ },
2983
+ {
2984
+ "epoch": 2.98,
2985
+ "grad_norm": 2.6012797355651855,
2986
+ "learning_rate": 5.082872928176796e-05,
2987
+ "loss": 0.2111,
2988
+ "step": 3780
2989
+ },
2990
+ {
2991
+ "epoch": 2.99,
2992
+ "grad_norm": 2.29512357711792,
2993
+ "learning_rate": 5.043409629044988e-05,
2994
+ "loss": 0.2987,
2995
+ "step": 3790
2996
+ },
2997
+ {
2998
+ "epoch": 3.0,
2999
+ "grad_norm": 5.790306568145752,
3000
+ "learning_rate": 5.0039463299131806e-05,
3001
+ "loss": 0.3414,
3002
+ "step": 3800
3003
+ },
3004
+ {
3005
+ "epoch": 3.0,
3006
+ "eval_accuracy": 0.8166567988948096,
3007
+ "eval_loss": 0.5749832987785339,
3008
+ "eval_runtime": 92.3841,
3009
+ "eval_samples_per_second": 54.847,
3010
+ "eval_steps_per_second": 6.863,
3011
+ "step": 3800
3012
+ },
3013
+ {
3014
+ "epoch": 3.01,
3015
+ "grad_norm": 4.339576721191406,
3016
+ "learning_rate": 4.9644830307813736e-05,
3017
+ "loss": 0.1144,
3018
+ "step": 3810
3019
+ },
3020
+ {
3021
+ "epoch": 3.01,
3022
+ "grad_norm": 0.7903636693954468,
3023
+ "learning_rate": 4.925019731649566e-05,
3024
+ "loss": 0.0767,
3025
+ "step": 3820
3026
+ },
3027
+ {
3028
+ "epoch": 3.02,
3029
+ "grad_norm": 2.8627429008483887,
3030
+ "learning_rate": 4.885556432517759e-05,
3031
+ "loss": 0.0859,
3032
+ "step": 3830
3033
+ },
3034
+ {
3035
+ "epoch": 3.03,
3036
+ "grad_norm": 4.783523082733154,
3037
+ "learning_rate": 4.8460931333859514e-05,
3038
+ "loss": 0.1071,
3039
+ "step": 3840
3040
+ },
3041
+ {
3042
+ "epoch": 3.04,
3043
+ "grad_norm": 0.7647588849067688,
3044
+ "learning_rate": 4.806629834254144e-05,
3045
+ "loss": 0.1531,
3046
+ "step": 3850
3047
+ },
3048
+ {
3049
+ "epoch": 3.05,
3050
+ "grad_norm": 2.228163719177246,
3051
+ "learning_rate": 4.767166535122337e-05,
3052
+ "loss": 0.1582,
3053
+ "step": 3860
3054
+ },
3055
+ {
3056
+ "epoch": 3.05,
3057
+ "grad_norm": 0.918712854385376,
3058
+ "learning_rate": 4.727703235990529e-05,
3059
+ "loss": 0.1197,
3060
+ "step": 3870
3061
+ },
3062
+ {
3063
+ "epoch": 3.06,
3064
+ "grad_norm": 4.140162467956543,
3065
+ "learning_rate": 4.6882399368587215e-05,
3066
+ "loss": 0.0733,
3067
+ "step": 3880
3068
+ },
3069
+ {
3070
+ "epoch": 3.07,
3071
+ "grad_norm": 2.8325231075286865,
3072
+ "learning_rate": 4.6487766377269145e-05,
3073
+ "loss": 0.1651,
3074
+ "step": 3890
3075
+ },
3076
+ {
3077
+ "epoch": 3.08,
3078
+ "grad_norm": 0.49401429295539856,
3079
+ "learning_rate": 4.609313338595106e-05,
3080
+ "loss": 0.1082,
3081
+ "step": 3900
3082
+ },
3083
+ {
3084
+ "epoch": 3.08,
3085
+ "eval_accuracy": 0.819617130451944,
3086
+ "eval_loss": 0.6201203465461731,
3087
+ "eval_runtime": 92.3164,
3088
+ "eval_samples_per_second": 54.887,
3089
+ "eval_steps_per_second": 6.868,
3090
+ "step": 3900
3091
+ },
3092
+ {
3093
+ "epoch": 3.09,
3094
+ "grad_norm": 0.34822145104408264,
3095
+ "learning_rate": 4.569850039463299e-05,
3096
+ "loss": 0.0457,
3097
+ "step": 3910
3098
+ },
3099
+ {
3100
+ "epoch": 3.09,
3101
+ "grad_norm": 1.0006756782531738,
3102
+ "learning_rate": 4.530386740331492e-05,
3103
+ "loss": 0.1194,
3104
+ "step": 3920
3105
+ },
3106
+ {
3107
+ "epoch": 3.1,
3108
+ "grad_norm": 0.7993775010108948,
3109
+ "learning_rate": 4.4909234411996846e-05,
3110
+ "loss": 0.1325,
3111
+ "step": 3930
3112
+ },
3113
+ {
3114
+ "epoch": 3.11,
3115
+ "grad_norm": 6.907358646392822,
3116
+ "learning_rate": 4.451460142067877e-05,
3117
+ "loss": 0.0605,
3118
+ "step": 3940
3119
+ },
3120
+ {
3121
+ "epoch": 3.12,
3122
+ "grad_norm": 1.2168715000152588,
3123
+ "learning_rate": 4.41199684293607e-05,
3124
+ "loss": 0.0968,
3125
+ "step": 3950
3126
+ },
3127
+ {
3128
+ "epoch": 3.13,
3129
+ "grad_norm": 0.48231208324432373,
3130
+ "learning_rate": 4.372533543804262e-05,
3131
+ "loss": 0.0868,
3132
+ "step": 3960
3133
+ },
3134
+ {
3135
+ "epoch": 3.13,
3136
+ "grad_norm": 0.09531491994857788,
3137
+ "learning_rate": 4.333070244672455e-05,
3138
+ "loss": 0.1161,
3139
+ "step": 3970
3140
+ },
3141
+ {
3142
+ "epoch": 3.14,
3143
+ "grad_norm": 3.662717342376709,
3144
+ "learning_rate": 4.293606945540648e-05,
3145
+ "loss": 0.1188,
3146
+ "step": 3980
3147
+ },
3148
+ {
3149
+ "epoch": 3.15,
3150
+ "grad_norm": 8.440380096435547,
3151
+ "learning_rate": 4.25414364640884e-05,
3152
+ "loss": 0.1319,
3153
+ "step": 3990
3154
+ },
3155
+ {
3156
+ "epoch": 3.16,
3157
+ "grad_norm": 2.019827127456665,
3158
+ "learning_rate": 4.2146803472770324e-05,
3159
+ "loss": 0.049,
3160
+ "step": 4000
3161
+ },
3162
+ {
3163
+ "epoch": 3.16,
3164
+ "eval_accuracy": 0.8160647325833826,
3165
+ "eval_loss": 0.6474657654762268,
3166
+ "eval_runtime": 92.4783,
3167
+ "eval_samples_per_second": 54.791,
3168
+ "eval_steps_per_second": 6.856,
3169
+ "step": 4000
3170
+ },
3171
+ {
3172
+ "epoch": 3.16,
3173
+ "grad_norm": 4.42202615737915,
3174
+ "learning_rate": 4.1752170481452254e-05,
3175
+ "loss": 0.0794,
3176
+ "step": 4010
3177
+ },
3178
+ {
3179
+ "epoch": 3.17,
3180
+ "grad_norm": 4.54102087020874,
3181
+ "learning_rate": 4.135753749013418e-05,
3182
+ "loss": 0.1152,
3183
+ "step": 4020
3184
+ },
3185
+ {
3186
+ "epoch": 3.18,
3187
+ "grad_norm": 0.31936466693878174,
3188
+ "learning_rate": 4.09629044988161e-05,
3189
+ "loss": 0.0945,
3190
+ "step": 4030
3191
+ },
3192
+ {
3193
+ "epoch": 3.19,
3194
+ "grad_norm": 0.12120810896158218,
3195
+ "learning_rate": 4.056827150749803e-05,
3196
+ "loss": 0.1196,
3197
+ "step": 4040
3198
+ },
3199
+ {
3200
+ "epoch": 3.2,
3201
+ "grad_norm": 0.3144413232803345,
3202
+ "learning_rate": 4.0173638516179955e-05,
3203
+ "loss": 0.1226,
3204
+ "step": 4050
3205
+ },
3206
+ {
3207
+ "epoch": 3.2,
3208
+ "grad_norm": 5.312731742858887,
3209
+ "learning_rate": 3.977900552486188e-05,
3210
+ "loss": 0.1807,
3211
+ "step": 4060
3212
+ },
3213
+ {
3214
+ "epoch": 3.21,
3215
+ "grad_norm": 0.05327579751610756,
3216
+ "learning_rate": 3.938437253354381e-05,
3217
+ "loss": 0.1521,
3218
+ "step": 4070
3219
+ },
3220
+ {
3221
+ "epoch": 3.22,
3222
+ "grad_norm": 1.8903406858444214,
3223
+ "learning_rate": 3.898973954222573e-05,
3224
+ "loss": 0.1712,
3225
+ "step": 4080
3226
+ },
3227
+ {
3228
+ "epoch": 3.23,
3229
+ "grad_norm": 0.07272280752658844,
3230
+ "learning_rate": 3.8595106550907656e-05,
3231
+ "loss": 0.0731,
3232
+ "step": 4090
3233
+ },
3234
+ {
3235
+ "epoch": 3.24,
3236
+ "grad_norm": 0.41295164823532104,
3237
+ "learning_rate": 3.8200473559589587e-05,
3238
+ "loss": 0.102,
3239
+ "step": 4100
3240
+ },
3241
+ {
3242
+ "epoch": 3.24,
3243
+ "eval_accuracy": 0.8097493585948293,
3244
+ "eval_loss": 0.6791070103645325,
3245
+ "eval_runtime": 92.1285,
3246
+ "eval_samples_per_second": 54.999,
3247
+ "eval_steps_per_second": 6.882,
3248
+ "step": 4100
3249
+ },
3250
+ {
3251
+ "epoch": 3.24,
3252
+ "grad_norm": 5.161396026611328,
3253
+ "learning_rate": 3.780584056827151e-05,
3254
+ "loss": 0.0824,
3255
+ "step": 4110
3256
+ },
3257
+ {
3258
+ "epoch": 3.25,
3259
+ "grad_norm": 0.2593359053134918,
3260
+ "learning_rate": 3.7411207576953434e-05,
3261
+ "loss": 0.1567,
3262
+ "step": 4120
3263
+ },
3264
+ {
3265
+ "epoch": 3.26,
3266
+ "grad_norm": 1.5720149278640747,
3267
+ "learning_rate": 3.7016574585635364e-05,
3268
+ "loss": 0.1422,
3269
+ "step": 4130
3270
+ },
3271
+ {
3272
+ "epoch": 3.27,
3273
+ "grad_norm": 6.186444282531738,
3274
+ "learning_rate": 3.662194159431729e-05,
3275
+ "loss": 0.0811,
3276
+ "step": 4140
3277
+ },
3278
+ {
3279
+ "epoch": 3.28,
3280
+ "grad_norm": 1.9182838201522827,
3281
+ "learning_rate": 3.622730860299921e-05,
3282
+ "loss": 0.1454,
3283
+ "step": 4150
3284
+ },
3285
+ {
3286
+ "epoch": 3.28,
3287
+ "grad_norm": 3.1142055988311768,
3288
+ "learning_rate": 3.583267561168114e-05,
3289
+ "loss": 0.1304,
3290
+ "step": 4160
3291
+ },
3292
+ {
3293
+ "epoch": 3.29,
3294
+ "grad_norm": 0.17880991101264954,
3295
+ "learning_rate": 3.5438042620363065e-05,
3296
+ "loss": 0.0791,
3297
+ "step": 4170
3298
+ },
3299
+ {
3300
+ "epoch": 3.3,
3301
+ "grad_norm": 3.2896628379821777,
3302
+ "learning_rate": 3.504340962904499e-05,
3303
+ "loss": 0.0903,
3304
+ "step": 4180
3305
+ },
3306
+ {
3307
+ "epoch": 3.31,
3308
+ "grad_norm": 3.1752073764801025,
3309
+ "learning_rate": 3.464877663772691e-05,
3310
+ "loss": 0.0988,
3311
+ "step": 4190
3312
+ },
3313
+ {
3314
+ "epoch": 3.31,
3315
+ "grad_norm": 1.282556414604187,
3316
+ "learning_rate": 3.425414364640884e-05,
3317
+ "loss": 0.0483,
3318
+ "step": 4200
3319
+ },
3320
+ {
3321
+ "epoch": 3.31,
3322
+ "eval_accuracy": 0.8215906848233668,
3323
+ "eval_loss": 0.6581845283508301,
3324
+ "eval_runtime": 92.2168,
3325
+ "eval_samples_per_second": 54.947,
3326
+ "eval_steps_per_second": 6.875,
3327
+ "step": 4200
3328
+ },
3329
+ {
3330
+ "epoch": 3.32,
3331
+ "grad_norm": 5.438603401184082,
3332
+ "learning_rate": 3.3859510655090766e-05,
3333
+ "loss": 0.084,
3334
+ "step": 4210
3335
+ },
3336
+ {
3337
+ "epoch": 3.33,
3338
+ "grad_norm": 0.6808891296386719,
3339
+ "learning_rate": 3.346487766377269e-05,
3340
+ "loss": 0.1193,
3341
+ "step": 4220
3342
+ },
3343
+ {
3344
+ "epoch": 3.34,
3345
+ "grad_norm": 0.4623468220233917,
3346
+ "learning_rate": 3.307024467245462e-05,
3347
+ "loss": 0.0575,
3348
+ "step": 4230
3349
+ },
3350
+ {
3351
+ "epoch": 3.35,
3352
+ "grad_norm": 0.5891864895820618,
3353
+ "learning_rate": 3.267561168113654e-05,
3354
+ "loss": 0.0353,
3355
+ "step": 4240
3356
+ },
3357
+ {
3358
+ "epoch": 3.35,
3359
+ "grad_norm": 3.8541829586029053,
3360
+ "learning_rate": 3.2280978689818467e-05,
3361
+ "loss": 0.1207,
3362
+ "step": 4250
3363
+ },
3364
+ {
3365
+ "epoch": 3.36,
3366
+ "grad_norm": 0.4876788854598999,
3367
+ "learning_rate": 3.18863456985004e-05,
3368
+ "loss": 0.0386,
3369
+ "step": 4260
3370
+ },
3371
+ {
3372
+ "epoch": 3.37,
3373
+ "grad_norm": 1.7822741270065308,
3374
+ "learning_rate": 3.149171270718232e-05,
3375
+ "loss": 0.0753,
3376
+ "step": 4270
3377
+ },
3378
+ {
3379
+ "epoch": 3.38,
3380
+ "grad_norm": 0.6429558396339417,
3381
+ "learning_rate": 3.1097079715864244e-05,
3382
+ "loss": 0.0799,
3383
+ "step": 4280
3384
+ },
3385
+ {
3386
+ "epoch": 3.39,
3387
+ "grad_norm": 6.067298889160156,
3388
+ "learning_rate": 3.0702446724546174e-05,
3389
+ "loss": 0.1465,
3390
+ "step": 4290
3391
+ },
3392
+ {
3393
+ "epoch": 3.39,
3394
+ "grad_norm": 3.5951452255249023,
3395
+ "learning_rate": 3.03078137332281e-05,
3396
+ "loss": 0.1204,
3397
+ "step": 4300
3398
+ },
3399
+ {
3400
+ "epoch": 3.39,
3401
+ "eval_accuracy": 0.8221827511347938,
3402
+ "eval_loss": 0.6602674722671509,
3403
+ "eval_runtime": 91.9479,
3404
+ "eval_samples_per_second": 55.107,
3405
+ "eval_steps_per_second": 6.895,
3406
+ "step": 4300
3407
+ },
3408
+ {
3409
+ "epoch": 3.4,
3410
+ "grad_norm": 0.12355990707874298,
3411
+ "learning_rate": 2.9913180741910025e-05,
3412
+ "loss": 0.0873,
3413
+ "step": 4310
3414
+ },
3415
+ {
3416
+ "epoch": 3.41,
3417
+ "grad_norm": 1.412264347076416,
3418
+ "learning_rate": 2.951854775059195e-05,
3419
+ "loss": 0.0222,
3420
+ "step": 4320
3421
+ },
3422
+ {
3423
+ "epoch": 3.42,
3424
+ "grad_norm": 0.25506913661956787,
3425
+ "learning_rate": 2.912391475927388e-05,
3426
+ "loss": 0.049,
3427
+ "step": 4330
3428
+ },
3429
+ {
3430
+ "epoch": 3.43,
3431
+ "grad_norm": 1.5517828464508057,
3432
+ "learning_rate": 2.8729281767955802e-05,
3433
+ "loss": 0.0323,
3434
+ "step": 4340
3435
+ },
3436
+ {
3437
+ "epoch": 3.43,
3438
+ "grad_norm": 9.105844497680664,
3439
+ "learning_rate": 2.833464877663773e-05,
3440
+ "loss": 0.1014,
3441
+ "step": 4350
3442
+ },
3443
+ {
3444
+ "epoch": 3.44,
3445
+ "grad_norm": 0.06447356194257736,
3446
+ "learning_rate": 2.7940015785319656e-05,
3447
+ "loss": 0.049,
3448
+ "step": 4360
3449
+ },
3450
+ {
3451
+ "epoch": 3.45,
3452
+ "grad_norm": 6.344263076782227,
3453
+ "learning_rate": 2.754538279400158e-05,
3454
+ "loss": 0.0841,
3455
+ "step": 4370
3456
+ },
3457
+ {
3458
+ "epoch": 3.46,
3459
+ "grad_norm": 0.45002487301826477,
3460
+ "learning_rate": 2.7150749802683506e-05,
3461
+ "loss": 0.1012,
3462
+ "step": 4380
3463
+ },
3464
+ {
3465
+ "epoch": 3.46,
3466
+ "grad_norm": 0.10588052868843079,
3467
+ "learning_rate": 2.6756116811365433e-05,
3468
+ "loss": 0.0076,
3469
+ "step": 4390
3470
+ },
3471
+ {
3472
+ "epoch": 3.47,
3473
+ "grad_norm": 3.3592453002929688,
3474
+ "learning_rate": 2.6361483820047357e-05,
3475
+ "loss": 0.0611,
3476
+ "step": 4400
3477
+ },
3478
+ {
3479
+ "epoch": 3.47,
3480
+ "eval_accuracy": 0.8190250641405171,
3481
+ "eval_loss": 0.717377781867981,
3482
+ "eval_runtime": 91.8985,
3483
+ "eval_samples_per_second": 55.137,
3484
+ "eval_steps_per_second": 6.899,
3485
+ "step": 4400
3486
+ },
3487
+ {
3488
+ "epoch": 3.48,
3489
+ "grad_norm": 0.4130847156047821,
3490
+ "learning_rate": 2.5966850828729284e-05,
3491
+ "loss": 0.1135,
3492
+ "step": 4410
3493
+ },
3494
+ {
3495
+ "epoch": 3.49,
3496
+ "grad_norm": 4.881414890289307,
3497
+ "learning_rate": 2.557221783741121e-05,
3498
+ "loss": 0.1324,
3499
+ "step": 4420
3500
+ },
3501
+ {
3502
+ "epoch": 3.5,
3503
+ "grad_norm": 2.830336332321167,
3504
+ "learning_rate": 2.5177584846093134e-05,
3505
+ "loss": 0.0686,
3506
+ "step": 4430
3507
+ },
3508
+ {
3509
+ "epoch": 3.5,
3510
+ "grad_norm": 0.41728538274765015,
3511
+ "learning_rate": 2.478295185477506e-05,
3512
+ "loss": 0.0649,
3513
+ "step": 4440
3514
+ },
3515
+ {
3516
+ "epoch": 3.51,
3517
+ "grad_norm": 2.339663505554199,
3518
+ "learning_rate": 2.4388318863456985e-05,
3519
+ "loss": 0.1187,
3520
+ "step": 4450
3521
+ },
3522
+ {
3523
+ "epoch": 3.52,
3524
+ "grad_norm": 0.06545936316251755,
3525
+ "learning_rate": 2.399368587213891e-05,
3526
+ "loss": 0.0995,
3527
+ "step": 4460
3528
+ },
3529
+ {
3530
+ "epoch": 3.53,
3531
+ "grad_norm": 0.9462890625,
3532
+ "learning_rate": 2.359905288082084e-05,
3533
+ "loss": 0.0796,
3534
+ "step": 4470
3535
+ },
3536
+ {
3537
+ "epoch": 3.54,
3538
+ "grad_norm": 3.4971237182617188,
3539
+ "learning_rate": 2.3204419889502762e-05,
3540
+ "loss": 0.062,
3541
+ "step": 4480
3542
+ },
3543
+ {
3544
+ "epoch": 3.54,
3545
+ "grad_norm": 3.370915412902832,
3546
+ "learning_rate": 2.280978689818469e-05,
3547
+ "loss": 0.1375,
3548
+ "step": 4490
3549
+ },
3550
+ {
3551
+ "epoch": 3.55,
3552
+ "grad_norm": 0.047411490231752396,
3553
+ "learning_rate": 2.2415153906866616e-05,
3554
+ "loss": 0.0555,
3555
+ "step": 4500
3556
+ },
3557
+ {
3558
+ "epoch": 3.55,
3559
+ "eval_accuracy": 0.8235642391947898,
3560
+ "eval_loss": 0.6841058135032654,
3561
+ "eval_runtime": 92.265,
3562
+ "eval_samples_per_second": 54.918,
3563
+ "eval_steps_per_second": 6.872,
3564
+ "step": 4500
3565
+ },
3566
+ {
3567
+ "epoch": 3.56,
3568
+ "grad_norm": 9.102856636047363,
3569
+ "learning_rate": 2.202052091554854e-05,
3570
+ "loss": 0.0936,
3571
+ "step": 4510
3572
+ },
3573
+ {
3574
+ "epoch": 3.57,
3575
+ "grad_norm": 1.5080801248550415,
3576
+ "learning_rate": 2.1625887924230466e-05,
3577
+ "loss": 0.0577,
3578
+ "step": 4520
3579
+ },
3580
+ {
3581
+ "epoch": 3.58,
3582
+ "grad_norm": 1.3888694047927856,
3583
+ "learning_rate": 2.1231254932912393e-05,
3584
+ "loss": 0.1027,
3585
+ "step": 4530
3586
+ },
3587
+ {
3588
+ "epoch": 3.58,
3589
+ "grad_norm": 3.6194889545440674,
3590
+ "learning_rate": 2.0836621941594317e-05,
3591
+ "loss": 0.0755,
3592
+ "step": 4540
3593
+ },
3594
+ {
3595
+ "epoch": 3.59,
3596
+ "grad_norm": 0.7071271538734436,
3597
+ "learning_rate": 2.0441988950276244e-05,
3598
+ "loss": 0.0276,
3599
+ "step": 4550
3600
+ },
3601
+ {
3602
+ "epoch": 3.6,
3603
+ "grad_norm": 0.07247108966112137,
3604
+ "learning_rate": 2.004735595895817e-05,
3605
+ "loss": 0.1272,
3606
+ "step": 4560
3607
+ },
3608
+ {
3609
+ "epoch": 3.61,
3610
+ "grad_norm": 1.932975172996521,
3611
+ "learning_rate": 1.9652722967640098e-05,
3612
+ "loss": 0.1341,
3613
+ "step": 4570
3614
+ },
3615
+ {
3616
+ "epoch": 3.61,
3617
+ "grad_norm": 0.6965919137001038,
3618
+ "learning_rate": 1.925808997632202e-05,
3619
+ "loss": 0.0614,
3620
+ "step": 4580
3621
+ },
3622
+ {
3623
+ "epoch": 3.62,
3624
+ "grad_norm": 3.4213573932647705,
3625
+ "learning_rate": 1.8863456985003948e-05,
3626
+ "loss": 0.0764,
3627
+ "step": 4590
3628
+ },
3629
+ {
3630
+ "epoch": 3.63,
3631
+ "grad_norm": 0.1402810513973236,
3632
+ "learning_rate": 1.8468823993685875e-05,
3633
+ "loss": 0.0188,
3634
+ "step": 4600
3635
+ },
3636
+ {
3637
+ "epoch": 3.63,
3638
+ "eval_accuracy": 0.8239589500690744,
3639
+ "eval_loss": 0.7009280920028687,
3640
+ "eval_runtime": 92.5146,
3641
+ "eval_samples_per_second": 54.77,
3642
+ "eval_steps_per_second": 6.853,
3643
+ "step": 4600
3644
+ },
3645
+ {
3646
+ "epoch": 3.64,
3647
+ "grad_norm": 0.07443057000637054,
3648
+ "learning_rate": 1.80741910023678e-05,
3649
+ "loss": 0.0846,
3650
+ "step": 4610
3651
+ },
3652
+ {
3653
+ "epoch": 3.65,
3654
+ "grad_norm": 0.20932039618492126,
3655
+ "learning_rate": 1.7679558011049725e-05,
3656
+ "loss": 0.0747,
3657
+ "step": 4620
3658
+ },
3659
+ {
3660
+ "epoch": 3.65,
3661
+ "grad_norm": 0.04756965488195419,
3662
+ "learning_rate": 1.7284925019731652e-05,
3663
+ "loss": 0.0628,
3664
+ "step": 4630
3665
+ },
3666
+ {
3667
+ "epoch": 3.66,
3668
+ "grad_norm": 13.026732444763184,
3669
+ "learning_rate": 1.6890292028413576e-05,
3670
+ "loss": 0.1348,
3671
+ "step": 4640
3672
+ },
3673
+ {
3674
+ "epoch": 3.67,
3675
+ "grad_norm": 0.08388219773769379,
3676
+ "learning_rate": 1.64956590370955e-05,
3677
+ "loss": 0.0186,
3678
+ "step": 4650
3679
+ },
3680
+ {
3681
+ "epoch": 3.68,
3682
+ "grad_norm": 0.08429060131311417,
3683
+ "learning_rate": 1.610102604577743e-05,
3684
+ "loss": 0.0342,
3685
+ "step": 4660
3686
+ },
3687
+ {
3688
+ "epoch": 3.69,
3689
+ "grad_norm": 0.22585107386112213,
3690
+ "learning_rate": 1.5706393054459353e-05,
3691
+ "loss": 0.0371,
3692
+ "step": 4670
3693
+ },
3694
+ {
3695
+ "epoch": 3.69,
3696
+ "grad_norm": 8.003403663635254,
3697
+ "learning_rate": 1.5311760063141277e-05,
3698
+ "loss": 0.1692,
3699
+ "step": 4680
3700
+ },
3701
+ {
3702
+ "epoch": 3.7,
3703
+ "grad_norm": 1.522824764251709,
3704
+ "learning_rate": 1.4917127071823205e-05,
3705
+ "loss": 0.0612,
3706
+ "step": 4690
3707
+ },
3708
+ {
3709
+ "epoch": 3.71,
3710
+ "grad_norm": 2.2803661823272705,
3711
+ "learning_rate": 1.452249408050513e-05,
3712
+ "loss": 0.1292,
3713
+ "step": 4700
3714
+ },
3715
+ {
3716
+ "epoch": 3.71,
3717
+ "eval_accuracy": 0.8204065522005132,
3718
+ "eval_loss": 0.703969419002533,
3719
+ "eval_runtime": 92.794,
3720
+ "eval_samples_per_second": 54.605,
3721
+ "eval_steps_per_second": 6.832,
3722
+ "step": 4700
3723
+ },
3724
+ {
3725
+ "epoch": 3.72,
3726
+ "grad_norm": 6.225048065185547,
3727
+ "learning_rate": 1.4127861089187056e-05,
3728
+ "loss": 0.0557,
3729
+ "step": 4710
3730
+ },
3731
+ {
3732
+ "epoch": 3.73,
3733
+ "grad_norm": 0.4492289423942566,
3734
+ "learning_rate": 1.3733228097868983e-05,
3735
+ "loss": 0.1578,
3736
+ "step": 4720
3737
+ },
3738
+ {
3739
+ "epoch": 3.73,
3740
+ "grad_norm": 1.797982096672058,
3741
+ "learning_rate": 1.3338595106550908e-05,
3742
+ "loss": 0.019,
3743
+ "step": 4730
3744
+ },
3745
+ {
3746
+ "epoch": 3.74,
3747
+ "grad_norm": 0.4045000970363617,
3748
+ "learning_rate": 1.2943962115232833e-05,
3749
+ "loss": 0.0614,
3750
+ "step": 4740
3751
+ },
3752
+ {
3753
+ "epoch": 3.75,
3754
+ "grad_norm": 0.08240614086389542,
3755
+ "learning_rate": 1.254932912391476e-05,
3756
+ "loss": 0.1421,
3757
+ "step": 4750
3758
+ },
3759
+ {
3760
+ "epoch": 3.76,
3761
+ "grad_norm": 4.642998695373535,
3762
+ "learning_rate": 1.2154696132596685e-05,
3763
+ "loss": 0.0562,
3764
+ "step": 4760
3765
+ },
3766
+ {
3767
+ "epoch": 3.76,
3768
+ "grad_norm": 0.16091428697109222,
3769
+ "learning_rate": 1.176006314127861e-05,
3770
+ "loss": 0.0638,
3771
+ "step": 4770
3772
+ },
3773
+ {
3774
+ "epoch": 3.77,
3775
+ "grad_norm": 1.0345463752746582,
3776
+ "learning_rate": 1.1365430149960538e-05,
3777
+ "loss": 0.0634,
3778
+ "step": 4780
3779
+ },
3780
+ {
3781
+ "epoch": 3.78,
3782
+ "grad_norm": 0.042663924396038055,
3783
+ "learning_rate": 1.0970797158642463e-05,
3784
+ "loss": 0.1053,
3785
+ "step": 4790
3786
+ },
3787
+ {
3788
+ "epoch": 3.79,
3789
+ "grad_norm": 1.7890504598617554,
3790
+ "learning_rate": 1.0576164167324388e-05,
3791
+ "loss": 0.0661,
3792
+ "step": 4800
3793
+ },
3794
+ {
3795
+ "epoch": 3.79,
3796
+ "eval_accuracy": 0.8237615946319321,
3797
+ "eval_loss": 0.7073862552642822,
3798
+ "eval_runtime": 91.8421,
3799
+ "eval_samples_per_second": 55.171,
3800
+ "eval_steps_per_second": 6.903,
3801
+ "step": 4800
3802
+ },
3803
+ {
3804
+ "epoch": 3.8,
3805
+ "grad_norm": 0.08139993995428085,
3806
+ "learning_rate": 1.0181531176006315e-05,
3807
+ "loss": 0.0623,
3808
+ "step": 4810
3809
+ },
3810
+ {
3811
+ "epoch": 3.8,
3812
+ "grad_norm": 0.02715960517525673,
3813
+ "learning_rate": 9.786898184688242e-06,
3814
+ "loss": 0.0532,
3815
+ "step": 4820
3816
+ },
3817
+ {
3818
+ "epoch": 3.81,
3819
+ "grad_norm": 5.507246494293213,
3820
+ "learning_rate": 9.392265193370165e-06,
3821
+ "loss": 0.0829,
3822
+ "step": 4830
3823
+ },
3824
+ {
3825
+ "epoch": 3.82,
3826
+ "grad_norm": 0.11962206661701202,
3827
+ "learning_rate": 8.997632202052092e-06,
3828
+ "loss": 0.0887,
3829
+ "step": 4840
3830
+ },
3831
+ {
3832
+ "epoch": 3.83,
3833
+ "grad_norm": 0.14435319602489471,
3834
+ "learning_rate": 8.602999210734018e-06,
3835
+ "loss": 0.0478,
3836
+ "step": 4850
3837
+ },
3838
+ {
3839
+ "epoch": 3.84,
3840
+ "grad_norm": 4.151905536651611,
3841
+ "learning_rate": 8.208366219415943e-06,
3842
+ "loss": 0.1795,
3843
+ "step": 4860
3844
+ },
3845
+ {
3846
+ "epoch": 3.84,
3847
+ "grad_norm": 0.10827145725488663,
3848
+ "learning_rate": 7.81373322809787e-06,
3849
+ "loss": 0.0566,
3850
+ "step": 4870
3851
+ },
3852
+ {
3853
+ "epoch": 3.85,
3854
+ "grad_norm": 0.5536698698997498,
3855
+ "learning_rate": 7.419100236779796e-06,
3856
+ "loss": 0.0602,
3857
+ "step": 4880
3858
+ },
3859
+ {
3860
+ "epoch": 3.86,
3861
+ "grad_norm": 0.1482585370540619,
3862
+ "learning_rate": 7.02446724546172e-06,
3863
+ "loss": 0.0995,
3864
+ "step": 4890
3865
+ },
3866
+ {
3867
+ "epoch": 3.87,
3868
+ "grad_norm": 0.6495879888534546,
3869
+ "learning_rate": 6.629834254143646e-06,
3870
+ "loss": 0.1061,
3871
+ "step": 4900
3872
+ },
3873
+ {
3874
+ "epoch": 3.87,
3875
+ "eval_accuracy": 0.82099861851194,
3876
+ "eval_loss": 0.6983892321586609,
3877
+ "eval_runtime": 92.4464,
3878
+ "eval_samples_per_second": 54.81,
3879
+ "eval_steps_per_second": 6.858,
3880
+ "step": 4900
3881
+ },
3882
+ {
3883
+ "epoch": 3.88,
3884
+ "grad_norm": 1.4518879652023315,
3885
+ "learning_rate": 6.235201262825572e-06,
3886
+ "loss": 0.0968,
3887
+ "step": 4910
3888
+ },
3889
+ {
3890
+ "epoch": 3.88,
3891
+ "grad_norm": 2.563098669052124,
3892
+ "learning_rate": 5.840568271507498e-06,
3893
+ "loss": 0.0877,
3894
+ "step": 4920
3895
+ },
3896
+ {
3897
+ "epoch": 3.89,
3898
+ "grad_norm": 5.312118053436279,
3899
+ "learning_rate": 5.445935280189424e-06,
3900
+ "loss": 0.1662,
3901
+ "step": 4930
3902
+ },
3903
+ {
3904
+ "epoch": 3.9,
3905
+ "grad_norm": 2.835618257522583,
3906
+ "learning_rate": 5.05130228887135e-06,
3907
+ "loss": 0.0757,
3908
+ "step": 4940
3909
+ },
3910
+ {
3911
+ "epoch": 3.91,
3912
+ "grad_norm": 1.9459186792373657,
3913
+ "learning_rate": 4.656669297553276e-06,
3914
+ "loss": 0.0844,
3915
+ "step": 4950
3916
+ },
3917
+ {
3918
+ "epoch": 3.91,
3919
+ "grad_norm": 1.6886509656906128,
3920
+ "learning_rate": 4.262036306235202e-06,
3921
+ "loss": 0.1373,
3922
+ "step": 4960
3923
+ },
3924
+ {
3925
+ "epoch": 3.92,
3926
+ "grad_norm": 9.272232055664062,
3927
+ "learning_rate": 3.867403314917127e-06,
3928
+ "loss": 0.0566,
3929
+ "step": 4970
3930
+ },
3931
+ {
3932
+ "epoch": 3.93,
3933
+ "grad_norm": 0.17531900107860565,
3934
+ "learning_rate": 3.472770323599053e-06,
3935
+ "loss": 0.0691,
3936
+ "step": 4980
3937
+ },
3938
+ {
3939
+ "epoch": 3.94,
3940
+ "grad_norm": 0.08186766505241394,
3941
+ "learning_rate": 3.0781373322809787e-06,
3942
+ "loss": 0.0586,
3943
+ "step": 4990
3944
+ },
3945
+ {
3946
+ "epoch": 3.95,
3947
+ "grad_norm": 3.012854814529419,
3948
+ "learning_rate": 2.683504340962905e-06,
3949
+ "loss": 0.0861,
3950
+ "step": 5000
3951
+ },
3952
+ {
3953
+ "epoch": 3.95,
3954
+ "eval_accuracy": 0.8229721728833629,
3955
+ "eval_loss": 0.6912640929222107,
3956
+ "eval_runtime": 91.3989,
3957
+ "eval_samples_per_second": 55.438,
3958
+ "eval_steps_per_second": 6.937,
3959
+ "step": 5000
3960
+ },
3961
+ {
3962
+ "epoch": 3.95,
3963
+ "grad_norm": 0.05941695719957352,
3964
+ "learning_rate": 2.2888713496448305e-06,
3965
+ "loss": 0.0622,
3966
+ "step": 5010
3967
+ },
3968
+ {
3969
+ "epoch": 3.96,
3970
+ "grad_norm": 7.365468502044678,
3971
+ "learning_rate": 1.8942383583267563e-06,
3972
+ "loss": 0.1702,
3973
+ "step": 5020
3974
+ },
3975
+ {
3976
+ "epoch": 3.97,
3977
+ "grad_norm": 2.009979724884033,
3978
+ "learning_rate": 1.499605367008682e-06,
3979
+ "loss": 0.0164,
3980
+ "step": 5030
3981
+ },
3982
+ {
3983
+ "epoch": 3.98,
3984
+ "grad_norm": 4.683029651641846,
3985
+ "learning_rate": 1.1049723756906078e-06,
3986
+ "loss": 0.0963,
3987
+ "step": 5040
3988
+ },
3989
+ {
3990
+ "epoch": 3.99,
3991
+ "grad_norm": 8.475516319274902,
3992
+ "learning_rate": 7.103393843725336e-07,
3993
+ "loss": 0.071,
3994
+ "step": 5050
3995
+ },
3996
+ {
3997
+ "epoch": 3.99,
3998
+ "grad_norm": 0.3904523253440857,
3999
+ "learning_rate": 3.1570639305445935e-07,
4000
+ "loss": 0.0701,
4001
+ "step": 5060
4002
+ },
4003
+ {
4004
+ "epoch": 4.0,
4005
+ "step": 5068,
4006
+ "total_flos": 6.281528488153842e+18,
4007
+ "train_loss": 0.4664627312864129,
4008
+ "train_runtime": 8576.6293,
4009
+ "train_samples_per_second": 9.451,
4010
+ "train_steps_per_second": 0.591
4011
+ }
4012
+ ],
4013
+ "logging_steps": 10,
4014
+ "max_steps": 5068,
4015
+ "num_input_tokens_seen": 0,
4016
+ "num_train_epochs": 4,
4017
+ "save_steps": 100,
4018
+ "total_flos": 6.281528488153842e+18,
4019
+ "train_batch_size": 16,
4020
+ "trial_name": null,
4021
+ "trial_params": null
4022
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94bf4cc7500dcc931037f77b38e40227f16449a080b18acb41cb69b3e754fba7
3
+ size 4920