Upload 19 files

aliceinwonderland-llama3/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: models\Meta-Llama-3-8b
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.8.2

aliceinwonderland-llama3/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "models\\Meta-Llama-3-8b",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

aliceinwonderland-llama3/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c8aea5a6c4a4dd8d4e7b4abc49349557f05d86a74217158bf968ca948c06ed5
+size 54572362

aliceinwonderland-llama3/checkpoint-13-loss-0_98/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: models\Meta-Llama-3-8b
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.8.2

aliceinwonderland-llama3/checkpoint-13-loss-0_98/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "models\\Meta-Llama-3-8b",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

aliceinwonderland-llama3/checkpoint-13-loss-0_98/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb176e0bbe60607d99ff9fd4795a52ceb55a6f0024bebd0f5852a24acc5e7121
+size 54572362

aliceinwonderland-llama3/checkpoint-13-loss-0_98/training_log.json

@@ -0,0 +1,14 @@
+{
+  "base_model_name": "Meta-Llama-3-8b",
+  "base_model_class": "LlamaForCausalLM",
+  "base_loaded_in_4bit": true,
+  "base_loaded_in_8bit": false,
+  "projections": "q, v",
+  "loss": 0.9825,
+  "grad_norm": 2.2934579849243164,
+  "learning_rate": 1.2e-07,
+  "epoch": 0.1326530612244898,
+  "current_steps": 12,
+  "current_steps_adjusted": 12,
+  "epoch_adjusted": 0.1326530612244898
+}

aliceinwonderland-llama3/checkpoint-13-loss-0_98/training_prompt.json

@@ -0,0 +1,3 @@
+{
+  "template_type": "raw_text"
+}

aliceinwonderland-llama3/checkpoint-392-loss-0_86/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: models\Meta-Llama-3-8b
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.8.2

aliceinwonderland-llama3/checkpoint-392-loss-0_86/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "models\\Meta-Llama-3-8b",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

aliceinwonderland-llama3/checkpoint-392-loss-0_86/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b5cfba2103baac5a1734f17ae2e0f95004cdb0ad415ba264862240a4900a127
+size 54572362

aliceinwonderland-llama3/checkpoint-392-loss-0_86/training_log.json

@@ -0,0 +1,14 @@
+{
+  "base_model_name": "Meta-Llama-3-8b",
+  "base_model_class": "LlamaForCausalLM",
+  "base_loaded_in_4bit": true,
+  "base_loaded_in_8bit": false,
+  "projections": "q, v",
+  "loss": 0.862,
+  "grad_norm": 3.4459314346313477,
+  "learning_rate": 2.5897435897435897e-07,
+  "epoch": 4.0,
+  "current_steps": 391,
+  "current_steps_adjusted": 391,
+  "epoch_adjusted": 4.0
+}

aliceinwonderland-llama3/checkpoint-392-loss-0_86/training_prompt.json

@@ -0,0 +1,3 @@
+{
+  "template_type": "raw_text"
+}

aliceinwonderland-llama3/runs/Jun04_16-40-59/events.out.tfevents.1717537260.DESKTOP-7QRHF82.4688.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e052f98b17ecd4340beb555eb5f48b1a6b16cfe76c599dda1421aa779e5ce7f1
+size 108486

aliceinwonderland-llama3/training_graph.json

@@ -0,0 +1,2948 @@
+[
+    {
+        "current_steps": 0,
+        "loss": 1.0375,
+        "learning_rate": 1e-08,
+        "epoch": 0.01020408163265306
+    },
+    {
+        "current_steps": 1,
+        "loss": 0.9218,
+        "learning_rate": 1e-08,
+        "epoch": 0.02040816326530612
+    },
+    {
+        "current_steps": 2,
+        "loss": 1.2099,
+        "learning_rate": 2e-08,
+        "epoch": 0.030612244897959183
+    },
+    {
+        "current_steps": 3,
+        "loss": 0.8966,
+        "learning_rate": 3e-08,
+        "epoch": 0.04081632653061224
+    },
+    {
+        "current_steps": 4,
+        "loss": 1.0577,
+        "learning_rate": 4e-08,
+        "epoch": 0.05102040816326531
+    },
+    {
+        "current_steps": 5,
+        "loss": 1.3639,
+        "learning_rate": 5e-08,
+        "epoch": 0.061224489795918366
+    },
+    {
+        "current_steps": 6,
+        "loss": 1.2809,
+        "learning_rate": 6e-08,
+        "epoch": 0.07142857142857142
+    },
+    {
+        "current_steps": 7,
+        "loss": 1.0623,
+        "learning_rate": 7e-08,
+        "epoch": 0.08163265306122448
+    },
+    {
+        "current_steps": 8,
+        "loss": 1.4736,
+        "learning_rate": 8e-08,
+        "epoch": 0.09183673469387756
+    },
+    {
+        "current_steps": 9,
+        "loss": 1.0629,
+        "learning_rate": 9e-08,
+        "epoch": 0.10204081632653061
+    },
+    {
+        "current_steps": 10,
+        "loss": 0.9894,
+        "learning_rate": 1e-07,
+        "epoch": 0.11224489795918367
+    },
+    {
+        "current_steps": 11,
+        "loss": 1.3042,
+        "learning_rate": 1.0999999999999999e-07,
+        "epoch": 0.12244897959183673
+    },
+    {
+        "current_steps": 12,
+        "loss": 0.9825,
+        "learning_rate": 1.2e-07,
+        "epoch": 0.1326530612244898
+    },
+    {
+        "current_steps": 13,
+        "loss": 0.9608,
+        "learning_rate": 1.3e-07,
+        "epoch": 0.14285714285714285
+    },
+    {
+        "current_steps": 14,
+        "loss": 1.1691,
+        "learning_rate": 1.4e-07,
+        "epoch": 0.15306122448979592
+    },
+    {
+        "current_steps": 15,
+        "loss": 0.9785,
+        "learning_rate": 1.5e-07,
+        "epoch": 0.16326530612244897
+    },
+    {
+        "current_steps": 16,
+        "loss": 1.0136,
+        "learning_rate": 1.6e-07,
+        "epoch": 0.17346938775510204
+    },
+    {
+        "current_steps": 17,
+        "loss": 1.0683,
+        "learning_rate": 1.7000000000000001e-07,
+        "epoch": 0.1836734693877551
+    },
+    {
+        "current_steps": 18,
+        "loss": 1.354,
+        "learning_rate": 1.8e-07,
+        "epoch": 0.19387755102040816
+    },
+    {
+        "current_steps": 19,
+        "loss": 0.8537,
+        "learning_rate": 1.8999999999999998e-07,
+        "epoch": 0.20408163265306123
+    },
+    {
+        "current_steps": 20,
+        "loss": 1.3124,
+        "learning_rate": 2e-07,
+        "epoch": 0.21428571428571427
+    },
+    {
+        "current_steps": 21,
+        "loss": 1.0973,
+        "learning_rate": 2.0999999999999997e-07,
+        "epoch": 0.22448979591836735
+    },
+    {
+        "current_steps": 22,
+        "loss": 0.8491,
+        "learning_rate": 2.1999999999999998e-07,
+        "epoch": 0.23469387755102042
+    },
+    {
+        "current_steps": 23,
+        "loss": 0.8702,
+        "learning_rate": 2.3e-07,
+        "epoch": 0.24489795918367346
+    },
+    {
+        "current_steps": 24,
+        "loss": 1.065,
+        "learning_rate": 2.4e-07,
+        "epoch": 0.25510204081632654
+    },
+    {
+        "current_steps": 25,
+        "loss": 0.9593,
+        "learning_rate": 2.5e-07,
+        "epoch": 0.2653061224489796
+    },
+    {
+        "current_steps": 26,
+        "loss": 1.0278,
+        "learning_rate": 2.6e-07,
+        "epoch": 0.2755102040816326
+    },
+    {
+        "current_steps": 27,
+        "loss": 1.0488,
+        "learning_rate": 2.7e-07,
+        "epoch": 0.2857142857142857
+    },
+    {
+        "current_steps": 28,
+        "loss": 0.8877,
+        "learning_rate": 2.8e-07,
+        "epoch": 0.29591836734693877
+    },
+    {
+        "current_steps": 29,
+        "loss": 1.2834,
+        "learning_rate": 2.9e-07,
+        "epoch": 0.30612244897959184
+    },
+    {
+        "current_steps": 30,
+        "loss": 1.1416,
+        "learning_rate": 3e-07,
+        "epoch": 0.3163265306122449
+    },
+    {
+        "current_steps": 31,
+        "loss": 1.2615,
+        "learning_rate": 3.1e-07,
+        "epoch": 0.32653061224489793
+    },
+    {
+        "current_steps": 32,
+        "loss": 1.522,
+        "learning_rate": 3.2e-07,
+        "epoch": 0.336734693877551
+    },
+    {
+        "current_steps": 33,
+        "loss": 1.2533,
+        "learning_rate": 3.3e-07,
+        "epoch": 0.3469387755102041
+    },
+    {
+        "current_steps": 34,
+        "loss": 1.3297,
+        "learning_rate": 3.4000000000000003e-07,
+        "epoch": 0.35714285714285715
+    },
+    {
+        "current_steps": 35,
+        "loss": 1.1923,
+        "learning_rate": 3.4000000000000003e-07,
+        "epoch": 0.3673469387755102
+    },
+    {
+        "current_steps": 36,
+        "loss": 1.0751,
+        "learning_rate": 3.5e-07,
+        "epoch": 0.37755102040816324
+    },
+    {
+        "current_steps": 37,
+        "loss": 0.9824,
+        "learning_rate": 3.6e-07,
+        "epoch": 0.3877551020408163
+    },
+    {
+        "current_steps": 38,
+        "loss": 1.2065,
+        "learning_rate": 3.7e-07,
+        "epoch": 0.3979591836734694
+    },
+    {
+        "current_steps": 39,
+        "loss": 0.6985,
+        "learning_rate": 3.7999999999999996e-07,
+        "epoch": 0.40816326530612246
+    },
+    {
+        "current_steps": 40,
+        "loss": 1.2638,
+        "learning_rate": 3.8999999999999997e-07,
+        "epoch": 0.41836734693877553
+    },
+    {
+        "current_steps": 41,
+        "loss": 1.0332,
+        "learning_rate": 4e-07,
+        "epoch": 0.42857142857142855
+    },
+    {
+        "current_steps": 42,
+        "loss": 0.9709,
+        "learning_rate": 4.0999999999999994e-07,
+        "epoch": 0.4387755102040816
+    },
+    {
+        "current_steps": 43,
+        "loss": 1.0852,
+        "learning_rate": 4.1999999999999995e-07,
+        "epoch": 0.4489795918367347
+    },
+    {
+        "current_steps": 44,
+        "loss": 1.0733,
+        "learning_rate": 4.2999999999999996e-07,
+        "epoch": 0.45918367346938777
+    },
+    {
+        "current_steps": 45,
+        "loss": 1.3022,
+        "learning_rate": 4.3999999999999997e-07,
+        "epoch": 0.46938775510204084
+    },
+    {
+        "current_steps": 46,
+        "loss": 1.0857,
+        "learning_rate": 4.5e-07,
+        "epoch": 0.47959183673469385
+    },
+    {
+        "current_steps": 47,
+        "loss": 1.1048,
+        "learning_rate": 4.6e-07,
+        "epoch": 0.4897959183673469
+    },
+    {
+        "current_steps": 48,
+        "loss": 0.8515,
+        "learning_rate": 4.6999999999999995e-07,
+        "epoch": 0.5
+    },
+    {
+        "current_steps": 49,
+        "loss": 1.0769,
+        "learning_rate": 4.8e-07,
+        "epoch": 0.5102040816326531
+    },
+    {
+        "current_steps": 50,
+        "loss": 1.2454,
+        "learning_rate": 4.9e-07,
+        "epoch": 0.5204081632653061
+    },
+    {
+        "current_steps": 51,
+        "loss": 1.309,
+        "learning_rate": 5e-07,
+        "epoch": 0.5306122448979592
+    },
+    {
+        "current_steps": 52,
+        "loss": 1.0071,
+        "learning_rate": 5.1e-07,
+        "epoch": 0.5408163265306123
+    },
+    {
+        "current_steps": 53,
+        "loss": 1.01,
+        "learning_rate": 5.2e-07,
+        "epoch": 0.5510204081632653
+    },
+    {
+        "current_steps": 54,
+        "loss": 0.9557,
+        "learning_rate": 5.3e-07,
+        "epoch": 0.5612244897959183
+    },
+    {
+        "current_steps": 55,
+        "loss": 1.0609,
+        "learning_rate": 5.4e-07,
+        "epoch": 0.5714285714285714
+    },
+    {
+        "current_steps": 56,
+        "loss": 0.8179,
+        "learning_rate": 5.5e-07,
+        "epoch": 0.5816326530612245
+    },
+    {
+        "current_steps": 57,
+        "loss": 1.0753,
+        "learning_rate": 5.6e-07,
+        "epoch": 0.5918367346938775
+    },
+    {
+        "current_steps": 58,
+        "loss": 0.9524,
+        "learning_rate": 5.699999999999999e-07,
+        "epoch": 0.6020408163265306
+    },
+    {
+        "current_steps": 59,
+        "loss": 1.0423,
+        "learning_rate": 5.8e-07,
+        "epoch": 0.6122448979591837
+    },
+    {
+        "current_steps": 60,
+        "loss": 1.1175,
+        "learning_rate": 5.9e-07,
+        "epoch": 0.6224489795918368
+    },
+    {
+        "current_steps": 61,
+        "loss": 1.1842,
+        "learning_rate": 6e-07,
+        "epoch": 0.6326530612244898
+    },
+    {
+        "current_steps": 62,
+        "loss": 0.9762,
+        "learning_rate": 6.1e-07,
+        "epoch": 0.6428571428571429
+    },
+    {
+        "current_steps": 63,
+        "loss": 0.9862,
+        "learning_rate": 6.2e-07,
+        "epoch": 0.6530612244897959
+    },
+    {
+        "current_steps": 64,
+        "loss": 1.3298,
+        "learning_rate": 6.3e-07,
+        "epoch": 0.6632653061224489
+    },
+    {
+        "current_steps": 65,
+        "loss": 0.892,
+        "learning_rate": 6.4e-07,
+        "epoch": 0.673469387755102
+    },
+    {
+        "current_steps": 66,
+        "loss": 1.3031,
+        "learning_rate": 6.5e-07,
+        "epoch": 0.6836734693877551
+    },
+    {
+        "current_steps": 67,
+        "loss": 1.0295,
+        "learning_rate": 6.6e-07,
+        "epoch": 0.6938775510204082
+    },
+    {
+        "current_steps": 68,
+        "loss": 1.0984,
+        "learning_rate": 6.7e-07,
+        "epoch": 0.7040816326530612
+    },
+    {
+        "current_steps": 69,
+        "loss": 0.961,
+        "learning_rate": 6.800000000000001e-07,
+        "epoch": 0.7142857142857143
+    },
+    {
+        "current_steps": 70,
+        "loss": 1.1315,
+        "learning_rate": 6.9e-07,
+        "epoch": 0.7244897959183674
+    },
+    {
+        "current_steps": 71,
+        "loss": 1.151,
+        "learning_rate": 7e-07,
+        "epoch": 0.7346938775510204
+    },
+    {
+        "current_steps": 72,
+        "loss": 1.1734,
+        "learning_rate": 7.1e-07,
+        "epoch": 0.7448979591836735
+    },
+    {
+        "current_steps": 73,
+        "loss": 1.2119,
+        "learning_rate": 7.2e-07,
+        "epoch": 0.7551020408163265
+    },
+    {
+        "current_steps": 74,
+        "loss": 1.2703,
+        "learning_rate": 7.3e-07,
+        "epoch": 0.7653061224489796
+    },
+    {
+        "current_steps": 75,
+        "loss": 1.171,
+        "learning_rate": 7.4e-07,
+        "epoch": 0.7755102040816326
+    },
+    {
+        "current_steps": 76,
+        "loss": 1.0916,
+        "learning_rate": 7.5e-07,
+        "epoch": 0.7857142857142857
+    },
+    {
+        "current_steps": 77,
+        "loss": 0.9059,
+        "learning_rate": 7.599999999999999e-07,
+        "epoch": 0.7959183673469388
+    },
+    {
+        "current_steps": 78,
+        "loss": 1.0347,
+        "learning_rate": 7.699999999999999e-07,
+        "epoch": 0.8061224489795918
+    },
+    {
+        "current_steps": 79,
+        "loss": 1.2085,
+        "learning_rate": 7.799999999999999e-07,
+        "epoch": 0.8163265306122449
+    },
+    {
+        "current_steps": 80,
+        "loss": 1.0081,
+        "learning_rate": 7.9e-07,
+        "epoch": 0.826530612244898
+    },
+    {
+        "current_steps": 81,
+        "loss": 1.0151,
+        "learning_rate": 8e-07,
+        "epoch": 0.8367346938775511
+    },
+    {
+        "current_steps": 82,
+        "loss": 1.0571,
+        "learning_rate": 8.1e-07,
+        "epoch": 0.8469387755102041
+    },
+    {
+        "current_steps": 83,
+        "loss": 1.3335,
+        "learning_rate": 8.199999999999999e-07,
+        "epoch": 0.8571428571428571
+    },
+    {
+        "current_steps": 84,
+        "loss": 1.0442,
+        "learning_rate": 8.299999999999999e-07,
+        "epoch": 0.8673469387755102
+    },
+    {
+        "current_steps": 85,
+        "loss": 0.9886,
+        "learning_rate": 8.399999999999999e-07,
+        "epoch": 0.8775510204081632
+    },
+    {
+        "current_steps": 86,
+        "loss": 1.1209,
+        "learning_rate": 8.499999999999999e-07,
+        "epoch": 0.8877551020408163
+    },
+    {
+        "current_steps": 87,
+        "loss": 0.9748,
+        "learning_rate": 8.599999999999999e-07,
+        "epoch": 0.8979591836734694
+    },
+    {
+        "current_steps": 88,
+        "loss": 0.9632,
+        "learning_rate": 8.699999999999999e-07,
+        "epoch": 0.9081632653061225
+    },
+    {
+        "current_steps": 89,
+        "loss": 1.3457,
+        "learning_rate": 8.799999999999999e-07,
+        "epoch": 0.9183673469387755
+    },
+    {
+        "current_steps": 90,
+        "loss": 0.9974,
+        "learning_rate": 8.9e-07,
+        "epoch": 0.9285714285714286
+    },
+    {
+        "current_steps": 91,
+        "loss": 1.1924,
+        "learning_rate": 9e-07,
+        "epoch": 0.9387755102040817
+    },
+    {
+        "current_steps": 92,
+        "loss": 1.2519,
+        "learning_rate": 9.1e-07,
+        "epoch": 0.9489795918367347
+    },
+    {
+        "current_steps": 93,
+        "loss": 1.0482,
+        "learning_rate": 9.2e-07,
+        "epoch": 0.9591836734693877
+    },
+    {
+        "current_steps": 94,
+        "loss": 0.9984,
+        "learning_rate": 9.3e-07,
+        "epoch": 0.9693877551020408
+    },
+    {
+        "current_steps": 95,
+        "loss": 1.2079,
+        "learning_rate": 9.399999999999999e-07,
+        "epoch": 0.9795918367346939
+    },
+    {
+        "current_steps": 96,
+        "loss": 0.9468,
+        "learning_rate": 9.499999999999999e-07,
+        "epoch": 0.9897959183673469
+    },
+    {
+        "current_steps": 97,
+        "loss": 0.9655,
+        "learning_rate": 9.6e-07,
+        "epoch": 1.0
+    },
+    {
+        "current_steps": 98,
+        "loss": 0.9911,
+        "learning_rate": 9.7e-07,
+        "epoch": 1.010204081632653
+    },
+    {
+        "current_steps": 99,
+        "loss": 1.019,
+        "learning_rate": 9.8e-07,
+        "epoch": 1.0204081632653061
+    },
+    {
+        "current_steps": 100,
+        "loss": 1.0679,
+        "learning_rate": 9.9e-07,
+        "epoch": 1.030612244897959
+    },
+    {
+        "current_steps": 101,
+        "loss": 0.8909,
+        "learning_rate": 1e-06,
+        "epoch": 1.0408163265306123
+    },
+    {
+        "current_steps": 102,
+        "loss": 1.1474,
+        "learning_rate": 9.974358974358974e-07,
+        "epoch": 1.0510204081632653
+    },
+    {
+        "current_steps": 103,
+        "loss": 0.9872,
+        "learning_rate": 9.948717948717949e-07,
+        "epoch": 1.0612244897959184
+    },
+    {
+        "current_steps": 104,
+        "loss": 0.8438,
+        "learning_rate": 9.923076923076923e-07,
+        "epoch": 1.0714285714285714
+    },
+    {
+        "current_steps": 105,
+        "loss": 0.9473,
+        "learning_rate": 9.897435897435898e-07,
+        "epoch": 1.0816326530612246
+    },
+    {
+        "current_steps": 106,
+        "loss": 1.0933,
+        "learning_rate": 9.871794871794872e-07,
+        "epoch": 1.0918367346938775
+    },
+    {
+        "current_steps": 107,
+        "loss": 0.8545,
+        "learning_rate": 9.846153846153847e-07,
+        "epoch": 1.1020408163265305
+    },
+    {
+        "current_steps": 108,
+        "loss": 1.1711,
+        "learning_rate": 9.820512820512819e-07,
+        "epoch": 1.1122448979591837
+    },
+    {
+        "current_steps": 109,
+        "loss": 0.905,
+        "learning_rate": 9.794871794871793e-07,
+        "epoch": 1.1224489795918366
+    },
+    {
+        "current_steps": 110,
+        "loss": 0.9026,
+        "learning_rate": 9.769230769230768e-07,
+        "epoch": 1.1326530612244898
+    },
+    {
+        "current_steps": 111,
+        "loss": 1.2263,
+        "learning_rate": 9.743589743589742e-07,
+        "epoch": 1.1428571428571428
+    },
+    {
+        "current_steps": 112,
+        "loss": 1.0883,
+        "learning_rate": 9.717948717948717e-07,
+        "epoch": 1.153061224489796
+    },
+    {
+        "current_steps": 113,
+        "loss": 1.1185,
+        "learning_rate": 9.692307692307691e-07,
+        "epoch": 1.163265306122449
+    },
+    {
+        "current_steps": 114,
+        "loss": 1.2212,
+        "learning_rate": 9.666666666666666e-07,
+        "epoch": 1.1734693877551021
+    },
+    {
+        "current_steps": 115,
+        "loss": 1.2142,
+        "learning_rate": 9.64102564102564e-07,
+        "epoch": 1.183673469387755
+    },
+    {
+        "current_steps": 116,
+        "loss": 1.8625,
+        "learning_rate": 9.615384615384615e-07,
+        "epoch": 1.193877551020408
+    },
+    {
+        "current_steps": 117,
+        "loss": 1.0982,
+        "learning_rate": 9.58974358974359e-07,
+        "epoch": 1.2040816326530612
+    },
+    {
+        "current_steps": 118,
+        "loss": 1.2912,
+        "learning_rate": 9.564102564102564e-07,
+        "epoch": 1.2142857142857142
+    },
+    {
+        "current_steps": 119,
+        "loss": 1.2424,
+        "learning_rate": 9.538461538461538e-07,
+        "epoch": 1.2244897959183674
+    },
+    {
+        "current_steps": 120,
+        "loss": 1.0336,
+        "learning_rate": 9.512820512820512e-07,
+        "epoch": 1.2346938775510203
+    },
+    {
+        "current_steps": 121,
+        "loss": 1.1639,
+        "learning_rate": 9.487179487179486e-07,
+        "epoch": 1.2448979591836735
+    },
+    {
+        "current_steps": 122,
+        "loss": 1.1928,
+        "learning_rate": 9.461538461538461e-07,
+        "epoch": 1.2551020408163265
+    },
+    {
+        "current_steps": 123,
+        "loss": 0.8534,
+        "learning_rate": 9.435897435897435e-07,
+        "epoch": 1.2653061224489797
+    },
+    {
+        "current_steps": 124,
+        "loss": 1.0712,
+        "learning_rate": 9.41025641025641e-07,
+        "epoch": 1.2755102040816326
+    },
+    {
+        "current_steps": 125,
+        "loss": 1.2151,
+        "learning_rate": 9.384615384615384e-07,
+        "epoch": 1.2857142857142856
+    },
+    {
+        "current_steps": 126,
+        "loss": 0.8883,
+        "learning_rate": 9.358974358974359e-07,
+        "epoch": 1.2959183673469388
+    },
+    {
+        "current_steps": 127,
+        "loss": 1.1596,
+        "learning_rate": 9.333333333333333e-07,
+        "epoch": 1.306122448979592
+    },
+    {
+        "current_steps": 128,
+        "loss": 1.0218,
+        "learning_rate": 9.307692307692308e-07,
+        "epoch": 1.316326530612245
+    },
+    {
+        "current_steps": 129,
+        "loss": 0.953,
+        "learning_rate": 9.282051282051282e-07,
+        "epoch": 1.3265306122448979
+    },
+    {
+        "current_steps": 130,
+        "loss": 0.9616,
+        "learning_rate": 9.282051282051282e-07,
+        "epoch": 1.336734693877551
+    },
+    {
+        "current_steps": 131,
+        "loss": 1.0368,
+        "learning_rate": 9.256410256410257e-07,
+        "epoch": 1.346938775510204
+    },
+    {
+        "current_steps": 132,
+        "loss": 1.008,
+        "learning_rate": 9.230769230769231e-07,
+        "epoch": 1.3571428571428572
+    },
+    {
+        "current_steps": 133,
+        "loss": 0.9845,
+        "learning_rate": 9.205128205128205e-07,
+        "epoch": 1.3673469387755102
+    },
+    {
+        "current_steps": 134,
+        "loss": 0.8404,
+        "learning_rate": 9.179487179487179e-07,
+        "epoch": 1.3775510204081631
+    },
+    {
+        "current_steps": 135,
+        "loss": 0.9514,
+        "learning_rate": 9.153846153846153e-07,
+        "epoch": 1.3877551020408163
+    },
+    {
+        "current_steps": 136,
+        "loss": 1.0032,
+        "learning_rate": 9.128205128205127e-07,
+        "epoch": 1.3979591836734695
+    },
+    {
+        "current_steps": 137,
+        "loss": 0.9905,
+        "learning_rate": 9.102564102564102e-07,
+        "epoch": 1.4081632653061225
+    },
+    {
+        "current_steps": 138,
+        "loss": 1.0208,
+        "learning_rate": 9.076923076923076e-07,
+        "epoch": 1.4183673469387754
+    },
+    {
+        "current_steps": 139,
+        "loss": 0.9982,
+        "learning_rate": 9.051282051282051e-07,
+        "epoch": 1.4285714285714286
+    },
+    {
+        "current_steps": 140,
+        "loss": 0.9697,
+        "learning_rate": 9.025641025641025e-07,
+        "epoch": 1.4387755102040816
+    },
+    {
+        "current_steps": 141,
+        "loss": 1.031,
+        "learning_rate": 9e-07,
+        "epoch": 1.4489795918367347
+    },
+    {
+        "current_steps": 142,
+        "loss": 1.1713,
+        "learning_rate": 8.974358974358974e-07,
+        "epoch": 1.4591836734693877
+    },
+    {
+        "current_steps": 143,
+        "loss": 1.3207,
+        "learning_rate": 8.948717948717949e-07,
+        "epoch": 1.469387755102041
+    },
+    {
+        "current_steps": 144,
+        "loss": 0.9822,
+        "learning_rate": 8.923076923076923e-07,
+        "epoch": 1.4795918367346939
+    },
+    {
+        "current_steps": 145,
+        "loss": 0.9969,
+        "learning_rate": 8.897435897435897e-07,
+        "epoch": 1.489795918367347
+    },
+    {
+        "current_steps": 146,
+        "loss": 1.216,
+        "learning_rate": 8.871794871794871e-07,
+        "epoch": 1.5
+    },
+    {
+        "current_steps": 147,
+        "loss": 1.0927,
+        "learning_rate": 8.846153846153846e-07,
+        "epoch": 1.510204081632653
+    },
+    {
+        "current_steps": 148,
+        "loss": 1.1464,
+        "learning_rate": 8.82051282051282e-07,
+        "epoch": 1.5204081632653061
+    },
+    {
+        "current_steps": 149,
+        "loss": 0.9045,
+        "learning_rate": 8.794871794871795e-07,
+        "epoch": 1.5306122448979593
+    },
+    {
+        "current_steps": 150,
+        "loss": 1.0216,
+        "learning_rate": 8.769230769230769e-07,
+        "epoch": 1.5408163265306123
+    },
+    {
+        "current_steps": 151,
+        "loss": 0.8687,
+        "learning_rate": 8.743589743589743e-07,
+        "epoch": 1.5510204081632653
+    },
+    {
+        "current_steps": 152,
+        "loss": 1.3796,
+        "learning_rate": 8.717948717948718e-07,
+        "epoch": 1.5612244897959182
+    },
+    {
+        "current_steps": 153,
+        "loss": 1.0517,
+        "learning_rate": 8.692307692307692e-07,
+        "epoch": 1.5714285714285714
+    },
+    {
+        "current_steps": 154,
+        "loss": 1.2114,
+        "learning_rate": 8.666666666666667e-07,
+        "epoch": 1.5816326530612246
+    },
+    {
+        "current_steps": 155,
+        "loss": 0.8193,
+        "learning_rate": 8.641025641025641e-07,
+        "epoch": 1.5918367346938775
+    },
+    {
+        "current_steps": 156,
+        "loss": 1.1385,
+        "learning_rate": 8.615384615384616e-07,
+        "epoch": 1.6020408163265305
+    },
+    {
+        "current_steps": 157,
+        "loss": 0.9739,
+        "learning_rate": 8.589743589743588e-07,
+        "epoch": 1.6122448979591837
+    },
+    {
+        "current_steps": 158,
+        "loss": 1.5266,
+        "learning_rate": 8.564102564102563e-07,
+        "epoch": 1.6224489795918369
+    },
+    {
+        "current_steps": 159,
+        "loss": 1.0021,
+        "learning_rate": 8.538461538461537e-07,
+        "epoch": 1.6326530612244898
+    },
+    {
+        "current_steps": 160,
+        "loss": 1.119,
+        "learning_rate": 8.512820512820512e-07,
+        "epoch": 1.6428571428571428
+    },
+    {
+        "current_steps": 161,
+        "loss": 0.9662,
+        "learning_rate": 8.487179487179486e-07,
+        "epoch": 1.6530612244897958
+    },
+    {
+        "current_steps": 162,
+        "loss": 1.1719,
+        "learning_rate": 8.461538461538461e-07,
+        "epoch": 1.663265306122449
+    },
+    {
+        "current_steps": 163,
+        "loss": 0.9716,
+        "learning_rate": 8.435897435897435e-07,
+        "epoch": 1.6734693877551021
+    },
+    {
+        "current_steps": 164,
+        "loss": 1.5846,
+        "learning_rate": 8.41025641025641e-07,
+        "epoch": 1.683673469387755
+    },
+    {
+        "current_steps": 165,
+        "loss": 0.8901,
+        "learning_rate": 8.384615384615384e-07,
+        "epoch": 1.693877551020408
+    },
+    {
+        "current_steps": 166,
+        "loss": 0.9545,
+        "learning_rate": 8.358974358974359e-07,
+        "epoch": 1.7040816326530612
+    },
+    {
+        "current_steps": 167,
+        "loss": 1.1515,
+        "learning_rate": 8.333333333333333e-07,
+        "epoch": 1.7142857142857144
+    },
+    {
+        "current_steps": 168,
+        "loss": 1.3852,
+        "learning_rate": 8.307692307692308e-07,
+        "epoch": 1.7244897959183674
+    },
+    {
+        "current_steps": 169,
+        "loss": 1.0304,
+        "learning_rate": 8.282051282051282e-07,
+        "epoch": 1.7346938775510203
+    },
+    {
+        "current_steps": 170,
+        "loss": 1.0555,
+        "learning_rate": 8.256410256410256e-07,
+        "epoch": 1.7448979591836735
+    },
+    {
+        "current_steps": 171,
+        "loss": 0.7715,
+        "learning_rate": 8.23076923076923e-07,
+        "epoch": 1.7551020408163265
+    },
+    {
+        "current_steps": 172,
+        "loss": 0.9879,
+        "learning_rate": 8.205128205128205e-07,
+        "epoch": 1.7653061224489797
+    },
+    {
+        "current_steps": 173,
+        "loss": 1.244,
+        "learning_rate": 8.179487179487179e-07,
+        "epoch": 1.7755102040816326
+    },
+    {
+        "current_steps": 174,
+        "loss": 1.1134,
+        "learning_rate": 8.153846153846154e-07,
+        "epoch": 1.7857142857142856
+    },
+    {
+        "current_steps": 175,
+        "loss": 1.1013,
+        "learning_rate": 8.128205128205128e-07,
+        "epoch": 1.7959183673469388
+    },
+    {
+        "current_steps": 176,
+        "loss": 1.1775,
+        "learning_rate": 8.102564102564103e-07,
+        "epoch": 1.806122448979592
+    },
+    {
+        "current_steps": 177,
+        "loss": 1.1815,
+        "learning_rate": 8.076923076923077e-07,
+        "epoch": 1.816326530612245
+    },
+    {
+        "current_steps": 178,
+        "loss": 1.1765,
+        "learning_rate": 8.051282051282052e-07,
+        "epoch": 1.8265306122448979
+    },
+    {
+        "current_steps": 179,
+        "loss": 0.9524,
+        "learning_rate": 8.025641025641025e-07,
+        "epoch": 1.836734693877551
+    },
+    {
+        "current_steps": 180,
+        "loss": 1.1465,
+        "learning_rate": 8e-07,
+        "epoch": 1.8469387755102042
+    },
+    {
+        "current_steps": 181,
+        "loss": 1.198,
+        "learning_rate": 7.974358974358974e-07,
+        "epoch": 1.8571428571428572
+    },
+    {
+        "current_steps": 182,
+        "loss": 1.0793,
+        "learning_rate": 7.948717948717948e-07,
+        "epoch": 1.8673469387755102
+    },
+    {
+        "current_steps": 183,
+        "loss": 1.2496,
+        "learning_rate": 7.923076923076922e-07,
+        "epoch": 1.8775510204081631
+    },
+    {
+        "current_steps": 184,
+        "loss": 0.986,
+        "learning_rate": 7.897435897435897e-07,
+        "epoch": 1.8877551020408163
+    },
+    {
+        "current_steps": 185,
+        "loss": 1.0159,
+        "learning_rate": 7.871794871794871e-07,
+        "epoch": 1.8979591836734695
+    },
+    {
+        "current_steps": 186,
+        "loss": 1.0971,
+        "learning_rate": 7.846153846153846e-07,
+        "epoch": 1.9081632653061225
+    },
+    {
+        "current_steps": 187,
+        "loss": 0.9174,
+        "learning_rate": 7.82051282051282e-07,
+        "epoch": 1.9183673469387754
+    },
+    {
+        "current_steps": 188,
+        "loss": 0.9513,
+        "learning_rate": 7.794871794871795e-07,
+        "epoch": 1.9285714285714286
+    },
+    {
+        "current_steps": 189,
+        "loss": 1.0736,
+        "learning_rate": 7.769230769230769e-07,
+        "epoch": 1.9387755102040818
+    },
+    {
+        "current_steps": 190,
+        "loss": 1.1306,
+        "learning_rate": 7.743589743589744e-07,
+        "epoch": 1.9489795918367347
+    },
+    {
+        "current_steps": 191,
+        "loss": 0.9481,
+        "learning_rate": 7.717948717948718e-07,
+        "epoch": 1.9591836734693877
+    },
+    {
+        "current_steps": 192,
+        "loss": 0.8325,
+        "learning_rate": 7.692307692307693e-07,
+        "epoch": 1.9693877551020407
+    },
+    {
+        "current_steps": 193,
+        "loss": 1.0885,
+        "learning_rate": 7.666666666666667e-07,
+        "epoch": 1.9795918367346939
+    },
+    {
+        "current_steps": 194,
+        "loss": 1.1409,
+        "learning_rate": 7.64102564102564e-07,
+        "epoch": 1.989795918367347
+    },
+    {
+        "current_steps": 195,
+        "loss": 0.7824,
+        "learning_rate": 7.615384615384615e-07,
+        "epoch": 2.0
+    },
+    {
+        "current_steps": 196,
+        "loss": 1.1916,
+        "learning_rate": 7.589743589743589e-07,
+        "epoch": 2.010204081632653
+    },
+    {
+        "current_steps": 197,
+        "loss": 1.1858,
+        "learning_rate": 7.564102564102564e-07,
+        "epoch": 2.020408163265306
+    },
+    {
+        "current_steps": 198,
+        "loss": 0.8909,
+        "learning_rate": 7.538461538461538e-07,
+        "epoch": 2.0306122448979593
+    },
+    {
+        "current_steps": 199,
+        "loss": 1.2537,
+        "learning_rate": 7.512820512820513e-07,
+        "epoch": 2.0408163265306123
+    },
+    {
+        "current_steps": 200,
+        "loss": 1.1669,
+        "learning_rate": 7.487179487179486e-07,
+        "epoch": 2.0510204081632653
+    },
+    {
+        "current_steps": 201,
+        "loss": 0.9486,
+        "learning_rate": 7.461538461538461e-07,
+        "epoch": 2.061224489795918
+    },
+    {
+        "current_steps": 202,
+        "loss": 0.9397,
+        "learning_rate": 7.435897435897435e-07,
+        "epoch": 2.0714285714285716
+    },
+    {
+        "current_steps": 203,
+        "loss": 1.0689,
+        "learning_rate": 7.41025641025641e-07,
+        "epoch": 2.0816326530612246
+    },
+    {
+        "current_steps": 204,
+        "loss": 0.8393,
+        "learning_rate": 7.384615384615384e-07,
+        "epoch": 2.0918367346938775
+    },
+    {
+        "current_steps": 205,
+        "loss": 0.9834,
+        "learning_rate": 7.358974358974359e-07,
+        "epoch": 2.1020408163265305
+    },
+    {
+        "current_steps": 206,
+        "loss": 0.9443,
+        "learning_rate": 7.333333333333332e-07,
+        "epoch": 2.1122448979591835
+    },
+    {
+        "current_steps": 207,
+        "loss": 0.8711,
+        "learning_rate": 7.307692307692307e-07,
+        "epoch": 2.122448979591837
+    },
+    {
+        "current_steps": 208,
+        "loss": 1.0088,
+        "learning_rate": 7.282051282051281e-07,
+        "epoch": 2.13265306122449
+    },
+    {
+        "current_steps": 209,
+        "loss": 0.9495,
+        "learning_rate": 7.256410256410256e-07,
+        "epoch": 2.142857142857143
+    },
+    {
+        "current_steps": 210,
+        "loss": 1.0246,
+        "learning_rate": 7.23076923076923e-07,
+        "epoch": 2.1530612244897958
+    },
+    {
+        "current_steps": 211,
+        "loss": 0.8712,
+        "learning_rate": 7.205128205128205e-07,
+        "epoch": 2.163265306122449
+    },
+    {
+        "current_steps": 212,
+        "loss": 1.0604,
+        "learning_rate": 7.179487179487179e-07,
+        "epoch": 2.173469387755102
+    },
+    {
+        "current_steps": 213,
+        "loss": 0.9848,
+        "learning_rate": 7.153846153846154e-07,
+        "epoch": 2.183673469387755
+    },
+    {
+        "current_steps": 214,
+        "loss": 1.0133,
+        "learning_rate": 7.128205128205128e-07,
+        "epoch": 2.193877551020408
+    },
+    {
+        "current_steps": 215,
+        "loss": 1.0367,
+        "learning_rate": 7.102564102564103e-07,
+        "epoch": 2.204081632653061
+    },
+    {
+        "current_steps": 216,
+        "loss": 1.0779,
+        "learning_rate": 7.076923076923077e-07,
+        "epoch": 2.2142857142857144
+    },
+    {
+        "current_steps": 217,
+        "loss": 1.1813,
+        "learning_rate": 7.051282051282052e-07,
+        "epoch": 2.2244897959183674
+    },
+    {
+        "current_steps": 218,
+        "loss": 1.209,
+        "learning_rate": 7.025641025641025e-07,
+        "epoch": 2.2346938775510203
+    },
+    {
+        "current_steps": 219,
+        "loss": 0.7289,
+        "learning_rate": 7e-07,
+        "epoch": 2.2448979591836733
+    },
+    {
+        "current_steps": 220,
+        "loss": 1.0283,
+        "learning_rate": 6.974358974358974e-07,
+        "epoch": 2.2551020408163267
+    },
+    {
+        "current_steps": 221,
+        "loss": 1.141,
+        "learning_rate": 6.948717948717948e-07,
+        "epoch": 2.2653061224489797
+    },
+    {
+        "current_steps": 222,
+        "loss": 0.8899,
+        "learning_rate": 6.923076923076922e-07,
+        "epoch": 2.2755102040816326
+    },
+    {
+        "current_steps": 223,
+        "loss": 1.0982,
+        "learning_rate": 6.897435897435897e-07,
+        "epoch": 2.2857142857142856
+    },
+    {
+        "current_steps": 224,
+        "loss": 0.9901,
+        "learning_rate": 6.871794871794871e-07,
+        "epoch": 2.295918367346939
+    },
+    {
+        "current_steps": 225,
+        "loss": 1.0405,
+        "learning_rate": 6.846153846153846e-07,
+        "epoch": 2.306122448979592
+    },
+    {
+        "current_steps": 226,
+        "loss": 0.8221,
+        "learning_rate": 6.82051282051282e-07,
+        "epoch": 2.316326530612245
+    },
+    {
+        "current_steps": 227,
+        "loss": 1.2293,
+        "learning_rate": 6.794871794871795e-07,
+        "epoch": 2.326530612244898
+    },
+    {
+        "current_steps": 228,
+        "loss": 1.2191,
+        "learning_rate": 6.769230769230769e-07,
+        "epoch": 2.336734693877551
+    },
+    {
+        "current_steps": 229,
+        "loss": 1.0375,
+        "learning_rate": 6.743589743589744e-07,
+        "epoch": 2.3469387755102042
+    },
+    {
+        "current_steps": 230,
+        "loss": 0.9397,
+        "learning_rate": 6.717948717948717e-07,
+        "epoch": 2.357142857142857
+    },
+    {
+        "current_steps": 231,
+        "loss": 0.9278,
+        "learning_rate": 6.692307692307692e-07,
+        "epoch": 2.36734693877551
+    },
+    {
+        "current_steps": 232,
+        "loss": 1.0717,
+        "learning_rate": 6.666666666666666e-07,
+        "epoch": 2.377551020408163
+    },
+    {
+        "current_steps": 233,
+        "loss": 1.1695,
+        "learning_rate": 6.64102564102564e-07,
+        "epoch": 2.387755102040816
+    },
+    {
+        "current_steps": 234,
+        "loss": 1.1653,
+        "learning_rate": 6.615384615384615e-07,
+        "epoch": 2.3979591836734695
+    },
+    {
+        "current_steps": 235,
+        "loss": 1.1682,
+        "learning_rate": 6.58974358974359e-07,
+        "epoch": 2.4081632653061225
+    },
+    {
+        "current_steps": 236,
+        "loss": 0.9805,
+        "learning_rate": 6.564102564102564e-07,
+        "epoch": 2.4183673469387754
+    },
+    {
+        "current_steps": 237,
+        "loss": 1.1046,
+        "learning_rate": 6.538461538461538e-07,
+        "epoch": 2.4285714285714284
+    },
+    {
+        "current_steps": 238,
+        "loss": 0.9734,
+        "learning_rate": 6.512820512820513e-07,
+        "epoch": 2.438775510204082
+    },
+    {
+        "current_steps": 239,
+        "loss": 1.2178,
+        "learning_rate": 6.487179487179487e-07,
+        "epoch": 2.4489795918367347
+    },
+    {
+        "current_steps": 240,
+        "loss": 1.1556,
+        "learning_rate": 6.461538461538462e-07,
+        "epoch": 2.4591836734693877
+    },
+    {
+        "current_steps": 241,
+        "loss": 1.0326,
+        "learning_rate": 6.435897435897436e-07,
+        "epoch": 2.4693877551020407
+    },
+    {
+        "current_steps": 242,
+        "loss": 0.9868,
+        "learning_rate": 6.410256410256411e-07,
+        "epoch": 2.479591836734694
+    },
+    {
+        "current_steps": 243,
+        "loss": 0.8947,
+        "learning_rate": 6.384615384615383e-07,
+        "epoch": 2.489795918367347
+    },
+    {
+        "current_steps": 244,
+        "loss": 0.9736,
+        "learning_rate": 6.358974358974358e-07,
+        "epoch": 2.5
+    },
+    {
+        "current_steps": 245,
+        "loss": 1.5294,
+        "learning_rate": 6.333333333333332e-07,
+        "epoch": 2.510204081632653
+    },
+    {
+        "current_steps": 246,
+        "loss": 1.176,
+        "learning_rate": 6.307692307692307e-07,
+        "epoch": 2.520408163265306
+    },
+    {
+        "current_steps": 247,
+        "loss": 1.0583,
+        "learning_rate": 6.282051282051281e-07,
+        "epoch": 2.5306122448979593
+    },
+    {
+        "current_steps": 248,
+        "loss": 0.9439,
+        "learning_rate": 6.256410256410256e-07,
+        "epoch": 2.5408163265306123
+    },
+    {
+        "current_steps": 249,
+        "loss": 0.9092,
+        "learning_rate": 6.23076923076923e-07,
+        "epoch": 2.5510204081632653
+    },
+    {
+        "current_steps": 250,
+        "loss": 1.2091,
+        "learning_rate": 6.205128205128205e-07,
+        "epoch": 2.561224489795918
+    },
+    {
+        "current_steps": 251,
+        "loss": 0.9285,
+        "learning_rate": 6.179487179487179e-07,
+        "epoch": 2.571428571428571
+    },
+    {
+        "current_steps": 252,
+        "loss": 1.039,
+        "learning_rate": 6.153846153846154e-07,
+        "epoch": 2.5816326530612246
+    },
+    {
+        "current_steps": 253,
+        "loss": 1.0148,
+        "learning_rate": 6.128205128205128e-07,
+        "epoch": 2.5918367346938775
+    },
+    {
+        "current_steps": 254,
+        "loss": 0.9658,
+        "learning_rate": 6.102564102564103e-07,
+        "epoch": 2.6020408163265305
+    },
+    {
+        "current_steps": 255,
+        "loss": 0.7872,
+        "learning_rate": 6.076923076923076e-07,
+        "epoch": 2.612244897959184
+    },
+    {
+        "current_steps": 256,
+        "loss": 0.9161,
+        "learning_rate": 6.051282051282051e-07,
+        "epoch": 2.622448979591837
+    },
+    {
+        "current_steps": 257,
+        "loss": 1.2506,
+        "learning_rate": 6.025641025641025e-07,
+        "epoch": 2.63265306122449
+    },
+    {
+        "current_steps": 258,
+        "loss": 0.969,
+        "learning_rate": 6e-07,
+        "epoch": 2.642857142857143
+    },
+    {
+        "current_steps": 259,
+        "loss": 0.8615,
+        "learning_rate": 5.974358974358974e-07,
+        "epoch": 2.6530612244897958
+    },
+    {
+        "current_steps": 260,
+        "loss": 0.8467,
+        "learning_rate": 5.948717948717949e-07,
+        "epoch": 2.663265306122449
+    },
+    {
+        "current_steps": 261,
+        "loss": 1.1803,
+        "learning_rate": 5.923076923076923e-07,
+        "epoch": 2.673469387755102
+    },
+    {
+        "current_steps": 262,
+        "loss": 0.6155,
+        "learning_rate": 5.897435897435898e-07,
+        "epoch": 2.683673469387755
+    },
+    {
+        "current_steps": 263,
+        "loss": 1.148,
+        "learning_rate": 5.871794871794872e-07,
+        "epoch": 2.693877551020408
+    },
+    {
+        "current_steps": 264,
+        "loss": 1.2103,
+        "learning_rate": 5.846153846153847e-07,
+        "epoch": 2.704081632653061
+    },
+    {
+        "current_steps": 265,
+        "loss": 0.8083,
+        "learning_rate": 5.82051282051282e-07,
+        "epoch": 2.7142857142857144
+    },
+    {
+        "current_steps": 266,
+        "loss": 0.9858,
+        "learning_rate": 5.794871794871795e-07,
+        "epoch": 2.7244897959183674
+    },
+    {
+        "current_steps": 267,
+        "loss": 0.9855,
+        "learning_rate": 5.769230769230768e-07,
+        "epoch": 2.7346938775510203
+    },
+    {
+        "current_steps": 268,
+        "loss": 1.1443,
+        "learning_rate": 5.743589743589743e-07,
+        "epoch": 2.7448979591836737
+    },
+    {
+        "current_steps": 269,
+        "loss": 0.8432,
+        "learning_rate": 5.717948717948717e-07,
+        "epoch": 2.7551020408163263
+    },
+    {
+        "current_steps": 270,
+        "loss": 1.1858,
+        "learning_rate": 5.692307692307692e-07,
+        "epoch": 2.7653061224489797
+    },
+    {
+        "current_steps": 271,
+        "loss": 1.5042,
+        "learning_rate": 5.666666666666666e-07,
+        "epoch": 2.7755102040816326
+    },
+    {
+        "current_steps": 272,
+        "loss": 1.0224,
+        "learning_rate": 5.641025641025641e-07,
+        "epoch": 2.7857142857142856
+    },
+    {
+        "current_steps": 273,
+        "loss": 1.243,
+        "learning_rate": 5.615384615384615e-07,
+        "epoch": 2.795918367346939
+    },
+    {
+        "current_steps": 274,
+        "loss": 0.9863,
+        "learning_rate": 5.58974358974359e-07,
+        "epoch": 2.806122448979592
+    },
+    {
+        "current_steps": 275,
+        "loss": 1.2529,
+        "learning_rate": 5.564102564102564e-07,
+        "epoch": 2.816326530612245
+    },
+    {
+        "current_steps": 276,
+        "loss": 0.9227,
+        "learning_rate": 5.538461538461539e-07,
+        "epoch": 2.826530612244898
+    },
+    {
+        "current_steps": 277,
+        "loss": 0.8991,
+        "learning_rate": 5.512820512820513e-07,
+        "epoch": 2.836734693877551
+    },
+    {
+        "current_steps": 278,
+        "loss": 0.9586,
+        "learning_rate": 5.487179487179488e-07,
+        "epoch": 2.8469387755102042
+    },
+    {
+        "current_steps": 279,
+        "loss": 0.9635,
+        "learning_rate": 5.461538461538461e-07,
+        "epoch": 2.857142857142857
+    },
+    {
+        "current_steps": 280,
+        "loss": 1.0263,
+        "learning_rate": 5.435897435897435e-07,
+        "epoch": 2.86734693877551
+    },
+    {
+        "current_steps": 281,
+        "loss": 0.9285,
+        "learning_rate": 5.41025641025641e-07,
+        "epoch": 2.877551020408163
+    },
+    {
+        "current_steps": 282,
+        "loss": 1.2307,
+        "learning_rate": 5.384615384615384e-07,
+        "epoch": 2.887755102040816
+    },
+    {
+        "current_steps": 283,
+        "loss": 1.0337,
+        "learning_rate": 5.358974358974359e-07,
+        "epoch": 2.8979591836734695
+    },
+    {
+        "current_steps": 284,
+        "loss": 0.9798,
+        "learning_rate": 5.333333333333333e-07,
+        "epoch": 2.9081632653061225
+    },
+    {
+        "current_steps": 285,
+        "loss": 1.0283,
+        "learning_rate": 5.307692307692308e-07,
+        "epoch": 2.9183673469387754
+    },
+    {
+        "current_steps": 286,
+        "loss": 0.946,
+        "learning_rate": 5.282051282051282e-07,
+        "epoch": 2.928571428571429
+    },
+    {
+        "current_steps": 287,
+        "loss": 0.9716,
+        "learning_rate": 5.256410256410256e-07,
+        "epoch": 2.938775510204082
+    },
+    {
+        "current_steps": 288,
+        "loss": 1.0291,
+        "learning_rate": 5.23076923076923e-07,
+        "epoch": 2.9489795918367347
+    },
+    {
+        "current_steps": 289,
+        "loss": 1.1638,
+        "learning_rate": 5.205128205128205e-07,
+        "epoch": 2.9591836734693877
+    },
+    {
+        "current_steps": 290,
+        "loss": 1.249,
+        "learning_rate": 5.179487179487179e-07,
+        "epoch": 2.9693877551020407
+    },
+    {
+        "current_steps": 291,
+        "loss": 1.1532,
+        "learning_rate": 5.153846153846153e-07,
+        "epoch": 2.979591836734694
+    },
+    {
+        "current_steps": 292,
+        "loss": 0.8845,
+        "learning_rate": 5.128205128205127e-07,
+        "epoch": 2.989795918367347
+    },
+    {
+        "current_steps": 293,
+        "loss": 1.2865,
+        "learning_rate": 5.102564102564102e-07,
+        "epoch": 3.0
+    },
+    {
+        "current_steps": 294,
+        "loss": 1.0558,
+        "learning_rate": 5.076923076923076e-07,
+        "epoch": 3.010204081632653
+    },
+    {
+        "current_steps": 295,
+        "loss": 0.8381,
+        "learning_rate": 5.051282051282051e-07,
+        "epoch": 3.020408163265306
+    },
+    {
+        "current_steps": 296,
+        "loss": 0.9348,
+        "learning_rate": 5.025641025641025e-07,
+        "epoch": 3.0306122448979593
+    },
+    {
+        "current_steps": 297,
+        "loss": 0.9659,
+        "learning_rate": 5e-07,
+        "epoch": 3.0408163265306123
+    },
+    {
+        "current_steps": 298,
+        "loss": 1.3105,
+        "learning_rate": 4.974358974358974e-07,
+        "epoch": 3.0510204081632653
+    },
+    {
+        "current_steps": 299,
+        "loss": 0.7721,
+        "learning_rate": 4.948717948717949e-07,
+        "epoch": 3.061224489795918
+    },
+    {
+        "current_steps": 300,
+        "loss": 0.9762,
+        "learning_rate": 4.923076923076923e-07,
+        "epoch": 3.0714285714285716
+    },
+    {
+        "current_steps": 301,
+        "loss": 0.9398,
+        "learning_rate": 4.897435897435897e-07,
+        "epoch": 3.0816326530612246
+    },
+    {
+        "current_steps": 302,
+        "loss": 1.2612,
+        "learning_rate": 4.871794871794871e-07,
+        "epoch": 3.0918367346938775
+    },
+    {
+        "current_steps": 303,
+        "loss": 1.2505,
+        "learning_rate": 4.846153846153846e-07,
+        "epoch": 3.1020408163265305
+    },
+    {
+        "current_steps": 304,
+        "loss": 1.1641,
+        "learning_rate": 4.82051282051282e-07,
+        "epoch": 3.1122448979591835
+    },
+    {
+        "current_steps": 305,
+        "loss": 1.0805,
+        "learning_rate": 4.794871794871795e-07,
+        "epoch": 3.122448979591837
+    },
+    {
+        "current_steps": 306,
+        "loss": 0.8856,
+        "learning_rate": 4.769230769230769e-07,
+        "epoch": 3.13265306122449
+    },
+    {
+        "current_steps": 307,
+        "loss": 1.1931,
+        "learning_rate": 4.743589743589743e-07,
+        "epoch": 3.142857142857143
+    },
+    {
+        "current_steps": 308,
+        "loss": 1.0618,
+        "learning_rate": 4.7179487179487176e-07,
+        "epoch": 3.1530612244897958
+    },
+    {
+        "current_steps": 309,
+        "loss": 0.9113,
+        "learning_rate": 4.692307692307692e-07,
+        "epoch": 3.163265306122449
+    },
+    {
+        "current_steps": 310,
+        "loss": 1.0872,
+        "learning_rate": 4.6666666666666666e-07,
+        "epoch": 3.173469387755102
+    },
+    {
+        "current_steps": 311,
+        "loss": 1.0619,
+        "learning_rate": 4.641025641025641e-07,
+        "epoch": 3.183673469387755
+    },
+    {
+        "current_steps": 312,
+        "loss": 0.8784,
+        "learning_rate": 4.6153846153846156e-07,
+        "epoch": 3.193877551020408
+    },
+    {
+        "current_steps": 313,
+        "loss": 1.0601,
+        "learning_rate": 4.5897435897435896e-07,
+        "epoch": 3.204081632653061
+    },
+    {
+        "current_steps": 314,
+        "loss": 0.959,
+        "learning_rate": 4.5641025641025636e-07,
+        "epoch": 3.2142857142857144
+    },
+    {
+        "current_steps": 315,
+        "loss": 1.0085,
+        "learning_rate": 4.538461538461538e-07,
+        "epoch": 3.2244897959183674
+    },
+    {
+        "current_steps": 316,
+        "loss": 1.0718,
+        "learning_rate": 4.5128205128205125e-07,
+        "epoch": 3.2346938775510203
+    },
+    {
+        "current_steps": 317,
+        "loss": 1.1865,
+        "learning_rate": 4.487179487179487e-07,
+        "epoch": 3.2448979591836733
+    },
+    {
+        "current_steps": 318,
+        "loss": 0.9815,
+        "learning_rate": 4.4615384615384615e-07,
+        "epoch": 3.2551020408163267
+    },
+    {
+        "current_steps": 319,
+        "loss": 1.1176,
+        "learning_rate": 4.4358974358974355e-07,
+        "epoch": 3.2653061224489797
+    },
+    {
+        "current_steps": 320,
+        "loss": 0.7826,
+        "learning_rate": 4.41025641025641e-07,
+        "epoch": 3.2755102040816326
+    },
+    {
+        "current_steps": 321,
+        "loss": 1.0717,
+        "learning_rate": 4.3846153846153845e-07,
+        "epoch": 3.2857142857142856
+    },
+    {
+        "current_steps": 322,
+        "loss": 0.9129,
+        "learning_rate": 4.358974358974359e-07,
+        "epoch": 3.295918367346939
+    },
+    {
+        "current_steps": 323,
+        "loss": 0.8037,
+        "learning_rate": 4.3333333333333335e-07,
+        "epoch": 3.306122448979592
+    },
+    {
+        "current_steps": 324,
+        "loss": 1.1173,
+        "learning_rate": 4.307692307692308e-07,
+        "epoch": 3.316326530612245
+    },
+    {
+        "current_steps": 325,
+        "loss": 1.2198,
+        "learning_rate": 4.2820512820512814e-07,
+        "epoch": 3.326530612244898
+    },
+    {
+        "current_steps": 326,
+        "loss": 1.0326,
+        "learning_rate": 4.256410256410256e-07,
+        "epoch": 3.336734693877551
+    },
+    {
+        "current_steps": 327,
+        "loss": 1.237,
+        "learning_rate": 4.2307692307692304e-07,
+        "epoch": 3.3469387755102042
+    },
+    {
+        "current_steps": 328,
+        "loss": 1.1247,
+        "learning_rate": 4.205128205128205e-07,
+        "epoch": 3.357142857142857
+    },
+    {
+        "current_steps": 329,
+        "loss": 1.0071,
+        "learning_rate": 4.1794871794871794e-07,
+        "epoch": 3.36734693877551
+    },
+    {
+        "current_steps": 330,
+        "loss": 1.2143,
+        "learning_rate": 4.153846153846154e-07,
+        "epoch": 3.377551020408163
+    },
+    {
+        "current_steps": 331,
+        "loss": 0.9502,
+        "learning_rate": 4.128205128205128e-07,
+        "epoch": 3.387755102040816
+    },
+    {
+        "current_steps": 332,
+        "loss": 0.8315,
+        "learning_rate": 4.1025641025641024e-07,
+        "epoch": 3.3979591836734695
+    },
+    {
+        "current_steps": 333,
+        "loss": 0.8832,
+        "learning_rate": 4.076923076923077e-07,
+        "epoch": 3.4081632653061225
+    },
+    {
+        "current_steps": 334,
+        "loss": 1.0922,
+        "learning_rate": 4.0512820512820514e-07,
+        "epoch": 3.4183673469387754
+    },
+    {
+        "current_steps": 335,
+        "loss": 0.8653,
+        "learning_rate": 4.025641025641026e-07,
+        "epoch": 3.4285714285714284
+    },
+    {
+        "current_steps": 336,
+        "loss": 0.8588,
+        "learning_rate": 4e-07,
+        "epoch": 3.438775510204082
+    },
+    {
+        "current_steps": 337,
+        "loss": 1.0424,
+        "learning_rate": 3.974358974358974e-07,
+        "epoch": 3.4489795918367347
+    },
+    {
+        "current_steps": 338,
+        "loss": 1.0364,
+        "learning_rate": 3.9487179487179483e-07,
+        "epoch": 3.4591836734693877
+    },
+    {
+        "current_steps": 339,
+        "loss": 1.1119,
+        "learning_rate": 3.923076923076923e-07,
+        "epoch": 3.4693877551020407
+    },
+    {
+        "current_steps": 340,
+        "loss": 1.412,
+        "learning_rate": 3.8974358974358973e-07,
+        "epoch": 3.479591836734694
+    },
+    {
+        "current_steps": 341,
+        "loss": 1.0895,
+        "learning_rate": 3.871794871794872e-07,
+        "epoch": 3.489795918367347
+    },
+    {
+        "current_steps": 342,
+        "loss": 0.937,
+        "learning_rate": 3.8461538461538463e-07,
+        "epoch": 3.5
+    },
+    {
+        "current_steps": 343,
+        "loss": 0.9634,
+        "learning_rate": 3.82051282051282e-07,
+        "epoch": 3.510204081632653
+    },
+    {
+        "current_steps": 344,
+        "loss": 1.0896,
+        "learning_rate": 3.7948717948717947e-07,
+        "epoch": 3.520408163265306
+    },
+    {
+        "current_steps": 345,
+        "loss": 1.1336,
+        "learning_rate": 3.769230769230769e-07,
+        "epoch": 3.5306122448979593
+    },
+    {
+        "current_steps": 346,
+        "loss": 1.1216,
+        "learning_rate": 3.743589743589743e-07,
+        "epoch": 3.5408163265306123
+    },
+    {
+        "current_steps": 347,
+        "loss": 1.0329,
+        "learning_rate": 3.7179487179487177e-07,
+        "epoch": 3.5510204081632653
+    },
+    {
+        "current_steps": 348,
+        "loss": 0.9592,
+        "learning_rate": 3.692307692307692e-07,
+        "epoch": 3.561224489795918
+    },
+    {
+        "current_steps": 349,
+        "loss": 1.1346,
+        "learning_rate": 3.666666666666666e-07,
+        "epoch": 3.571428571428571
+    },
+    {
+        "current_steps": 350,
+        "loss": 0.9627,
+        "learning_rate": 3.6410256410256406e-07,
+        "epoch": 3.5816326530612246
+    },
+    {
+        "current_steps": 351,
+        "loss": 0.8698,
+        "learning_rate": 3.615384615384615e-07,
+        "epoch": 3.5918367346938775
+    },
+    {
+        "current_steps": 352,
+        "loss": 1.0556,
+        "learning_rate": 3.5897435897435896e-07,
+        "epoch": 3.6020408163265305
+    },
+    {
+        "current_steps": 353,
+        "loss": 1.1182,
+        "learning_rate": 3.564102564102564e-07,
+        "epoch": 3.612244897959184
+    },
+    {
+        "current_steps": 354,
+        "loss": 1.1447,
+        "learning_rate": 3.5384615384615386e-07,
+        "epoch": 3.622448979591837
+    },
+    {
+        "current_steps": 355,
+        "loss": 0.8987,
+        "learning_rate": 3.5128205128205126e-07,
+        "epoch": 3.63265306122449
+    },
+    {
+        "current_steps": 356,
+        "loss": 0.8758,
+        "learning_rate": 3.487179487179487e-07,
+        "epoch": 3.642857142857143
+    },
+    {
+        "current_steps": 357,
+        "loss": 1.7681,
+        "learning_rate": 3.461538461538461e-07,
+        "epoch": 3.6530612244897958
+    },
+    {
+        "current_steps": 358,
+        "loss": 1.0395,
+        "learning_rate": 3.4358974358974356e-07,
+        "epoch": 3.663265306122449
+    },
+    {
+        "current_steps": 359,
+        "loss": 0.7837,
+        "learning_rate": 3.41025641025641e-07,
+        "epoch": 3.673469387755102
+    },
+    {
+        "current_steps": 360,
+        "loss": 1.0153,
+        "learning_rate": 3.3846153846153845e-07,
+        "epoch": 3.683673469387755
+    },
+    {
+        "current_steps": 361,
+        "loss": 1.1471,
+        "learning_rate": 3.3589743589743585e-07,
+        "epoch": 3.693877551020408
+    },
+    {
+        "current_steps": 362,
+        "loss": 0.8569,
+        "learning_rate": 3.333333333333333e-07,
+        "epoch": 3.704081632653061
+    },
+    {
+        "current_steps": 363,
+        "loss": 1.0616,
+        "learning_rate": 3.3076923076923075e-07,
+        "epoch": 3.7142857142857144
+    },
+    {
+        "current_steps": 364,
+        "loss": 1.0197,
+        "learning_rate": 3.282051282051282e-07,
+        "epoch": 3.7244897959183674
+    },
+    {
+        "current_steps": 365,
+        "loss": 0.7505,
+        "learning_rate": 3.2564102564102565e-07,
+        "epoch": 3.7346938775510203
+    },
+    {
+        "current_steps": 366,
+        "loss": 1.1823,
+        "learning_rate": 3.230769230769231e-07,
+        "epoch": 3.7448979591836737
+    },
+    {
+        "current_steps": 367,
+        "loss": 1.0465,
+        "learning_rate": 3.2051282051282055e-07,
+        "epoch": 3.7551020408163263
+    },
+    {
+        "current_steps": 368,
+        "loss": 1.2175,
+        "learning_rate": 3.179487179487179e-07,
+        "epoch": 3.7653061224489797
+    },
+    {
+        "current_steps": 369,
+        "loss": 0.7914,
+        "learning_rate": 3.1538461538461534e-07,
+        "epoch": 3.7755102040816326
+    },
+    {
+        "current_steps": 370,
+        "loss": 1.0506,
+        "learning_rate": 3.128205128205128e-07,
+        "epoch": 3.7857142857142856
+    },
+    {
+        "current_steps": 371,
+        "loss": 1.2272,
+        "learning_rate": 3.1025641025641024e-07,
+        "epoch": 3.795918367346939
+    },
+    {
+        "current_steps": 372,
+        "loss": 0.9431,
+        "learning_rate": 3.076923076923077e-07,
+        "epoch": 3.806122448979592
+    },
+    {
+        "current_steps": 373,
+        "loss": 1.0228,
+        "learning_rate": 3.0512820512820514e-07,
+        "epoch": 3.816326530612245
+    },
+    {
+        "current_steps": 374,
+        "loss": 0.8733,
+        "learning_rate": 3.0256410256410254e-07,
+        "epoch": 3.826530612244898
+    },
+    {
+        "current_steps": 375,
+        "loss": 1.1344,
+        "learning_rate": 3e-07,
+        "epoch": 3.836734693877551
+    },
+    {
+        "current_steps": 376,
+        "loss": 1.0388,
+        "learning_rate": 2.9743589743589744e-07,
+        "epoch": 3.8469387755102042
+    },
+    {
+        "current_steps": 377,
+        "loss": 1.0039,
+        "learning_rate": 2.948717948717949e-07,
+        "epoch": 3.857142857142857
+    },
+    {
+        "current_steps": 378,
+        "loss": 1.161,
+        "learning_rate": 2.9230769230769234e-07,
+        "epoch": 3.86734693877551
+    },
+    {
+        "current_steps": 379,
+        "loss": 0.7736,
+        "learning_rate": 2.8974358974358973e-07,
+        "epoch": 3.877551020408163
+    },
+    {
+        "current_steps": 380,
+        "loss": 1.0812,
+        "learning_rate": 2.8717948717948713e-07,
+        "epoch": 3.887755102040816
+    },
+    {
+        "current_steps": 381,
+        "loss": 1.067,
+        "learning_rate": 2.846153846153846e-07,
+        "epoch": 3.8979591836734695
+    },
+    {
+        "current_steps": 382,
+        "loss": 0.8961,
+        "learning_rate": 2.8205128205128203e-07,
+        "epoch": 3.9081632653061225
+    },
+    {
+        "current_steps": 383,
+        "loss": 0.7864,
+        "learning_rate": 2.794871794871795e-07,
+        "epoch": 3.9183673469387754
+    },
+    {
+        "current_steps": 384,
+        "loss": 0.8673,
+        "learning_rate": 2.7692307692307693e-07,
+        "epoch": 3.928571428571429
+    },
+    {
+        "current_steps": 385,
+        "loss": 0.9608,
+        "learning_rate": 2.743589743589744e-07,
+        "epoch": 3.938775510204082
+    },
+    {
+        "current_steps": 386,
+        "loss": 1.0788,
+        "learning_rate": 2.7179487179487177e-07,
+        "epoch": 3.9489795918367347
+    },
+    {
+        "current_steps": 387,
+        "loss": 1.2868,
+        "learning_rate": 2.692307692307692e-07,
+        "epoch": 3.9591836734693877
+    },
+    {
+        "current_steps": 388,
+        "loss": 0.8304,
+        "learning_rate": 2.6666666666666667e-07,
+        "epoch": 3.9693877551020407
+    },
+    {
+        "current_steps": 389,
+        "loss": 0.7779,
+        "learning_rate": 2.641025641025641e-07,
+        "epoch": 3.979591836734694
+    },
+    {
+        "current_steps": 390,
+        "loss": 0.7525,
+        "learning_rate": 2.615384615384615e-07,
+        "epoch": 3.989795918367347
+    },
+    {
+        "current_steps": 391,
+        "loss": 0.862,
+        "learning_rate": 2.5897435897435897e-07,
+        "epoch": 4.0
+    },
+    {
+        "current_steps": 392,
+        "loss": 1.0776,
+        "learning_rate": 2.5641025641025636e-07,
+        "epoch": 4.010204081632653
+    },
+    {
+        "current_steps": 393,
+        "loss": 0.9312,
+        "learning_rate": 2.538461538461538e-07,
+        "epoch": 4.020408163265306
+    },
+    {
+        "current_steps": 394,
+        "loss": 1.3004,
+        "learning_rate": 2.5128205128205126e-07,
+        "epoch": 4.030612244897959
+    },
+    {
+        "current_steps": 395,
+        "loss": 1.2295,
+        "learning_rate": 2.487179487179487e-07,
+        "epoch": 4.040816326530612
+    },
+    {
+        "current_steps": 396,
+        "loss": 1.0688,
+        "learning_rate": 2.4615384615384616e-07,
+        "epoch": 4.051020408163265
+    },
+    {
+        "current_steps": 397,
+        "loss": 0.906,
+        "learning_rate": 2.4358974358974356e-07,
+        "epoch": 4.061224489795919
+    },
+    {
+        "current_steps": 398,
+        "loss": 0.8381,
+        "learning_rate": 2.41025641025641e-07,
+        "epoch": 4.071428571428571
+    },
+    {
+        "current_steps": 399,
+        "loss": 0.9278,
+        "learning_rate": 2.3846153846153846e-07,
+        "epoch": 4.081632653061225
+    },
+    {
+        "current_steps": 400,
+        "loss": 1.3734,
+        "learning_rate": 2.3589743589743588e-07,
+        "epoch": 4.091836734693878
+    },
+    {
+        "current_steps": 401,
+        "loss": 1.2469,
+        "learning_rate": 2.3333333333333333e-07,
+        "epoch": 4.1020408163265305
+    },
+    {
+        "current_steps": 402,
+        "loss": 1.3845,
+        "learning_rate": 2.3076923076923078e-07,
+        "epoch": 4.112244897959184
+    },
+    {
+        "current_steps": 403,
+        "loss": 0.9578,
+        "learning_rate": 2.2820512820512818e-07,
+        "epoch": 4.122448979591836
+    },
+    {
+        "current_steps": 404,
+        "loss": 0.9949,
+        "learning_rate": 2.2564102564102563e-07,
+        "epoch": 4.13265306122449
+    },
+    {
+        "current_steps": 405,
+        "loss": 1.0655,
+        "learning_rate": 2.2307692307692308e-07,
+        "epoch": 4.142857142857143
+    },
+    {
+        "current_steps": 406,
+        "loss": 0.8242,
+        "learning_rate": 2.205128205128205e-07,
+        "epoch": 4.153061224489796
+    },
+    {
+        "current_steps": 407,
+        "loss": 1.1328,
+        "learning_rate": 2.1794871794871795e-07,
+        "epoch": 4.163265306122449
+    },
+    {
+        "current_steps": 408,
+        "loss": 0.9127,
+        "learning_rate": 2.153846153846154e-07,
+        "epoch": 4.173469387755102
+    },
+    {
+        "current_steps": 409,
+        "loss": 0.9174,
+        "learning_rate": 2.128205128205128e-07,
+        "epoch": 4.183673469387755
+    },
+    {
+        "current_steps": 410,
+        "loss": 1.2596,
+        "learning_rate": 2.1025641025641025e-07,
+        "epoch": 4.1938775510204085
+    },
+    {
+        "current_steps": 411,
+        "loss": 1.109,
+        "learning_rate": 2.076923076923077e-07,
+        "epoch": 4.204081632653061
+    },
+    {
+        "current_steps": 412,
+        "loss": 1.313,
+        "learning_rate": 2.0512820512820512e-07,
+        "epoch": 4.214285714285714
+    },
+    {
+        "current_steps": 413,
+        "loss": 1.0135,
+        "learning_rate": 2.0256410256410257e-07,
+        "epoch": 4.224489795918367
+    },
+    {
+        "current_steps": 414,
+        "loss": 0.9048,
+        "learning_rate": 2e-07,
+        "epoch": 4.23469387755102
+    },
+    {
+        "current_steps": 415,
+        "loss": 1.1162,
+        "learning_rate": 1.9743589743589741e-07,
+        "epoch": 4.244897959183674
+    },
+    {
+        "current_steps": 416,
+        "loss": 0.8849,
+        "learning_rate": 1.9487179487179486e-07,
+        "epoch": 4.255102040816326
+    },
+    {
+        "current_steps": 417,
+        "loss": 0.8803,
+        "learning_rate": 1.9230769230769231e-07,
+        "epoch": 4.26530612244898
+    },
+    {
+        "current_steps": 418,
+        "loss": 0.9497,
+        "learning_rate": 1.8974358974358974e-07,
+        "epoch": 4.275510204081632
+    },
+    {
+        "current_steps": 419,
+        "loss": 0.9216,
+        "learning_rate": 1.8717948717948716e-07,
+        "epoch": 4.285714285714286
+    },
+    {
+        "current_steps": 420,
+        "loss": 0.8915,
+        "learning_rate": 1.846153846153846e-07,
+        "epoch": 4.295918367346939
+    },
+    {
+        "current_steps": 421,
+        "loss": 0.9967,
+        "learning_rate": 1.8205128205128203e-07,
+        "epoch": 4.3061224489795915
+    },
+    {
+        "current_steps": 422,
+        "loss": 0.9392,
+        "learning_rate": 1.7948717948717948e-07,
+        "epoch": 4.316326530612245
+    },
+    {
+        "current_steps": 423,
+        "loss": 1.1217,
+        "learning_rate": 1.7692307692307693e-07,
+        "epoch": 4.326530612244898
+    },
+    {
+        "current_steps": 424,
+        "loss": 1.0389,
+        "learning_rate": 1.7435897435897435e-07,
+        "epoch": 4.336734693877551
+    },
+    {
+        "current_steps": 425,
+        "loss": 1.106,
+        "learning_rate": 1.7179487179487178e-07,
+        "epoch": 4.346938775510204
+    },
+    {
+        "current_steps": 426,
+        "loss": 1.0438,
+        "learning_rate": 1.6923076923076923e-07,
+        "epoch": 4.357142857142857
+    },
+    {
+        "current_steps": 427,
+        "loss": 1.0671,
+        "learning_rate": 1.6666666666666665e-07,
+        "epoch": 4.36734693877551
+    },
+    {
+        "current_steps": 428,
+        "loss": 1.0963,
+        "learning_rate": 1.641025641025641e-07,
+        "epoch": 4.377551020408164
+    },
+    {
+        "current_steps": 429,
+        "loss": 1.0008,
+        "learning_rate": 1.6153846153846155e-07,
+        "epoch": 4.387755102040816
+    },
+    {
+        "current_steps": 430,
+        "loss": 0.8875,
+        "learning_rate": 1.5897435897435895e-07,
+        "epoch": 4.3979591836734695
+    },
+    {
+        "current_steps": 431,
+        "loss": 0.989,
+        "learning_rate": 1.564102564102564e-07,
+        "epoch": 4.408163265306122
+    },
+    {
+        "current_steps": 432,
+        "loss": 1.0361,
+        "learning_rate": 1.5384615384615385e-07,
+        "epoch": 4.418367346938775
+    },
+    {
+        "current_steps": 433,
+        "loss": 0.9075,
+        "learning_rate": 1.5128205128205127e-07,
+        "epoch": 4.428571428571429
+    },
+    {
+        "current_steps": 434,
+        "loss": 0.9151,
+        "learning_rate": 1.4871794871794872e-07,
+        "epoch": 4.438775510204081
+    },
+    {
+        "current_steps": 435,
+        "loss": 1.2658,
+        "learning_rate": 1.4615384615384617e-07,
+        "epoch": 4.448979591836735
+    },
+    {
+        "current_steps": 436,
+        "loss": 0.9476,
+        "learning_rate": 1.4358974358974356e-07,
+        "epoch": 4.459183673469388
+    },
+    {
+        "current_steps": 437,
+        "loss": 1.2257,
+        "learning_rate": 1.4102564102564101e-07,
+        "epoch": 4.469387755102041
+    },
+    {
+        "current_steps": 438,
+        "loss": 1.0466,
+        "learning_rate": 1.3846153846153846e-07,
+        "epoch": 4.479591836734694
+    },
+    {
+        "current_steps": 439,
+        "loss": 1.0721,
+        "learning_rate": 1.3589743589743589e-07,
+        "epoch": 4.489795918367347
+    },
+    {
+        "current_steps": 440,
+        "loss": 0.9636,
+        "learning_rate": 1.3333333333333334e-07,
+        "epoch": 4.5
+    },
+    {
+        "current_steps": 441,
+        "loss": 0.7291,
+        "learning_rate": 1.3076923076923076e-07,
+        "epoch": 4.510204081632653
+    },
+    {
+        "current_steps": 442,
+        "loss": 1.1522,
+        "learning_rate": 1.2820512820512818e-07,
+        "epoch": 4.520408163265306
+    },
+    {
+        "current_steps": 443,
+        "loss": 0.8466,
+        "learning_rate": 1.2564102564102563e-07,
+        "epoch": 4.530612244897959
+    },
+    {
+        "current_steps": 444,
+        "loss": 0.9945,
+        "learning_rate": 1.2307692307692308e-07,
+        "epoch": 4.540816326530612
+    },
+    {
+        "current_steps": 445,
+        "loss": 0.9127,
+        "learning_rate": 1.205128205128205e-07,
+        "epoch": 4.551020408163265
+    },
+    {
+        "current_steps": 446,
+        "loss": 0.6726,
+        "learning_rate": 1.1794871794871794e-07,
+        "epoch": 4.561224489795919
+    },
+    {
+        "current_steps": 447,
+        "loss": 1.2726,
+        "learning_rate": 1.1538461538461539e-07,
+        "epoch": 4.571428571428571
+    },
+    {
+        "current_steps": 448,
+        "loss": 0.9189,
+        "learning_rate": 1.1282051282051281e-07,
+        "epoch": 4.581632653061225
+    },
+    {
+        "current_steps": 449,
+        "loss": 0.7115,
+        "learning_rate": 1.1025641025641025e-07,
+        "epoch": 4.591836734693878
+    },
+    {
+        "current_steps": 450,
+        "loss": 0.8172,
+        "learning_rate": 1.076923076923077e-07,
+        "epoch": 4.6020408163265305
+    },
+    {
+        "current_steps": 451,
+        "loss": 0.9128,
+        "learning_rate": 1.0512820512820512e-07,
+        "epoch": 4.612244897959184
+    },
+    {
+        "current_steps": 452,
+        "loss": 0.8185,
+        "learning_rate": 1.0256410256410256e-07,
+        "epoch": 4.622448979591836
+    },
+    {
+        "current_steps": 453,
+        "loss": 0.6656,
+        "learning_rate": 1e-07,
+        "epoch": 4.63265306122449
+    },
+    {
+        "current_steps": 454,
+        "loss": 0.9927,
+        "learning_rate": 9.743589743589743e-08,
+        "epoch": 4.642857142857143
+    },
+    {
+        "current_steps": 455,
+        "loss": 1.038,
+        "learning_rate": 9.487179487179487e-08,
+        "epoch": 4.653061224489796
+    },
+    {
+        "current_steps": 456,
+        "loss": 1.0286,
+        "learning_rate": 9.23076923076923e-08,
+        "epoch": 4.663265306122449
+    },
+    {
+        "current_steps": 457,
+        "loss": 0.9324,
+        "learning_rate": 8.974358974358974e-08,
+        "epoch": 4.673469387755102
+    },
+    {
+        "current_steps": 458,
+        "loss": 1.1796,
+        "learning_rate": 8.717948717948718e-08,
+        "epoch": 4.683673469387755
+    },
+    {
+        "current_steps": 459,
+        "loss": 1.0709,
+        "learning_rate": 8.461538461538461e-08,
+        "epoch": 4.6938775510204085
+    },
+    {
+        "current_steps": 460,
+        "loss": 1.2764,
+        "learning_rate": 8.205128205128205e-08,
+        "epoch": 4.704081632653061
+    },
+    {
+        "current_steps": 461,
+        "loss": 0.9408,
+        "learning_rate": 7.948717948717947e-08,
+        "epoch": 4.714285714285714
+    },
+    {
+        "current_steps": 462,
+        "loss": 1.0265,
+        "learning_rate": 7.692307692307692e-08,
+        "epoch": 4.724489795918368
+    },
+    {
+        "current_steps": 463,
+        "loss": 0.8105,
+        "learning_rate": 7.435897435897436e-08,
+        "epoch": 4.73469387755102
+    },
+    {
+        "current_steps": 464,
+        "loss": 0.8581,
+        "learning_rate": 7.179487179487178e-08,
+        "epoch": 4.744897959183674
+    },
+    {
+        "current_steps": 465,
+        "loss": 1.0193,
+        "learning_rate": 6.923076923076923e-08,
+        "epoch": 4.755102040816326
+    },
+    {
+        "current_steps": 466,
+        "loss": 0.9166,
+        "learning_rate": 6.666666666666667e-08,
+        "epoch": 4.76530612244898
+    },
+    {
+        "current_steps": 467,
+        "loss": 1.0751,
+        "learning_rate": 6.410256410256409e-08,
+        "epoch": 4.775510204081632
+    },
+    {
+        "current_steps": 468,
+        "loss": 1.0601,
+        "learning_rate": 6.153846153846154e-08,
+        "epoch": 4.785714285714286
+    },
+    {
+        "current_steps": 469,
+        "loss": 1.2884,
+        "learning_rate": 5.897435897435897e-08,
+        "epoch": 4.795918367346939
+    },
+    {
+        "current_steps": 470,
+        "loss": 0.9581,
+        "learning_rate": 5.641025641025641e-08,
+        "epoch": 4.8061224489795915
+    },
+    {
+        "current_steps": 471,
+        "loss": 1.0762,
+        "learning_rate": 5.384615384615385e-08,
+        "epoch": 4.816326530612245
+    },
+    {
+        "current_steps": 472,
+        "loss": 0.9197,
+        "learning_rate": 5.128205128205128e-08,
+        "epoch": 4.826530612244898
+    },
+    {
+        "current_steps": 473,
+        "loss": 1.0248,
+        "learning_rate": 4.8717948717948716e-08,
+        "epoch": 4.836734693877551
+    },
+    {
+        "current_steps": 474,
+        "loss": 1.0387,
+        "learning_rate": 4.615384615384615e-08,
+        "epoch": 4.846938775510204
+    },
+    {
+        "current_steps": 475,
+        "loss": 0.9627,
+        "learning_rate": 4.358974358974359e-08,
+        "epoch": 4.857142857142857
+    },
+    {
+        "current_steps": 476,
+        "loss": 0.951,
+        "learning_rate": 4.1025641025641025e-08,
+        "epoch": 4.86734693877551
+    },
+    {
+        "current_steps": 477,
+        "loss": 1.0213,
+        "learning_rate": 3.846153846153846e-08,
+        "epoch": 4.877551020408164
+    },
+    {
+        "current_steps": 478,
+        "loss": 1.4383,
+        "learning_rate": 3.589743589743589e-08,
+        "epoch": 4.887755102040816
+    },
+    {
+        "current_steps": 479,
+        "loss": 1.0652,
+        "learning_rate": 3.3333333333333334e-08,
+        "epoch": 4.8979591836734695
+    },
+    {
+        "current_steps": 480,
+        "loss": 0.9408,
+        "learning_rate": 3.076923076923077e-08,
+        "epoch": 4.908163265306122
+    },
+    {
+        "current_steps": 481,
+        "loss": 1.2111,
+        "learning_rate": 2.8205128205128203e-08,
+        "epoch": 4.918367346938775
+    },
+    {
+        "current_steps": 482,
+        "loss": 0.9162,
+        "learning_rate": 2.564102564102564e-08,
+        "epoch": 4.928571428571429
+    },
+    {
+        "current_steps": 483,
+        "loss": 0.9021,
+        "learning_rate": 2.3076923076923076e-08,
+        "epoch": 4.938775510204081
+    },
+    {
+        "current_steps": 484,
+        "loss": 1.2209,
+        "learning_rate": 2.0512820512820512e-08,
+        "epoch": 4.948979591836735
+    },
+    {
+        "current_steps": 485,
+        "loss": 0.8742,
+        "learning_rate": 1.7948717948717946e-08,
+        "epoch": 4.959183673469388
+    },
+    {
+        "current_steps": 486,
+        "loss": 1.0272,
+        "learning_rate": 1.5384615384615385e-08,
+        "epoch": 4.969387755102041
+    },
+    {
+        "current_steps": 487,
+        "loss": 1.3311,
+        "learning_rate": 1.282051282051282e-08,
+        "epoch": 4.979591836734694
+    },
+    {
+        "current_steps": 488,
+        "loss": 0.7024,
+        "learning_rate": 1.0256410256410256e-08,
+        "epoch": 4.989795918367347
+    },
+    {
+        "current_steps": 489,
+        "loss": 1.0515,
+        "learning_rate": 7.692307692307693e-09,
+        "epoch": 5.0
+    },
+    {
+        "current_steps": 489,
+        "loss": 1.0515,
+        "learning_rate": 7.692307692307693e-09,
+        "epoch": 5.0
+    }
+]

aliceinwonderland-llama3/training_log.json

@@ -0,0 +1,19 @@
+{
+  "base_model_name": "Meta-Llama-3-8b",
+  "base_model_class": "LlamaForCausalLM",
+  "base_loaded_in_4bit": true,
+  "base_loaded_in_8bit": false,
+  "projections": "q, v",
+  "loss": 1.0515,
+  "grad_norm": 3.200089693069458,
+  "learning_rate": 7.692307692307693e-09,
+  "epoch": 5.0,
+  "current_steps": 489,
+  "current_steps_adjusted": 489,
+  "epoch_adjusted": 5.0,
+  "train_runtime": 779.2785,
+  "train_samples_per_second": 2.502,
+  "train_steps_per_second": 0.629,
+  "total_flos": 2.2519579410432e+16,
+  "train_loss": 1.0483642434587284
+}

aliceinwonderland-llama3/training_parameters.json

@@ -0,0 +1,37 @@
+{
+  "lora_name": "aliceinwonderland-llama3",
+  "always_override": true,
+  "save_steps": 0,
+  "micro_batch_size": 4,
+  "batch_size": 0,
+  "epochs": 5,
+  "learning_rate": "1e-6",
+  "lr_scheduler_type": "linear",
+  "lora_rank": 32,
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "cutoff_len": 256,
+  "dataset": "None",
+  "eval_dataset": "None",
+  "format": "None",
+  "eval_steps": 100,
+  "raw_text_file": "aliceandwonderland",
+  "higher_rank_limit": false,
+  "warmup_steps": 100,
+  "optimizer": "adamw_torch",
+  "hard_cut_string": "\\n\\n\\n",
+  "train_only_after": "",
+  "stop_at_loss": 0,
+  "add_eos_token": false,
+  "min_chars": 20,
+  "report_to": "None",
+  "precize_slicing_overlap": true,
+  "add_eos_token_type": "Every Block",
+  "save_steps_under_loss": 1.8,
+  "add_bos_token": true,
+  "training_projection": "q-v",
+  "sliding_window": false,
+  "warmup_ratio": 0,
+  "grad_accumulation": 1,
+  "neft_noise_alpha": 0
+}

aliceinwonderland-llama3/training_prompt.json

@@ -0,0 +1,3 @@
+{
+  "template_type": "raw_text"
+}