Upload 26 files

aliceinwonderland/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: models\Llama-2-13b-hf
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.8.2

aliceinwonderland/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "models\\Llama-2-13b-hf",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

aliceinwonderland/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ec95cd40d9469ab7d56f444483c09b8727f0c834258363b46306bb4387fd5dd
+size 104915722

aliceinwonderland/checkpoint-15-loss-1_17/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: models\Llama-2-13b-hf
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.8.2

aliceinwonderland/checkpoint-15-loss-1_17/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "models\\Llama-2-13b-hf",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

aliceinwonderland/checkpoint-15-loss-1_17/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67856ecab2080beca8aca0a5e47b291f0805c418a022b4b2a97bb80d7f901ee7
+size 104915722

aliceinwonderland/checkpoint-15-loss-1_17/training_log.json

@@ -0,0 +1,19 @@
+{
+  "base_model_name": "Llama-2-13b-hf",
+  "base_model_class": "LlamaForCausalLM",
+  "base_loaded_in_4bit": true,
+  "base_loaded_in_8bit": false,
+  "projections": "q, v",
+  "loss": 1.1716,
+  "grad_norm": 1.0258234739303589,
+  "learning_rate": 1.3e-07,
+  "epoch": 0.13392857142857142,
+  "current_steps": 14,
+  "current_steps_adjusted": 14,
+  "epoch_adjusted": 0.13392857142857142,
+  "train_runtime": 60.8524,
+  "train_samples_per_second": 7.313,
+  "train_steps_per_second": 1.841,
+  "total_flos": 1819849670000640.0,
+  "train_loss": 0.7478187213773313
+}

aliceinwonderland/checkpoint-15-loss-1_17/training_prompt.json

@@ -0,0 +1,3 @@
+{
+  "template_type": "raw_text"
+}

aliceinwonderland/checkpoint-19-loss-0_90/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: models\Llama-2-13b-hf
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.8.2

aliceinwonderland/checkpoint-19-loss-0_90/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "models\\Llama-2-13b-hf",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

aliceinwonderland/checkpoint-19-loss-0_90/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33728eded59d53993902b4320dd496b739df642eebfd71df1f2fded242f1cf5e
+size 104915722

aliceinwonderland/checkpoint-19-loss-0_90/training_log.json

@@ -0,0 +1,19 @@
+{
+  "base_model_name": "Llama-2-13b-hf",
+  "base_model_class": "LlamaForCausalLM",
+  "base_loaded_in_4bit": true,
+  "base_loaded_in_8bit": false,
+  "projections": "q, v",
+  "loss": 0.9004,
+  "grad_norm": 0.8880526423454285,
+  "learning_rate": 1.7000000000000001e-07,
+  "epoch": 0.16964285714285715,
+  "current_steps": 18,
+  "current_steps_adjusted": 18,
+  "epoch_adjusted": 0.16964285714285715,
+  "train_runtime": 60.8524,
+  "train_samples_per_second": 7.313,
+  "train_steps_per_second": 1.841,
+  "total_flos": 1819849670000640.0,
+  "train_loss": 0.7478187213773313
+}

aliceinwonderland/checkpoint-19-loss-0_90/training_prompt.json

@@ -0,0 +1,3 @@
+{
+  "template_type": "raw_text"
+}

aliceinwonderland/checkpoint-23-loss-0_60/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: models\Llama-2-13b-hf
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.8.2

aliceinwonderland/checkpoint-23-loss-0_60/adapter_config.json

@@ -0,0 +1,27 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "models\\Llama-2-13b-hf",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_rslora": false
+}

aliceinwonderland/checkpoint-23-loss-0_60/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bfb41f5ef71b08818902ba0f659dca42b6e428c3fca8dc04cccd89f096be730b
+size 104915722

aliceinwonderland/checkpoint-23-loss-0_60/training_log.json

@@ -0,0 +1,19 @@
+{
+  "base_model_name": "Llama-2-13b-hf",
+  "base_model_class": "LlamaForCausalLM",
+  "base_loaded_in_4bit": true,
+  "base_loaded_in_8bit": false,
+  "projections": "q, v",
+  "loss": 0.6049,
+  "grad_norm": 1.030413269996643,
+  "learning_rate": 2.0999999999999997e-07,
+  "epoch": 0.20535714285714285,
+  "current_steps": 22,
+  "current_steps_adjusted": 22,
+  "epoch_adjusted": 0.20535714285714285,
+  "train_runtime": 60.8524,
+  "train_samples_per_second": 7.313,
+  "train_steps_per_second": 1.841,
+  "total_flos": 1819849670000640.0,
+  "train_loss": 0.7478187213773313
+}

aliceinwonderland/checkpoint-23-loss-0_60/training_prompt.json

@@ -0,0 +1,3 @@
+{
+  "template_type": "raw_text"
+}

aliceinwonderland/runs/Jun04_00-27-53/events.out.tfevents.1717478875.DESKTOP-7QRHF82.5780.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d9c48c3ff261ea97428f58f99fdcc72538405b49e4ca84c9eb991bd5963f9b3
+size 10741

aliceinwonderland/runs/Jun04_00-32-30/events.out.tfevents.1717479151.DESKTOP-7QRHF82.5780.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8acbbef9133cbacd0f4cc53e3cd9da98fe8510f03648ccc3a82c169bf8daa6c2
+size 10326

aliceinwonderland/runs/Jun04_00-34-12/events.out.tfevents.1717479252.DESKTOP-7QRHF82.5780.2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a194065673c018cd47b84d86a22c61ccebe7dc95cda5512cd1658d56cec6938
+size 123223

aliceinwonderland/training_graph.json

@@ -0,0 +1,3368 @@
+[
+    {
+        "current_steps": 0,
+        "loss": 0.6046,
+        "learning_rate": 1e-08,
+        "epoch": 0.008928571428571428
+    },
+    {
+        "current_steps": 1,
+        "loss": 0.6431,
+        "learning_rate": 2e-08,
+        "epoch": 0.017857142857142856
+    },
+    {
+        "current_steps": 2,
+        "loss": 0.6447,
+        "learning_rate": 3e-08,
+        "epoch": 0.026785714285714284
+    },
+    {
+        "current_steps": 3,
+        "loss": 0.7972,
+        "learning_rate": 4e-08,
+        "epoch": 0.03571428571428571
+    },
+    {
+        "current_steps": 4,
+        "loss": 0.6911,
+        "learning_rate": 5e-08,
+        "epoch": 0.044642857142857144
+    },
+    {
+        "current_steps": 5,
+        "loss": 0.8546,
+        "learning_rate": 6e-08,
+        "epoch": 0.05357142857142857
+    },
+    {
+        "current_steps": 6,
+        "loss": 0.7624,
+        "learning_rate": 6e-08,
+        "epoch": 0.0625
+    },
+    {
+        "current_steps": 7,
+        "loss": 0.6565,
+        "learning_rate": 7e-08,
+        "epoch": 0.07142857142857142
+    },
+    {
+        "current_steps": 8,
+        "loss": 0.6789,
+        "learning_rate": 8e-08,
+        "epoch": 0.08035714285714286
+    },
+    {
+        "current_steps": 9,
+        "loss": 0.8562,
+        "learning_rate": 8e-08,
+        "epoch": 0.08928571428571429
+    },
+    {
+        "current_steps": 10,
+        "loss": 0.8084,
+        "learning_rate": 9e-08,
+        "epoch": 0.09821428571428571
+    },
+    {
+        "current_steps": 11,
+        "loss": 0.7024,
+        "learning_rate": 1e-07,
+        "epoch": 0.10714285714285714
+    },
+    {
+        "current_steps": 12,
+        "loss": 0.7454,
+        "learning_rate": 1.0999999999999999e-07,
+        "epoch": 0.11607142857142858
+    },
+    {
+        "current_steps": 13,
+        "loss": 0.5896,
+        "learning_rate": 1.2e-07,
+        "epoch": 0.125
+    },
+    {
+        "current_steps": 14,
+        "loss": 1.1716,
+        "learning_rate": 1.3e-07,
+        "epoch": 0.13392857142857142
+    },
+    {
+        "current_steps": 15,
+        "loss": 0.8561,
+        "learning_rate": 1.4e-07,
+        "epoch": 0.14285714285714285
+    },
+    {
+        "current_steps": 16,
+        "loss": 0.9048,
+        "learning_rate": 1.5e-07,
+        "epoch": 0.15178571428571427
+    },
+    {
+        "current_steps": 17,
+        "loss": 0.6079,
+        "learning_rate": 1.6e-07,
+        "epoch": 0.16071428571428573
+    },
+    {
+        "current_steps": 18,
+        "loss": 0.9004,
+        "learning_rate": 1.7000000000000001e-07,
+        "epoch": 0.16964285714285715
+    },
+    {
+        "current_steps": 19,
+        "loss": 0.5512,
+        "learning_rate": 1.8e-07,
+        "epoch": 0.17857142857142858
+    },
+    {
+        "current_steps": 20,
+        "loss": 0.7782,
+        "learning_rate": 1.8999999999999998e-07,
+        "epoch": 0.1875
+    },
+    {
+        "current_steps": 21,
+        "loss": 0.7905,
+        "learning_rate": 2e-07,
+        "epoch": 0.19642857142857142
+    },
+    {
+        "current_steps": 22,
+        "loss": 0.6049,
+        "learning_rate": 2.0999999999999997e-07,
+        "epoch": 0.20535714285714285
+    },
+    {
+        "current_steps": 23,
+        "loss": 0.685,
+        "learning_rate": 2.1999999999999998e-07,
+        "epoch": 0.21428571428571427
+    },
+    {
+        "current_steps": 24,
+        "loss": 0.8171,
+        "learning_rate": 2.3e-07,
+        "epoch": 0.22321428571428573
+    },
+    {
+        "current_steps": 25,
+        "loss": 0.8018,
+        "learning_rate": 2.4e-07,
+        "epoch": 0.23214285714285715
+    },
+    {
+        "current_steps": 26,
+        "loss": 0.4959,
+        "learning_rate": 2.5e-07,
+        "epoch": 0.24107142857142858
+    },
+    {
+        "current_steps": 27,
+        "loss": 0.6348,
+        "learning_rate": 2.6e-07,
+        "epoch": 0.25
+    },
+    {
+        "current_steps": 28,
+        "loss": 0.8005,
+        "learning_rate": 2.7e-07,
+        "epoch": 0.25892857142857145
+    },
+    {
+        "current_steps": 29,
+        "loss": 0.6777,
+        "learning_rate": 2.8e-07,
+        "epoch": 0.26785714285714285
+    },
+    {
+        "current_steps": 30,
+        "loss": 0.9042,
+        "learning_rate": 2.9e-07,
+        "epoch": 0.2767857142857143
+    },
+    {
+        "current_steps": 31,
+        "loss": 0.6491,
+        "learning_rate": 3e-07,
+        "epoch": 0.2857142857142857
+    },
+    {
+        "current_steps": 32,
+        "loss": 1.0966,
+        "learning_rate": 3.1e-07,
+        "epoch": 0.29464285714285715
+    },
+    {
+        "current_steps": 33,
+        "loss": 0.7451,
+        "learning_rate": 3.2e-07,
+        "epoch": 0.30357142857142855
+    },
+    {
+        "current_steps": 34,
+        "loss": 1.1446,
+        "learning_rate": 3.2e-07,
+        "epoch": 0.3125
+    },
+    {
+        "current_steps": 35,
+        "loss": 0.7644,
+        "learning_rate": 3.3e-07,
+        "epoch": 0.32142857142857145
+    },
+    {
+        "current_steps": 36,
+        "loss": 0.7742,
+        "learning_rate": 3.4000000000000003e-07,
+        "epoch": 0.33035714285714285
+    },
+    {
+        "current_steps": 37,
+        "loss": 0.8247,
+        "learning_rate": 3.5e-07,
+        "epoch": 0.3392857142857143
+    },
+    {
+        "current_steps": 38,
+        "loss": 0.8667,
+        "learning_rate": 3.6e-07,
+        "epoch": 0.3482142857142857
+    },
+    {
+        "current_steps": 39,
+        "loss": 0.8309,
+        "learning_rate": 3.7e-07,
+        "epoch": 0.35714285714285715
+    },
+    {
+        "current_steps": 40,
+        "loss": 0.5913,
+        "learning_rate": 3.7999999999999996e-07,
+        "epoch": 0.36607142857142855
+    },
+    {
+        "current_steps": 41,
+        "loss": 0.5562,
+        "learning_rate": 3.8999999999999997e-07,
+        "epoch": 0.375
+    },
+    {
+        "current_steps": 42,
+        "loss": 1.6276,
+        "learning_rate": 4e-07,
+        "epoch": 0.38392857142857145
+    },
+    {
+        "current_steps": 43,
+        "loss": 0.682,
+        "learning_rate": 4.0999999999999994e-07,
+        "epoch": 0.39285714285714285
+    },
+    {
+        "current_steps": 44,
+        "loss": 0.8022,
+        "learning_rate": 4.1999999999999995e-07,
+        "epoch": 0.4017857142857143
+    },
+    {
+        "current_steps": 45,
+        "loss": 0.6702,
+        "learning_rate": 4.2999999999999996e-07,
+        "epoch": 0.4107142857142857
+    },
+    {
+        "current_steps": 46,
+        "loss": 0.6993,
+        "learning_rate": 4.3999999999999997e-07,
+        "epoch": 0.41964285714285715
+    },
+    {
+        "current_steps": 47,
+        "loss": 0.9685,
+        "learning_rate": 4.5e-07,
+        "epoch": 0.42857142857142855
+    },
+    {
+        "current_steps": 48,
+        "loss": 0.6637,
+        "learning_rate": 4.6e-07,
+        "epoch": 0.4375
+    },
+    {
+        "current_steps": 49,
+        "loss": 0.908,
+        "learning_rate": 4.6999999999999995e-07,
+        "epoch": 0.44642857142857145
+    },
+    {
+        "current_steps": 50,
+        "loss": 0.8683,
+        "learning_rate": 4.8e-07,
+        "epoch": 0.45535714285714285
+    },
+    {
+        "current_steps": 51,
+        "loss": 0.9243,
+        "learning_rate": 4.9e-07,
+        "epoch": 0.4642857142857143
+    },
+    {
+        "current_steps": 52,
+        "loss": 0.7933,
+        "learning_rate": 5e-07,
+        "epoch": 0.4732142857142857
+    },
+    {
+        "current_steps": 53,
+        "loss": 0.5856,
+        "learning_rate": 5.1e-07,
+        "epoch": 0.48214285714285715
+    },
+    {
+        "current_steps": 54,
+        "loss": 0.7097,
+        "learning_rate": 5.2e-07,
+        "epoch": 0.49107142857142855
+    },
+    {
+        "current_steps": 55,
+        "loss": 0.6476,
+        "learning_rate": 5.3e-07,
+        "epoch": 0.5
+    },
+    {
+        "current_steps": 56,
+        "loss": 0.8212,
+        "learning_rate": 5.4e-07,
+        "epoch": 0.5089285714285714
+    },
+    {
+        "current_steps": 57,
+        "loss": 0.7932,
+        "learning_rate": 5.5e-07,
+        "epoch": 0.5178571428571429
+    },
+    {
+        "current_steps": 58,
+        "loss": 0.8155,
+        "learning_rate": 5.6e-07,
+        "epoch": 0.5267857142857143
+    },
+    {
+        "current_steps": 59,
+        "loss": 0.5644,
+        "learning_rate": 5.699999999999999e-07,
+        "epoch": 0.5357142857142857
+    },
+    {
+        "current_steps": 60,
+        "loss": 0.8935,
+        "learning_rate": 5.8e-07,
+        "epoch": 0.5446428571428571
+    },
+    {
+        "current_steps": 61,
+        "loss": 0.6935,
+        "learning_rate": 5.9e-07,
+        "epoch": 0.5535714285714286
+    },
+    {
+        "current_steps": 62,
+        "loss": 0.6186,
+        "learning_rate": 6e-07,
+        "epoch": 0.5625
+    },
+    {
+        "current_steps": 63,
+        "loss": 0.7528,
+        "learning_rate": 6.1e-07,
+        "epoch": 0.5714285714285714
+    },
+    {
+        "current_steps": 64,
+        "loss": 0.7043,
+        "learning_rate": 6.2e-07,
+        "epoch": 0.5803571428571429
+    },
+    {
+        "current_steps": 65,
+        "loss": 0.5926,
+        "learning_rate": 6.3e-07,
+        "epoch": 0.5892857142857143
+    },
+    {
+        "current_steps": 66,
+        "loss": 0.7927,
+        "learning_rate": 6.4e-07,
+        "epoch": 0.5982142857142857
+    },
+    {
+        "current_steps": 67,
+        "loss": 0.5625,
+        "learning_rate": 6.5e-07,
+        "epoch": 0.6071428571428571
+    },
+    {
+        "current_steps": 68,
+        "loss": 0.707,
+        "learning_rate": 6.6e-07,
+        "epoch": 0.6160714285714286
+    },
+    {
+        "current_steps": 69,
+        "loss": 0.7023,
+        "learning_rate": 6.7e-07,
+        "epoch": 0.625
+    },
+    {
+        "current_steps": 70,
+        "loss": 0.586,
+        "learning_rate": 6.800000000000001e-07,
+        "epoch": 0.6339285714285714
+    },
+    {
+        "current_steps": 71,
+        "loss": 0.5741,
+        "learning_rate": 6.9e-07,
+        "epoch": 0.6428571428571429
+    },
+    {
+        "current_steps": 72,
+        "loss": 1.086,
+        "learning_rate": 7e-07,
+        "epoch": 0.6517857142857143
+    },
+    {
+        "current_steps": 73,
+        "loss": 0.6381,
+        "learning_rate": 7.1e-07,
+        "epoch": 0.6607142857142857
+    },
+    {
+        "current_steps": 74,
+        "loss": 0.7509,
+        "learning_rate": 7.2e-07,
+        "epoch": 0.6696428571428571
+    },
+    {
+        "current_steps": 75,
+        "loss": 0.8276,
+        "learning_rate": 7.3e-07,
+        "epoch": 0.6785714285714286
+    },
+    {
+        "current_steps": 76,
+        "loss": 0.7623,
+        "learning_rate": 7.4e-07,
+        "epoch": 0.6875
+    },
+    {
+        "current_steps": 77,
+        "loss": 0.9499,
+        "learning_rate": 7.5e-07,
+        "epoch": 0.6964285714285714
+    },
+    {
+        "current_steps": 78,
+        "loss": 0.8563,
+        "learning_rate": 7.599999999999999e-07,
+        "epoch": 0.7053571428571429
+    },
+    {
+        "current_steps": 79,
+        "loss": 0.6512,
+        "learning_rate": 7.699999999999999e-07,
+        "epoch": 0.7142857142857143
+    },
+    {
+        "current_steps": 80,
+        "loss": 0.843,
+        "learning_rate": 7.799999999999999e-07,
+        "epoch": 0.7232142857142857
+    },
+    {
+        "current_steps": 81,
+        "loss": 0.7272,
+        "learning_rate": 7.9e-07,
+        "epoch": 0.7321428571428571
+    },
+    {
+        "current_steps": 82,
+        "loss": 0.5161,
+        "learning_rate": 8e-07,
+        "epoch": 0.7410714285714286
+    },
+    {
+        "current_steps": 83,
+        "loss": 0.8293,
+        "learning_rate": 8.1e-07,
+        "epoch": 0.75
+    },
+    {
+        "current_steps": 84,
+        "loss": 0.8704,
+        "learning_rate": 8.199999999999999e-07,
+        "epoch": 0.7589285714285714
+    },
+    {
+        "current_steps": 85,
+        "loss": 0.7255,
+        "learning_rate": 8.299999999999999e-07,
+        "epoch": 0.7678571428571429
+    },
+    {
+        "current_steps": 86,
+        "loss": 0.6252,
+        "learning_rate": 8.399999999999999e-07,
+        "epoch": 0.7767857142857143
+    },
+    {
+        "current_steps": 87,
+        "loss": 0.8116,
+        "learning_rate": 8.499999999999999e-07,
+        "epoch": 0.7857142857142857
+    },
+    {
+        "current_steps": 88,
+        "loss": 0.7703,
+        "learning_rate": 8.599999999999999e-07,
+        "epoch": 0.7946428571428571
+    },
+    {
+        "current_steps": 89,
+        "loss": 0.6496,
+        "learning_rate": 8.699999999999999e-07,
+        "epoch": 0.8035714285714286
+    },
+    {
+        "current_steps": 90,
+        "loss": 0.8585,
+        "learning_rate": 8.799999999999999e-07,
+        "epoch": 0.8125
+    },
+    {
+        "current_steps": 91,
+        "loss": 0.905,
+        "learning_rate": 8.9e-07,
+        "epoch": 0.8214285714285714
+    },
+    {
+        "current_steps": 92,
+        "loss": 0.9139,
+        "learning_rate": 9e-07,
+        "epoch": 0.8303571428571429
+    },
+    {
+        "current_steps": 93,
+        "loss": 0.9925,
+        "learning_rate": 9.1e-07,
+        "epoch": 0.8392857142857143
+    },
+    {
+        "current_steps": 94,
+        "loss": 0.7344,
+        "learning_rate": 9.2e-07,
+        "epoch": 0.8482142857142857
+    },
+    {
+        "current_steps": 95,
+        "loss": 0.7477,
+        "learning_rate": 9.3e-07,
+        "epoch": 0.8571428571428571
+    },
+    {
+        "current_steps": 96,
+        "loss": 0.671,
+        "learning_rate": 9.399999999999999e-07,
+        "epoch": 0.8660714285714286
+    },
+    {
+        "current_steps": 97,
+        "loss": 0.9654,
+        "learning_rate": 9.499999999999999e-07,
+        "epoch": 0.875
+    },
+    {
+        "current_steps": 98,
+        "loss": 0.6788,
+        "learning_rate": 9.6e-07,
+        "epoch": 0.8839285714285714
+    },
+    {
+        "current_steps": 99,
+        "loss": 0.764,
+        "learning_rate": 9.7e-07,
+        "epoch": 0.8928571428571429
+    },
+    {
+        "current_steps": 100,
+        "loss": 0.7536,
+        "learning_rate": 9.8e-07,
+        "epoch": 0.9017857142857143
+    },
+    {
+        "current_steps": 101,
+        "loss": 0.6409,
+        "learning_rate": 9.9e-07,
+        "epoch": 0.9107142857142857
+    },
+    {
+        "current_steps": 102,
+        "loss": 0.904,
+        "learning_rate": 1e-06,
+        "epoch": 0.9196428571428571
+    },
+    {
+        "current_steps": 103,
+        "loss": 0.7079,
+        "learning_rate": 9.978260869565217e-07,
+        "epoch": 0.9285714285714286
+    },
+    {
+        "current_steps": 104,
+        "loss": 0.748,
+        "learning_rate": 9.956521739130434e-07,
+        "epoch": 0.9375
+    },
+    {
+        "current_steps": 105,
+        "loss": 0.7228,
+        "learning_rate": 9.934782608695653e-07,
+        "epoch": 0.9464285714285714
+    },
+    {
+        "current_steps": 106,
+        "loss": 0.722,
+        "learning_rate": 9.91304347826087e-07,
+        "epoch": 0.9553571428571429
+    },
+    {
+        "current_steps": 107,
+        "loss": 0.8011,
+        "learning_rate": 9.891304347826085e-07,
+        "epoch": 0.9642857142857143
+    },
+    {
+        "current_steps": 108,
+        "loss": 0.8125,
+        "learning_rate": 9.869565217391304e-07,
+        "epoch": 0.9732142857142857
+    },
+    {
+        "current_steps": 109,
+        "loss": 0.8091,
+        "learning_rate": 9.847826086956522e-07,
+        "epoch": 0.9821428571428571
+    },
+    {
+        "current_steps": 110,
+        "loss": 0.9399,
+        "learning_rate": 9.826086956521739e-07,
+        "epoch": 0.9910714285714286
+    },
+    {
+        "current_steps": 111,
+        "loss": 1.0917,
+        "learning_rate": 9.804347826086956e-07,
+        "epoch": 1.0
+    },
+    {
+        "current_steps": 112,
+        "loss": 0.9014,
+        "learning_rate": 9.782608695652173e-07,
+        "epoch": 1.0089285714285714
+    },
+    {
+        "current_steps": 113,
+        "loss": 0.873,
+        "learning_rate": 9.782608695652173e-07,
+        "epoch": 1.0178571428571428
+    },
+    {
+        "current_steps": 114,
+        "loss": 0.7153,
+        "learning_rate": 9.76086956521739e-07,
+        "epoch": 1.0267857142857142
+    },
+    {
+        "current_steps": 115,
+        "loss": 0.8828,
+        "learning_rate": 9.73913043478261e-07,
+        "epoch": 1.0357142857142858
+    },
+    {
+        "current_steps": 116,
+        "loss": 1.0329,
+        "learning_rate": 9.717391304347827e-07,
+        "epoch": 1.0446428571428572
+    },
+    {
+        "current_steps": 117,
+        "loss": 1.057,
+        "learning_rate": 9.695652173913042e-07,
+        "epoch": 1.0535714285714286
+    },
+    {
+        "current_steps": 118,
+        "loss": 0.8047,
+        "learning_rate": 9.67391304347826e-07,
+        "epoch": 1.0625
+    },
+    {
+        "current_steps": 119,
+        "loss": 0.7098,
+        "learning_rate": 9.652173913043478e-07,
+        "epoch": 1.0714285714285714
+    },
+    {
+        "current_steps": 120,
+        "loss": 1.094,
+        "learning_rate": 9.630434782608695e-07,
+        "epoch": 1.0803571428571428
+    },
+    {
+        "current_steps": 121,
+        "loss": 0.7521,
+        "learning_rate": 9.608695652173912e-07,
+        "epoch": 1.0892857142857142
+    },
+    {
+        "current_steps": 122,
+        "loss": 0.9738,
+        "learning_rate": 9.58695652173913e-07,
+        "epoch": 1.0982142857142858
+    },
+    {
+        "current_steps": 123,
+        "loss": 0.5577,
+        "learning_rate": 9.565217391304349e-07,
+        "epoch": 1.1071428571428572
+    },
+    {
+        "current_steps": 124,
+        "loss": 1.046,
+        "learning_rate": 9.543478260869566e-07,
+        "epoch": 1.1160714285714286
+    },
+    {
+        "current_steps": 125,
+        "loss": 0.597,
+        "learning_rate": 9.521739130434783e-07,
+        "epoch": 1.125
+    },
+    {
+        "current_steps": 126,
+        "loss": 0.7996,
+        "learning_rate": 9.499999999999999e-07,
+        "epoch": 1.1339285714285714
+    },
+    {
+        "current_steps": 127,
+        "loss": 0.9885,
+        "learning_rate": 9.478260869565216e-07,
+        "epoch": 1.1428571428571428
+    },
+    {
+        "current_steps": 128,
+        "loss": 0.6274,
+        "learning_rate": 9.456521739130434e-07,
+        "epoch": 1.1517857142857142
+    },
+    {
+        "current_steps": 129,
+        "loss": 0.8557,
+        "learning_rate": 9.434782608695652e-07,
+        "epoch": 1.1607142857142858
+    },
+    {
+        "current_steps": 130,
+        "loss": 0.702,
+        "learning_rate": 9.41304347826087e-07,
+        "epoch": 1.1696428571428572
+    },
+    {
+        "current_steps": 131,
+        "loss": 0.6905,
+        "learning_rate": 9.391304347826087e-07,
+        "epoch": 1.1785714285714286
+    },
+    {
+        "current_steps": 132,
+        "loss": 0.5707,
+        "learning_rate": 9.369565217391304e-07,
+        "epoch": 1.1875
+    },
+    {
+        "current_steps": 133,
+        "loss": 0.6121,
+        "learning_rate": 9.347826086956522e-07,
+        "epoch": 1.1964285714285714
+    },
+    {
+        "current_steps": 134,
+        "loss": 0.8348,
+        "learning_rate": 9.326086956521738e-07,
+        "epoch": 1.2053571428571428
+    },
+    {
+        "current_steps": 135,
+        "loss": 0.8768,
+        "learning_rate": 9.304347826086955e-07,
+        "epoch": 1.2142857142857142
+    },
+    {
+        "current_steps": 136,
+        "loss": 0.5648,
+        "learning_rate": 9.282608695652174e-07,
+        "epoch": 1.2232142857142858
+    },
+    {
+        "current_steps": 137,
+        "loss": 0.6316,
+        "learning_rate": 9.260869565217391e-07,
+        "epoch": 1.2321428571428572
+    },
+    {
+        "current_steps": 138,
+        "loss": 1.1728,
+        "learning_rate": 9.239130434782608e-07,
+        "epoch": 1.2410714285714286
+    },
+    {
+        "current_steps": 139,
+        "loss": 0.7299,
+        "learning_rate": 9.217391304347826e-07,
+        "epoch": 1.25
+    },
+    {
+        "current_steps": 140,
+        "loss": 0.6284,
+        "learning_rate": 9.195652173913043e-07,
+        "epoch": 1.2589285714285714
+    },
+    {
+        "current_steps": 141,
+        "loss": 0.6366,
+        "learning_rate": 9.17391304347826e-07,
+        "epoch": 1.2678571428571428
+    },
+    {
+        "current_steps": 142,
+        "loss": 0.7357,
+        "learning_rate": 9.152173913043479e-07,
+        "epoch": 1.2767857142857144
+    },
+    {
+        "current_steps": 143,
+        "loss": 0.8618,
+        "learning_rate": 9.130434782608695e-07,
+        "epoch": 1.2857142857142856
+    },
+    {
+        "current_steps": 144,
+        "loss": 0.6803,
+        "learning_rate": 9.108695652173912e-07,
+        "epoch": 1.2946428571428572
+    },
+    {
+        "current_steps": 145,
+        "loss": 0.8093,
+        "learning_rate": 9.08695652173913e-07,
+        "epoch": 1.3035714285714286
+    },
+    {
+        "current_steps": 146,
+        "loss": 0.6808,
+        "learning_rate": 9.065217391304347e-07,
+        "epoch": 1.3125
+    },
+    {
+        "current_steps": 147,
+        "loss": 0.7173,
+        "learning_rate": 9.043478260869564e-07,
+        "epoch": 1.3214285714285714
+    },
+    {
+        "current_steps": 148,
+        "loss": 0.6964,
+        "learning_rate": 9.021739130434782e-07,
+        "epoch": 1.3303571428571428
+    },
+    {
+        "current_steps": 149,
+        "loss": 0.5458,
+        "learning_rate": 9e-07,
+        "epoch": 1.3392857142857144
+    },
+    {
+        "current_steps": 150,
+        "loss": 0.5362,
+        "learning_rate": 8.978260869565218e-07,
+        "epoch": 1.3482142857142856
+    },
+    {
+        "current_steps": 151,
+        "loss": 0.7248,
+        "learning_rate": 8.956521739130435e-07,
+        "epoch": 1.3571428571428572
+    },
+    {
+        "current_steps": 152,
+        "loss": 0.9701,
+        "learning_rate": 8.934782608695651e-07,
+        "epoch": 1.3660714285714286
+    },
+    {
+        "current_steps": 153,
+        "loss": 0.6072,
+        "learning_rate": 8.913043478260869e-07,
+        "epoch": 1.375
+    },
+    {
+        "current_steps": 154,
+        "loss": 0.8135,
+        "learning_rate": 8.891304347826086e-07,
+        "epoch": 1.3839285714285714
+    },
+    {
+        "current_steps": 155,
+        "loss": 0.6519,
+        "learning_rate": 8.869565217391303e-07,
+        "epoch": 1.3928571428571428
+    },
+    {
+        "current_steps": 156,
+        "loss": 0.7911,
+        "learning_rate": 8.847826086956522e-07,
+        "epoch": 1.4017857142857144
+    },
+    {
+        "current_steps": 157,
+        "loss": 0.7084,
+        "learning_rate": 8.826086956521739e-07,
+        "epoch": 1.4107142857142856
+    },
+    {
+        "current_steps": 158,
+        "loss": 0.6062,
+        "learning_rate": 8.804347826086956e-07,
+        "epoch": 1.4196428571428572
+    },
+    {
+        "current_steps": 159,
+        "loss": 0.5372,
+        "learning_rate": 8.782608695652174e-07,
+        "epoch": 1.4285714285714286
+    },
+    {
+        "current_steps": 160,
+        "loss": 0.7001,
+        "learning_rate": 8.760869565217391e-07,
+        "epoch": 1.4375
+    },
+    {
+        "current_steps": 161,
+        "loss": 0.628,
+        "learning_rate": 8.739130434782607e-07,
+        "epoch": 1.4464285714285714
+    },
+    {
+        "current_steps": 162,
+        "loss": 0.6766,
+        "learning_rate": 8.717391304347826e-07,
+        "epoch": 1.4553571428571428
+    },
+    {
+        "current_steps": 163,
+        "loss": 0.7406,
+        "learning_rate": 8.695652173913043e-07,
+        "epoch": 1.4642857142857144
+    },
+    {
+        "current_steps": 164,
+        "loss": 0.7032,
+        "learning_rate": 8.67391304347826e-07,
+        "epoch": 1.4732142857142856
+    },
+    {
+        "current_steps": 165,
+        "loss": 0.8338,
+        "learning_rate": 8.652173913043478e-07,
+        "epoch": 1.4821428571428572
+    },
+    {
+        "current_steps": 166,
+        "loss": 0.6067,
+        "learning_rate": 8.630434782608695e-07,
+        "epoch": 1.4910714285714286
+    },
+    {
+        "current_steps": 167,
+        "loss": 0.6988,
+        "learning_rate": 8.608695652173913e-07,
+        "epoch": 1.5
+    },
+    {
+        "current_steps": 168,
+        "loss": 0.6294,
+        "learning_rate": 8.586956521739131e-07,
+        "epoch": 1.5089285714285714
+    },
+    {
+        "current_steps": 169,
+        "loss": 0.7358,
+        "learning_rate": 8.565217391304348e-07,
+        "epoch": 1.5178571428571428
+    },
+    {
+        "current_steps": 170,
+        "loss": 0.7709,
+        "learning_rate": 8.543478260869565e-07,
+        "epoch": 1.5267857142857144
+    },
+    {
+        "current_steps": 171,
+        "loss": 0.8913,
+        "learning_rate": 8.521739130434782e-07,
+        "epoch": 1.5357142857142856
+    },
+    {
+        "current_steps": 172,
+        "loss": 0.697,
+        "learning_rate": 8.499999999999999e-07,
+        "epoch": 1.5446428571428572
+    },
+    {
+        "current_steps": 173,
+        "loss": 0.7902,
+        "learning_rate": 8.478260869565217e-07,
+        "epoch": 1.5535714285714286
+    },
+    {
+        "current_steps": 174,
+        "loss": 0.7858,
+        "learning_rate": 8.456521739130434e-07,
+        "epoch": 1.5625
+    },
+    {
+        "current_steps": 175,
+        "loss": 0.8903,
+        "learning_rate": 8.434782608695652e-07,
+        "epoch": 1.5714285714285714
+    },
+    {
+        "current_steps": 176,
+        "loss": 0.8324,
+        "learning_rate": 8.41304347826087e-07,
+        "epoch": 1.5803571428571428
+    },
+    {
+        "current_steps": 177,
+        "loss": 0.7323,
+        "learning_rate": 8.391304347826087e-07,
+        "epoch": 1.5892857142857144
+    },
+    {
+        "current_steps": 178,
+        "loss": 0.7527,
+        "learning_rate": 8.369565217391304e-07,
+        "epoch": 1.5982142857142856
+    },
+    {
+        "current_steps": 179,
+        "loss": 0.8336,
+        "learning_rate": 8.347826086956521e-07,
+        "epoch": 1.6071428571428572
+    },
+    {
+        "current_steps": 180,
+        "loss": 0.7886,
+        "learning_rate": 8.326086956521738e-07,
+        "epoch": 1.6160714285714286
+    },
+    {
+        "current_steps": 181,
+        "loss": 0.7455,
+        "learning_rate": 8.304347826086955e-07,
+        "epoch": 1.625
+    },
+    {
+        "current_steps": 182,
+        "loss": 0.7702,
+        "learning_rate": 8.282608695652174e-07,
+        "epoch": 1.6339285714285714
+    },
+    {
+        "current_steps": 183,
+        "loss": 0.6935,
+        "learning_rate": 8.260869565217391e-07,
+        "epoch": 1.6428571428571428
+    },
+    {
+        "current_steps": 184,
+        "loss": 0.6778,
+        "learning_rate": 8.239130434782609e-07,
+        "epoch": 1.6517857142857144
+    },
+    {
+        "current_steps": 185,
+        "loss": 0.7623,
+        "learning_rate": 8.217391304347826e-07,
+        "epoch": 1.6607142857142856
+    },
+    {
+        "current_steps": 186,
+        "loss": 0.8068,
+        "learning_rate": 8.195652173913043e-07,
+        "epoch": 1.6696428571428572
+    },
+    {
+        "current_steps": 187,
+        "loss": 0.6384,
+        "learning_rate": 8.173913043478261e-07,
+        "epoch": 1.6785714285714286
+    },
+    {
+        "current_steps": 188,
+        "loss": 0.9876,
+        "learning_rate": 8.152173913043478e-07,
+        "epoch": 1.6875
+    },
+    {
+        "current_steps": 189,
+        "loss": 0.5316,
+        "learning_rate": 8.130434782608695e-07,
+        "epoch": 1.6964285714285714
+    },
+    {
+        "current_steps": 190,
+        "loss": 0.6117,
+        "learning_rate": 8.108695652173913e-07,
+        "epoch": 1.7053571428571428
+    },
+    {
+        "current_steps": 191,
+        "loss": 0.5897,
+        "learning_rate": 8.08695652173913e-07,
+        "epoch": 1.7142857142857144
+    },
+    {
+        "current_steps": 192,
+        "loss": 0.7045,
+        "learning_rate": 8.065217391304347e-07,
+        "epoch": 1.7232142857142856
+    },
+    {
+        "current_steps": 193,
+        "loss": 0.7491,
+        "learning_rate": 8.043478260869565e-07,
+        "epoch": 1.7321428571428572
+    },
+    {
+        "current_steps": 194,
+        "loss": 0.8067,
+        "learning_rate": 8.021739130434782e-07,
+        "epoch": 1.7410714285714286
+    },
+    {
+        "current_steps": 195,
+        "loss": 0.9085,
+        "learning_rate": 8e-07,
+        "epoch": 1.75
+    },
+    {
+        "current_steps": 196,
+        "loss": 0.7977,
+        "learning_rate": 7.978260869565217e-07,
+        "epoch": 1.7589285714285714
+    },
+    {
+        "current_steps": 197,
+        "loss": 0.7509,
+        "learning_rate": 7.956521739130434e-07,
+        "epoch": 1.7678571428571428
+    },
+    {
+        "current_steps": 198,
+        "loss": 0.7048,
+        "learning_rate": 7.934782608695651e-07,
+        "epoch": 1.7767857142857144
+    },
+    {
+        "current_steps": 199,
+        "loss": 0.6452,
+        "learning_rate": 7.913043478260869e-07,
+        "epoch": 1.7857142857142856
+    },
+    {
+        "current_steps": 200,
+        "loss": 0.7265,
+        "learning_rate": 7.891304347826086e-07,
+        "epoch": 1.7946428571428572
+    },
+    {
+        "current_steps": 201,
+        "loss": 0.7936,
+        "learning_rate": 7.869565217391305e-07,
+        "epoch": 1.8035714285714286
+    },
+    {
+        "current_steps": 202,
+        "loss": 0.7336,
+        "learning_rate": 7.847826086956522e-07,
+        "epoch": 1.8125
+    },
+    {
+        "current_steps": 203,
+        "loss": 0.6462,
+        "learning_rate": 7.826086956521739e-07,
+        "epoch": 1.8214285714285714
+    },
+    {
+        "current_steps": 204,
+        "loss": 0.579,
+        "learning_rate": 7.804347826086957e-07,
+        "epoch": 1.8303571428571428
+    },
+    {
+        "current_steps": 205,
+        "loss": 0.6014,
+        "learning_rate": 7.782608695652173e-07,
+        "epoch": 1.8392857142857144
+    },
+    {
+        "current_steps": 206,
+        "loss": 0.684,
+        "learning_rate": 7.76086956521739e-07,
+        "epoch": 1.8482142857142856
+    },
+    {
+        "current_steps": 207,
+        "loss": 0.5932,
+        "learning_rate": 7.739130434782608e-07,
+        "epoch": 1.8571428571428572
+    },
+    {
+        "current_steps": 208,
+        "loss": 0.7736,
+        "learning_rate": 7.717391304347826e-07,
+        "epoch": 1.8660714285714286
+    },
+    {
+        "current_steps": 209,
+        "loss": 0.7601,
+        "learning_rate": 7.695652173913043e-07,
+        "epoch": 1.875
+    },
+    {
+        "current_steps": 210,
+        "loss": 0.8428,
+        "learning_rate": 7.673913043478261e-07,
+        "epoch": 1.8839285714285714
+    },
+    {
+        "current_steps": 211,
+        "loss": 0.8017,
+        "learning_rate": 7.652173913043478e-07,
+        "epoch": 1.8928571428571428
+    },
+    {
+        "current_steps": 212,
+        "loss": 0.5998,
+        "learning_rate": 7.630434782608695e-07,
+        "epoch": 1.9017857142857144
+    },
+    {
+        "current_steps": 213,
+        "loss": 0.9071,
+        "learning_rate": 7.608695652173913e-07,
+        "epoch": 1.9107142857142856
+    },
+    {
+        "current_steps": 214,
+        "loss": 0.8255,
+        "learning_rate": 7.58695652173913e-07,
+        "epoch": 1.9196428571428572
+    },
+    {
+        "current_steps": 215,
+        "loss": 0.9256,
+        "learning_rate": 7.565217391304347e-07,
+        "epoch": 1.9285714285714286
+    },
+    {
+        "current_steps": 216,
+        "loss": 0.6745,
+        "learning_rate": 7.543478260869565e-07,
+        "epoch": 1.9375
+    },
+    {
+        "current_steps": 217,
+        "loss": 0.6372,
+        "learning_rate": 7.521739130434782e-07,
+        "epoch": 1.9464285714285714
+    },
+    {
+        "current_steps": 218,
+        "loss": 0.6495,
+        "learning_rate": 7.5e-07,
+        "epoch": 1.9553571428571428
+    },
+    {
+        "current_steps": 219,
+        "loss": 0.6054,
+        "learning_rate": 7.478260869565217e-07,
+        "epoch": 1.9642857142857144
+    },
+    {
+        "current_steps": 220,
+        "loss": 0.9751,
+        "learning_rate": 7.478260869565217e-07,
+        "epoch": 1.9732142857142856
+    },
+    {
+        "current_steps": 221,
+        "loss": 0.6258,
+        "learning_rate": 7.456521739130434e-07,
+        "epoch": 1.9821428571428572
+    },
+    {
+        "current_steps": 222,
+        "loss": 0.794,
+        "learning_rate": 7.434782608695653e-07,
+        "epoch": 1.9910714285714286
+    },
+    {
+        "current_steps": 223,
+        "loss": 0.9991,
+        "learning_rate": 7.41304347826087e-07,
+        "epoch": 2.0
+    },
+    {
+        "current_steps": 224,
+        "loss": 0.8048,
+        "learning_rate": 7.391304347826086e-07,
+        "epoch": 2.0089285714285716
+    },
+    {
+        "current_steps": 225,
+        "loss": 0.8439,
+        "learning_rate": 7.369565217391304e-07,
+        "epoch": 2.017857142857143
+    },
+    {
+        "current_steps": 226,
+        "loss": 0.7546,
+        "learning_rate": 7.347826086956521e-07,
+        "epoch": 2.0267857142857144
+    },
+    {
+        "current_steps": 227,
+        "loss": 0.8195,
+        "learning_rate": 7.326086956521738e-07,
+        "epoch": 2.0357142857142856
+    },
+    {
+        "current_steps": 228,
+        "loss": 0.6988,
+        "learning_rate": 7.304347826086957e-07,
+        "epoch": 2.044642857142857
+    },
+    {
+        "current_steps": 229,
+        "loss": 0.8419,
+        "learning_rate": 7.282608695652174e-07,
+        "epoch": 2.0535714285714284
+    },
+    {
+        "current_steps": 230,
+        "loss": 0.6133,
+        "learning_rate": 7.260869565217391e-07,
+        "epoch": 2.0625
+    },
+    {
+        "current_steps": 231,
+        "loss": 0.6307,
+        "learning_rate": 7.239130434782609e-07,
+        "epoch": 2.0714285714285716
+    },
+    {
+        "current_steps": 232,
+        "loss": 0.7852,
+        "learning_rate": 7.217391304347826e-07,
+        "epoch": 2.080357142857143
+    },
+    {
+        "current_steps": 233,
+        "loss": 0.4894,
+        "learning_rate": 7.195652173913042e-07,
+        "epoch": 2.0892857142857144
+    },
+    {
+        "current_steps": 234,
+        "loss": 0.6806,
+        "learning_rate": 7.17391304347826e-07,
+        "epoch": 2.0982142857142856
+    },
+    {
+        "current_steps": 235,
+        "loss": 0.7798,
+        "learning_rate": 7.152173913043478e-07,
+        "epoch": 2.107142857142857
+    },
+    {
+        "current_steps": 236,
+        "loss": 0.934,
+        "learning_rate": 7.130434782608695e-07,
+        "epoch": 2.1160714285714284
+    },
+    {
+        "current_steps": 237,
+        "loss": 0.8044,
+        "learning_rate": 7.108695652173913e-07,
+        "epoch": 2.125
+    },
+    {
+        "current_steps": 238,
+        "loss": 0.8984,
+        "learning_rate": 7.08695652173913e-07,
+        "epoch": 2.1339285714285716
+    },
+    {
+        "current_steps": 239,
+        "loss": 0.7468,
+        "learning_rate": 7.065217391304348e-07,
+        "epoch": 2.142857142857143
+    },
+    {
+        "current_steps": 240,
+        "loss": 0.744,
+        "learning_rate": 7.043478260869565e-07,
+        "epoch": 2.1517857142857144
+    },
+    {
+        "current_steps": 241,
+        "loss": 0.5531,
+        "learning_rate": 7.021739130434783e-07,
+        "epoch": 2.1607142857142856
+    },
+    {
+        "current_steps": 242,
+        "loss": 0.8155,
+        "learning_rate": 7e-07,
+        "epoch": 2.169642857142857
+    },
+    {
+        "current_steps": 243,
+        "loss": 0.7626,
+        "learning_rate": 6.978260869565217e-07,
+        "epoch": 2.1785714285714284
+    },
+    {
+        "current_steps": 244,
+        "loss": 0.5438,
+        "learning_rate": 6.956521739130434e-07,
+        "epoch": 2.1875
+    },
+    {
+        "current_steps": 245,
+        "loss": 0.7638,
+        "learning_rate": 6.934782608695652e-07,
+        "epoch": 2.1964285714285716
+    },
+    {
+        "current_steps": 246,
+        "loss": 0.5092,
+        "learning_rate": 6.913043478260869e-07,
+        "epoch": 2.205357142857143
+    },
+    {
+        "current_steps": 247,
+        "loss": 0.7026,
+        "learning_rate": 6.891304347826086e-07,
+        "epoch": 2.2142857142857144
+    },
+    {
+        "current_steps": 248,
+        "loss": 0.727,
+        "learning_rate": 6.869565217391305e-07,
+        "epoch": 2.2232142857142856
+    },
+    {
+        "current_steps": 249,
+        "loss": 0.6229,
+        "learning_rate": 6.847826086956522e-07,
+        "epoch": 2.232142857142857
+    },
+    {
+        "current_steps": 250,
+        "loss": 0.6695,
+        "learning_rate": 6.826086956521738e-07,
+        "epoch": 2.2410714285714284
+    },
+    {
+        "current_steps": 251,
+        "loss": 0.6603,
+        "learning_rate": 6.804347826086956e-07,
+        "epoch": 2.25
+    },
+    {
+        "current_steps": 252,
+        "loss": 0.7804,
+        "learning_rate": 6.782608695652173e-07,
+        "epoch": 2.2589285714285716
+    },
+    {
+        "current_steps": 253,
+        "loss": 0.9138,
+        "learning_rate": 6.76086956521739e-07,
+        "epoch": 2.267857142857143
+    },
+    {
+        "current_steps": 254,
+        "loss": 0.7793,
+        "learning_rate": 6.739130434782609e-07,
+        "epoch": 2.2767857142857144
+    },
+    {
+        "current_steps": 255,
+        "loss": 0.7045,
+        "learning_rate": 6.717391304347826e-07,
+        "epoch": 2.2857142857142856
+    },
+    {
+        "current_steps": 256,
+        "loss": 0.8594,
+        "learning_rate": 6.695652173913044e-07,
+        "epoch": 2.294642857142857
+    },
+    {
+        "current_steps": 257,
+        "loss": 0.9529,
+        "learning_rate": 6.673913043478261e-07,
+        "epoch": 2.3035714285714284
+    },
+    {
+        "current_steps": 258,
+        "loss": 0.7477,
+        "learning_rate": 6.652173913043478e-07,
+        "epoch": 2.3125
+    },
+    {
+        "current_steps": 259,
+        "loss": 0.7676,
+        "learning_rate": 6.630434782608695e-07,
+        "epoch": 2.3214285714285716
+    },
+    {
+        "current_steps": 260,
+        "loss": 0.6468,
+        "learning_rate": 6.608695652173912e-07,
+        "epoch": 2.330357142857143
+    },
+    {
+        "current_steps": 261,
+        "loss": 0.6665,
+        "learning_rate": 6.58695652173913e-07,
+        "epoch": 2.3392857142857144
+    },
+    {
+        "current_steps": 262,
+        "loss": 0.838,
+        "learning_rate": 6.565217391304348e-07,
+        "epoch": 2.3482142857142856
+    },
+    {
+        "current_steps": 263,
+        "loss": 0.7129,
+        "learning_rate": 6.543478260869565e-07,
+        "epoch": 2.357142857142857
+    },
+    {
+        "current_steps": 264,
+        "loss": 0.8685,
+        "learning_rate": 6.521739130434782e-07,
+        "epoch": 2.3660714285714284
+    },
+    {
+        "current_steps": 265,
+        "loss": 0.7224,
+        "learning_rate": 6.5e-07,
+        "epoch": 2.375
+    },
+    {
+        "current_steps": 266,
+        "loss": 0.7037,
+        "learning_rate": 6.478260869565217e-07,
+        "epoch": 2.3839285714285716
+    },
+    {
+        "current_steps": 267,
+        "loss": 0.5596,
+        "learning_rate": 6.456521739130435e-07,
+        "epoch": 2.392857142857143
+    },
+    {
+        "current_steps": 268,
+        "loss": 0.8887,
+        "learning_rate": 6.434782608695652e-07,
+        "epoch": 2.4017857142857144
+    },
+    {
+        "current_steps": 269,
+        "loss": 0.6721,
+        "learning_rate": 6.413043478260869e-07,
+        "epoch": 2.4107142857142856
+    },
+    {
+        "current_steps": 270,
+        "loss": 0.7387,
+        "learning_rate": 6.391304347826086e-07,
+        "epoch": 2.419642857142857
+    },
+    {
+        "current_steps": 271,
+        "loss": 0.6304,
+        "learning_rate": 6.369565217391304e-07,
+        "epoch": 2.4285714285714284
+    },
+    {
+        "current_steps": 272,
+        "loss": 0.7563,
+        "learning_rate": 6.347826086956521e-07,
+        "epoch": 2.4375
+    },
+    {
+        "current_steps": 273,
+        "loss": 0.6833,
+        "learning_rate": 6.326086956521739e-07,
+        "epoch": 2.4464285714285716
+    },
+    {
+        "current_steps": 274,
+        "loss": 0.722,
+        "learning_rate": 6.304347826086957e-07,
+        "epoch": 2.455357142857143
+    },
+    {
+        "current_steps": 275,
+        "loss": 0.8583,
+        "learning_rate": 6.282608695652174e-07,
+        "epoch": 2.4642857142857144
+    },
+    {
+        "current_steps": 276,
+        "loss": 0.8988,
+        "learning_rate": 6.260869565217392e-07,
+        "epoch": 2.4732142857142856
+    },
+    {
+        "current_steps": 277,
+        "loss": 0.6269,
+        "learning_rate": 6.239130434782608e-07,
+        "epoch": 2.482142857142857
+    },
+    {
+        "current_steps": 278,
+        "loss": 0.473,
+        "learning_rate": 6.217391304347825e-07,
+        "epoch": 2.4910714285714284
+    },
+    {
+        "current_steps": 279,
+        "loss": 0.7065,
+        "learning_rate": 6.195652173913043e-07,
+        "epoch": 2.5
+    },
+    {
+        "current_steps": 280,
+        "loss": 0.7912,
+        "learning_rate": 6.17391304347826e-07,
+        "epoch": 2.508928571428571
+    },
+    {
+        "current_steps": 281,
+        "loss": 0.6589,
+        "learning_rate": 6.152173913043478e-07,
+        "epoch": 2.517857142857143
+    },
+    {
+        "current_steps": 282,
+        "loss": 0.5908,
+        "learning_rate": 6.130434782608696e-07,
+        "epoch": 2.5267857142857144
+    },
+    {
+        "current_steps": 283,
+        "loss": 0.839,
+        "learning_rate": 6.108695652173913e-07,
+        "epoch": 2.5357142857142856
+    },
+    {
+        "current_steps": 284,
+        "loss": 0.9573,
+        "learning_rate": 6.08695652173913e-07,
+        "epoch": 2.544642857142857
+    },
+    {
+        "current_steps": 285,
+        "loss": 0.8881,
+        "learning_rate": 6.065217391304348e-07,
+        "epoch": 2.553571428571429
+    },
+    {
+        "current_steps": 286,
+        "loss": 0.5213,
+        "learning_rate": 6.043478260869564e-07,
+        "epoch": 2.5625
+    },
+    {
+        "current_steps": 287,
+        "loss": 0.5668,
+        "learning_rate": 6.021739130434782e-07,
+        "epoch": 2.571428571428571
+    },
+    {
+        "current_steps": 288,
+        "loss": 0.6856,
+        "learning_rate": 6e-07,
+        "epoch": 2.580357142857143
+    },
+    {
+        "current_steps": 289,
+        "loss": 0.6793,
+        "learning_rate": 5.978260869565217e-07,
+        "epoch": 2.5892857142857144
+    },
+    {
+        "current_steps": 290,
+        "loss": 0.6176,
+        "learning_rate": 5.956521739130435e-07,
+        "epoch": 2.5982142857142856
+    },
+    {
+        "current_steps": 291,
+        "loss": 0.5633,
+        "learning_rate": 5.934782608695652e-07,
+        "epoch": 2.607142857142857
+    },
+    {
+        "current_steps": 292,
+        "loss": 0.8512,
+        "learning_rate": 5.913043478260869e-07,
+        "epoch": 2.616071428571429
+    },
+    {
+        "current_steps": 293,
+        "loss": 0.9664,
+        "learning_rate": 5.891304347826088e-07,
+        "epoch": 2.625
+    },
+    {
+        "current_steps": 294,
+        "loss": 0.6124,
+        "learning_rate": 5.869565217391305e-07,
+        "epoch": 2.633928571428571
+    },
+    {
+        "current_steps": 295,
+        "loss": 0.6244,
+        "learning_rate": 5.847826086956521e-07,
+        "epoch": 2.642857142857143
+    },
+    {
+        "current_steps": 296,
+        "loss": 0.7879,
+        "learning_rate": 5.826086956521739e-07,
+        "epoch": 2.6517857142857144
+    },
+    {
+        "current_steps": 297,
+        "loss": 0.6862,
+        "learning_rate": 5.804347826086956e-07,
+        "epoch": 2.6607142857142856
+    },
+    {
+        "current_steps": 298,
+        "loss": 0.6368,
+        "learning_rate": 5.782608695652173e-07,
+        "epoch": 2.669642857142857
+    },
+    {
+        "current_steps": 299,
+        "loss": 0.8478,
+        "learning_rate": 5.760869565217391e-07,
+        "epoch": 2.678571428571429
+    },
+    {
+        "current_steps": 300,
+        "loss": 0.6466,
+        "learning_rate": 5.739130434782609e-07,
+        "epoch": 2.6875
+    },
+    {
+        "current_steps": 301,
+        "loss": 0.7323,
+        "learning_rate": 5.717391304347826e-07,
+        "epoch": 2.696428571428571
+    },
+    {
+        "current_steps": 302,
+        "loss": 0.7611,
+        "learning_rate": 5.695652173913044e-07,
+        "epoch": 2.705357142857143
+    },
+    {
+        "current_steps": 303,
+        "loss": 0.7075,
+        "learning_rate": 5.673913043478261e-07,
+        "epoch": 2.7142857142857144
+    },
+    {
+        "current_steps": 304,
+        "loss": 0.5448,
+        "learning_rate": 5.652173913043477e-07,
+        "epoch": 2.7232142857142856
+    },
+    {
+        "current_steps": 305,
+        "loss": 0.704,
+        "learning_rate": 5.630434782608695e-07,
+        "epoch": 2.732142857142857
+    },
+    {
+        "current_steps": 306,
+        "loss": 0.8591,
+        "learning_rate": 5.608695652173912e-07,
+        "epoch": 2.741071428571429
+    },
+    {
+        "current_steps": 307,
+        "loss": 0.6702,
+        "learning_rate": 5.58695652173913e-07,
+        "epoch": 2.75
+    },
+    {
+        "current_steps": 308,
+        "loss": 0.6652,
+        "learning_rate": 5.565217391304348e-07,
+        "epoch": 2.758928571428571
+    },
+    {
+        "current_steps": 309,
+        "loss": 0.7208,
+        "learning_rate": 5.543478260869565e-07,
+        "epoch": 2.767857142857143
+    },
+    {
+        "current_steps": 310,
+        "loss": 0.7334,
+        "learning_rate": 5.521739130434783e-07,
+        "epoch": 2.7767857142857144
+    },
+    {
+        "current_steps": 311,
+        "loss": 0.865,
+        "learning_rate": 5.5e-07,
+        "epoch": 2.7857142857142856
+    },
+    {
+        "current_steps": 312,
+        "loss": 0.5955,
+        "learning_rate": 5.478260869565216e-07,
+        "epoch": 2.794642857142857
+    },
+    {
+        "current_steps": 313,
+        "loss": 0.5059,
+        "learning_rate": 5.456521739130435e-07,
+        "epoch": 2.803571428571429
+    },
+    {
+        "current_steps": 314,
+        "loss": 1.0855,
+        "learning_rate": 5.434782608695652e-07,
+        "epoch": 2.8125
+    },
+    {
+        "current_steps": 315,
+        "loss": 0.7484,
+        "learning_rate": 5.413043478260869e-07,
+        "epoch": 2.821428571428571
+    },
+    {
+        "current_steps": 316,
+        "loss": 0.8017,
+        "learning_rate": 5.391304347826087e-07,
+        "epoch": 2.830357142857143
+    },
+    {
+        "current_steps": 317,
+        "loss": 0.7272,
+        "learning_rate": 5.369565217391304e-07,
+        "epoch": 2.8392857142857144
+    },
+    {
+        "current_steps": 318,
+        "loss": 0.6897,
+        "learning_rate": 5.347826086956521e-07,
+        "epoch": 2.8482142857142856
+    },
+    {
+        "current_steps": 319,
+        "loss": 0.634,
+        "learning_rate": 5.32608695652174e-07,
+        "epoch": 2.857142857142857
+    },
+    {
+        "current_steps": 320,
+        "loss": 0.7684,
+        "learning_rate": 5.304347826086957e-07,
+        "epoch": 2.866071428571429
+    },
+    {
+        "current_steps": 321,
+        "loss": 0.5758,
+        "learning_rate": 5.282608695652173e-07,
+        "epoch": 2.875
+    },
+    {
+        "current_steps": 322,
+        "loss": 0.687,
+        "learning_rate": 5.260869565217391e-07,
+        "epoch": 2.883928571428571
+    },
+    {
+        "current_steps": 323,
+        "loss": 0.6942,
+        "learning_rate": 5.239130434782608e-07,
+        "epoch": 2.892857142857143
+    },
+    {
+        "current_steps": 324,
+        "loss": 0.7698,
+        "learning_rate": 5.217391304347825e-07,
+        "epoch": 2.9017857142857144
+    },
+    {
+        "current_steps": 325,
+        "loss": 0.815,
+        "learning_rate": 5.195652173913043e-07,
+        "epoch": 2.9107142857142856
+    },
+    {
+        "current_steps": 326,
+        "loss": 0.6837,
+        "learning_rate": 5.173913043478261e-07,
+        "epoch": 2.919642857142857
+    },
+    {
+        "current_steps": 327,
+        "loss": 0.7103,
+        "learning_rate": 5.152173913043479e-07,
+        "epoch": 2.928571428571429
+    },
+    {
+        "current_steps": 328,
+        "loss": 0.6798,
+        "learning_rate": 5.130434782608696e-07,
+        "epoch": 2.9375
+    },
+    {
+        "current_steps": 329,
+        "loss": 0.767,
+        "learning_rate": 5.108695652173913e-07,
+        "epoch": 2.946428571428571
+    },
+    {
+        "current_steps": 330,
+        "loss": 0.6161,
+        "learning_rate": 5.08695652173913e-07,
+        "epoch": 2.955357142857143
+    },
+    {
+        "current_steps": 331,
+        "loss": 0.6607,
+        "learning_rate": 5.065217391304347e-07,
+        "epoch": 2.9642857142857144
+    },
+    {
+        "current_steps": 332,
+        "loss": 0.6875,
+        "learning_rate": 5.043478260869564e-07,
+        "epoch": 2.9732142857142856
+    },
+    {
+        "current_steps": 333,
+        "loss": 0.746,
+        "learning_rate": 5.021739130434783e-07,
+        "epoch": 2.982142857142857
+    },
+    {
+        "current_steps": 334,
+        "loss": 0.6093,
+        "learning_rate": 5e-07,
+        "epoch": 2.991071428571429
+    },
+    {
+        "current_steps": 335,
+        "loss": 0.5599,
+        "learning_rate": 4.978260869565217e-07,
+        "epoch": 3.0
+    },
+    {
+        "current_steps": 336,
+        "loss": 0.5985,
+        "learning_rate": 4.956521739130435e-07,
+        "epoch": 3.0089285714285716
+    },
+    {
+        "current_steps": 337,
+        "loss": 0.6692,
+        "learning_rate": 4.934782608695652e-07,
+        "epoch": 3.017857142857143
+    },
+    {
+        "current_steps": 338,
+        "loss": 0.5887,
+        "learning_rate": 4.913043478260869e-07,
+        "epoch": 3.0267857142857144
+    },
+    {
+        "current_steps": 339,
+        "loss": 0.5831,
+        "learning_rate": 4.891304347826087e-07,
+        "epoch": 3.0357142857142856
+    },
+    {
+        "current_steps": 340,
+        "loss": 0.5424,
+        "learning_rate": 4.869565217391305e-07,
+        "epoch": 3.044642857142857
+    },
+    {
+        "current_steps": 341,
+        "loss": 1.0041,
+        "learning_rate": 4.847826086956521e-07,
+        "epoch": 3.0535714285714284
+    },
+    {
+        "current_steps": 342,
+        "loss": 0.6989,
+        "learning_rate": 4.826086956521739e-07,
+        "epoch": 3.0625
+    },
+    {
+        "current_steps": 343,
+        "loss": 0.7104,
+        "learning_rate": 4.804347826086956e-07,
+        "epoch": 3.0714285714285716
+    },
+    {
+        "current_steps": 344,
+        "loss": 0.6493,
+        "learning_rate": 4.782608695652174e-07,
+        "epoch": 3.080357142857143
+    },
+    {
+        "current_steps": 345,
+        "loss": 0.8018,
+        "learning_rate": 4.7608695652173915e-07,
+        "epoch": 3.0892857142857144
+    },
+    {
+        "current_steps": 346,
+        "loss": 0.638,
+        "learning_rate": 4.739130434782608e-07,
+        "epoch": 3.0982142857142856
+    },
+    {
+        "current_steps": 347,
+        "loss": 0.7714,
+        "learning_rate": 4.717391304347826e-07,
+        "epoch": 3.107142857142857
+    },
+    {
+        "current_steps": 348,
+        "loss": 0.7103,
+        "learning_rate": 4.6956521739130434e-07,
+        "epoch": 3.1160714285714284
+    },
+    {
+        "current_steps": 349,
+        "loss": 0.5937,
+        "learning_rate": 4.673913043478261e-07,
+        "epoch": 3.125
+    },
+    {
+        "current_steps": 350,
+        "loss": 0.7256,
+        "learning_rate": 4.6521739130434777e-07,
+        "epoch": 3.1339285714285716
+    },
+    {
+        "current_steps": 351,
+        "loss": 0.864,
+        "learning_rate": 4.6304347826086954e-07,
+        "epoch": 3.142857142857143
+    },
+    {
+        "current_steps": 352,
+        "loss": 0.7429,
+        "learning_rate": 4.608695652173913e-07,
+        "epoch": 3.1517857142857144
+    },
+    {
+        "current_steps": 353,
+        "loss": 0.6658,
+        "learning_rate": 4.58695652173913e-07,
+        "epoch": 3.1607142857142856
+    },
+    {
+        "current_steps": 354,
+        "loss": 0.647,
+        "learning_rate": 4.5652173913043473e-07,
+        "epoch": 3.169642857142857
+    },
+    {
+        "current_steps": 355,
+        "loss": 0.7772,
+        "learning_rate": 4.543478260869565e-07,
+        "epoch": 3.1785714285714284
+    },
+    {
+        "current_steps": 356,
+        "loss": 0.6939,
+        "learning_rate": 4.521739130434782e-07,
+        "epoch": 3.1875
+    },
+    {
+        "current_steps": 357,
+        "loss": 0.5744,
+        "learning_rate": 4.5e-07,
+        "epoch": 3.1964285714285716
+    },
+    {
+        "current_steps": 358,
+        "loss": 0.7193,
+        "learning_rate": 4.4782608695652175e-07,
+        "epoch": 3.205357142857143
+    },
+    {
+        "current_steps": 359,
+        "loss": 0.667,
+        "learning_rate": 4.4565217391304346e-07,
+        "epoch": 3.2142857142857144
+    },
+    {
+        "current_steps": 360,
+        "loss": 0.6671,
+        "learning_rate": 4.434782608695652e-07,
+        "epoch": 3.2232142857142856
+    },
+    {
+        "current_steps": 361,
+        "loss": 0.8531,
+        "learning_rate": 4.4130434782608694e-07,
+        "epoch": 3.232142857142857
+    },
+    {
+        "current_steps": 362,
+        "loss": 0.6706,
+        "learning_rate": 4.391304347826087e-07,
+        "epoch": 3.2410714285714284
+    },
+    {
+        "current_steps": 363,
+        "loss": 0.8786,
+        "learning_rate": 4.3695652173913037e-07,
+        "epoch": 3.25
+    },
+    {
+        "current_steps": 364,
+        "loss": 0.6281,
+        "learning_rate": 4.3478260869565214e-07,
+        "epoch": 3.2589285714285716
+    },
+    {
+        "current_steps": 365,
+        "loss": 0.8648,
+        "learning_rate": 4.326086956521739e-07,
+        "epoch": 3.267857142857143
+    },
+    {
+        "current_steps": 366,
+        "loss": 0.5872,
+        "learning_rate": 4.3043478260869567e-07,
+        "epoch": 3.2767857142857144
+    },
+    {
+        "current_steps": 367,
+        "loss": 0.5874,
+        "learning_rate": 4.282608695652174e-07,
+        "epoch": 3.2857142857142856
+    },
+    {
+        "current_steps": 368,
+        "loss": 0.7057,
+        "learning_rate": 4.260869565217391e-07,
+        "epoch": 3.294642857142857
+    },
+    {
+        "current_steps": 369,
+        "loss": 0.6076,
+        "learning_rate": 4.2391304347826086e-07,
+        "epoch": 3.3035714285714284
+    },
+    {
+        "current_steps": 370,
+        "loss": 0.7514,
+        "learning_rate": 4.217391304347826e-07,
+        "epoch": 3.3125
+    },
+    {
+        "current_steps": 371,
+        "loss": 0.689,
+        "learning_rate": 4.1956521739130434e-07,
+        "epoch": 3.3214285714285716
+    },
+    {
+        "current_steps": 372,
+        "loss": 0.7074,
+        "learning_rate": 4.1739130434782606e-07,
+        "epoch": 3.330357142857143
+    },
+    {
+        "current_steps": 373,
+        "loss": 0.6425,
+        "learning_rate": 4.1521739130434777e-07,
+        "epoch": 3.3392857142857144
+    },
+    {
+        "current_steps": 374,
+        "loss": 0.5247,
+        "learning_rate": 4.1304347826086954e-07,
+        "epoch": 3.3482142857142856
+    },
+    {
+        "current_steps": 375,
+        "loss": 0.7755,
+        "learning_rate": 4.108695652173913e-07,
+        "epoch": 3.357142857142857
+    },
+    {
+        "current_steps": 376,
+        "loss": 0.7774,
+        "learning_rate": 4.0869565217391307e-07,
+        "epoch": 3.3660714285714284
+    },
+    {
+        "current_steps": 377,
+        "loss": 0.6871,
+        "learning_rate": 4.0652173913043473e-07,
+        "epoch": 3.375
+    },
+    {
+        "current_steps": 378,
+        "loss": 0.566,
+        "learning_rate": 4.043478260869565e-07,
+        "epoch": 3.3839285714285716
+    },
+    {
+        "current_steps": 379,
+        "loss": 1.0922,
+        "learning_rate": 4.0217391304347827e-07,
+        "epoch": 3.392857142857143
+    },
+    {
+        "current_steps": 380,
+        "loss": 0.5958,
+        "learning_rate": 4e-07,
+        "epoch": 3.4017857142857144
+    },
+    {
+        "current_steps": 381,
+        "loss": 0.9182,
+        "learning_rate": 3.978260869565217e-07,
+        "epoch": 3.4107142857142856
+    },
+    {
+        "current_steps": 382,
+        "loss": 0.7356,
+        "learning_rate": 3.9565217391304346e-07,
+        "epoch": 3.419642857142857
+    },
+    {
+        "current_steps": 383,
+        "loss": 0.8677,
+        "learning_rate": 3.9347826086956523e-07,
+        "epoch": 3.4285714285714284
+    },
+    {
+        "current_steps": 384,
+        "loss": 0.6885,
+        "learning_rate": 3.9130434782608694e-07,
+        "epoch": 3.4375
+    },
+    {
+        "current_steps": 385,
+        "loss": 0.7982,
+        "learning_rate": 3.8913043478260866e-07,
+        "epoch": 3.4464285714285716
+    },
+    {
+        "current_steps": 386,
+        "loss": 0.8466,
+        "learning_rate": 3.869565217391304e-07,
+        "epoch": 3.455357142857143
+    },
+    {
+        "current_steps": 387,
+        "loss": 0.4563,
+        "learning_rate": 3.8478260869565214e-07,
+        "epoch": 3.4642857142857144
+    },
+    {
+        "current_steps": 388,
+        "loss": 0.7675,
+        "learning_rate": 3.826086956521739e-07,
+        "epoch": 3.4732142857142856
+    },
+    {
+        "current_steps": 389,
+        "loss": 0.7642,
+        "learning_rate": 3.8043478260869567e-07,
+        "epoch": 3.482142857142857
+    },
+    {
+        "current_steps": 390,
+        "loss": 0.6065,
+        "learning_rate": 3.7826086956521733e-07,
+        "epoch": 3.4910714285714284
+    },
+    {
+        "current_steps": 391,
+        "loss": 0.6121,
+        "learning_rate": 3.760869565217391e-07,
+        "epoch": 3.5
+    },
+    {
+        "current_steps": 392,
+        "loss": 0.8562,
+        "learning_rate": 3.7391304347826087e-07,
+        "epoch": 3.508928571428571
+    },
+    {
+        "current_steps": 393,
+        "loss": 0.8169,
+        "learning_rate": 3.7173913043478263e-07,
+        "epoch": 3.517857142857143
+    },
+    {
+        "current_steps": 394,
+        "loss": 0.7264,
+        "learning_rate": 3.695652173913043e-07,
+        "epoch": 3.5267857142857144
+    },
+    {
+        "current_steps": 395,
+        "loss": 0.6761,
+        "learning_rate": 3.6739130434782606e-07,
+        "epoch": 3.5357142857142856
+    },
+    {
+        "current_steps": 396,
+        "loss": 0.485,
+        "learning_rate": 3.6521739130434783e-07,
+        "epoch": 3.544642857142857
+    },
+    {
+        "current_steps": 397,
+        "loss": 0.6992,
+        "learning_rate": 3.6304347826086954e-07,
+        "epoch": 3.553571428571429
+    },
+    {
+        "current_steps": 398,
+        "loss": 0.6543,
+        "learning_rate": 3.608695652173913e-07,
+        "epoch": 3.5625
+    },
+    {
+        "current_steps": 399,
+        "loss": 0.6019,
+        "learning_rate": 3.58695652173913e-07,
+        "epoch": 3.571428571428571
+    },
+    {
+        "current_steps": 400,
+        "loss": 0.8135,
+        "learning_rate": 3.5652173913043474e-07,
+        "epoch": 3.580357142857143
+    },
+    {
+        "current_steps": 401,
+        "loss": 0.5053,
+        "learning_rate": 3.543478260869565e-07,
+        "epoch": 3.5892857142857144
+    },
+    {
+        "current_steps": 402,
+        "loss": 0.6121,
+        "learning_rate": 3.5217391304347827e-07,
+        "epoch": 3.5982142857142856
+    },
+    {
+        "current_steps": 403,
+        "loss": 0.5648,
+        "learning_rate": 3.5e-07,
+        "epoch": 3.607142857142857
+    },
+    {
+        "current_steps": 404,
+        "loss": 0.6023,
+        "learning_rate": 3.478260869565217e-07,
+        "epoch": 3.616071428571429
+    },
+    {
+        "current_steps": 405,
+        "loss": 0.7843,
+        "learning_rate": 3.4565217391304346e-07,
+        "epoch": 3.625
+    },
+    {
+        "current_steps": 406,
+        "loss": 0.6902,
+        "learning_rate": 3.4347826086956523e-07,
+        "epoch": 3.633928571428571
+    },
+    {
+        "current_steps": 407,
+        "loss": 0.6103,
+        "learning_rate": 3.413043478260869e-07,
+        "epoch": 3.642857142857143
+    },
+    {
+        "current_steps": 408,
+        "loss": 0.759,
+        "learning_rate": 3.3913043478260866e-07,
+        "epoch": 3.6517857142857144
+    },
+    {
+        "current_steps": 409,
+        "loss": 0.7823,
+        "learning_rate": 3.369565217391304e-07,
+        "epoch": 3.6607142857142856
+    },
+    {
+        "current_steps": 410,
+        "loss": 0.8021,
+        "learning_rate": 3.347826086956522e-07,
+        "epoch": 3.669642857142857
+    },
+    {
+        "current_steps": 411,
+        "loss": 0.5927,
+        "learning_rate": 3.326086956521739e-07,
+        "epoch": 3.678571428571429
+    },
+    {
+        "current_steps": 412,
+        "loss": 0.6503,
+        "learning_rate": 3.304347826086956e-07,
+        "epoch": 3.6875
+    },
+    {
+        "current_steps": 413,
+        "loss": 0.886,
+        "learning_rate": 3.282608695652174e-07,
+        "epoch": 3.696428571428571
+    },
+    {
+        "current_steps": 414,
+        "loss": 0.6331,
+        "learning_rate": 3.260869565217391e-07,
+        "epoch": 3.705357142857143
+    },
+    {
+        "current_steps": 415,
+        "loss": 0.7633,
+        "learning_rate": 3.2391304347826087e-07,
+        "epoch": 3.7142857142857144
+    },
+    {
+        "current_steps": 416,
+        "loss": 0.6538,
+        "learning_rate": 3.217391304347826e-07,
+        "epoch": 3.7232142857142856
+    },
+    {
+        "current_steps": 417,
+        "loss": 0.6156,
+        "learning_rate": 3.195652173913043e-07,
+        "epoch": 3.732142857142857
+    },
+    {
+        "current_steps": 418,
+        "loss": 0.6973,
+        "learning_rate": 3.1739130434782606e-07,
+        "epoch": 3.741071428571429
+    },
+    {
+        "current_steps": 419,
+        "loss": 0.6521,
+        "learning_rate": 3.1521739130434783e-07,
+        "epoch": 3.75
+    },
+    {
+        "current_steps": 420,
+        "loss": 0.6931,
+        "learning_rate": 3.130434782608696e-07,
+        "epoch": 3.758928571428571
+    },
+    {
+        "current_steps": 421,
+        "loss": 0.8192,
+        "learning_rate": 3.1086956521739126e-07,
+        "epoch": 3.767857142857143
+    },
+    {
+        "current_steps": 422,
+        "loss": 0.5986,
+        "learning_rate": 3.08695652173913e-07,
+        "epoch": 3.7767857142857144
+    },
+    {
+        "current_steps": 423,
+        "loss": 0.9986,
+        "learning_rate": 3.065217391304348e-07,
+        "epoch": 3.7857142857142856
+    },
+    {
+        "current_steps": 424,
+        "loss": 0.7645,
+        "learning_rate": 3.043478260869565e-07,
+        "epoch": 3.794642857142857
+    },
+    {
+        "current_steps": 425,
+        "loss": 0.6489,
+        "learning_rate": 3.021739130434782e-07,
+        "epoch": 3.803571428571429
+    },
+    {
+        "current_steps": 426,
+        "loss": 0.5974,
+        "learning_rate": 3e-07,
+        "epoch": 3.8125
+    },
+    {
+        "current_steps": 427,
+        "loss": 0.7392,
+        "learning_rate": 2.9782608695652175e-07,
+        "epoch": 3.821428571428571
+    },
+    {
+        "current_steps": 428,
+        "loss": 0.7813,
+        "learning_rate": 2.9565217391304347e-07,
+        "epoch": 3.830357142857143
+    },
+    {
+        "current_steps": 429,
+        "loss": 0.7818,
+        "learning_rate": 2.9347826086956523e-07,
+        "epoch": 3.8392857142857144
+    },
+    {
+        "current_steps": 430,
+        "loss": 1.0693,
+        "learning_rate": 2.9130434782608695e-07,
+        "epoch": 3.8482142857142856
+    },
+    {
+        "current_steps": 431,
+        "loss": 0.6324,
+        "learning_rate": 2.8913043478260866e-07,
+        "epoch": 3.857142857142857
+    },
+    {
+        "current_steps": 432,
+        "loss": 0.5228,
+        "learning_rate": 2.8695652173913043e-07,
+        "epoch": 3.866071428571429
+    },
+    {
+        "current_steps": 433,
+        "loss": 0.6631,
+        "learning_rate": 2.847826086956522e-07,
+        "epoch": 3.875
+    },
+    {
+        "current_steps": 434,
+        "loss": 0.6685,
+        "learning_rate": 2.8260869565217386e-07,
+        "epoch": 3.883928571428571
+    },
+    {
+        "current_steps": 435,
+        "loss": 0.6566,
+        "learning_rate": 2.804347826086956e-07,
+        "epoch": 3.892857142857143
+    },
+    {
+        "current_steps": 436,
+        "loss": 0.6169,
+        "learning_rate": 2.782608695652174e-07,
+        "epoch": 3.9017857142857144
+    },
+    {
+        "current_steps": 437,
+        "loss": 0.5012,
+        "learning_rate": 2.7608695652173916e-07,
+        "epoch": 3.9107142857142856
+    },
+    {
+        "current_steps": 438,
+        "loss": 0.637,
+        "learning_rate": 2.739130434782608e-07,
+        "epoch": 3.919642857142857
+    },
+    {
+        "current_steps": 439,
+        "loss": 0.7777,
+        "learning_rate": 2.717391304347826e-07,
+        "epoch": 3.928571428571429
+    },
+    {
+        "current_steps": 440,
+        "loss": 0.6963,
+        "learning_rate": 2.6956521739130435e-07,
+        "epoch": 3.9375
+    },
+    {
+        "current_steps": 441,
+        "loss": 0.5398,
+        "learning_rate": 2.6739130434782607e-07,
+        "epoch": 3.946428571428571
+    },
+    {
+        "current_steps": 442,
+        "loss": 1.0029,
+        "learning_rate": 2.6521739130434783e-07,
+        "epoch": 3.955357142857143
+    },
+    {
+        "current_steps": 443,
+        "loss": 0.8166,
+        "learning_rate": 2.6304347826086955e-07,
+        "epoch": 3.9642857142857144
+    },
+    {
+        "current_steps": 444,
+        "loss": 0.8981,
+        "learning_rate": 2.6086956521739126e-07,
+        "epoch": 3.9732142857142856
+    },
+    {
+        "current_steps": 445,
+        "loss": 0.536,
+        "learning_rate": 2.5869565217391303e-07,
+        "epoch": 3.982142857142857
+    },
+    {
+        "current_steps": 446,
+        "loss": 0.7719,
+        "learning_rate": 2.565217391304348e-07,
+        "epoch": 3.991071428571429
+    },
+    {
+        "current_steps": 447,
+        "loss": 3.9574,
+        "learning_rate": 2.565217391304348e-07,
+        "epoch": 4.0
+    },
+    {
+        "current_steps": 448,
+        "loss": 0.6567,
+        "learning_rate": 2.543478260869565e-07,
+        "epoch": 4.008928571428571
+    },
+    {
+        "current_steps": 449,
+        "loss": 0.8622,
+        "learning_rate": 2.521739130434782e-07,
+        "epoch": 4.017857142857143
+    },
+    {
+        "current_steps": 450,
+        "loss": 0.5737,
+        "learning_rate": 2.5e-07,
+        "epoch": 4.026785714285714
+    },
+    {
+        "current_steps": 451,
+        "loss": 0.736,
+        "learning_rate": 2.4782608695652176e-07,
+        "epoch": 4.035714285714286
+    },
+    {
+        "current_steps": 452,
+        "loss": 0.8457,
+        "learning_rate": 2.4565217391304347e-07,
+        "epoch": 4.044642857142857
+    },
+    {
+        "current_steps": 453,
+        "loss": 0.7416,
+        "learning_rate": 2.4347826086956524e-07,
+        "epoch": 4.053571428571429
+    },
+    {
+        "current_steps": 454,
+        "loss": 1.0355,
+        "learning_rate": 2.4130434782608695e-07,
+        "epoch": 4.0625
+    },
+    {
+        "current_steps": 455,
+        "loss": 0.7162,
+        "learning_rate": 2.391304347826087e-07,
+        "epoch": 4.071428571428571
+    },
+    {
+        "current_steps": 456,
+        "loss": 0.8163,
+        "learning_rate": 2.369565217391304e-07,
+        "epoch": 4.080357142857143
+    },
+    {
+        "current_steps": 457,
+        "loss": 0.5188,
+        "learning_rate": 2.3478260869565217e-07,
+        "epoch": 4.089285714285714
+    },
+    {
+        "current_steps": 458,
+        "loss": 0.9544,
+        "learning_rate": 2.3260869565217389e-07,
+        "epoch": 4.098214285714286
+    },
+    {
+        "current_steps": 459,
+        "loss": 0.6205,
+        "learning_rate": 2.3043478260869565e-07,
+        "epoch": 4.107142857142857
+    },
+    {
+        "current_steps": 460,
+        "loss": 0.6643,
+        "learning_rate": 2.2826086956521737e-07,
+        "epoch": 4.116071428571429
+    },
+    {
+        "current_steps": 461,
+        "loss": 0.6465,
+        "learning_rate": 2.260869565217391e-07,
+        "epoch": 4.125
+    },
+    {
+        "current_steps": 462,
+        "loss": 0.6697,
+        "learning_rate": 2.2391304347826087e-07,
+        "epoch": 4.133928571428571
+    },
+    {
+        "current_steps": 463,
+        "loss": 0.7041,
+        "learning_rate": 2.217391304347826e-07,
+        "epoch": 4.142857142857143
+    },
+    {
+        "current_steps": 464,
+        "loss": 0.802,
+        "learning_rate": 2.1956521739130435e-07,
+        "epoch": 4.151785714285714
+    },
+    {
+        "current_steps": 465,
+        "loss": 0.623,
+        "learning_rate": 2.1739130434782607e-07,
+        "epoch": 4.160714285714286
+    },
+    {
+        "current_steps": 466,
+        "loss": 0.6071,
+        "learning_rate": 2.1521739130434783e-07,
+        "epoch": 4.169642857142857
+    },
+    {
+        "current_steps": 467,
+        "loss": 0.718,
+        "learning_rate": 2.1304347826086955e-07,
+        "epoch": 4.178571428571429
+    },
+    {
+        "current_steps": 468,
+        "loss": 0.6337,
+        "learning_rate": 2.108695652173913e-07,
+        "epoch": 4.1875
+    },
+    {
+        "current_steps": 469,
+        "loss": 0.5689,
+        "learning_rate": 2.0869565217391303e-07,
+        "epoch": 4.196428571428571
+    },
+    {
+        "current_steps": 470,
+        "loss": 0.62,
+        "learning_rate": 2.0652173913043477e-07,
+        "epoch": 4.205357142857143
+    },
+    {
+        "current_steps": 471,
+        "loss": 1.0191,
+        "learning_rate": 2.0434782608695654e-07,
+        "epoch": 4.214285714285714
+    },
+    {
+        "current_steps": 472,
+        "loss": 0.6678,
+        "learning_rate": 2.0217391304347825e-07,
+        "epoch": 4.223214285714286
+    },
+    {
+        "current_steps": 473,
+        "loss": 0.6296,
+        "learning_rate": 2e-07,
+        "epoch": 4.232142857142857
+    },
+    {
+        "current_steps": 474,
+        "loss": 0.884,
+        "learning_rate": 1.9782608695652173e-07,
+        "epoch": 4.241071428571429
+    },
+    {
+        "current_steps": 475,
+        "loss": 0.7207,
+        "learning_rate": 1.9565217391304347e-07,
+        "epoch": 4.25
+    },
+    {
+        "current_steps": 476,
+        "loss": 0.6856,
+        "learning_rate": 1.934782608695652e-07,
+        "epoch": 4.258928571428571
+    },
+    {
+        "current_steps": 477,
+        "loss": 0.6314,
+        "learning_rate": 1.9130434782608695e-07,
+        "epoch": 4.267857142857143
+    },
+    {
+        "current_steps": 478,
+        "loss": 0.5759,
+        "learning_rate": 1.8913043478260867e-07,
+        "epoch": 4.276785714285714
+    },
+    {
+        "current_steps": 479,
+        "loss": 0.6925,
+        "learning_rate": 1.8695652173913043e-07,
+        "epoch": 4.285714285714286
+    },
+    {
+        "current_steps": 480,
+        "loss": 0.6237,
+        "learning_rate": 1.8478260869565215e-07,
+        "epoch": 4.294642857142857
+    },
+    {
+        "current_steps": 481,
+        "loss": 0.6666,
+        "learning_rate": 1.8260869565217391e-07,
+        "epoch": 4.303571428571429
+    },
+    {
+        "current_steps": 482,
+        "loss": 0.709,
+        "learning_rate": 1.8043478260869565e-07,
+        "epoch": 4.3125
+    },
+    {
+        "current_steps": 483,
+        "loss": 0.8078,
+        "learning_rate": 1.7826086956521737e-07,
+        "epoch": 4.321428571428571
+    },
+    {
+        "current_steps": 484,
+        "loss": 0.7355,
+        "learning_rate": 1.7608695652173914e-07,
+        "epoch": 4.330357142857143
+    },
+    {
+        "current_steps": 485,
+        "loss": 0.8901,
+        "learning_rate": 1.7391304347826085e-07,
+        "epoch": 4.339285714285714
+    },
+    {
+        "current_steps": 486,
+        "loss": 0.565,
+        "learning_rate": 1.7173913043478262e-07,
+        "epoch": 4.348214285714286
+    },
+    {
+        "current_steps": 487,
+        "loss": 0.6396,
+        "learning_rate": 1.6956521739130433e-07,
+        "epoch": 4.357142857142857
+    },
+    {
+        "current_steps": 488,
+        "loss": 0.531,
+        "learning_rate": 1.673913043478261e-07,
+        "epoch": 4.366071428571429
+    },
+    {
+        "current_steps": 489,
+        "loss": 0.5726,
+        "learning_rate": 1.652173913043478e-07,
+        "epoch": 4.375
+    },
+    {
+        "current_steps": 490,
+        "loss": 0.602,
+        "learning_rate": 1.6304347826086955e-07,
+        "epoch": 4.383928571428571
+    },
+    {
+        "current_steps": 491,
+        "loss": 0.7032,
+        "learning_rate": 1.608695652173913e-07,
+        "epoch": 4.392857142857143
+    },
+    {
+        "current_steps": 492,
+        "loss": 0.8984,
+        "learning_rate": 1.5869565217391303e-07,
+        "epoch": 4.401785714285714
+    },
+    {
+        "current_steps": 493,
+        "loss": 0.5913,
+        "learning_rate": 1.565217391304348e-07,
+        "epoch": 4.410714285714286
+    },
+    {
+        "current_steps": 494,
+        "loss": 0.6021,
+        "learning_rate": 1.543478260869565e-07,
+        "epoch": 4.419642857142857
+    },
+    {
+        "current_steps": 495,
+        "loss": 0.7554,
+        "learning_rate": 1.5217391304347825e-07,
+        "epoch": 4.428571428571429
+    },
+    {
+        "current_steps": 496,
+        "loss": 0.8683,
+        "learning_rate": 1.5e-07,
+        "epoch": 4.4375
+    },
+    {
+        "current_steps": 497,
+        "loss": 0.5465,
+        "learning_rate": 1.4782608695652173e-07,
+        "epoch": 4.446428571428571
+    },
+    {
+        "current_steps": 498,
+        "loss": 0.6903,
+        "learning_rate": 1.4565217391304347e-07,
+        "epoch": 4.455357142857143
+    },
+    {
+        "current_steps": 499,
+        "loss": 0.4821,
+        "learning_rate": 1.4347826086956521e-07,
+        "epoch": 4.464285714285714
+    },
+    {
+        "current_steps": 500,
+        "loss": 0.6731,
+        "learning_rate": 1.4130434782608693e-07,
+        "epoch": 4.473214285714286
+    },
+    {
+        "current_steps": 501,
+        "loss": 0.7423,
+        "learning_rate": 1.391304347826087e-07,
+        "epoch": 4.482142857142857
+    },
+    {
+        "current_steps": 502,
+        "loss": 0.6967,
+        "learning_rate": 1.369565217391304e-07,
+        "epoch": 4.491071428571429
+    },
+    {
+        "current_steps": 503,
+        "loss": 0.5918,
+        "learning_rate": 1.3478260869565218e-07,
+        "epoch": 4.5
+    },
+    {
+        "current_steps": 504,
+        "loss": 0.8028,
+        "learning_rate": 1.3260869565217392e-07,
+        "epoch": 4.508928571428571
+    },
+    {
+        "current_steps": 505,
+        "loss": 0.9578,
+        "learning_rate": 1.3043478260869563e-07,
+        "epoch": 4.517857142857143
+    },
+    {
+        "current_steps": 506,
+        "loss": 0.6187,
+        "learning_rate": 1.282608695652174e-07,
+        "epoch": 4.526785714285714
+    },
+    {
+        "current_steps": 507,
+        "loss": 0.6426,
+        "learning_rate": 1.260869565217391e-07,
+        "epoch": 4.535714285714286
+    },
+    {
+        "current_steps": 508,
+        "loss": 0.5835,
+        "learning_rate": 1.2391304347826088e-07,
+        "epoch": 4.544642857142857
+    },
+    {
+        "current_steps": 509,
+        "loss": 0.7218,
+        "learning_rate": 1.2173913043478262e-07,
+        "epoch": 4.553571428571429
+    },
+    {
+        "current_steps": 510,
+        "loss": 0.812,
+        "learning_rate": 1.1956521739130436e-07,
+        "epoch": 4.5625
+    },
+    {
+        "current_steps": 511,
+        "loss": 0.5526,
+        "learning_rate": 1.1739130434782609e-07,
+        "epoch": 4.571428571428571
+    },
+    {
+        "current_steps": 512,
+        "loss": 0.8554,
+        "learning_rate": 1.1521739130434783e-07,
+        "epoch": 4.580357142857143
+    },
+    {
+        "current_steps": 513,
+        "loss": 0.7209,
+        "learning_rate": 1.1304347826086955e-07,
+        "epoch": 4.589285714285714
+    },
+    {
+        "current_steps": 514,
+        "loss": 0.7154,
+        "learning_rate": 1.108695652173913e-07,
+        "epoch": 4.598214285714286
+    },
+    {
+        "current_steps": 515,
+        "loss": 0.7147,
+        "learning_rate": 1.0869565217391303e-07,
+        "epoch": 4.607142857142857
+    },
+    {
+        "current_steps": 516,
+        "loss": 0.6997,
+        "learning_rate": 1.0652173913043477e-07,
+        "epoch": 4.616071428571429
+    },
+    {
+        "current_steps": 517,
+        "loss": 0.6283,
+        "learning_rate": 1.0434782608695651e-07,
+        "epoch": 4.625
+    },
+    {
+        "current_steps": 518,
+        "loss": 0.6279,
+        "learning_rate": 1.0217391304347827e-07,
+        "epoch": 4.633928571428571
+    },
+    {
+        "current_steps": 519,
+        "loss": 0.8152,
+        "learning_rate": 1e-07,
+        "epoch": 4.642857142857143
+    },
+    {
+        "current_steps": 520,
+        "loss": 0.6155,
+        "learning_rate": 9.782608695652174e-08,
+        "epoch": 4.651785714285714
+    },
+    {
+        "current_steps": 521,
+        "loss": 0.4727,
+        "learning_rate": 9.565217391304348e-08,
+        "epoch": 4.660714285714286
+    },
+    {
+        "current_steps": 522,
+        "loss": 0.7457,
+        "learning_rate": 9.347826086956522e-08,
+        "epoch": 4.669642857142857
+    },
+    {
+        "current_steps": 523,
+        "loss": 0.9712,
+        "learning_rate": 9.130434782608696e-08,
+        "epoch": 4.678571428571429
+    },
+    {
+        "current_steps": 524,
+        "loss": 0.7759,
+        "learning_rate": 8.913043478260868e-08,
+        "epoch": 4.6875
+    },
+    {
+        "current_steps": 525,
+        "loss": 0.6597,
+        "learning_rate": 8.695652173913042e-08,
+        "epoch": 4.696428571428571
+    },
+    {
+        "current_steps": 526,
+        "loss": 0.6258,
+        "learning_rate": 8.478260869565216e-08,
+        "epoch": 4.705357142857143
+    },
+    {
+        "current_steps": 527,
+        "loss": 0.6443,
+        "learning_rate": 8.26086956521739e-08,
+        "epoch": 4.714285714285714
+    },
+    {
+        "current_steps": 528,
+        "loss": 0.5547,
+        "learning_rate": 8.043478260869565e-08,
+        "epoch": 4.723214285714286
+    },
+    {
+        "current_steps": 529,
+        "loss": 0.7149,
+        "learning_rate": 7.82608695652174e-08,
+        "epoch": 4.732142857142857
+    },
+    {
+        "current_steps": 530,
+        "loss": 0.6138,
+        "learning_rate": 7.608695652173913e-08,
+        "epoch": 4.741071428571429
+    },
+    {
+        "current_steps": 531,
+        "loss": 0.8032,
+        "learning_rate": 7.391304347826087e-08,
+        "epoch": 4.75
+    },
+    {
+        "current_steps": 532,
+        "loss": 0.7141,
+        "learning_rate": 7.173913043478261e-08,
+        "epoch": 4.758928571428571
+    },
+    {
+        "current_steps": 533,
+        "loss": 0.724,
+        "learning_rate": 6.956521739130435e-08,
+        "epoch": 4.767857142857143
+    },
+    {
+        "current_steps": 534,
+        "loss": 0.7707,
+        "learning_rate": 6.739130434782609e-08,
+        "epoch": 4.776785714285714
+    },
+    {
+        "current_steps": 535,
+        "loss": 0.6754,
+        "learning_rate": 6.521739130434782e-08,
+        "epoch": 4.785714285714286
+    },
+    {
+        "current_steps": 536,
+        "loss": 0.5861,
+        "learning_rate": 6.304347826086956e-08,
+        "epoch": 4.794642857142857
+    },
+    {
+        "current_steps": 537,
+        "loss": 0.8395,
+        "learning_rate": 6.086956521739131e-08,
+        "epoch": 4.803571428571429
+    },
+    {
+        "current_steps": 538,
+        "loss": 0.7642,
+        "learning_rate": 5.869565217391304e-08,
+        "epoch": 4.8125
+    },
+    {
+        "current_steps": 539,
+        "loss": 0.735,
+        "learning_rate": 5.6521739130434777e-08,
+        "epoch": 4.821428571428571
+    },
+    {
+        "current_steps": 540,
+        "loss": 0.6153,
+        "learning_rate": 5.434782608695652e-08,
+        "epoch": 4.830357142857143
+    },
+    {
+        "current_steps": 541,
+        "loss": 0.6299,
+        "learning_rate": 5.217391304347826e-08,
+        "epoch": 4.839285714285714
+    },
+    {
+        "current_steps": 542,
+        "loss": 1.078,
+        "learning_rate": 5e-08,
+        "epoch": 4.848214285714286
+    },
+    {
+        "current_steps": 543,
+        "loss": 0.7314,
+        "learning_rate": 4.782608695652174e-08,
+        "epoch": 4.857142857142857
+    },
+    {
+        "current_steps": 544,
+        "loss": 0.8515,
+        "learning_rate": 4.565217391304348e-08,
+        "epoch": 4.866071428571429
+    },
+    {
+        "current_steps": 545,
+        "loss": 0.5401,
+        "learning_rate": 4.347826086956521e-08,
+        "epoch": 4.875
+    },
+    {
+        "current_steps": 546,
+        "loss": 0.7315,
+        "learning_rate": 4.130434782608695e-08,
+        "epoch": 4.883928571428571
+    },
+    {
+        "current_steps": 547,
+        "loss": 0.6113,
+        "learning_rate": 3.91304347826087e-08,
+        "epoch": 4.892857142857143
+    },
+    {
+        "current_steps": 548,
+        "loss": 0.6239,
+        "learning_rate": 3.6956521739130433e-08,
+        "epoch": 4.901785714285714
+    },
+    {
+        "current_steps": 549,
+        "loss": 0.7292,
+        "learning_rate": 3.4782608695652174e-08,
+        "epoch": 4.910714285714286
+    },
+    {
+        "current_steps": 550,
+        "loss": 0.5297,
+        "learning_rate": 3.260869565217391e-08,
+        "epoch": 4.919642857142857
+    },
+    {
+        "current_steps": 551,
+        "loss": 0.6269,
+        "learning_rate": 3.0434782608695655e-08,
+        "epoch": 4.928571428571429
+    },
+    {
+        "current_steps": 552,
+        "loss": 0.6724,
+        "learning_rate": 2.8260869565217388e-08,
+        "epoch": 4.9375
+    },
+    {
+        "current_steps": 553,
+        "loss": 0.5109,
+        "learning_rate": 2.608695652173913e-08,
+        "epoch": 4.946428571428571
+    },
+    {
+        "current_steps": 554,
+        "loss": 0.9446,
+        "learning_rate": 2.391304347826087e-08,
+        "epoch": 4.955357142857143
+    },
+    {
+        "current_steps": 555,
+        "loss": 0.6897,
+        "learning_rate": 2.1739130434782606e-08,
+        "epoch": 4.964285714285714
+    },
+    {
+        "current_steps": 556,
+        "loss": 0.5511,
+        "learning_rate": 1.956521739130435e-08,
+        "epoch": 4.973214285714286
+    },
+    {
+        "current_steps": 557,
+        "loss": 0.7246,
+        "learning_rate": 1.7391304347826087e-08,
+        "epoch": 4.982142857142857
+    },
+    {
+        "current_steps": 558,
+        "loss": 0.6332,
+        "learning_rate": 1.5217391304347827e-08,
+        "epoch": 4.991071428571429
+    },
+    {
+        "current_steps": 559,
+        "loss": 1.0499,
+        "learning_rate": 1.3043478260869564e-08,
+        "epoch": 5.0
+    },
+    {
+        "current_steps": 559,
+        "loss": 1.0499,
+        "learning_rate": 1.3043478260869564e-08,
+        "epoch": 5.0
+    }
+]

aliceinwonderland/training_log.json

@@ -0,0 +1,19 @@
+{
+  "base_model_name": "Llama-2-13b-hf",
+  "base_model_class": "LlamaForCausalLM",
+  "base_loaded_in_4bit": true,
+  "base_loaded_in_8bit": false,
+  "projections": "q, v",
+  "loss": 1.0499,
+  "grad_norm": 5.645450592041016,
+  "learning_rate": 1.3043478260869564e-08,
+  "epoch": 5.0,
+  "current_steps": 559,
+  "current_steps_adjusted": 559,
+  "epoch_adjusted": 5.0,
+  "train_runtime": 1468.5439,
+  "train_samples_per_second": 1.515,
+  "train_steps_per_second": 0.381,
+  "total_flos": 4.4012668649472e+16,
+  "train_loss": 0.7355319578732763
+}

aliceinwonderland/training_parameters.json

@@ -0,0 +1,37 @@
+{
+  "lora_name": "aliceinwonderland",
+  "always_override": true,
+  "save_steps": 0,
+  "micro_batch_size": 4,
+  "batch_size": 0,
+  "epochs": 5,
+  "learning_rate": "1e-6",
+  "lr_scheduler_type": "linear",
+  "lora_rank": 32,
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "cutoff_len": 256,
+  "dataset": "None",
+  "eval_dataset": "None",
+  "format": "None",
+  "eval_steps": 100,
+  "raw_text_file": "aliceandwonderland",
+  "higher_rank_limit": false,
+  "warmup_steps": 100,
+  "optimizer": "adamw_torch",
+  "hard_cut_string": "\\n\\n\\n",
+  "train_only_after": "",
+  "stop_at_loss": 0,
+  "add_eos_token": false,
+  "min_chars": 20,
+  "report_to": "None",
+  "precize_slicing_overlap": true,
+  "add_eos_token_type": "Every Block",
+  "save_steps_under_loss": 1.8,
+  "add_bos_token": true,
+  "training_projection": "q-v",
+  "sliding_window": false,
+  "warmup_ratio": 0,
+  "grad_accumulation": 1,
+  "neft_noise_alpha": 0
+}

aliceinwonderland/training_prompt.json

@@ -0,0 +1,3 @@
+{
+  "template_type": "raw_text"
+}