Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: EleutherAI/gpt-neo-1.3B
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "EleutherAI/gpt-neo-1.3B",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e433843af10a9fa7f33dfde08bf020b75cdb128d06819a87b09fadfc0ba438c8
+size 9457000

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6870e8d3f6b9c0e0dbd9080ac6822a4781bc3826a14063c9731fb3b4b94eafb
+size 9481338

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:614afc6b0f9d27b5da1c1fd3f3e401421b91b96f5f5c65c0a835446078a80550
+size 14180

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7234e8581dffef02a92b7ea26d1c8c37881ca263044b5b5344c959d6b93880f4
+size 1064

special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|endoftext|>",
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 2048,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

trainer_state.json

@@ -0,0 +1,1468 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.74247491638796,
+  "eval_steps": 200,
+  "global_step": 20500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.013377926421404682,
+      "grad_norm": 0.9171162843704224,
+      "learning_rate": 7.05e-06,
+      "loss": 2.16,
+      "step": 100
+    },
+    {
+      "epoch": 0.026755852842809364,
+      "grad_norm": 0.4195552170276642,
+      "learning_rate": 1.41e-05,
+      "loss": 1.7242,
+      "step": 200
+    },
+    {
+      "epoch": 0.04013377926421405,
+      "grad_norm": 0.40102216601371765,
+      "learning_rate": 1.4036557930258718e-05,
+      "loss": 1.2567,
+      "step": 300
+    },
+    {
+      "epoch": 0.05351170568561873,
+      "grad_norm": 0.5191290378570557,
+      "learning_rate": 1.3973115860517437e-05,
+      "loss": 1.0338,
+      "step": 400
+    },
+    {
+      "epoch": 0.06688963210702341,
+      "grad_norm": 0.356404185295105,
+      "learning_rate": 1.3909673790776152e-05,
+      "loss": 0.8353,
+      "step": 500
+    },
+    {
+      "epoch": 0.0802675585284281,
+      "grad_norm": 0.4750893712043762,
+      "learning_rate": 1.3846231721034871e-05,
+      "loss": 0.7277,
+      "step": 600
+    },
+    {
+      "epoch": 0.09364548494983277,
+      "grad_norm": 0.27046123147010803,
+      "learning_rate": 1.3782789651293589e-05,
+      "loss": 0.6938,
+      "step": 700
+    },
+    {
+      "epoch": 0.10702341137123746,
+      "grad_norm": 0.22247174382209778,
+      "learning_rate": 1.3719347581552308e-05,
+      "loss": 0.6622,
+      "step": 800
+    },
+    {
+      "epoch": 0.12040133779264214,
+      "grad_norm": 0.2817244827747345,
+      "learning_rate": 1.3655905511811023e-05,
+      "loss": 0.639,
+      "step": 900
+    },
+    {
+      "epoch": 0.13377926421404682,
+      "grad_norm": 0.31701573729515076,
+      "learning_rate": 1.3592463442069742e-05,
+      "loss": 0.6269,
+      "step": 1000
+    },
+    {
+      "epoch": 0.14715719063545152,
+      "grad_norm": 0.2620750367641449,
+      "learning_rate": 1.352902137232846e-05,
+      "loss": 0.6095,
+      "step": 1100
+    },
+    {
+      "epoch": 0.1605351170568562,
+      "grad_norm": 0.31606021523475647,
+      "learning_rate": 1.3465579302587177e-05,
+      "loss": 0.6336,
+      "step": 1200
+    },
+    {
+      "epoch": 0.17391304347826086,
+      "grad_norm": 0.2545430362224579,
+      "learning_rate": 1.3402137232845896e-05,
+      "loss": 0.6061,
+      "step": 1300
+    },
+    {
+      "epoch": 0.18729096989966554,
+      "grad_norm": 0.1961711347103119,
+      "learning_rate": 1.3338695163104611e-05,
+      "loss": 0.6155,
+      "step": 1400
+    },
+    {
+      "epoch": 0.20066889632107024,
+      "grad_norm": 0.3367158770561218,
+      "learning_rate": 1.327525309336333e-05,
+      "loss": 0.6045,
+      "step": 1500
+    },
+    {
+      "epoch": 0.2140468227424749,
+      "grad_norm": 0.23051577806472778,
+      "learning_rate": 1.3211811023622048e-05,
+      "loss": 0.5899,
+      "step": 1600
+    },
+    {
+      "epoch": 0.22742474916387959,
+      "grad_norm": 0.198777973651886,
+      "learning_rate": 1.3148368953880767e-05,
+      "loss": 0.5787,
+      "step": 1700
+    },
+    {
+      "epoch": 0.2408026755852843,
+      "grad_norm": 0.20267942547798157,
+      "learning_rate": 1.3084926884139484e-05,
+      "loss": 0.5825,
+      "step": 1800
+    },
+    {
+      "epoch": 0.25418060200668896,
+      "grad_norm": 0.19377346336841583,
+      "learning_rate": 1.30214848143982e-05,
+      "loss": 0.5571,
+      "step": 1900
+    },
+    {
+      "epoch": 0.26755852842809363,
+      "grad_norm": 0.2121889889240265,
+      "learning_rate": 1.2958042744656918e-05,
+      "loss": 0.5843,
+      "step": 2000
+    },
+    {
+      "epoch": 0.2809364548494983,
+      "grad_norm": 0.28105607628822327,
+      "learning_rate": 1.2894600674915636e-05,
+      "loss": 0.5766,
+      "step": 2100
+    },
+    {
+      "epoch": 0.29431438127090304,
+      "grad_norm": 0.22681908309459686,
+      "learning_rate": 1.2831158605174355e-05,
+      "loss": 0.6059,
+      "step": 2200
+    },
+    {
+      "epoch": 0.3076923076923077,
+      "grad_norm": 0.21604150533676147,
+      "learning_rate": 1.276771653543307e-05,
+      "loss": 0.5724,
+      "step": 2300
+    },
+    {
+      "epoch": 0.3210702341137124,
+      "grad_norm": 0.24839992821216583,
+      "learning_rate": 1.270427446569179e-05,
+      "loss": 0.5921,
+      "step": 2400
+    },
+    {
+      "epoch": 0.33444816053511706,
+      "grad_norm": 0.1966131031513214,
+      "learning_rate": 1.2640832395950507e-05,
+      "loss": 0.553,
+      "step": 2500
+    },
+    {
+      "epoch": 0.34782608695652173,
+      "grad_norm": 0.21823905408382416,
+      "learning_rate": 1.2577390326209224e-05,
+      "loss": 0.5772,
+      "step": 2600
+    },
+    {
+      "epoch": 0.3612040133779264,
+      "grad_norm": 0.19366396963596344,
+      "learning_rate": 1.2513948256467943e-05,
+      "loss": 0.574,
+      "step": 2700
+    },
+    {
+      "epoch": 0.3745819397993311,
+      "grad_norm": 0.2325657606124878,
+      "learning_rate": 1.2450506186726658e-05,
+      "loss": 0.5573,
+      "step": 2800
+    },
+    {
+      "epoch": 0.3879598662207358,
+      "grad_norm": 0.1978592425584793,
+      "learning_rate": 1.2387064116985377e-05,
+      "loss": 0.5728,
+      "step": 2900
+    },
+    {
+      "epoch": 0.4013377926421405,
+      "grad_norm": 0.25745445489883423,
+      "learning_rate": 1.2323622047244095e-05,
+      "loss": 0.5659,
+      "step": 3000
+    },
+    {
+      "epoch": 0.41471571906354515,
+      "grad_norm": 0.25649788975715637,
+      "learning_rate": 1.2260179977502814e-05,
+      "loss": 0.5573,
+      "step": 3100
+    },
+    {
+      "epoch": 0.4280936454849498,
+      "grad_norm": 0.2513531744480133,
+      "learning_rate": 1.2196737907761531e-05,
+      "loss": 0.5505,
+      "step": 3200
+    },
+    {
+      "epoch": 0.4414715719063545,
+      "grad_norm": 0.3094983398914337,
+      "learning_rate": 1.2133295838020248e-05,
+      "loss": 0.5708,
+      "step": 3300
+    },
+    {
+      "epoch": 0.45484949832775917,
+      "grad_norm": 0.2737010717391968,
+      "learning_rate": 1.2069853768278965e-05,
+      "loss": 0.5586,
+      "step": 3400
+    },
+    {
+      "epoch": 0.4682274247491639,
+      "grad_norm": 0.23205673694610596,
+      "learning_rate": 1.2006411698537683e-05,
+      "loss": 0.5509,
+      "step": 3500
+    },
+    {
+      "epoch": 0.4816053511705686,
+      "grad_norm": 0.23067252337932587,
+      "learning_rate": 1.1942969628796402e-05,
+      "loss": 0.5622,
+      "step": 3600
+    },
+    {
+      "epoch": 0.49498327759197325,
+      "grad_norm": 0.25091397762298584,
+      "learning_rate": 1.1879527559055117e-05,
+      "loss": 0.5468,
+      "step": 3700
+    },
+    {
+      "epoch": 0.5083612040133779,
+      "grad_norm": 0.20600271224975586,
+      "learning_rate": 1.1816085489313836e-05,
+      "loss": 0.5649,
+      "step": 3800
+    },
+    {
+      "epoch": 0.5217391304347826,
+      "grad_norm": 0.26764076948165894,
+      "learning_rate": 1.1752643419572554e-05,
+      "loss": 0.5713,
+      "step": 3900
+    },
+    {
+      "epoch": 0.5351170568561873,
+      "grad_norm": 0.22692909836769104,
+      "learning_rate": 1.1689201349831273e-05,
+      "loss": 0.5495,
+      "step": 4000
+    },
+    {
+      "epoch": 0.5484949832775919,
+      "grad_norm": 0.2563936710357666,
+      "learning_rate": 1.162575928008999e-05,
+      "loss": 0.5557,
+      "step": 4100
+    },
+    {
+      "epoch": 0.5618729096989966,
+      "grad_norm": 0.23220312595367432,
+      "learning_rate": 1.1562317210348705e-05,
+      "loss": 0.542,
+      "step": 4200
+    },
+    {
+      "epoch": 0.5752508361204013,
+      "grad_norm": 0.19552694261074066,
+      "learning_rate": 1.1498875140607424e-05,
+      "loss": 0.5645,
+      "step": 4300
+    },
+    {
+      "epoch": 0.5886287625418061,
+      "grad_norm": 0.27880212664604187,
+      "learning_rate": 1.1435433070866142e-05,
+      "loss": 0.538,
+      "step": 4400
+    },
+    {
+      "epoch": 0.6020066889632107,
+      "grad_norm": 0.2759273052215576,
+      "learning_rate": 1.137199100112486e-05,
+      "loss": 0.554,
+      "step": 4500
+    },
+    {
+      "epoch": 0.6153846153846154,
+      "grad_norm": 0.21541567146778107,
+      "learning_rate": 1.1308548931383578e-05,
+      "loss": 0.5598,
+      "step": 4600
+    },
+    {
+      "epoch": 0.6287625418060201,
+      "grad_norm": 0.2586516737937927,
+      "learning_rate": 1.1245106861642295e-05,
+      "loss": 0.5558,
+      "step": 4700
+    },
+    {
+      "epoch": 0.6421404682274248,
+      "grad_norm": 0.355333149433136,
+      "learning_rate": 1.1181664791901013e-05,
+      "loss": 0.5529,
+      "step": 4800
+    },
+    {
+      "epoch": 0.6555183946488294,
+      "grad_norm": 0.2383916974067688,
+      "learning_rate": 1.111822272215973e-05,
+      "loss": 0.5409,
+      "step": 4900
+    },
+    {
+      "epoch": 0.6688963210702341,
+      "grad_norm": 0.24805516004562378,
+      "learning_rate": 1.1054780652418449e-05,
+      "loss": 0.5546,
+      "step": 5000
+    },
+    {
+      "epoch": 0.6822742474916388,
+      "grad_norm": 0.25741276144981384,
+      "learning_rate": 1.0991338582677164e-05,
+      "loss": 0.5393,
+      "step": 5100
+    },
+    {
+      "epoch": 0.6956521739130435,
+      "grad_norm": 0.2695322334766388,
+      "learning_rate": 1.0927896512935883e-05,
+      "loss": 0.5526,
+      "step": 5200
+    },
+    {
+      "epoch": 0.7090301003344481,
+      "grad_norm": 0.3145568072795868,
+      "learning_rate": 1.08644544431946e-05,
+      "loss": 0.541,
+      "step": 5300
+    },
+    {
+      "epoch": 0.7224080267558528,
+      "grad_norm": 0.2946968972682953,
+      "learning_rate": 1.080101237345332e-05,
+      "loss": 0.5463,
+      "step": 5400
+    },
+    {
+      "epoch": 0.7357859531772575,
+      "grad_norm": 0.24955065548419952,
+      "learning_rate": 1.0737570303712037e-05,
+      "loss": 0.5423,
+      "step": 5500
+    },
+    {
+      "epoch": 0.7491638795986622,
+      "grad_norm": 0.24316702783107758,
+      "learning_rate": 1.0674128233970754e-05,
+      "loss": 0.5445,
+      "step": 5600
+    },
+    {
+      "epoch": 0.7625418060200669,
+      "grad_norm": 0.25755423307418823,
+      "learning_rate": 1.0610686164229471e-05,
+      "loss": 0.5396,
+      "step": 5700
+    },
+    {
+      "epoch": 0.7759197324414716,
+      "grad_norm": 0.25169652700424194,
+      "learning_rate": 1.0547244094488189e-05,
+      "loss": 0.5416,
+      "step": 5800
+    },
+    {
+      "epoch": 0.7892976588628763,
+      "grad_norm": 0.27045324444770813,
+      "learning_rate": 1.0483802024746908e-05,
+      "loss": 0.5358,
+      "step": 5900
+    },
+    {
+      "epoch": 0.802675585284281,
+      "grad_norm": 0.24991023540496826,
+      "learning_rate": 1.0420359955005625e-05,
+      "loss": 0.5346,
+      "step": 6000
+    },
+    {
+      "epoch": 0.8160535117056856,
+      "grad_norm": 0.26907652616500854,
+      "learning_rate": 1.0356917885264342e-05,
+      "loss": 0.5435,
+      "step": 6100
+    },
+    {
+      "epoch": 0.8294314381270903,
+      "grad_norm": 0.23370976746082306,
+      "learning_rate": 1.029347581552306e-05,
+      "loss": 0.5465,
+      "step": 6200
+    },
+    {
+      "epoch": 0.842809364548495,
+      "grad_norm": 0.2467469573020935,
+      "learning_rate": 1.0230033745781779e-05,
+      "loss": 0.5321,
+      "step": 6300
+    },
+    {
+      "epoch": 0.8561872909698997,
+      "grad_norm": 0.2975829839706421,
+      "learning_rate": 1.0166591676040496e-05,
+      "loss": 0.5454,
+      "step": 6400
+    },
+    {
+      "epoch": 0.8695652173913043,
+      "grad_norm": 0.2862960398197174,
+      "learning_rate": 1.0103149606299211e-05,
+      "loss": 0.5364,
+      "step": 6500
+    },
+    {
+      "epoch": 0.882943143812709,
+      "grad_norm": 0.2974550724029541,
+      "learning_rate": 1.003970753655793e-05,
+      "loss": 0.5551,
+      "step": 6600
+    },
+    {
+      "epoch": 0.8963210702341137,
+      "grad_norm": 0.3028159737586975,
+      "learning_rate": 9.976265466816648e-06,
+      "loss": 0.545,
+      "step": 6700
+    },
+    {
+      "epoch": 0.9096989966555183,
+      "grad_norm": 0.2919917404651642,
+      "learning_rate": 9.912823397075367e-06,
+      "loss": 0.5321,
+      "step": 6800
+    },
+    {
+      "epoch": 0.9230769230769231,
+      "grad_norm": 0.2511245012283325,
+      "learning_rate": 9.849381327334084e-06,
+      "loss": 0.5437,
+      "step": 6900
+    },
+    {
+      "epoch": 0.9364548494983278,
+      "grad_norm": 0.2747950553894043,
+      "learning_rate": 9.785939257592801e-06,
+      "loss": 0.5515,
+      "step": 7000
+    },
+    {
+      "epoch": 0.9498327759197325,
+      "grad_norm": 0.28584617376327515,
+      "learning_rate": 9.722497187851519e-06,
+      "loss": 0.5372,
+      "step": 7100
+    },
+    {
+      "epoch": 0.9632107023411371,
+      "grad_norm": 0.3047812283039093,
+      "learning_rate": 9.659055118110236e-06,
+      "loss": 0.5497,
+      "step": 7200
+    },
+    {
+      "epoch": 0.9765886287625418,
+      "grad_norm": 0.27929434180259705,
+      "learning_rate": 9.595613048368955e-06,
+      "loss": 0.5343,
+      "step": 7300
+    },
+    {
+      "epoch": 0.9899665551839465,
+      "grad_norm": 0.26125389337539673,
+      "learning_rate": 9.532170978627672e-06,
+      "loss": 0.5345,
+      "step": 7400
+    },
+    {
+      "epoch": 1.0033444816053512,
+      "grad_norm": 0.26159632205963135,
+      "learning_rate": 9.46872890888639e-06,
+      "loss": 0.5367,
+      "step": 7500
+    },
+    {
+      "epoch": 1.0167224080267558,
+      "grad_norm": 0.2183777093887329,
+      "learning_rate": 9.405286839145107e-06,
+      "loss": 0.5299,
+      "step": 7600
+    },
+    {
+      "epoch": 1.0301003344481605,
+      "grad_norm": 0.29552075266838074,
+      "learning_rate": 9.341844769403826e-06,
+      "loss": 0.5233,
+      "step": 7700
+    },
+    {
+      "epoch": 1.0434782608695652,
+      "grad_norm": 0.30615565180778503,
+      "learning_rate": 9.278402699662543e-06,
+      "loss": 0.556,
+      "step": 7800
+    },
+    {
+      "epoch": 1.0568561872909699,
+      "grad_norm": 0.27595141530036926,
+      "learning_rate": 9.21496062992126e-06,
+      "loss": 0.5394,
+      "step": 7900
+    },
+    {
+      "epoch": 1.0702341137123745,
+      "grad_norm": 0.3372192978858948,
+      "learning_rate": 9.151518560179978e-06,
+      "loss": 0.5228,
+      "step": 8000
+    },
+    {
+      "epoch": 1.0836120401337792,
+      "grad_norm": 0.25586509704589844,
+      "learning_rate": 9.088076490438695e-06,
+      "loss": 0.5441,
+      "step": 8100
+    },
+    {
+      "epoch": 1.0969899665551839,
+      "grad_norm": 0.2440231442451477,
+      "learning_rate": 9.024634420697414e-06,
+      "loss": 0.5469,
+      "step": 8200
+    },
+    {
+      "epoch": 1.1103678929765886,
+      "grad_norm": 0.21979352831840515,
+      "learning_rate": 8.961192350956131e-06,
+      "loss": 0.5295,
+      "step": 8300
+    },
+    {
+      "epoch": 1.1237458193979932,
+      "grad_norm": 0.3029584586620331,
+      "learning_rate": 8.897750281214848e-06,
+      "loss": 0.5334,
+      "step": 8400
+    },
+    {
+      "epoch": 1.137123745819398,
+      "grad_norm": 0.2796266973018646,
+      "learning_rate": 8.834308211473566e-06,
+      "loss": 0.5346,
+      "step": 8500
+    },
+    {
+      "epoch": 1.1505016722408028,
+      "grad_norm": 0.2513901889324188,
+      "learning_rate": 8.770866141732285e-06,
+      "loss": 0.5271,
+      "step": 8600
+    },
+    {
+      "epoch": 1.1638795986622075,
+      "grad_norm": 0.32265734672546387,
+      "learning_rate": 8.707424071991002e-06,
+      "loss": 0.5426,
+      "step": 8700
+    },
+    {
+      "epoch": 1.1772575250836121,
+      "grad_norm": 0.26003068685531616,
+      "learning_rate": 8.64398200224972e-06,
+      "loss": 0.5484,
+      "step": 8800
+    },
+    {
+      "epoch": 1.1906354515050168,
+      "grad_norm": 0.28756487369537354,
+      "learning_rate": 8.580539932508436e-06,
+      "loss": 0.5459,
+      "step": 8900
+    },
+    {
+      "epoch": 1.2040133779264215,
+      "grad_norm": 0.2572449743747711,
+      "learning_rate": 8.517097862767154e-06,
+      "loss": 0.5371,
+      "step": 9000
+    },
+    {
+      "epoch": 1.2173913043478262,
+      "grad_norm": 0.3304179012775421,
+      "learning_rate": 8.453655793025873e-06,
+      "loss": 0.5319,
+      "step": 9100
+    },
+    {
+      "epoch": 1.2307692307692308,
+      "grad_norm": 0.30001091957092285,
+      "learning_rate": 8.39021372328459e-06,
+      "loss": 0.5441,
+      "step": 9200
+    },
+    {
+      "epoch": 1.2441471571906355,
+      "grad_norm": 0.29723092913627625,
+      "learning_rate": 8.326771653543307e-06,
+      "loss": 0.5252,
+      "step": 9300
+    },
+    {
+      "epoch": 1.2575250836120402,
+      "grad_norm": 0.3556860685348511,
+      "learning_rate": 8.263329583802025e-06,
+      "loss": 0.5369,
+      "step": 9400
+    },
+    {
+      "epoch": 1.2709030100334449,
+      "grad_norm": 0.3591911792755127,
+      "learning_rate": 8.199887514060742e-06,
+      "loss": 0.5352,
+      "step": 9500
+    },
+    {
+      "epoch": 1.2842809364548495,
+      "grad_norm": 0.3007011115550995,
+      "learning_rate": 8.13644544431946e-06,
+      "loss": 0.5418,
+      "step": 9600
+    },
+    {
+      "epoch": 1.2976588628762542,
+      "grad_norm": 0.2747509479522705,
+      "learning_rate": 8.073003374578178e-06,
+      "loss": 0.5377,
+      "step": 9700
+    },
+    {
+      "epoch": 1.3110367892976589,
+      "grad_norm": 0.32818782329559326,
+      "learning_rate": 8.009561304836895e-06,
+      "loss": 0.5226,
+      "step": 9800
+    },
+    {
+      "epoch": 1.3244147157190636,
+      "grad_norm": 0.33487051725387573,
+      "learning_rate": 7.946119235095613e-06,
+      "loss": 0.5314,
+      "step": 9900
+    },
+    {
+      "epoch": 1.3377926421404682,
+      "grad_norm": 0.25218817591667175,
+      "learning_rate": 7.882677165354332e-06,
+      "loss": 0.5329,
+      "step": 10000
+    },
+    {
+      "epoch": 1.351170568561873,
+      "grad_norm": 0.3178349733352661,
+      "learning_rate": 7.819235095613049e-06,
+      "loss": 0.5278,
+      "step": 10100
+    },
+    {
+      "epoch": 1.3645484949832776,
+      "grad_norm": 0.2949560284614563,
+      "learning_rate": 7.755793025871766e-06,
+      "loss": 0.5453,
+      "step": 10200
+    },
+    {
+      "epoch": 1.3779264214046822,
+      "grad_norm": 0.3159886300563812,
+      "learning_rate": 7.692350956130484e-06,
+      "loss": 0.5382,
+      "step": 10300
+    },
+    {
+      "epoch": 1.391304347826087,
+      "grad_norm": 0.32193896174430847,
+      "learning_rate": 7.628908886389202e-06,
+      "loss": 0.5365,
+      "step": 10400
+    },
+    {
+      "epoch": 1.4046822742474916,
+      "grad_norm": 0.27942079305648804,
+      "learning_rate": 7.56546681664792e-06,
+      "loss": 0.5276,
+      "step": 10500
+    },
+    {
+      "epoch": 1.4180602006688963,
+      "grad_norm": 0.28040722012519836,
+      "learning_rate": 7.502024746906637e-06,
+      "loss": 0.5281,
+      "step": 10600
+    },
+    {
+      "epoch": 1.431438127090301,
+      "grad_norm": 0.3048112094402313,
+      "learning_rate": 7.4385826771653535e-06,
+      "loss": 0.5347,
+      "step": 10700
+    },
+    {
+      "epoch": 1.4448160535117056,
+      "grad_norm": 0.3182535767555237,
+      "learning_rate": 7.375140607424072e-06,
+      "loss": 0.5444,
+      "step": 10800
+    },
+    {
+      "epoch": 1.4581939799331103,
+      "grad_norm": 0.6139577031135559,
+      "learning_rate": 7.31169853768279e-06,
+      "loss": 0.5292,
+      "step": 10900
+    },
+    {
+      "epoch": 1.471571906354515,
+      "grad_norm": 0.26481977105140686,
+      "learning_rate": 7.248256467941508e-06,
+      "loss": 0.5409,
+      "step": 11000
+    },
+    {
+      "epoch": 1.4849498327759196,
+      "grad_norm": 0.2902706265449524,
+      "learning_rate": 7.184814398200226e-06,
+      "loss": 0.5525,
+      "step": 11100
+    },
+    {
+      "epoch": 1.4983277591973243,
+      "grad_norm": 0.3402746021747589,
+      "learning_rate": 7.1213723284589425e-06,
+      "loss": 0.5267,
+      "step": 11200
+    },
+    {
+      "epoch": 1.511705685618729,
+      "grad_norm": 0.2799215018749237,
+      "learning_rate": 7.057930258717661e-06,
+      "loss": 0.5247,
+      "step": 11300
+    },
+    {
+      "epoch": 1.5250836120401337,
+      "grad_norm": 0.38291794061660767,
+      "learning_rate": 6.994488188976379e-06,
+      "loss": 0.5376,
+      "step": 11400
+    },
+    {
+      "epoch": 1.5384615384615383,
+      "grad_norm": 0.28172844648361206,
+      "learning_rate": 6.931046119235095e-06,
+      "loss": 0.5279,
+      "step": 11500
+    },
+    {
+      "epoch": 1.551839464882943,
+      "grad_norm": 0.42768093943595886,
+      "learning_rate": 6.867604049493813e-06,
+      "loss": 0.5189,
+      "step": 11600
+    },
+    {
+      "epoch": 1.5652173913043477,
+      "grad_norm": 0.2789342701435089,
+      "learning_rate": 6.8041619797525314e-06,
+      "loss": 0.528,
+      "step": 11700
+    },
+    {
+      "epoch": 1.5785953177257523,
+      "grad_norm": 0.30609387159347534,
+      "learning_rate": 6.740719910011249e-06,
+      "loss": 0.5381,
+      "step": 11800
+    },
+    {
+      "epoch": 1.5919732441471572,
+      "grad_norm": 0.3225257098674774,
+      "learning_rate": 6.677277840269967e-06,
+      "loss": 0.5472,
+      "step": 11900
+    },
+    {
+      "epoch": 1.605351170568562,
+      "grad_norm": 0.3523938059806824,
+      "learning_rate": 6.613835770528684e-06,
+      "loss": 0.5269,
+      "step": 12000
+    },
+    {
+      "epoch": 1.6187290969899666,
+      "grad_norm": 0.31391632556915283,
+      "learning_rate": 6.550393700787402e-06,
+      "loss": 0.539,
+      "step": 12100
+    },
+    {
+      "epoch": 1.6321070234113713,
+      "grad_norm": 0.3466421365737915,
+      "learning_rate": 6.4869516310461196e-06,
+      "loss": 0.5228,
+      "step": 12200
+    },
+    {
+      "epoch": 1.645484949832776,
+      "grad_norm": 0.31395405530929565,
+      "learning_rate": 6.423509561304837e-06,
+      "loss": 0.5199,
+      "step": 12300
+    },
+    {
+      "epoch": 1.6588628762541806,
+      "grad_norm": 0.2847534120082855,
+      "learning_rate": 6.360067491563555e-06,
+      "loss": 0.5233,
+      "step": 12400
+    },
+    {
+      "epoch": 1.6722408026755853,
+      "grad_norm": 0.28372418880462646,
+      "learning_rate": 6.296625421822272e-06,
+      "loss": 0.5287,
+      "step": 12500
+    },
+    {
+      "epoch": 1.68561872909699,
+      "grad_norm": 0.30443572998046875,
+      "learning_rate": 6.23318335208099e-06,
+      "loss": 0.5307,
+      "step": 12600
+    },
+    {
+      "epoch": 1.6989966555183946,
+      "grad_norm": 0.2609320878982544,
+      "learning_rate": 6.169741282339708e-06,
+      "loss": 0.5342,
+      "step": 12700
+    },
+    {
+      "epoch": 1.7123745819397993,
+      "grad_norm": 0.33631429076194763,
+      "learning_rate": 6.106299212598426e-06,
+      "loss": 0.5276,
+      "step": 12800
+    },
+    {
+      "epoch": 1.725752508361204,
+      "grad_norm": 0.3686012923717499,
+      "learning_rate": 6.042857142857143e-06,
+      "loss": 0.5469,
+      "step": 12900
+    },
+    {
+      "epoch": 1.7391304347826086,
+      "grad_norm": 0.31965962052345276,
+      "learning_rate": 5.97941507311586e-06,
+      "loss": 0.5225,
+      "step": 13000
+    },
+    {
+      "epoch": 1.7525083612040135,
+      "grad_norm": 0.3114967346191406,
+      "learning_rate": 5.9159730033745785e-06,
+      "loss": 0.5284,
+      "step": 13100
+    },
+    {
+      "epoch": 1.7658862876254182,
+      "grad_norm": 0.2994590997695923,
+      "learning_rate": 5.852530933633296e-06,
+      "loss": 0.522,
+      "step": 13200
+    },
+    {
+      "epoch": 1.779264214046823,
+      "grad_norm": 0.2821991443634033,
+      "learning_rate": 5.789088863892014e-06,
+      "loss": 0.5331,
+      "step": 13300
+    },
+    {
+      "epoch": 1.7926421404682276,
+      "grad_norm": 0.3011917769908905,
+      "learning_rate": 5.725646794150731e-06,
+      "loss": 0.53,
+      "step": 13400
+    },
+    {
+      "epoch": 1.8060200668896322,
+      "grad_norm": 0.35369744896888733,
+      "learning_rate": 5.662204724409449e-06,
+      "loss": 0.5214,
+      "step": 13500
+    },
+    {
+      "epoch": 1.819397993311037,
+      "grad_norm": 0.4051268696784973,
+      "learning_rate": 5.598762654668167e-06,
+      "loss": 0.523,
+      "step": 13600
+    },
+    {
+      "epoch": 1.8327759197324416,
+      "grad_norm": 0.291557252407074,
+      "learning_rate": 5.535320584926885e-06,
+      "loss": 0.5168,
+      "step": 13700
+    },
+    {
+      "epoch": 1.8461538461538463,
+      "grad_norm": 0.3216087222099304,
+      "learning_rate": 5.471878515185602e-06,
+      "loss": 0.5173,
+      "step": 13800
+    },
+    {
+      "epoch": 1.859531772575251,
+      "grad_norm": 0.30401530861854553,
+      "learning_rate": 5.408436445444319e-06,
+      "loss": 0.53,
+      "step": 13900
+    },
+    {
+      "epoch": 1.8729096989966556,
+      "grad_norm": 0.317964106798172,
+      "learning_rate": 5.3449943757030374e-06,
+      "loss": 0.5338,
+      "step": 14000
+    },
+    {
+      "epoch": 1.8862876254180603,
+      "grad_norm": 0.29879599809646606,
+      "learning_rate": 5.281552305961755e-06,
+      "loss": 0.5167,
+      "step": 14100
+    },
+    {
+      "epoch": 1.899665551839465,
+      "grad_norm": 0.32501399517059326,
+      "learning_rate": 5.218110236220473e-06,
+      "loss": 0.5403,
+      "step": 14200
+    },
+    {
+      "epoch": 1.9130434782608696,
+      "grad_norm": 0.3261101543903351,
+      "learning_rate": 5.15466816647919e-06,
+      "loss": 0.5393,
+      "step": 14300
+    },
+    {
+      "epoch": 1.9264214046822743,
+      "grad_norm": 0.3254980742931366,
+      "learning_rate": 5.091226096737908e-06,
+      "loss": 0.532,
+      "step": 14400
+    },
+    {
+      "epoch": 1.939799331103679,
+      "grad_norm": 0.3090830445289612,
+      "learning_rate": 5.0277840269966256e-06,
+      "loss": 0.523,
+      "step": 14500
+    },
+    {
+      "epoch": 1.9531772575250836,
+      "grad_norm": 0.3061154782772064,
+      "learning_rate": 4.964341957255343e-06,
+      "loss": 0.543,
+      "step": 14600
+    },
+    {
+      "epoch": 1.9665551839464883,
+      "grad_norm": 0.3317748010158539,
+      "learning_rate": 4.900899887514061e-06,
+      "loss": 0.5294,
+      "step": 14700
+    },
+    {
+      "epoch": 1.979933110367893,
+      "grad_norm": 0.31979861855506897,
+      "learning_rate": 4.837457817772778e-06,
+      "loss": 0.5267,
+      "step": 14800
+    },
+    {
+      "epoch": 1.9933110367892977,
+      "grad_norm": 0.3413035571575165,
+      "learning_rate": 4.774015748031496e-06,
+      "loss": 0.5249,
+      "step": 14900
+    },
+    {
+      "epoch": 2.0066889632107023,
+      "grad_norm": 0.3047432601451874,
+      "learning_rate": 4.710573678290214e-06,
+      "loss": 0.5245,
+      "step": 15000
+    },
+    {
+      "epoch": 2.020066889632107,
+      "grad_norm": 0.41027894616127014,
+      "learning_rate": 4.647131608548932e-06,
+      "loss": 0.5254,
+      "step": 15100
+    },
+    {
+      "epoch": 2.0334448160535117,
+      "grad_norm": 0.32369574904441833,
+      "learning_rate": 4.58368953880765e-06,
+      "loss": 0.5129,
+      "step": 15200
+    },
+    {
+      "epoch": 2.0468227424749164,
+      "grad_norm": 0.35738885402679443,
+      "learning_rate": 4.520247469066366e-06,
+      "loss": 0.5297,
+      "step": 15300
+    },
+    {
+      "epoch": 2.060200668896321,
+      "grad_norm": 0.3717624545097351,
+      "learning_rate": 4.4568053993250845e-06,
+      "loss": 0.523,
+      "step": 15400
+    },
+    {
+      "epoch": 2.0735785953177257,
+      "grad_norm": 0.4179551601409912,
+      "learning_rate": 4.393363329583802e-06,
+      "loss": 0.5262,
+      "step": 15500
+    },
+    {
+      "epoch": 2.0869565217391304,
+      "grad_norm": 0.28650280833244324,
+      "learning_rate": 4.32992125984252e-06,
+      "loss": 0.5173,
+      "step": 15600
+    },
+    {
+      "epoch": 2.100334448160535,
+      "grad_norm": 0.3133101463317871,
+      "learning_rate": 4.266479190101237e-06,
+      "loss": 0.5371,
+      "step": 15700
+    },
+    {
+      "epoch": 2.1137123745819397,
+      "grad_norm": 0.28342047333717346,
+      "learning_rate": 4.203037120359955e-06,
+      "loss": 0.5305,
+      "step": 15800
+    },
+    {
+      "epoch": 2.1270903010033444,
+      "grad_norm": 0.3236102759838104,
+      "learning_rate": 4.1395950506186735e-06,
+      "loss": 0.5365,
+      "step": 15900
+    },
+    {
+      "epoch": 2.140468227424749,
+      "grad_norm": 0.2886063754558563,
+      "learning_rate": 4.076152980877391e-06,
+      "loss": 0.5264,
+      "step": 16000
+    },
+    {
+      "epoch": 2.1538461538461537,
+      "grad_norm": 0.32379692792892456,
+      "learning_rate": 4.012710911136108e-06,
+      "loss": 0.5311,
+      "step": 16100
+    },
+    {
+      "epoch": 2.1672240802675584,
+      "grad_norm": 0.33309829235076904,
+      "learning_rate": 3.949268841394825e-06,
+      "loss": 0.5253,
+      "step": 16200
+    },
+    {
+      "epoch": 2.180602006688963,
+      "grad_norm": 0.4101524353027344,
+      "learning_rate": 3.8858267716535435e-06,
+      "loss": 0.5302,
+      "step": 16300
+    },
+    {
+      "epoch": 2.1939799331103678,
+      "grad_norm": 0.3807373344898224,
+      "learning_rate": 3.822384701912261e-06,
+      "loss": 0.5178,
+      "step": 16400
+    },
+    {
+      "epoch": 2.2073578595317724,
+      "grad_norm": 0.32313257455825806,
+      "learning_rate": 3.758942632170979e-06,
+      "loss": 0.5158,
+      "step": 16500
+    },
+    {
+      "epoch": 2.220735785953177,
+      "grad_norm": 0.26424798369407654,
+      "learning_rate": 3.6955005624296966e-06,
+      "loss": 0.519,
+      "step": 16600
+    },
+    {
+      "epoch": 2.234113712374582,
+      "grad_norm": 0.4011124074459076,
+      "learning_rate": 3.632058492688414e-06,
+      "loss": 0.5195,
+      "step": 16700
+    },
+    {
+      "epoch": 2.2474916387959865,
+      "grad_norm": 0.3362158238887787,
+      "learning_rate": 3.568616422947132e-06,
+      "loss": 0.5373,
+      "step": 16800
+    },
+    {
+      "epoch": 2.260869565217391,
+      "grad_norm": 0.3145718276500702,
+      "learning_rate": 3.5051743532058493e-06,
+      "loss": 0.5405,
+      "step": 16900
+    },
+    {
+      "epoch": 2.274247491638796,
+      "grad_norm": 0.3200240731239319,
+      "learning_rate": 3.4417322834645674e-06,
+      "loss": 0.5299,
+      "step": 17000
+    },
+    {
+      "epoch": 2.2876254180602005,
+      "grad_norm": 0.37791967391967773,
+      "learning_rate": 3.3782902137232847e-06,
+      "loss": 0.5237,
+      "step": 17100
+    },
+    {
+      "epoch": 2.3010033444816056,
+      "grad_norm": 0.3673136532306671,
+      "learning_rate": 3.3148481439820024e-06,
+      "loss": 0.5324,
+      "step": 17200
+    },
+    {
+      "epoch": 2.3143812709030103,
+      "grad_norm": 0.39470499753952026,
+      "learning_rate": 3.25140607424072e-06,
+      "loss": 0.541,
+      "step": 17300
+    },
+    {
+      "epoch": 2.327759197324415,
+      "grad_norm": 0.3546443283557892,
+      "learning_rate": 3.187964004499438e-06,
+      "loss": 0.5279,
+      "step": 17400
+    },
+    {
+      "epoch": 2.3411371237458196,
+      "grad_norm": 0.5016634464263916,
+      "learning_rate": 3.124521934758155e-06,
+      "loss": 0.5353,
+      "step": 17500
+    },
+    {
+      "epoch": 2.3545150501672243,
+      "grad_norm": 0.40390917658805847,
+      "learning_rate": 3.061079865016873e-06,
+      "loss": 0.5304,
+      "step": 17600
+    },
+    {
+      "epoch": 2.367892976588629,
+      "grad_norm": 0.28355857729911804,
+      "learning_rate": 2.997637795275591e-06,
+      "loss": 0.5407,
+      "step": 17700
+    },
+    {
+      "epoch": 2.3812709030100336,
+      "grad_norm": 0.3010845482349396,
+      "learning_rate": 2.9341957255343082e-06,
+      "loss": 0.5195,
+      "step": 17800
+    },
+    {
+      "epoch": 2.3946488294314383,
+      "grad_norm": 0.2997265160083771,
+      "learning_rate": 2.870753655793026e-06,
+      "loss": 0.5132,
+      "step": 17900
+    },
+    {
+      "epoch": 2.408026755852843,
+      "grad_norm": 0.2978328466415405,
+      "learning_rate": 2.8073115860517436e-06,
+      "loss": 0.5138,
+      "step": 18000
+    },
+    {
+      "epoch": 2.4214046822742477,
+      "grad_norm": 0.45286494493484497,
+      "learning_rate": 2.7438695163104614e-06,
+      "loss": 0.525,
+      "step": 18100
+    },
+    {
+      "epoch": 2.4347826086956523,
+      "grad_norm": 0.3671605885028839,
+      "learning_rate": 2.6804274465691786e-06,
+      "loss": 0.5116,
+      "step": 18200
+    },
+    {
+      "epoch": 2.448160535117057,
+      "grad_norm": 0.38759517669677734,
+      "learning_rate": 2.6169853768278963e-06,
+      "loss": 0.5252,
+      "step": 18300
+    },
+    {
+      "epoch": 2.4615384615384617,
+      "grad_norm": 0.34254246950149536,
+      "learning_rate": 2.5535433070866145e-06,
+      "loss": 0.533,
+      "step": 18400
+    },
+    {
+      "epoch": 2.4749163879598663,
+      "grad_norm": 0.4010314345359802,
+      "learning_rate": 2.490101237345332e-06,
+      "loss": 0.5378,
+      "step": 18500
+    },
+    {
+      "epoch": 2.488294314381271,
+      "grad_norm": 0.37069615721702576,
+      "learning_rate": 2.4266591676040495e-06,
+      "loss": 0.5377,
+      "step": 18600
+    },
+    {
+      "epoch": 2.5016722408026757,
+      "grad_norm": 0.3937046229839325,
+      "learning_rate": 2.363217097862767e-06,
+      "loss": 0.5195,
+      "step": 18700
+    },
+    {
+      "epoch": 2.5150501672240804,
+      "grad_norm": 0.29711517691612244,
+      "learning_rate": 2.299775028121485e-06,
+      "loss": 0.5227,
+      "step": 18800
+    },
+    {
+      "epoch": 2.528428093645485,
+      "grad_norm": 0.4100757837295532,
+      "learning_rate": 2.2363329583802026e-06,
+      "loss": 0.5354,
+      "step": 18900
+    },
+    {
+      "epoch": 2.5418060200668897,
+      "grad_norm": 0.37300094962120056,
+      "learning_rate": 2.17289088863892e-06,
+      "loss": 0.5084,
+      "step": 19000
+    },
+    {
+      "epoch": 2.5551839464882944,
+      "grad_norm": 0.42899075150489807,
+      "learning_rate": 2.109448818897638e-06,
+      "loss": 0.5302,
+      "step": 19100
+    },
+    {
+      "epoch": 2.568561872909699,
+      "grad_norm": 0.35038477182388306,
+      "learning_rate": 2.0460067491563557e-06,
+      "loss": 0.5193,
+      "step": 19200
+    },
+    {
+      "epoch": 2.5819397993311037,
+      "grad_norm": 0.42968684434890747,
+      "learning_rate": 1.9825646794150734e-06,
+      "loss": 0.5342,
+      "step": 19300
+    },
+    {
+      "epoch": 2.5953177257525084,
+      "grad_norm": 0.33188048005104065,
+      "learning_rate": 1.9191226096737907e-06,
+      "loss": 0.5129,
+      "step": 19400
+    },
+    {
+      "epoch": 2.608695652173913,
+      "grad_norm": 0.341915100812912,
+      "learning_rate": 1.8556805399325084e-06,
+      "loss": 0.5236,
+      "step": 19500
+    },
+    {
+      "epoch": 2.6220735785953178,
+      "grad_norm": 0.3550443649291992,
+      "learning_rate": 1.7922384701912261e-06,
+      "loss": 0.5349,
+      "step": 19600
+    },
+    {
+      "epoch": 2.6354515050167224,
+      "grad_norm": 0.3079615533351898,
+      "learning_rate": 1.7287964004499438e-06,
+      "loss": 0.5153,
+      "step": 19700
+    },
+    {
+      "epoch": 2.648829431438127,
+      "grad_norm": 0.43465080857276917,
+      "learning_rate": 1.6653543307086615e-06,
+      "loss": 0.5324,
+      "step": 19800
+    },
+    {
+      "epoch": 2.6622073578595318,
+      "grad_norm": 0.2985840439796448,
+      "learning_rate": 1.601912260967379e-06,
+      "loss": 0.5228,
+      "step": 19900
+    },
+    {
+      "epoch": 2.6755852842809364,
+      "grad_norm": 0.3450627624988556,
+      "learning_rate": 1.538470191226097e-06,
+      "loss": 0.5205,
+      "step": 20000
+    },
+    {
+      "epoch": 2.688963210702341,
+      "grad_norm": 0.3443169593811035,
+      "learning_rate": 1.4750281214848145e-06,
+      "loss": 0.5292,
+      "step": 20100
+    },
+    {
+      "epoch": 2.702341137123746,
+      "grad_norm": 0.3672339618206024,
+      "learning_rate": 1.4115860517435322e-06,
+      "loss": 0.5193,
+      "step": 20200
+    },
+    {
+      "epoch": 2.7157190635451505,
+      "grad_norm": 0.40497252345085144,
+      "learning_rate": 1.3481439820022499e-06,
+      "loss": 0.5132,
+      "step": 20300
+    },
+    {
+      "epoch": 2.729096989966555,
+      "grad_norm": 0.32014942169189453,
+      "learning_rate": 1.2847019122609674e-06,
+      "loss": 0.551,
+      "step": 20400
+    },
+    {
+      "epoch": 2.74247491638796,
+      "grad_norm": 0.3468855619430542,
+      "learning_rate": 1.221259842519685e-06,
+      "loss": 0.5245,
+      "step": 20500
+    }
+  ],
+  "logging_steps": 100,
+  "max_steps": 22425,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.524970641256612e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:083f0dbbc9deccdb17ae5c46bd7f4464e84155c3450aa2425eced4342371c141
+size 5368