Training in progress, step 242, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen1.5-1.8B
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

last-checkpoint/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen1.5-1.8B",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "gate_proj",
+    "down_proj",
+    "up_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": true
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ccb69d146cca0ee55c25ca7289daaf9854722df23722734395cbf1c24d79234
+size 30026872

last-checkpoint/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33ee41350dc62543c3496c438dffcc724c1ca1a20ad5fb1c8299643f70f96bd8
+size 15611412

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89d497815af807e7701e03a210800a14c62a1ee6598594a4323a0e5703ec3d75
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d95c201305ecc99e6c3028eba2cf0ad151cc73d97b4dab70e299744306eb4611
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcfe42da0a4497e8b2b172c1f9f4ec423a46dc12907f4349c55025f670422ba9
+size 11418266

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1735 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.2505175983436853,
+  "eval_steps": 242,
+  "global_step": 242,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0010351966873706005,
+      "grad_norm": 2.8279857635498047,
+      "learning_rate": 2e-05,
+      "loss": 2.0876,
+      "step": 1
+    },
+    {
+      "epoch": 0.002070393374741201,
+      "grad_norm": 4.2900238037109375,
+      "learning_rate": 4e-05,
+      "loss": 2.4727,
+      "step": 2
+    },
+    {
+      "epoch": 0.003105590062111801,
+      "grad_norm": 4.697054862976074,
+      "learning_rate": 6e-05,
+      "loss": 2.2611,
+      "step": 3
+    },
+    {
+      "epoch": 0.004140786749482402,
+      "grad_norm": 2.949223279953003,
+      "learning_rate": 8e-05,
+      "loss": 2.1478,
+      "step": 4
+    },
+    {
+      "epoch": 0.005175983436853002,
+      "grad_norm": 3.132077217102051,
+      "learning_rate": 0.0001,
+      "loss": 2.0165,
+      "step": 5
+    },
+    {
+      "epoch": 0.006211180124223602,
+      "grad_norm": 3.589725971221924,
+      "learning_rate": 0.00012,
+      "loss": 1.7078,
+      "step": 6
+    },
+    {
+      "epoch": 0.007246376811594203,
+      "grad_norm": 3.078153133392334,
+      "learning_rate": 0.00014,
+      "loss": 1.4859,
+      "step": 7
+    },
+    {
+      "epoch": 0.008281573498964804,
+      "grad_norm": 2.8390965461730957,
+      "learning_rate": 0.00016,
+      "loss": 1.1642,
+      "step": 8
+    },
+    {
+      "epoch": 0.009316770186335404,
+      "grad_norm": 2.8201794624328613,
+      "learning_rate": 0.00018,
+      "loss": 0.8378,
+      "step": 9
+    },
+    {
+      "epoch": 0.010351966873706004,
+      "grad_norm": 3.251537322998047,
+      "learning_rate": 0.0002,
+      "loss": 0.8669,
+      "step": 10
+    },
+    {
+      "epoch": 0.011387163561076604,
+      "grad_norm": 3.7539002895355225,
+      "learning_rate": 0.00019999946004996418,
+      "loss": 1.1587,
+      "step": 11
+    },
+    {
+      "epoch": 0.012422360248447204,
+      "grad_norm": 3.67051100730896,
+      "learning_rate": 0.00019999784020568754,
+      "loss": 0.6544,
+      "step": 12
+    },
+    {
+      "epoch": 0.013457556935817806,
+      "grad_norm": 2.276019334793091,
+      "learning_rate": 0.00019999514048466284,
+      "loss": 0.7244,
+      "step": 13
+    },
+    {
+      "epoch": 0.014492753623188406,
+      "grad_norm": 2.2836315631866455,
+      "learning_rate": 0.00019999136091604434,
+      "loss": 0.5898,
+      "step": 14
+    },
+    {
+      "epoch": 0.015527950310559006,
+      "grad_norm": 1.9440338611602783,
+      "learning_rate": 0.00019998650154064764,
+      "loss": 0.6185,
+      "step": 15
+    },
+    {
+      "epoch": 0.016563146997929608,
+      "grad_norm": 1.9915348291397095,
+      "learning_rate": 0.0001999805624109491,
+      "loss": 0.3567,
+      "step": 16
+    },
+    {
+      "epoch": 0.017598343685300208,
+      "grad_norm": 1.803039312362671,
+      "learning_rate": 0.0001999735435910854,
+      "loss": 0.362,
+      "step": 17
+    },
+    {
+      "epoch": 0.018633540372670808,
+      "grad_norm": 2.339751720428467,
+      "learning_rate": 0.00019996544515685281,
+      "loss": 0.5677,
+      "step": 18
+    },
+    {
+      "epoch": 0.019668737060041408,
+      "grad_norm": 2.809717893600464,
+      "learning_rate": 0.00019995626719570626,
+      "loss": 1.0207,
+      "step": 19
+    },
+    {
+      "epoch": 0.020703933747412008,
+      "grad_norm": 1.8107366561889648,
+      "learning_rate": 0.00019994600980675862,
+      "loss": 0.7298,
+      "step": 20
+    },
+    {
+      "epoch": 0.021739130434782608,
+      "grad_norm": 2.0857794284820557,
+      "learning_rate": 0.0001999346731007794,
+      "loss": 1.0046,
+      "step": 21
+    },
+    {
+      "epoch": 0.022774327122153208,
+      "grad_norm": 1.9504820108413696,
+      "learning_rate": 0.00019992225720019376,
+      "loss": 0.5604,
+      "step": 22
+    },
+    {
+      "epoch": 0.023809523809523808,
+      "grad_norm": 1.5326741933822632,
+      "learning_rate": 0.00019990876223908093,
+      "loss": 0.6794,
+      "step": 23
+    },
+    {
+      "epoch": 0.024844720496894408,
+      "grad_norm": 1.9039870500564575,
+      "learning_rate": 0.00019989418836317304,
+      "loss": 0.6101,
+      "step": 24
+    },
+    {
+      "epoch": 0.025879917184265012,
+      "grad_norm": 2.3426167964935303,
+      "learning_rate": 0.00019987853572985342,
+      "loss": 0.6149,
+      "step": 25
+    },
+    {
+      "epoch": 0.026915113871635612,
+      "grad_norm": 2.251236915588379,
+      "learning_rate": 0.00019986180450815485,
+      "loss": 0.8201,
+      "step": 26
+    },
+    {
+      "epoch": 0.027950310559006212,
+      "grad_norm": 2.3338782787323,
+      "learning_rate": 0.00019984399487875778,
+      "loss": 0.6447,
+      "step": 27
+    },
+    {
+      "epoch": 0.028985507246376812,
+      "grad_norm": 2.2376973628997803,
+      "learning_rate": 0.00019982510703398843,
+      "loss": 0.7586,
+      "step": 28
+    },
+    {
+      "epoch": 0.030020703933747412,
+      "grad_norm": 2.5092544555664062,
+      "learning_rate": 0.00019980514117781667,
+      "loss": 0.4779,
+      "step": 29
+    },
+    {
+      "epoch": 0.031055900621118012,
+      "grad_norm": 1.733722448348999,
+      "learning_rate": 0.00019978409752585376,
+      "loss": 0.5806,
+      "step": 30
+    },
+    {
+      "epoch": 0.032091097308488616,
+      "grad_norm": 2.0580904483795166,
+      "learning_rate": 0.00019976197630535014,
+      "loss": 0.6438,
+      "step": 31
+    },
+    {
+      "epoch": 0.033126293995859216,
+      "grad_norm": 2.017098903656006,
+      "learning_rate": 0.00019973877775519285,
+      "loss": 0.9921,
+      "step": 32
+    },
+    {
+      "epoch": 0.034161490683229816,
+      "grad_norm": 1.927955150604248,
+      "learning_rate": 0.0001997145021259031,
+      "loss": 0.4482,
+      "step": 33
+    },
+    {
+      "epoch": 0.035196687370600416,
+      "grad_norm": 2.332653760910034,
+      "learning_rate": 0.00019968914967963337,
+      "loss": 0.5519,
+      "step": 34
+    },
+    {
+      "epoch": 0.036231884057971016,
+      "grad_norm": 1.8889377117156982,
+      "learning_rate": 0.0001996627206901648,
+      "loss": 0.5633,
+      "step": 35
+    },
+    {
+      "epoch": 0.037267080745341616,
+      "grad_norm": 1.4307236671447754,
+      "learning_rate": 0.00019963521544290403,
+      "loss": 0.248,
+      "step": 36
+    },
+    {
+      "epoch": 0.038302277432712216,
+      "grad_norm": 1.662349820137024,
+      "learning_rate": 0.00019960663423488026,
+      "loss": 0.3661,
+      "step": 37
+    },
+    {
+      "epoch": 0.039337474120082816,
+      "grad_norm": 1.6614302396774292,
+      "learning_rate": 0.00019957697737474196,
+      "loss": 0.2634,
+      "step": 38
+    },
+    {
+      "epoch": 0.040372670807453416,
+      "grad_norm": 1.7808960676193237,
+      "learning_rate": 0.0001995462451827536,
+      "loss": 0.3277,
+      "step": 39
+    },
+    {
+      "epoch": 0.041407867494824016,
+      "grad_norm": 1.5505648851394653,
+      "learning_rate": 0.00019951443799079215,
+      "loss": 0.3677,
+      "step": 40
+    },
+    {
+      "epoch": 0.042443064182194616,
+      "grad_norm": 2.4089574813842773,
+      "learning_rate": 0.0001994815561423435,
+      "loss": 0.5753,
+      "step": 41
+    },
+    {
+      "epoch": 0.043478260869565216,
+      "grad_norm": 1.8493282794952393,
+      "learning_rate": 0.00019944759999249872,
+      "loss": 0.392,
+      "step": 42
+    },
+    {
+      "epoch": 0.044513457556935816,
+      "grad_norm": 1.7924100160598755,
+      "learning_rate": 0.0001994125699079503,
+      "loss": 0.1795,
+      "step": 43
+    },
+    {
+      "epoch": 0.045548654244306416,
+      "grad_norm": 1.8189613819122314,
+      "learning_rate": 0.00019937646626698823,
+      "loss": 0.1544,
+      "step": 44
+    },
+    {
+      "epoch": 0.046583850931677016,
+      "grad_norm": 1.3129770755767822,
+      "learning_rate": 0.00019933928945949564,
+      "loss": 0.3141,
+      "step": 45
+    },
+    {
+      "epoch": 0.047619047619047616,
+      "grad_norm": 1.254351258277893,
+      "learning_rate": 0.000199301039886945,
+      "loss": 0.3286,
+      "step": 46
+    },
+    {
+      "epoch": 0.048654244306418216,
+      "grad_norm": 1.3479806184768677,
+      "learning_rate": 0.0001992617179623934,
+      "loss": 0.1986,
+      "step": 47
+    },
+    {
+      "epoch": 0.049689440993788817,
+      "grad_norm": 1.016732931137085,
+      "learning_rate": 0.00019922132411047833,
+      "loss": 0.2908,
+      "step": 48
+    },
+    {
+      "epoch": 0.050724637681159424,
+      "grad_norm": 1.4215621948242188,
+      "learning_rate": 0.0001991798587674131,
+      "loss": 0.4213,
+      "step": 49
+    },
+    {
+      "epoch": 0.051759834368530024,
+      "grad_norm": 1.3761650323867798,
+      "learning_rate": 0.0001991373223809819,
+      "loss": 0.6013,
+      "step": 50
+    },
+    {
+      "epoch": 0.052795031055900624,
+      "grad_norm": 2.655026435852051,
+      "learning_rate": 0.00019909371541053524,
+      "loss": 0.4882,
+      "step": 51
+    },
+    {
+      "epoch": 0.053830227743271224,
+      "grad_norm": 1.8184716701507568,
+      "learning_rate": 0.00019904903832698484,
+      "loss": 0.2912,
+      "step": 52
+    },
+    {
+      "epoch": 0.054865424430641824,
+      "grad_norm": 1.7437933683395386,
+      "learning_rate": 0.0001990032916127985,
+      "loss": 0.2745,
+      "step": 53
+    },
+    {
+      "epoch": 0.055900621118012424,
+      "grad_norm": 1.6080623865127563,
+      "learning_rate": 0.00019895647576199506,
+      "loss": 0.1551,
+      "step": 54
+    },
+    {
+      "epoch": 0.056935817805383024,
+      "grad_norm": 1.4400877952575684,
+      "learning_rate": 0.0001989085912801389,
+      "loss": 0.3579,
+      "step": 55
+    },
+    {
+      "epoch": 0.057971014492753624,
+      "grad_norm": 1.0851860046386719,
+      "learning_rate": 0.00019885963868433463,
+      "loss": 0.1092,
+      "step": 56
+    },
+    {
+      "epoch": 0.059006211180124224,
+      "grad_norm": 1.5754159688949585,
+      "learning_rate": 0.00019880961850322128,
+      "loss": 0.1132,
+      "step": 57
+    },
+    {
+      "epoch": 0.060041407867494824,
+      "grad_norm": 1.2764484882354736,
+      "learning_rate": 0.00019875853127696692,
+      "loss": 0.274,
+      "step": 58
+    },
+    {
+      "epoch": 0.061076604554865424,
+      "grad_norm": 1.4094544649124146,
+      "learning_rate": 0.00019870637755726244,
+      "loss": 0.1342,
+      "step": 59
+    },
+    {
+      "epoch": 0.062111801242236024,
+      "grad_norm": 1.3471510410308838,
+      "learning_rate": 0.000198653157907316,
+      "loss": 0.128,
+      "step": 60
+    },
+    {
+      "epoch": 0.06314699792960662,
+      "grad_norm": 1.1818960905075073,
+      "learning_rate": 0.00019859887290184656,
+      "loss": 0.3254,
+      "step": 61
+    },
+    {
+      "epoch": 0.06418219461697723,
+      "grad_norm": 1.3816760778427124,
+      "learning_rate": 0.00019854352312707798,
+      "loss": 0.1952,
+      "step": 62
+    },
+    {
+      "epoch": 0.06521739130434782,
+      "grad_norm": 1.4949595928192139,
+      "learning_rate": 0.00019848710918073247,
+      "loss": 0.3095,
+      "step": 63
+    },
+    {
+      "epoch": 0.06625258799171843,
+      "grad_norm": 1.4246385097503662,
+      "learning_rate": 0.00019842963167202433,
+      "loss": 0.5965,
+      "step": 64
+    },
+    {
+      "epoch": 0.06728778467908902,
+      "grad_norm": 1.0460829734802246,
+      "learning_rate": 0.00019837109122165317,
+      "loss": 0.1189,
+      "step": 65
+    },
+    {
+      "epoch": 0.06832298136645963,
+      "grad_norm": 1.0461266040802002,
+      "learning_rate": 0.0001983114884617974,
+      "loss": 0.363,
+      "step": 66
+    },
+    {
+      "epoch": 0.06935817805383022,
+      "grad_norm": 1.0448253154754639,
+      "learning_rate": 0.00019825082403610725,
+      "loss": 0.2444,
+      "step": 67
+    },
+    {
+      "epoch": 0.07039337474120083,
+      "grad_norm": 0.717854917049408,
+      "learning_rate": 0.0001981890985996979,
+      "loss": 0.0706,
+      "step": 68
+    },
+    {
+      "epoch": 0.07142857142857142,
+      "grad_norm": 1.1387605667114258,
+      "learning_rate": 0.00019812631281914233,
+      "loss": 0.1764,
+      "step": 69
+    },
+    {
+      "epoch": 0.07246376811594203,
+      "grad_norm": 1.1424511671066284,
+      "learning_rate": 0.0001980624673724643,
+      "loss": 0.4214,
+      "step": 70
+    },
+    {
+      "epoch": 0.07349896480331262,
+      "grad_norm": 0.9396776556968689,
+      "learning_rate": 0.0001979975629491308,
+      "loss": 0.2,
+      "step": 71
+    },
+    {
+      "epoch": 0.07453416149068323,
+      "grad_norm": 0.9749090671539307,
+      "learning_rate": 0.00019793160025004475,
+      "loss": 0.1032,
+      "step": 72
+    },
+    {
+      "epoch": 0.07556935817805382,
+      "grad_norm": 0.9403194189071655,
+      "learning_rate": 0.00019786457998753737,
+      "loss": 0.1099,
+      "step": 73
+    },
+    {
+      "epoch": 0.07660455486542443,
+      "grad_norm": 1.2667652368545532,
+      "learning_rate": 0.00019779650288536058,
+      "loss": 0.6156,
+      "step": 74
+    },
+    {
+      "epoch": 0.07763975155279502,
+      "grad_norm": 1.3092149496078491,
+      "learning_rate": 0.000197727369678679,
+      "loss": 0.3501,
+      "step": 75
+    },
+    {
+      "epoch": 0.07867494824016563,
+      "grad_norm": 1.2405564785003662,
+      "learning_rate": 0.00019765718111406218,
+      "loss": 0.2177,
+      "step": 76
+    },
+    {
+      "epoch": 0.07971014492753623,
+      "grad_norm": 1.030535101890564,
+      "learning_rate": 0.00019758593794947648,
+      "loss": 0.2898,
+      "step": 77
+    },
+    {
+      "epoch": 0.08074534161490683,
+      "grad_norm": 0.6831692457199097,
+      "learning_rate": 0.00019751364095427692,
+      "loss": 0.0612,
+      "step": 78
+    },
+    {
+      "epoch": 0.08178053830227744,
+      "grad_norm": 1.4834831953048706,
+      "learning_rate": 0.0001974402909091988,
+      "loss": 0.3575,
+      "step": 79
+    },
+    {
+      "epoch": 0.08281573498964803,
+      "grad_norm": 1.2691038846969604,
+      "learning_rate": 0.00019736588860634925,
+      "loss": 0.0888,
+      "step": 80
+    },
+    {
+      "epoch": 0.08385093167701864,
+      "grad_norm": 0.9828580021858215,
+      "learning_rate": 0.00019729043484919883,
+      "loss": 0.2182,
+      "step": 81
+    },
+    {
+      "epoch": 0.08488612836438923,
+      "grad_norm": 1.3540080785751343,
+      "learning_rate": 0.00019721393045257277,
+      "loss": 0.1912,
+      "step": 82
+    },
+    {
+      "epoch": 0.08592132505175984,
+      "grad_norm": 1.0908024311065674,
+      "learning_rate": 0.000197136376242642,
+      "loss": 0.09,
+      "step": 83
+    },
+    {
+      "epoch": 0.08695652173913043,
+      "grad_norm": 0.8948976993560791,
+      "learning_rate": 0.00019705777305691456,
+      "loss": 0.1546,
+      "step": 84
+    },
+    {
+      "epoch": 0.08799171842650104,
+      "grad_norm": 0.8032020926475525,
+      "learning_rate": 0.00019697812174422632,
+      "loss": 0.104,
+      "step": 85
+    },
+    {
+      "epoch": 0.08902691511387163,
+      "grad_norm": 1.1196469068527222,
+      "learning_rate": 0.00019689742316473182,
+      "loss": 0.1588,
+      "step": 86
+    },
+    {
+      "epoch": 0.09006211180124224,
+      "grad_norm": 1.1886346340179443,
+      "learning_rate": 0.00019681567818989506,
+      "loss": 0.2584,
+      "step": 87
+    },
+    {
+      "epoch": 0.09109730848861283,
+      "grad_norm": 1.280592441558838,
+      "learning_rate": 0.00019673288770248013,
+      "loss": 0.4856,
+      "step": 88
+    },
+    {
+      "epoch": 0.09213250517598344,
+      "grad_norm": 1.5554847717285156,
+      "learning_rate": 0.00019664905259654156,
+      "loss": 0.1078,
+      "step": 89
+    },
+    {
+      "epoch": 0.09316770186335403,
+      "grad_norm": 0.960267186164856,
+      "learning_rate": 0.0001965641737774147,
+      "loss": 0.1269,
+      "step": 90
+    },
+    {
+      "epoch": 0.09420289855072464,
+      "grad_norm": 1.036106824874878,
+      "learning_rate": 0.00019647825216170597,
+      "loss": 0.1289,
+      "step": 91
+    },
+    {
+      "epoch": 0.09523809523809523,
+      "grad_norm": 0.7929521203041077,
+      "learning_rate": 0.00019639128867728298,
+      "loss": 0.0946,
+      "step": 92
+    },
+    {
+      "epoch": 0.09627329192546584,
+      "grad_norm": 0.6613808870315552,
+      "learning_rate": 0.00019630328426326448,
+      "loss": 0.0876,
+      "step": 93
+    },
+    {
+      "epoch": 0.09730848861283643,
+      "grad_norm": 0.83968186378479,
+      "learning_rate": 0.00019621423987001014,
+      "loss": 0.107,
+      "step": 94
+    },
+    {
+      "epoch": 0.09834368530020704,
+      "grad_norm": 0.9722933173179626,
+      "learning_rate": 0.00019612415645911047,
+      "loss": 0.2009,
+      "step": 95
+    },
+    {
+      "epoch": 0.09937888198757763,
+      "grad_norm": 0.6649318933486938,
+      "learning_rate": 0.00019603303500337628,
+      "loss": 0.0872,
+      "step": 96
+    },
+    {
+      "epoch": 0.10041407867494824,
+      "grad_norm": 0.9452038407325745,
+      "learning_rate": 0.00019594087648682824,
+      "loss": 0.1039,
+      "step": 97
+    },
+    {
+      "epoch": 0.10144927536231885,
+      "grad_norm": 0.6946438550949097,
+      "learning_rate": 0.00019584768190468625,
+      "loss": 0.0863,
+      "step": 98
+    },
+    {
+      "epoch": 0.10248447204968944,
+      "grad_norm": 0.8984018564224243,
+      "learning_rate": 0.0001957534522633586,
+      "loss": 0.2128,
+      "step": 99
+    },
+    {
+      "epoch": 0.10351966873706005,
+      "grad_norm": 0.9533600211143494,
+      "learning_rate": 0.00019565818858043136,
+      "loss": 0.1912,
+      "step": 100
+    },
+    {
+      "epoch": 0.10455486542443064,
+      "grad_norm": 1.6711399555206299,
+      "learning_rate": 0.00019556189188465702,
+      "loss": 0.7777,
+      "step": 101
+    },
+    {
+      "epoch": 0.10559006211180125,
+      "grad_norm": 1.2199429273605347,
+      "learning_rate": 0.00019546456321594376,
+      "loss": 0.2692,
+      "step": 102
+    },
+    {
+      "epoch": 0.10662525879917184,
+      "grad_norm": 0.9783166646957397,
+      "learning_rate": 0.0001953662036253438,
+      "loss": 0.3838,
+      "step": 103
+    },
+    {
+      "epoch": 0.10766045548654245,
+      "grad_norm": 0.6032301187515259,
+      "learning_rate": 0.00019526681417504258,
+      "loss": 0.0687,
+      "step": 104
+    },
+    {
+      "epoch": 0.10869565217391304,
+      "grad_norm": 0.7837541699409485,
+      "learning_rate": 0.0001951663959383468,
+      "loss": 0.086,
+      "step": 105
+    },
+    {
+      "epoch": 0.10973084886128365,
+      "grad_norm": 0.826560914516449,
+      "learning_rate": 0.00019506494999967298,
+      "loss": 0.363,
+      "step": 106
+    },
+    {
+      "epoch": 0.11076604554865424,
+      "grad_norm": 0.5679432153701782,
+      "learning_rate": 0.000194962477454536,
+      "loss": 0.0507,
+      "step": 107
+    },
+    {
+      "epoch": 0.11180124223602485,
+      "grad_norm": 0.8615655899047852,
+      "learning_rate": 0.00019485897940953688,
+      "loss": 0.2809,
+      "step": 108
+    },
+    {
+      "epoch": 0.11283643892339544,
+      "grad_norm": 0.9143649935722351,
+      "learning_rate": 0.0001947544569823511,
+      "loss": 0.0751,
+      "step": 109
+    },
+    {
+      "epoch": 0.11387163561076605,
+      "grad_norm": 1.1551470756530762,
+      "learning_rate": 0.00019464891130171647,
+      "loss": 0.4063,
+      "step": 110
+    },
+    {
+      "epoch": 0.11490683229813664,
+      "grad_norm": 0.9268538355827332,
+      "learning_rate": 0.0001945423435074208,
+      "loss": 0.2959,
+      "step": 111
+    },
+    {
+      "epoch": 0.11594202898550725,
+      "grad_norm": 0.7617860436439514,
+      "learning_rate": 0.00019443475475028983,
+      "loss": 0.0846,
+      "step": 112
+    },
+    {
+      "epoch": 0.11697722567287784,
+      "grad_norm": 1.0101971626281738,
+      "learning_rate": 0.00019432614619217459,
+      "loss": 0.2787,
+      "step": 113
+    },
+    {
+      "epoch": 0.11801242236024845,
+      "grad_norm": 1.0200468301773071,
+      "learning_rate": 0.000194216519005939,
+      "loss": 0.2736,
+      "step": 114
+    },
+    {
+      "epoch": 0.11904761904761904,
+      "grad_norm": 0.6387790441513062,
+      "learning_rate": 0.0001941058743754471,
+      "loss": 0.0604,
+      "step": 115
+    },
+    {
+      "epoch": 0.12008281573498965,
+      "grad_norm": 0.5903948545455933,
+      "learning_rate": 0.00019399421349555035,
+      "loss": 0.0606,
+      "step": 116
+    },
+    {
+      "epoch": 0.12111801242236025,
+      "grad_norm": 0.877464234828949,
+      "learning_rate": 0.00019388153757207471,
+      "loss": 0.2858,
+      "step": 117
+    },
+    {
+      "epoch": 0.12215320910973085,
+      "grad_norm": 0.6367958188056946,
+      "learning_rate": 0.00019376784782180746,
+      "loss": 0.0706,
+      "step": 118
+    },
+    {
+      "epoch": 0.12318840579710146,
+      "grad_norm": 0.9316298961639404,
+      "learning_rate": 0.0001936531454724844,
+      "loss": 0.2209,
+      "step": 119
+    },
+    {
+      "epoch": 0.12422360248447205,
+      "grad_norm": 0.4951368570327759,
+      "learning_rate": 0.00019353743176277622,
+      "loss": 0.0466,
+      "step": 120
+    },
+    {
+      "epoch": 0.12525879917184266,
+      "grad_norm": 0.614039421081543,
+      "learning_rate": 0.00019342070794227536,
+      "loss": 0.0664,
+      "step": 121
+    },
+    {
+      "epoch": 0.12629399585921325,
+      "grad_norm": 0.3867522180080414,
+      "learning_rate": 0.00019330297527148246,
+      "loss": 0.058,
+      "step": 122
+    },
+    {
+      "epoch": 0.12732919254658384,
+      "grad_norm": 1.0258592367172241,
+      "learning_rate": 0.00019318423502179272,
+      "loss": 0.0948,
+      "step": 123
+    },
+    {
+      "epoch": 0.12836438923395446,
+      "grad_norm": 1.0882457494735718,
+      "learning_rate": 0.00019306448847548216,
+      "loss": 0.3615,
+      "step": 124
+    },
+    {
+      "epoch": 0.12939958592132506,
+      "grad_norm": 0.7189700603485107,
+      "learning_rate": 0.00019294373692569383,
+      "loss": 0.0952,
+      "step": 125
+    },
+    {
+      "epoch": 0.13043478260869565,
+      "grad_norm": 0.5546021461486816,
+      "learning_rate": 0.0001928219816764238,
+      "loss": 0.0631,
+      "step": 126
+    },
+    {
+      "epoch": 0.13146997929606624,
+      "grad_norm": 1.1376755237579346,
+      "learning_rate": 0.0001926992240425071,
+      "loss": 0.5956,
+      "step": 127
+    },
+    {
+      "epoch": 0.13250517598343686,
+      "grad_norm": 1.056764006614685,
+      "learning_rate": 0.0001925754653496035,
+      "loss": 0.3271,
+      "step": 128
+    },
+    {
+      "epoch": 0.13354037267080746,
+      "grad_norm": 0.8348942399024963,
+      "learning_rate": 0.00019245070693418322,
+      "loss": 0.2654,
+      "step": 129
+    },
+    {
+      "epoch": 0.13457556935817805,
+      "grad_norm": 0.6183813810348511,
+      "learning_rate": 0.00019232495014351246,
+      "loss": 0.0966,
+      "step": 130
+    },
+    {
+      "epoch": 0.13561076604554864,
+      "grad_norm": 0.7051968574523926,
+      "learning_rate": 0.00019219819633563891,
+      "loss": 0.0866,
+      "step": 131
+    },
+    {
+      "epoch": 0.13664596273291926,
+      "grad_norm": 0.396007239818573,
+      "learning_rate": 0.00019207044687937703,
+      "loss": 0.0639,
+      "step": 132
+    },
+    {
+      "epoch": 0.13768115942028986,
+      "grad_norm": 0.4146755337715149,
+      "learning_rate": 0.0001919417031542933,
+      "loss": 0.0675,
+      "step": 133
+    },
+    {
+      "epoch": 0.13871635610766045,
+      "grad_norm": 0.9342963695526123,
+      "learning_rate": 0.00019181196655069127,
+      "loss": 0.0927,
+      "step": 134
+    },
+    {
+      "epoch": 0.13975155279503104,
+      "grad_norm": 1.3310648202896118,
+      "learning_rate": 0.00019168123846959666,
+      "loss": 0.2268,
+      "step": 135
+    },
+    {
+      "epoch": 0.14078674948240166,
+      "grad_norm": 0.6448948383331299,
+      "learning_rate": 0.00019154952032274206,
+      "loss": 0.0747,
+      "step": 136
+    },
+    {
+      "epoch": 0.14182194616977226,
+      "grad_norm": 0.6340884566307068,
+      "learning_rate": 0.00019141681353255184,
+      "loss": 0.0996,
+      "step": 137
+    },
+    {
+      "epoch": 0.14285714285714285,
+      "grad_norm": 0.5412713289260864,
+      "learning_rate": 0.00019128311953212678,
+      "loss": 0.0686,
+      "step": 138
+    },
+    {
+      "epoch": 0.14389233954451347,
+      "grad_norm": 0.4002693295478821,
+      "learning_rate": 0.00019114843976522842,
+      "loss": 0.0668,
+      "step": 139
+    },
+    {
+      "epoch": 0.14492753623188406,
+      "grad_norm": 0.5548096895217896,
+      "learning_rate": 0.00019101277568626374,
+      "loss": 0.0691,
+      "step": 140
+    },
+    {
+      "epoch": 0.14596273291925466,
+      "grad_norm": 0.9885902404785156,
+      "learning_rate": 0.00019087612876026908,
+      "loss": 0.235,
+      "step": 141
+    },
+    {
+      "epoch": 0.14699792960662525,
+      "grad_norm": 0.8336735963821411,
+      "learning_rate": 0.00019073850046289484,
+      "loss": 0.2889,
+      "step": 142
+    },
+    {
+      "epoch": 0.14803312629399587,
+      "grad_norm": 0.41383275389671326,
+      "learning_rate": 0.00019059989228038902,
+      "loss": 0.0606,
+      "step": 143
+    },
+    {
+      "epoch": 0.14906832298136646,
+      "grad_norm": 0.955814003944397,
+      "learning_rate": 0.0001904603057095815,
+      "loss": 0.3338,
+      "step": 144
+    },
+    {
+      "epoch": 0.15010351966873706,
+      "grad_norm": 1.192601203918457,
+      "learning_rate": 0.0001903197422578678,
+      "loss": 0.0888,
+      "step": 145
+    },
+    {
+      "epoch": 0.15113871635610765,
+      "grad_norm": 0.4909359812736511,
+      "learning_rate": 0.0001901782034431927,
+      "loss": 0.0783,
+      "step": 146
+    },
+    {
+      "epoch": 0.15217391304347827,
+      "grad_norm": 0.6072865724563599,
+      "learning_rate": 0.00019003569079403395,
+      "loss": 0.089,
+      "step": 147
+    },
+    {
+      "epoch": 0.15320910973084886,
+      "grad_norm": 0.5831778645515442,
+      "learning_rate": 0.00018989220584938573,
+      "loss": 0.0687,
+      "step": 148
+    },
+    {
+      "epoch": 0.15424430641821946,
+      "grad_norm": 0.46526455879211426,
+      "learning_rate": 0.00018974775015874213,
+      "loss": 0.0771,
+      "step": 149
+    },
+    {
+      "epoch": 0.15527950310559005,
+      "grad_norm": 1.0278894901275635,
+      "learning_rate": 0.00018960232528208022,
+      "loss": 0.4491,
+      "step": 150
+    },
+    {
+      "epoch": 0.15631469979296067,
+      "grad_norm": 1.3906394243240356,
+      "learning_rate": 0.00018945593278984333,
+      "loss": 0.7141,
+      "step": 151
+    },
+    {
+      "epoch": 0.15734989648033126,
+      "grad_norm": 0.8323919773101807,
+      "learning_rate": 0.00018930857426292412,
+      "loss": 0.1682,
+      "step": 152
+    },
+    {
+      "epoch": 0.15838509316770186,
+      "grad_norm": 0.6343721151351929,
+      "learning_rate": 0.0001891602512926474,
+      "loss": 0.0703,
+      "step": 153
+    },
+    {
+      "epoch": 0.15942028985507245,
+      "grad_norm": 0.6276781558990479,
+      "learning_rate": 0.00018901096548075305,
+      "loss": 0.1694,
+      "step": 154
+    },
+    {
+      "epoch": 0.16045548654244307,
+      "grad_norm": 0.8469846248626709,
+      "learning_rate": 0.00018886071843937866,
+      "loss": 0.2834,
+      "step": 155
+    },
+    {
+      "epoch": 0.16149068322981366,
+      "grad_norm": 0.4798016846179962,
+      "learning_rate": 0.00018870951179104212,
+      "loss": 0.0523,
+      "step": 156
+    },
+    {
+      "epoch": 0.16252587991718426,
+      "grad_norm": 0.3915042281150818,
+      "learning_rate": 0.00018855734716862417,
+      "loss": 0.0437,
+      "step": 157
+    },
+    {
+      "epoch": 0.16356107660455488,
+      "grad_norm": 0.4857017397880554,
+      "learning_rate": 0.00018840422621535066,
+      "loss": 0.064,
+      "step": 158
+    },
+    {
+      "epoch": 0.16459627329192547,
+      "grad_norm": 0.7294060587882996,
+      "learning_rate": 0.00018825015058477481,
+      "loss": 0.2458,
+      "step": 159
+    },
+    {
+      "epoch": 0.16563146997929606,
+      "grad_norm": 0.7387810945510864,
+      "learning_rate": 0.00018809512194075957,
+      "loss": 0.2508,
+      "step": 160
+    },
+    {
+      "epoch": 0.16666666666666666,
+      "grad_norm": 0.4389411509037018,
+      "learning_rate": 0.00018793914195745933,
+      "loss": 0.0594,
+      "step": 161
+    },
+    {
+      "epoch": 0.16770186335403728,
+      "grad_norm": 0.35686302185058594,
+      "learning_rate": 0.00018778221231930203,
+      "loss": 0.0518,
+      "step": 162
+    },
+    {
+      "epoch": 0.16873706004140787,
+      "grad_norm": 0.41090115904808044,
+      "learning_rate": 0.00018762433472097097,
+      "loss": 0.0367,
+      "step": 163
+    },
+    {
+      "epoch": 0.16977225672877846,
+      "grad_norm": 0.3962973952293396,
+      "learning_rate": 0.0001874655108673864,
+      "loss": 0.05,
+      "step": 164
+    },
+    {
+      "epoch": 0.17080745341614906,
+      "grad_norm": 0.2737535536289215,
+      "learning_rate": 0.00018730574247368732,
+      "loss": 0.0508,
+      "step": 165
+    },
+    {
+      "epoch": 0.17184265010351968,
+      "grad_norm": 0.7850186824798584,
+      "learning_rate": 0.0001871450312652126,
+      "loss": 0.3198,
+      "step": 166
+    },
+    {
+      "epoch": 0.17287784679089027,
+      "grad_norm": 0.8100550174713135,
+      "learning_rate": 0.00018698337897748283,
+      "loss": 0.2636,
+      "step": 167
+    },
+    {
+      "epoch": 0.17391304347826086,
+      "grad_norm": 0.4040709435939789,
+      "learning_rate": 0.0001868207873561811,
+      "loss": 0.0519,
+      "step": 168
+    },
+    {
+      "epoch": 0.17494824016563146,
+      "grad_norm": 0.7439549565315247,
+      "learning_rate": 0.00018665725815713443,
+      "loss": 0.2672,
+      "step": 169
+    },
+    {
+      "epoch": 0.17598343685300208,
+      "grad_norm": 0.7501465082168579,
+      "learning_rate": 0.00018649279314629483,
+      "loss": 0.2977,
+      "step": 170
+    },
+    {
+      "epoch": 0.17701863354037267,
+      "grad_norm": 0.6557707786560059,
+      "learning_rate": 0.00018632739409972003,
+      "loss": 0.0825,
+      "step": 171
+    },
+    {
+      "epoch": 0.17805383022774326,
+      "grad_norm": 0.8155187368392944,
+      "learning_rate": 0.00018616106280355444,
+      "loss": 0.1967,
+      "step": 172
+    },
+    {
+      "epoch": 0.17908902691511386,
+      "grad_norm": 0.4431547522544861,
+      "learning_rate": 0.00018599380105400982,
+      "loss": 0.0412,
+      "step": 173
+    },
+    {
+      "epoch": 0.18012422360248448,
+      "grad_norm": 0.586902916431427,
+      "learning_rate": 0.00018582561065734604,
+      "loss": 0.1568,
+      "step": 174
+    },
+    {
+      "epoch": 0.18115942028985507,
+      "grad_norm": 0.8832204937934875,
+      "learning_rate": 0.00018565649342985118,
+      "loss": 0.0845,
+      "step": 175
+    },
+    {
+      "epoch": 0.18219461697722567,
+      "grad_norm": 0.4738234281539917,
+      "learning_rate": 0.00018548645119782238,
+      "loss": 0.0527,
+      "step": 176
+    },
+    {
+      "epoch": 0.18322981366459629,
+      "grad_norm": 0.5710988640785217,
+      "learning_rate": 0.0001853154857975458,
+      "loss": 0.0673,
+      "step": 177
+    },
+    {
+      "epoch": 0.18426501035196688,
+      "grad_norm": 0.7422975301742554,
+      "learning_rate": 0.0001851435990752769,
+      "loss": 0.0959,
+      "step": 178
+    },
+    {
+      "epoch": 0.18530020703933747,
+      "grad_norm": 0.5633299946784973,
+      "learning_rate": 0.0001849707928872206,
+      "loss": 0.0612,
+      "step": 179
+    },
+    {
+      "epoch": 0.18633540372670807,
+      "grad_norm": 0.9179796576499939,
+      "learning_rate": 0.00018479706909951094,
+      "loss": 0.3594,
+      "step": 180
+    },
+    {
+      "epoch": 0.1873706004140787,
+      "grad_norm": 0.807579755783081,
+      "learning_rate": 0.0001846224295881913,
+      "loss": 0.2619,
+      "step": 181
+    },
+    {
+      "epoch": 0.18840579710144928,
+      "grad_norm": 0.6724291443824768,
+      "learning_rate": 0.00018444687623919386,
+      "loss": 0.242,
+      "step": 182
+    },
+    {
+      "epoch": 0.18944099378881987,
+      "grad_norm": 0.6845970153808594,
+      "learning_rate": 0.00018427041094831937,
+      "loss": 0.0827,
+      "step": 183
+    },
+    {
+      "epoch": 0.19047619047619047,
+      "grad_norm": 0.6740816831588745,
+      "learning_rate": 0.00018409303562121662,
+      "loss": 0.1934,
+      "step": 184
+    },
+    {
+      "epoch": 0.1915113871635611,
+      "grad_norm": 1.090834617614746,
+      "learning_rate": 0.00018391475217336193,
+      "loss": 0.4414,
+      "step": 185
+    },
+    {
+      "epoch": 0.19254658385093168,
+      "grad_norm": 0.5466878414154053,
+      "learning_rate": 0.0001837355625300383,
+      "loss": 0.0783,
+      "step": 186
+    },
+    {
+      "epoch": 0.19358178053830227,
+      "grad_norm": 0.931419312953949,
+      "learning_rate": 0.00018355546862631493,
+      "loss": 0.2274,
+      "step": 187
+    },
+    {
+      "epoch": 0.19461697722567287,
+      "grad_norm": 0.5122804641723633,
+      "learning_rate": 0.00018337447240702594,
+      "loss": 0.0661,
+      "step": 188
+    },
+    {
+      "epoch": 0.1956521739130435,
+      "grad_norm": 0.5255113840103149,
+      "learning_rate": 0.00018319257582674964,
+      "loss": 0.0564,
+      "step": 189
+    },
+    {
+      "epoch": 0.19668737060041408,
+      "grad_norm": 0.34425199031829834,
+      "learning_rate": 0.00018300978084978735,
+      "loss": 0.0674,
+      "step": 190
+    },
+    {
+      "epoch": 0.19772256728778467,
+      "grad_norm": 0.7382312417030334,
+      "learning_rate": 0.00018282608945014217,
+      "loss": 0.2796,
+      "step": 191
+    },
+    {
+      "epoch": 0.19875776397515527,
+      "grad_norm": 0.2415027916431427,
+      "learning_rate": 0.0001826415036114976,
+      "loss": 0.0546,
+      "step": 192
+    },
+    {
+      "epoch": 0.1997929606625259,
+      "grad_norm": 0.28614097833633423,
+      "learning_rate": 0.0001824560253271963,
+      "loss": 0.0519,
+      "step": 193
+    },
+    {
+      "epoch": 0.20082815734989648,
+      "grad_norm": 0.29192984104156494,
+      "learning_rate": 0.00018226965660021836,
+      "loss": 0.0548,
+      "step": 194
+    },
+    {
+      "epoch": 0.20186335403726707,
+      "grad_norm": 0.3177396059036255,
+      "learning_rate": 0.00018208239944315978,
+      "loss": 0.0635,
+      "step": 195
+    },
+    {
+      "epoch": 0.2028985507246377,
+      "grad_norm": 0.3965662121772766,
+      "learning_rate": 0.0001818942558782108,
+      "loss": 0.0668,
+      "step": 196
+    },
+    {
+      "epoch": 0.2039337474120083,
+      "grad_norm": 0.7662672400474548,
+      "learning_rate": 0.00018170522793713387,
+      "loss": 0.2526,
+      "step": 197
+    },
+    {
+      "epoch": 0.20496894409937888,
+      "grad_norm": 0.24819397926330566,
+      "learning_rate": 0.00018151531766124186,
+      "loss": 0.0562,
+      "step": 198
+    },
+    {
+      "epoch": 0.20600414078674947,
+      "grad_norm": 0.4133829176425934,
+      "learning_rate": 0.000181324527101376,
+      "loss": 0.0697,
+      "step": 199
+    },
+    {
+      "epoch": 0.2070393374741201,
+      "grad_norm": 0.8483571410179138,
+      "learning_rate": 0.00018113285831788365,
+      "loss": 0.2901,
+      "step": 200
+    },
+    {
+      "epoch": 0.2080745341614907,
+      "grad_norm": 1.0263965129852295,
+      "learning_rate": 0.00018094031338059617,
+      "loss": 0.7179,
+      "step": 201
+    },
+    {
+      "epoch": 0.20910973084886128,
+      "grad_norm": 0.6207789182662964,
+      "learning_rate": 0.00018074689436880644,
+      "loss": 0.1783,
+      "step": 202
+    },
+    {
+      "epoch": 0.21014492753623187,
+      "grad_norm": 0.925800085067749,
+      "learning_rate": 0.00018055260337124652,
+      "loss": 0.4417,
+      "step": 203
+    },
+    {
+      "epoch": 0.2111801242236025,
+      "grad_norm": 0.8356115221977234,
+      "learning_rate": 0.0001803574424860651,
+      "loss": 0.2758,
+      "step": 204
+    },
+    {
+      "epoch": 0.2122153209109731,
+      "grad_norm": 0.370069682598114,
+      "learning_rate": 0.0001801614138208046,
+      "loss": 0.0415,
+      "step": 205
+    },
+    {
+      "epoch": 0.21325051759834368,
+      "grad_norm": 0.7497729659080505,
+      "learning_rate": 0.0001799645194923788,
+      "loss": 0.2545,
+      "step": 206
+    },
+    {
+      "epoch": 0.21428571428571427,
+      "grad_norm": 0.9252378344535828,
+      "learning_rate": 0.00017976676162704966,
+      "loss": 0.4847,
+      "step": 207
+    },
+    {
+      "epoch": 0.2153209109730849,
+      "grad_norm": 0.3721665143966675,
+      "learning_rate": 0.0001795681423604045,
+      "loss": 0.0494,
+      "step": 208
+    },
+    {
+      "epoch": 0.2163561076604555,
+      "grad_norm": 0.9003026485443115,
+      "learning_rate": 0.00017936866383733298,
+      "loss": 0.3615,
+      "step": 209
+    },
+    {
+      "epoch": 0.21739130434782608,
+      "grad_norm": 0.36214712262153625,
+      "learning_rate": 0.00017916832821200375,
+      "loss": 0.0416,
+      "step": 210
+    },
+    {
+      "epoch": 0.21842650103519667,
+      "grad_norm": 1.1551343202590942,
+      "learning_rate": 0.00017896713764784143,
+      "loss": 0.519,
+      "step": 211
+    },
+    {
+      "epoch": 0.2194616977225673,
+      "grad_norm": 0.48404979705810547,
+      "learning_rate": 0.000178765094317503,
+      "loss": 0.0636,
+      "step": 212
+    },
+    {
+      "epoch": 0.2204968944099379,
+      "grad_norm": 0.30762040615081787,
+      "learning_rate": 0.00017856220040285458,
+      "loss": 0.0471,
+      "step": 213
+    },
+    {
+      "epoch": 0.22153209109730848,
+      "grad_norm": 0.3564504086971283,
+      "learning_rate": 0.00017835845809494768,
+      "loss": 0.0487,
+      "step": 214
+    },
+    {
+      "epoch": 0.2225672877846791,
+      "grad_norm": 0.7309280037879944,
+      "learning_rate": 0.00017815386959399565,
+      "loss": 0.2647,
+      "step": 215
+    },
+    {
+      "epoch": 0.2236024844720497,
+      "grad_norm": 0.7573635578155518,
+      "learning_rate": 0.0001779484371093498,
+      "loss": 0.2635,
+      "step": 216
+    },
+    {
+      "epoch": 0.2246376811594203,
+      "grad_norm": 0.33542540669441223,
+      "learning_rate": 0.00017774216285947576,
+      "loss": 0.0477,
+      "step": 217
+    },
+    {
+      "epoch": 0.22567287784679088,
+      "grad_norm": 0.2970454692840576,
+      "learning_rate": 0.00017753504907192923,
+      "loss": 0.0399,
+      "step": 218
+    },
+    {
+      "epoch": 0.2267080745341615,
+      "grad_norm": 0.49652576446533203,
+      "learning_rate": 0.00017732709798333221,
+      "loss": 0.1162,
+      "step": 219
+    },
+    {
+      "epoch": 0.2277432712215321,
+      "grad_norm": 0.4261191785335541,
+      "learning_rate": 0.0001771183118393486,
+      "loss": 0.0506,
+      "step": 220
+    },
+    {
+      "epoch": 0.2287784679089027,
+      "grad_norm": 0.7045575976371765,
+      "learning_rate": 0.00017690869289466017,
+      "loss": 0.2771,
+      "step": 221
+    },
+    {
+      "epoch": 0.22981366459627328,
+      "grad_norm": 0.31663334369659424,
+      "learning_rate": 0.00017669824341294202,
+      "loss": 0.0541,
+      "step": 222
+    },
+    {
+      "epoch": 0.2308488612836439,
+      "grad_norm": 0.2934422194957733,
+      "learning_rate": 0.00017648696566683824,
+      "loss": 0.0495,
+      "step": 223
+    },
+    {
+      "epoch": 0.2318840579710145,
+      "grad_norm": 0.2615799605846405,
+      "learning_rate": 0.00017627486193793742,
+      "loss": 0.0439,
+      "step": 224
+    },
+    {
+      "epoch": 0.2329192546583851,
+      "grad_norm": 0.37029555439949036,
+      "learning_rate": 0.00017606193451674785,
+      "loss": 0.0513,
+      "step": 225
+    },
+    {
+      "epoch": 0.23395445134575568,
+      "grad_norm": 0.29134050011634827,
+      "learning_rate": 0.00017584818570267284,
+      "loss": 0.0533,
+      "step": 226
+    },
+    {
+      "epoch": 0.2349896480331263,
+      "grad_norm": 0.27495524287223816,
+      "learning_rate": 0.00017563361780398613,
+      "loss": 0.0551,
+      "step": 227
+    },
+    {
+      "epoch": 0.2360248447204969,
+      "grad_norm": 0.3584780693054199,
+      "learning_rate": 0.00017541823313780647,
+      "loss": 0.059,
+      "step": 228
+    },
+    {
+      "epoch": 0.2370600414078675,
+      "grad_norm": 0.8626394271850586,
+      "learning_rate": 0.00017520203403007312,
+      "loss": 0.2204,
+      "step": 229
+    },
+    {
+      "epoch": 0.23809523809523808,
+      "grad_norm": 0.5317603349685669,
+      "learning_rate": 0.0001749850228155203,
+      "loss": 0.0744,
+      "step": 230
+    },
+    {
+      "epoch": 0.2391304347826087,
+      "grad_norm": 0.26228058338165283,
+      "learning_rate": 0.0001747672018376524,
+      "loss": 0.0557,
+      "step": 231
+    },
+    {
+      "epoch": 0.2401656314699793,
+      "grad_norm": 0.6008849143981934,
+      "learning_rate": 0.00017454857344871824,
+      "loss": 0.1827,
+      "step": 232
+    },
+    {
+      "epoch": 0.2412008281573499,
+      "grad_norm": 0.377302348613739,
+      "learning_rate": 0.00017432914000968592,
+      "loss": 0.0641,
+      "step": 233
+    },
+    {
+      "epoch": 0.2422360248447205,
+      "grad_norm": 0.4601781666278839,
+      "learning_rate": 0.00017410890389021736,
+      "loss": 0.0789,
+      "step": 234
+    },
+    {
+      "epoch": 0.2432712215320911,
+      "grad_norm": 0.9442905783653259,
+      "learning_rate": 0.00017388786746864256,
+      "loss": 0.3775,
+      "step": 235
+    },
+    {
+      "epoch": 0.2443064182194617,
+      "grad_norm": 0.8534306883811951,
+      "learning_rate": 0.000173666033131934,
+      "loss": 0.3152,
+      "step": 236
+    },
+    {
+      "epoch": 0.2453416149068323,
+      "grad_norm": 0.41121482849121094,
+      "learning_rate": 0.00017344340327568082,
+      "loss": 0.0689,
+      "step": 237
+    },
+    {
+      "epoch": 0.2463768115942029,
+      "grad_norm": 0.4545801877975464,
+      "learning_rate": 0.000173219980304063,
+      "loss": 0.0773,
+      "step": 238
+    },
+    {
+      "epoch": 0.2474120082815735,
+      "grad_norm": 1.0891170501708984,
+      "learning_rate": 0.0001729957666298254,
+      "loss": 0.4631,
+      "step": 239
+    },
+    {
+      "epoch": 0.2484472049689441,
+      "grad_norm": 0.2948625385761261,
+      "learning_rate": 0.0001727707646742516,
+      "loss": 0.0536,
+      "step": 240
+    },
+    {
+      "epoch": 0.2494824016563147,
+      "grad_norm": 0.3318866193294525,
+      "learning_rate": 0.00017254497686713797,
+      "loss": 0.0512,
+      "step": 241
+    },
+    {
+      "epoch": 0.2505175983436853,
+      "grad_norm": 0.7138605117797852,
+      "learning_rate": 0.0001723184056467671,
+      "loss": 0.1842,
+      "step": 242
+    },
+    {
+      "epoch": 0.2505175983436853,
+      "eval_loss": 0.11420517414808273,
+      "eval_runtime": 20.1367,
+      "eval_samples_per_second": 40.424,
+      "eval_steps_per_second": 20.212,
+      "step": 242
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 966,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 242,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.973242747145421e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29251127f2f5a637aaadba1d318611f17a23cac5949147be2fd33c09518eab1c
+size 6840