Upload checkpoint from SD-INP

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
+math/inp/SD-INP/math_self_distill_INP_u0.0-0.4_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
+math/inp/SD-INP/math_self_distill_INP_u0.6-1.0_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
+math/inp/SD-INP/math_self_distill_INP_u0.8-1.0_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:923eada7d8666b16cdb81ccbce193006e87c644c1311c0bb70fcee4e466bcaf6
+size 2406624648

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e196d386984beb9b9b0cbccdbedf17de6a642dc844fa690e62ee66d86702e865
+size 2406624648

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2809dca82201366f46ff3e62960cfebf78eeb97c0027c2186529aee226320034
+size 671304442

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cedbb24eb08e412f2b6567529f919723c479356a0b4861fb1f0133d92b4e4aa
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0d03a2071fbec29bd91c0dd6369cc6159f83e9366dcf2d4966bac1b9db09adc
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d8cdfceac9f7917b978dca661a3b8e04187faea5d5f6bd7b462d61d8234d57f
+size 1064

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/trainer_state.json

@@ -0,0 +1,263 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.21333333333333335,
+  "eval_steps": 500,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7686590318728121,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7938352275807403,
+      "learning_rate": 3e-06,
+      "loss": 1.1776,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5329017310026074,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.0625,
+      "kd_loss": 0.6616400777624165,
+      "learning_rate": 3e-06,
+      "loss": 0.881,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5311922653714646,
+      "epoch": 0.064,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.7425047810846863,
+      "learning_rate": 3e-06,
+      "loss": 0.8917,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46332676129907213,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.828125,
+      "kd_loss": 0.5697841891103053,
+      "learning_rate": 3e-06,
+      "loss": 0.7561,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6159080647946553,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.4999190036984601,
+      "learning_rate": 3e-06,
+      "loss": 0.8703,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6686879911097549,
+      "epoch": 0.128,
+      "grad_norm": 0.66796875,
+      "kd_loss": 0.6171865205901298,
+      "learning_rate": 3e-06,
+      "loss": 1.0246,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913988395589967,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 2.53125,
+      "kd_loss": 0.6814423620926391,
+      "learning_rate": 3e-06,
+      "loss": 1.1566,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5689800441559327,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.33984375,
+      "kd_loss": 0.49904182197715274,
+      "learning_rate": 3e-06,
+      "loss": 0.7152,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9389124085606454,
+      "epoch": 0.192,
+      "grad_norm": 0.6796875,
+      "kd_loss": 0.8731421671935546,
+      "learning_rate": 3e-06,
+      "loss": 1.2336,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5309442258150738,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.71875,
+      "kd_loss": 0.6812383219526226,
+      "learning_rate": 3e-06,
+      "loss": 1.035,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7d6e2353a4a011ee9d5626d3ff19f8d774bfd347bf01c9b971a0eaf1aee9d41
+size 7608

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:feb09cf4d8586a26359cf03f3b57878115b440f05ebdfecf952d45efca75c609
+size 2406624648

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc4e5a10f641ed6e4b5aa157f96fd10098529be4d65733089a527ccc9b2b768d
+size 671304442

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35f0d8cdba91c0873e4ce9bf07f955fed9abee001f34f2ac1f984f19666a371b
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd53d865b58debfbcc9fd888322bfe451d0ae8651ddb49493b0508f88a0f3e6b
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29d9aa99505fc60c0db1b9cdacaa08b06e8a85c8aaaab4e389667a719fafb9bf
+size 1064

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/trainer_state.json

@@ -0,0 +1,2333 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.1365333333333334,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7686590318728121,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7938352275807403,
+      "learning_rate": 3e-06,
+      "loss": 1.1776,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5329017310026074,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.0625,
+      "kd_loss": 0.6616400777624165,
+      "learning_rate": 3e-06,
+      "loss": 0.881,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5311922653714646,
+      "epoch": 0.064,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.7425047810846863,
+      "learning_rate": 3e-06,
+      "loss": 0.8917,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46332676129907213,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.828125,
+      "kd_loss": 0.5697841891103053,
+      "learning_rate": 3e-06,
+      "loss": 0.7561,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6159080647946553,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.4999190036984601,
+      "learning_rate": 3e-06,
+      "loss": 0.8703,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6686879911097549,
+      "epoch": 0.128,
+      "grad_norm": 0.66796875,
+      "kd_loss": 0.6171865205901298,
+      "learning_rate": 3e-06,
+      "loss": 1.0246,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913988395589967,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 2.53125,
+      "kd_loss": 0.6814423620926391,
+      "learning_rate": 3e-06,
+      "loss": 1.1566,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5689800441559327,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.33984375,
+      "kd_loss": 0.49904182197715274,
+      "learning_rate": 3e-06,
+      "loss": 0.7152,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9389124085606454,
+      "epoch": 0.192,
+      "grad_norm": 0.6796875,
+      "kd_loss": 0.8731421671935546,
+      "learning_rate": 3e-06,
+      "loss": 1.2336,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5309442258150738,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.71875,
+      "kd_loss": 0.6812383219526226,
+      "learning_rate": 3e-06,
+      "loss": 1.035,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    },
+    {
+      "avg_mask_ratio": 0.5232905174256303,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.5232905174256303,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.54909596200132,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 1.1875,
+      "kd_loss": 0.7053092580745443,
+      "learning_rate": 3e-06,
+      "loss": 1.0537,
+      "masked_tokens": 106.4375,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 106.4375
+    },
+    {
+      "avg_mask_ratio": 0.4815562474541366,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4815562474541366,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5007095821860276,
+      "epoch": 0.256,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.6486993154838274,
+      "learning_rate": 3e-06,
+      "loss": 0.9178,
+      "masked_tokens": 102.2,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 102.2
+    },
+    {
+      "avg_mask_ratio": 0.47414465841138737,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.47414465841138737,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.47370488773524355,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.4394873990022063,
+      "learning_rate": 3e-06,
+      "loss": 0.6913,
+      "masked_tokens": 94.7875,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 94.7875
+    },
+    {
+      "avg_mask_ratio": 0.523321858420968,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.523321858420968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9210947082143207,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 1.4375,
+      "kd_loss": 0.8009161149782358,
+      "learning_rate": 3e-06,
+      "loss": 1.2177,
+      "masked_tokens": 135.4,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 135.4
+    },
+    {
+      "avg_mask_ratio": 0.4975809322553687,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4975809322553687,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6427831699944363,
+      "epoch": 0.32,
+      "grad_norm": 0.1591796875,
+      "kd_loss": 0.6877767233976414,
+      "learning_rate": 3e-06,
+      "loss": 1.0534,
+      "masked_tokens": 129.925,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 129.925
+    },
+    {
+      "avg_mask_ratio": 0.5385947977076284,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5385947977076284,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9350756403351624,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.86328125,
+      "kd_loss": 0.7191342834705636,
+      "learning_rate": 3e-06,
+      "loss": 1.0812,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.5177937666652724,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.5177937666652724,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7097591370865757,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.83984375,
+      "kd_loss": 0.698952960721499,
+      "learning_rate": 3e-06,
+      "loss": 1.0164,
+      "masked_tokens": 93.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 93.225
+    },
+    {
+      "avg_mask_ratio": 0.4772969324782025,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.4772969324782025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6852161650983248,
+      "epoch": 0.384,
+      "grad_norm": 0.90234375,
+      "kd_loss": 0.6290313734622742,
+      "learning_rate": 3e-06,
+      "loss": 0.9438,
+      "masked_tokens": 108.8375,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 108.8375
+    },
+    {
+      "avg_mask_ratio": 0.5173690344206989,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.5173690344206989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6193340622402956,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.6558441586004846,
+      "learning_rate": 3e-06,
+      "loss": 0.959,
+      "masked_tokens": 114.0625,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 114.0625
+    },
+    {
+      "avg_mask_ratio": 0.49981915440876035,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.49981915440876035,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5121921868557535,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.40625,
+      "kd_loss": 0.5239893537682292,
+      "learning_rate": 3e-06,
+      "loss": 0.7682,
+      "masked_tokens": 101.175,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.484982778178528,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.484982778178528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.487391425676833,
+      "epoch": 0.448,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.5454409581767322,
+      "learning_rate": 3e-06,
+      "loss": 0.7901,
+      "masked_tokens": 101.2,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.5744095016038046,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5744095016038046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7552260112240674,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 1.1015625,
+      "kd_loss": 0.7122204179657274,
+      "learning_rate": 3e-06,
+      "loss": 1.1559,
+      "masked_tokens": 129.4,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 129.4
+    },
+    {
+      "avg_mask_ratio": 0.4629370831884444,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4629370831884444,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3121085743832509,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.81640625,
+      "kd_loss": 0.5816853262786026,
+      "learning_rate": 3e-06,
+      "loss": 0.8351,
+      "masked_tokens": 110.5125,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 110.5125
+    },
+    {
+      "avg_mask_ratio": 0.499816512214602,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.499816512214602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45488183712676006,
+      "epoch": 0.512,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.621679913950502,
+      "learning_rate": 3e-06,
+      "loss": 0.9475,
+      "masked_tokens": 110.075,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.075
+    },
+    {
+      "avg_mask_ratio": 0.4744578254292719,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4744578254292719,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40257708527117303,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.181640625,
+      "kd_loss": 0.5342776940590724,
+      "learning_rate": 3e-06,
+      "loss": 0.7566,
+      "masked_tokens": 117.1125,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 117.1125
+    },
+    {
+      "avg_mask_ratio": 0.4852474880579393,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.4852474880579393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3536207471866419,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.73046875,
+      "kd_loss": 0.5287767746266354,
+      "learning_rate": 3e-06,
+      "loss": 0.6987,
+      "masked_tokens": 113.725,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 113.725
+    },
+    {
+      "avg_mask_ratio": 0.565397203550674,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.565397203550674,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6109200767528364,
+      "epoch": 0.576,
+      "grad_norm": 1.0703125,
+      "kd_loss": 0.825409283463523,
+      "learning_rate": 3e-06,
+      "loss": 1.197,
+      "masked_tokens": 124.7125,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 124.7125
+    },
+    {
+      "avg_mask_ratio": 0.4814051762456074,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4814051762456074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48712714778754956,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.5236038082813025,
+      "learning_rate": 3e-06,
+      "loss": 0.8092,
+      "masked_tokens": 129.975,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 129.975
+    },
+    {
+      "avg_mask_ratio": 0.4164489531540312,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.4164489531540312,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1603455784160815,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.0849609375,
+      "kd_loss": 0.459246705797786,
+      "learning_rate": 3e-06,
+      "loss": 0.5704,
+      "masked_tokens": 100.625,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.625
+    },
+    {
+      "avg_mask_ratio": 0.47973727830685675,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.47973727830685675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4429863921482934,
+      "epoch": 0.64,
+      "grad_norm": 0.9609375,
+      "kd_loss": 0.6260649390371327,
+      "learning_rate": 3e-06,
+      "loss": 0.847,
+      "masked_tokens": 107.2375,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 107.2375
+    },
+    {
+      "avg_mask_ratio": 0.514206234831363,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.514206234831363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5035652598873412,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.51953125,
+      "kd_loss": 0.6732076569831407,
+      "learning_rate": 3e-06,
+      "loss": 0.9616,
+      "masked_tokens": 88.925,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 88.925
+    },
+    {
+      "avg_mask_ratio": 0.5238390378654003,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.5238390378654003,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.511856697372275,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.8046875,
+      "kd_loss": 0.7122418887767707,
+      "learning_rate": 3e-06,
+      "loss": 1.0805,
+      "masked_tokens": 123.25,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 123.25
+    },
+    {
+      "avg_mask_ratio": 0.5409158666618168,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.5409158666618168,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4557006843067654,
+      "epoch": 0.704,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.681670887517248,
+      "learning_rate": 3e-06,
+      "loss": 0.9853,
+      "masked_tokens": 132.2625,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 132.2625
+    },
+    {
+      "avg_mask_ratio": 0.47903697268920953,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.47903697268920953,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5934374530742617,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 1.265625,
+      "kd_loss": 0.7481572654819957,
+      "learning_rate": 3e-06,
+      "loss": 0.973,
+      "masked_tokens": 114.6375,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 114.6375
+    },
+    {
+      "avg_mask_ratio": 0.5254506973840762,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.5254506973840762,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6267570938103404,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 1.25,
+      "kd_loss": 0.7243690301205706,
+      "learning_rate": 3e-06,
+      "loss": 1.0663,
+      "masked_tokens": 129.7,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 129.7
+    },
+    {
+      "avg_mask_ratio": 0.5038800648180768,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.5038800648180768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37673365626956185,
+      "epoch": 0.768,
+      "grad_norm": 0.1943359375,
+      "kd_loss": 0.7419421046464777,
+      "learning_rate": 3e-06,
+      "loss": 0.8731,
+      "masked_tokens": 118.8375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 118.8375
+    },
+    {
+      "avg_mask_ratio": 0.5092529703164473,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.5092529703164473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5048238590722349,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.1875,
+      "kd_loss": 0.6675537923798401,
+      "learning_rate": 3e-06,
+      "loss": 0.8751,
+      "masked_tokens": 130.4375,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 130.4375
+    },
+    {
+      "avg_mask_ratio": 0.47521690553985535,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.47521690553985535,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3009114894711729,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.96875,
+      "kd_loss": 0.5576462655375508,
+      "learning_rate": 3e-06,
+      "loss": 0.7398,
+      "masked_tokens": 99.7125,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 99.7125
+    },
+    {
+      "avg_mask_ratio": 0.5089340912294574,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.5089340912294574,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42893290086574326,
+      "epoch": 0.832,
+      "grad_norm": 0.345703125,
+      "kd_loss": 0.6829758724064732,
+      "learning_rate": 3e-06,
+      "loss": 0.9627,
+      "masked_tokens": 115.925,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 115.925
+    },
+    {
+      "avg_mask_ratio": 0.5041010878514498,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.5041010878514498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5260561651751459,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 1.484375,
+      "kd_loss": 0.7314142643579544,
+      "learning_rate": 3e-06,
+      "loss": 1.0058,
+      "masked_tokens": 124.875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 124.875
+    },
+    {
+      "avg_mask_ratio": 0.5127229066158179,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.5127229066158179,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7525517523653207,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.7839959781571068,
+      "learning_rate": 3e-06,
+      "loss": 1.0779,
+      "masked_tokens": 123.575,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 123.575
+    },
+    {
+      "avg_mask_ratio": 0.47257317856419834,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.47257317856419834,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.27125484775717723,
+      "epoch": 0.896,
+      "grad_norm": 0.4609375,
+      "kd_loss": 0.5007113358164134,
+      "learning_rate": 3e-06,
+      "loss": 0.6359,
+      "masked_tokens": 90.175,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 90.175
+    },
+    {
+      "avg_mask_ratio": 0.49090774822980165,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.49090774822980165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4832683085802046,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.7578125,
+      "kd_loss": 0.579282811923622,
+      "learning_rate": 3e-06,
+      "loss": 0.8552,
+      "masked_tokens": 108.85,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4731982925441116,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.4731982925441116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5204018854022934,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.6328125,
+      "kd_loss": 0.7500040688079025,
+      "learning_rate": 3e-06,
+      "loss": 0.9602,
+      "masked_tokens": 111.5125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 111.5125
+    },
+    {
+      "avg_mask_ratio": 0.4575169428717345,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4575169428717345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.406052071983504,
+      "epoch": 0.96,
+      "grad_norm": 0.54296875,
+      "kd_loss": 0.6662414132848653,
+      "learning_rate": 3e-06,
+      "loss": 0.8269,
+      "masked_tokens": 107.8875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 107.8875
+    },
+    {
+      "avg_mask_ratio": 0.5099512930959463,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.5099512930959463,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37280656259052875,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.1357421875,
+      "kd_loss": 0.5824788354082557,
+      "learning_rate": 3e-06,
+      "loss": 0.8147,
+      "masked_tokens": 109.275,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 109.275
+    },
+    {
+      "avg_mask_ratio": 0.5108432768334058,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.5108432768334058,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4123959210827912,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.7391241128261429,
+      "learning_rate": 3e-06,
+      "loss": 0.9907,
+      "masked_tokens": 111.75,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 111.75
+    },
+    {
+      "avg_mask_ratio": 0.49879020540975033,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.49879020540975033,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4003649194146419,
+      "epoch": 1.0256,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.7427737473112416,
+      "learning_rate": 3e-06,
+      "loss": 0.9109,
+      "masked_tokens": 121.6875,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 121.6875
+    },
+    {
+      "avg_mask_ratio": 0.48607371354009954,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.48607371354009954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44578953055922454,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.7265625,
+      "kd_loss": 0.616808931784999,
+      "learning_rate": 3e-06,
+      "loss": 0.8475,
+      "masked_tokens": 102.1625,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 102.1625
+    },
+    {
+      "avg_mask_ratio": 0.5385718538891524,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5385718538891524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4503419364663387,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 2.3125,
+      "kd_loss": 0.7079652768573397,
+      "learning_rate": 3e-06,
+      "loss": 0.9016,
+      "masked_tokens": 127.6125,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.6125
+    },
+    {
+      "avg_mask_ratio": 0.5356179510476068,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5356179510476068,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5114419424304287,
+      "epoch": 1.0896,
+      "grad_norm": 2.421875,
+      "kd_loss": 0.8057545678497263,
+      "learning_rate": 3e-06,
+      "loss": 0.9692,
+      "masked_tokens": 136.725,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 136.725
+    },
+    {
+      "avg_mask_ratio": 0.4930020817089826,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4930020817089826,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45815266029704843,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.73046875,
+      "kd_loss": 0.6372618476128536,
+      "learning_rate": 3e-06,
+      "loss": 0.8813,
+      "masked_tokens": 91.975,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 91.975
+    },
+    {
+      "avg_mask_ratio": 0.4998604157241061,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4998604157241061,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5302904470116345,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.793973692091648,
+      "learning_rate": 3e-06,
+      "loss": 1.0187,
+      "masked_tokens": 103.4125,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 103.4125
+    },
+    {
+      "avg_mask_ratio": 0.5236943962518126,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.5236943962518126,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5931689006157967,
+      "epoch": 1.1536,
+      "grad_norm": 0.52734375,
+      "kd_loss": 0.6770309421569891,
+      "learning_rate": 3e-06,
+      "loss": 0.9785,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.4774137590778992,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4774137590778992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33368192758867965,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.552957196540001,
+      "learning_rate": 3e-06,
+      "loss": 0.7195,
+      "masked_tokens": 101.425,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 101.425
+    },
+    {
+      "avg_mask_ratio": 0.41173738130601123,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.41173738130601123,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37927565402829944,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.6514939159315418,
+      "learning_rate": 3e-06,
+      "loss": 0.8086,
+      "masked_tokens": 102.0375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 102.0375
+    },
+    {
+      "avg_mask_ratio": 0.4797614786075428,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.4797614786075428,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38391759186294505,
+      "epoch": 1.2176,
+      "grad_norm": 0.6875,
+      "kd_loss": 0.5477132451650505,
+      "learning_rate": 3e-06,
+      "loss": 0.7759,
+      "masked_tokens": 112.25,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 112.25
+    },
+    {
+      "avg_mask_ratio": 0.4974610014585778,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4974610014585778,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4613554118723869,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.8125,
+      "kd_loss": 0.7624371517440522,
+      "learning_rate": 3e-06,
+      "loss": 0.8935,
+      "masked_tokens": 134.025,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 134.025
+    },
+    {
+      "avg_mask_ratio": 0.5565119812032208,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5565119812032208,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45800332796356996,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.84765625,
+      "kd_loss": 0.7569947588504874,
+      "learning_rate": 3e-06,
+      "loss": 0.9895,
+      "masked_tokens": 126.1375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 126.1375
+    },
+    {
+      "avg_mask_ratio": 0.4784870075061917,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.4784870075061917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42934841778910593,
+      "epoch": 1.2816,
+      "grad_norm": 0.8203125,
+      "kd_loss": 0.6223680850909716,
+      "learning_rate": 3e-06,
+      "loss": 0.8811,
+      "masked_tokens": 113.875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 113.875
+    },
+    {
+      "avg_mask_ratio": 0.5459770569577813,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5459770569577813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46207297699419314,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.193359375,
+      "kd_loss": 0.7765240747485223,
+      "learning_rate": 3e-06,
+      "loss": 1.0488,
+      "masked_tokens": 115.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 115.85
+    },
+    {
+      "avg_mask_ratio": 0.5268841385375709,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.5268841385375709,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5177706574260241,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.42578125,
+      "kd_loss": 0.7318520774833814,
+      "learning_rate": 3e-06,
+      "loss": 1.029,
+      "masked_tokens": 114.6625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 114.6625
+    },
+    {
+      "avg_mask_ratio": 0.5629246362368576,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5629246362368576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5840083432511903,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.140625,
+      "kd_loss": 0.7630814963876361,
+      "learning_rate": 3e-06,
+      "loss": 1.2155,
+      "masked_tokens": 130.675,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.675
+    },
+    {
+      "avg_mask_ratio": 0.5119291188195347,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.5119291188195347,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3960001973351496,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.431640625,
+      "kd_loss": 0.5936955793389871,
+      "learning_rate": 3e-06,
+      "loss": 0.7667,
+      "masked_tokens": 125.9,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 125.9
+    },
+    {
+      "avg_mask_ratio": 0.5539714884362184,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5539714884362184,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6942642153400925,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.625,
+      "kd_loss": 0.8088986897017776,
+      "learning_rate": 3e-06,
+      "loss": 1.1162,
+      "masked_tokens": 131.7625,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 131.7625
+    },
+    {
+      "avg_mask_ratio": 0.5141558598377742,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.5141558598377742,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43362003258180265,
+      "epoch": 1.4096,
+      "grad_norm": 1.1875,
+      "kd_loss": 0.6850645583552251,
+      "learning_rate": 3e-06,
+      "loss": 0.9688,
+      "masked_tokens": 133.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 133.35
+    },
+    {
+      "avg_mask_ratio": 0.4895282822311856,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.4895282822311856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40907790855981147,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 1.1953125,
+      "kd_loss": 0.5458992913151632,
+      "learning_rate": 3e-06,
+      "loss": 0.7579,
+      "masked_tokens": 123.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 123.125
+    },
+    {
+      "avg_mask_ratio": 0.4951617428450845,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4951617428450845,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4818705332485024,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.45703125,
+      "kd_loss": 0.6341485966125283,
+      "learning_rate": 3e-06,
+      "loss": 0.8653,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.5100495176156983,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.5100495176156983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5174321293302568,
+      "epoch": 1.4736,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.692481756274401,
+      "learning_rate": 3e-06,
+      "loss": 0.8852,
+      "masked_tokens": 115.8875,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 115.8875
+    },
+    {
+      "avg_mask_ratio": 0.5639110118616373,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5639110118616373,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4771150118128048,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.46484375,
+      "kd_loss": 0.9550692923163296,
+      "learning_rate": 3e-06,
+      "loss": 1.1428,
+      "masked_tokens": 138.2,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 138.2
+    },
+    {
+      "avg_mask_ratio": 0.5292218026472255,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.5292218026472255,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.36868086418378426,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.283203125,
+      "kd_loss": 0.6208373298897982,
+      "learning_rate": 3e-06,
+      "loss": 0.782,
+      "masked_tokens": 113.375,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 113.375
+    },
+    {
+      "avg_mask_ratio": 0.5125403102487326,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.5125403102487326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5343423100649431,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 1.1796875,
+      "kd_loss": 0.8048794609300558,
+      "learning_rate": 3e-06,
+      "loss": 1.0332,
+      "masked_tokens": 115.675,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 115.675
+    },
+    {
+      "avg_mask_ratio": 0.48811948703369124,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.48811948703369124,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5609710454625769,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.6331229786760091,
+      "learning_rate": 3e-06,
+      "loss": 0.906,
+      "masked_tokens": 106.65,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.5547609420493245,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5547609420493245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6117596920593087,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7415648018846696,
+      "learning_rate": 3e-06,
+      "loss": 1.0796,
+      "masked_tokens": 100.775,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 100.775
+    },
+    {
+      "avg_mask_ratio": 0.44697874613921157,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.44697874613921157,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4524557980651025,
+      "epoch": 1.6016,
+      "grad_norm": 0.8125,
+      "kd_loss": 0.7427308069233959,
+      "learning_rate": 3e-06,
+      "loss": 0.8892,
+      "masked_tokens": 99.5,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 99.5
+    },
+    {
+      "avg_mask_ratio": 0.49905171967693607,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.49905171967693607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5331612331144584,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.869146594414876,
+      "learning_rate": 3e-06,
+      "loss": 1.024,
+      "masked_tokens": 111.525,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 111.525
+    },
+    {
+      "avg_mask_ratio": 0.4734679562970996,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4734679562970996,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3263147723814427,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.78515625,
+      "kd_loss": 0.5851220395212977,
+      "learning_rate": 3e-06,
+      "loss": 0.6745,
+      "masked_tokens": 124.1625,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 124.1625
+    },
+    {
+      "avg_mask_ratio": 0.45589545626135075,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.45589545626135075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40714966794125756,
+      "epoch": 1.6656,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.5390054064211469,
+      "learning_rate": 3e-06,
+      "loss": 0.6662,
+      "masked_tokens": 120.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 120.2
+    },
+    {
+      "avg_mask_ratio": 0.4975356309209019,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4975356309209019,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3924752975523688,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.125,
+      "kd_loss": 0.5501754963899657,
+      "learning_rate": 3e-06,
+      "loss": 0.7353,
+      "masked_tokens": 107.35,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 107.35
+    },
+    {
+      "avg_mask_ratio": 0.4948011673986912,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.4948011673986912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.32645424191614436,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 1.125,
+      "kd_loss": 0.5133585692653468,
+      "learning_rate": 3e-06,
+      "loss": 0.7248,
+      "masked_tokens": 109.1875,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 109.1875
+    },
+    {
+      "avg_mask_ratio": 0.5321399106411263,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5321399106411263,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5308010719869571,
+      "epoch": 1.7296,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.6684280144268996,
+      "learning_rate": 3e-06,
+      "loss": 0.9887,
+      "masked_tokens": 123.0375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 123.0375
+    },
+    {
+      "avg_mask_ratio": 0.5357416228158399,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5357416228158399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5018791363456672,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.8031356525036244,
+      "learning_rate": 3e-06,
+      "loss": 1.0867,
+      "masked_tokens": 121.5625,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 121.5625
+    },
+    {
+      "avg_mask_ratio": 0.5232944375369698,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.5232944375369698,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4925852923925049,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.9140625,
+      "kd_loss": 0.796732622892182,
+      "learning_rate": 3e-06,
+      "loss": 1.0096,
+      "masked_tokens": 138.675,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 138.675
+    },
+    {
+      "avg_mask_ratio": 0.5344608084415086,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5344608084415086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39800737908125255,
+      "epoch": 1.7936,
+      "grad_norm": 0.2041015625,
+      "kd_loss": 0.6124706103168371,
+      "learning_rate": 3e-06,
+      "loss": 0.806,
+      "masked_tokens": 119.225,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 119.225
+    },
+    {
+      "avg_mask_ratio": 0.5352845921181142,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5352845921181142,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.398157004356932,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.5720241727458018,
+      "learning_rate": 3e-06,
+      "loss": 0.7641,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4979630701942369,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4979630701942369,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4882415359192748,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.75,
+      "kd_loss": 0.6685384457871464,
+      "learning_rate": 3e-06,
+      "loss": 0.8746,
+      "masked_tokens": 111.275,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 111.275
+    },
+    {
+      "avg_mask_ratio": 0.5208624298567883,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.5208624298567883,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3722814081119509,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.90234375,
+      "kd_loss": 0.6259868353982938,
+      "learning_rate": 3e-06,
+      "loss": 0.931,
+      "masked_tokens": 119.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 119.8875
+    },
+    {
+      "avg_mask_ratio": 0.4884064760175534,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.4884064760175534,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3424368775197877,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.9453125,
+      "kd_loss": 0.7612382256288385,
+      "learning_rate": 3e-06,
+      "loss": 0.9297,
+      "masked_tokens": 97.0,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 97.0
+    },
+    {
+      "avg_mask_ratio": 0.47601241993543225,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.47601241993543225,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.29439299476740644,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.279296875,
+      "kd_loss": 0.5778021513873682,
+      "learning_rate": 3e-06,
+      "loss": 0.6641,
+      "masked_tokens": 107.1375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 107.1375
+    },
+    {
+      "avg_mask_ratio": 0.4820589871611446,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4820589871611446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4219410733627285,
+      "epoch": 1.9216,
+      "grad_norm": 0.51953125,
+      "kd_loss": 0.6610599886284263,
+      "learning_rate": 3e-06,
+      "loss": 0.8875,
+      "masked_tokens": 106.675,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 106.675
+    },
+    {
+      "avg_mask_ratio": 0.5009820312960074,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.5009820312960074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4484953544199811,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.423828125,
+      "kd_loss": 0.6393605162026915,
+      "learning_rate": 3e-06,
+      "loss": 0.8387,
+      "masked_tokens": 129.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 129.1625
+    },
+    {
+      "avg_mask_ratio": 0.4952817424898967,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4952817424898967,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4003621553950552,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 1.28125,
+      "kd_loss": 0.7165493822423741,
+      "learning_rate": 3e-06,
+      "loss": 0.87,
+      "masked_tokens": 114.2625,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 114.2625
+    },
+    {
+      "avg_mask_ratio": 0.5025755434762686,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.5025755434762686,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44728768827070553,
+      "epoch": 1.9856,
+      "grad_norm": 0.380859375,
+      "kd_loss": 0.5998596335327193,
+      "learning_rate": 3e-06,
+      "loss": 0.769,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5328231096001608,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5328231096001608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34552624383357916,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.6015625,
+      "kd_loss": 0.6963629411807882,
+      "learning_rate": 3e-06,
+      "loss": 0.9754,
+      "masked_tokens": 126.51190476190476,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 126.51190476190476
+    },
+    {
+      "avg_mask_ratio": 0.46634063599049114,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.46634063599049114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3470186025385317,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.6648836274001951,
+      "learning_rate": 3e-06,
+      "loss": 0.8308,
+      "masked_tokens": 111.15,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 111.15
+    },
+    {
+      "avg_mask_ratio": 0.5202614731155336,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.5202614731155336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39255777003332354,
+      "epoch": 2.0512,
+      "grad_norm": 0.9921875,
+      "kd_loss": 0.8403500239691084,
+      "learning_rate": 3e-06,
+      "loss": 1.0217,
+      "masked_tokens": 146.275,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 146.275
+    },
+    {
+      "avg_mask_ratio": 0.4732307325524744,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4732307325524744,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4136752946935417,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 1.6015625,
+      "kd_loss": 0.5665071224577105,
+      "learning_rate": 3e-06,
+      "loss": 0.7771,
+      "masked_tokens": 111.6375,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 111.6375
+    },
+    {
+      "avg_mask_ratio": 0.45015103057958183,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.45015103057958183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24581733565152036,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.515625,
+      "kd_loss": 0.5551535871100896,
+      "learning_rate": 3e-06,
+      "loss": 0.697,
+      "masked_tokens": 102.2875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 102.2875
+    },
+    {
+      "avg_mask_ratio": 0.48529006402241065,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.48529006402241065,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2608388370660464,
+      "epoch": 2.1152,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.5421745884898016,
+      "learning_rate": 3e-06,
+      "loss": 0.6504,
+      "masked_tokens": 123.075,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 123.075
+    },
+    {
+      "avg_mask_ratio": 0.4696127205621451,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4696127205621451,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3479914085991481,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.5546875,
+      "kd_loss": 0.5542144843038216,
+      "learning_rate": 3e-06,
+      "loss": 0.7371,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 103.0875
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7d6e2353a4a011ee9d5626d3ff19f8d774bfd347bf01c9b971a0eaf1aee9d41
+size 7608

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a6fd53c55cb113343d72476bc74f6734b1982008cde0738aa2265be4473fec7
+size 2406624648

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee9fbcc325609bcabcc07e77329ec4ba532ab44384defaacb57291f2bd81281e
+size 671304442

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4d951b3681768d0a5bb5b4a429126b0d534a20a49c0499d63f2afab759f4fb3
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b99783a29a620816bb0a632d5d2cf8313aa70711e5da7dbe41120d24b53c799f
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25795e3b7374d0f6abdd7ab4b34fbf7ab0447ba73c04014500c2ab8b5acec5b4
+size 1064

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/trainer_state.json

@@ -0,0 +1,2563 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.3498666666666668,
+  "eval_steps": 500,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7686590318728121,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7938352275807403,
+      "learning_rate": 3e-06,
+      "loss": 1.1776,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5329017310026074,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.0625,
+      "kd_loss": 0.6616400777624165,
+      "learning_rate": 3e-06,
+      "loss": 0.881,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5311922653714646,
+      "epoch": 0.064,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.7425047810846863,
+      "learning_rate": 3e-06,
+      "loss": 0.8917,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46332676129907213,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.828125,
+      "kd_loss": 0.5697841891103053,
+      "learning_rate": 3e-06,
+      "loss": 0.7561,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6159080647946553,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.4999190036984601,
+      "learning_rate": 3e-06,
+      "loss": 0.8703,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6686879911097549,
+      "epoch": 0.128,
+      "grad_norm": 0.66796875,
+      "kd_loss": 0.6171865205901298,
+      "learning_rate": 3e-06,
+      "loss": 1.0246,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913988395589967,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 2.53125,
+      "kd_loss": 0.6814423620926391,
+      "learning_rate": 3e-06,
+      "loss": 1.1566,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5689800441559327,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.33984375,
+      "kd_loss": 0.49904182197715274,
+      "learning_rate": 3e-06,
+      "loss": 0.7152,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9389124085606454,
+      "epoch": 0.192,
+      "grad_norm": 0.6796875,
+      "kd_loss": 0.8731421671935546,
+      "learning_rate": 3e-06,
+      "loss": 1.2336,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5309442258150738,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.71875,
+      "kd_loss": 0.6812383219526226,
+      "learning_rate": 3e-06,
+      "loss": 1.035,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    },
+    {
+      "avg_mask_ratio": 0.5232905174256303,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.5232905174256303,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.54909596200132,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 1.1875,
+      "kd_loss": 0.7053092580745443,
+      "learning_rate": 3e-06,
+      "loss": 1.0537,
+      "masked_tokens": 106.4375,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 106.4375
+    },
+    {
+      "avg_mask_ratio": 0.4815562474541366,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4815562474541366,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5007095821860276,
+      "epoch": 0.256,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.6486993154838274,
+      "learning_rate": 3e-06,
+      "loss": 0.9178,
+      "masked_tokens": 102.2,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 102.2
+    },
+    {
+      "avg_mask_ratio": 0.47414465841138737,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.47414465841138737,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.47370488773524355,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.4394873990022063,
+      "learning_rate": 3e-06,
+      "loss": 0.6913,
+      "masked_tokens": 94.7875,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 94.7875
+    },
+    {
+      "avg_mask_ratio": 0.523321858420968,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.523321858420968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9210947082143207,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 1.4375,
+      "kd_loss": 0.8009161149782358,
+      "learning_rate": 3e-06,
+      "loss": 1.2177,
+      "masked_tokens": 135.4,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 135.4
+    },
+    {
+      "avg_mask_ratio": 0.4975809322553687,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4975809322553687,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6427831699944363,
+      "epoch": 0.32,
+      "grad_norm": 0.1591796875,
+      "kd_loss": 0.6877767233976414,
+      "learning_rate": 3e-06,
+      "loss": 1.0534,
+      "masked_tokens": 129.925,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 129.925
+    },
+    {
+      "avg_mask_ratio": 0.5385947977076284,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5385947977076284,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9350756403351624,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.86328125,
+      "kd_loss": 0.7191342834705636,
+      "learning_rate": 3e-06,
+      "loss": 1.0812,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.5177937666652724,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.5177937666652724,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7097591370865757,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.83984375,
+      "kd_loss": 0.698952960721499,
+      "learning_rate": 3e-06,
+      "loss": 1.0164,
+      "masked_tokens": 93.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 93.225
+    },
+    {
+      "avg_mask_ratio": 0.4772969324782025,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.4772969324782025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6852161650983248,
+      "epoch": 0.384,
+      "grad_norm": 0.90234375,
+      "kd_loss": 0.6290313734622742,
+      "learning_rate": 3e-06,
+      "loss": 0.9438,
+      "masked_tokens": 108.8375,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 108.8375
+    },
+    {
+      "avg_mask_ratio": 0.5173690344206989,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.5173690344206989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6193340622402956,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.6558441586004846,
+      "learning_rate": 3e-06,
+      "loss": 0.959,
+      "masked_tokens": 114.0625,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 114.0625
+    },
+    {
+      "avg_mask_ratio": 0.49981915440876035,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.49981915440876035,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5121921868557535,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.40625,
+      "kd_loss": 0.5239893537682292,
+      "learning_rate": 3e-06,
+      "loss": 0.7682,
+      "masked_tokens": 101.175,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.484982778178528,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.484982778178528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.487391425676833,
+      "epoch": 0.448,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.5454409581767322,
+      "learning_rate": 3e-06,
+      "loss": 0.7901,
+      "masked_tokens": 101.2,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.5744095016038046,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5744095016038046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7552260112240674,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 1.1015625,
+      "kd_loss": 0.7122204179657274,
+      "learning_rate": 3e-06,
+      "loss": 1.1559,
+      "masked_tokens": 129.4,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 129.4
+    },
+    {
+      "avg_mask_ratio": 0.4629370831884444,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4629370831884444,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3121085743832509,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.81640625,
+      "kd_loss": 0.5816853262786026,
+      "learning_rate": 3e-06,
+      "loss": 0.8351,
+      "masked_tokens": 110.5125,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 110.5125
+    },
+    {
+      "avg_mask_ratio": 0.499816512214602,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.499816512214602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45488183712676006,
+      "epoch": 0.512,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.621679913950502,
+      "learning_rate": 3e-06,
+      "loss": 0.9475,
+      "masked_tokens": 110.075,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.075
+    },
+    {
+      "avg_mask_ratio": 0.4744578254292719,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4744578254292719,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40257708527117303,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.181640625,
+      "kd_loss": 0.5342776940590724,
+      "learning_rate": 3e-06,
+      "loss": 0.7566,
+      "masked_tokens": 117.1125,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 117.1125
+    },
+    {
+      "avg_mask_ratio": 0.4852474880579393,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.4852474880579393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3536207471866419,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.73046875,
+      "kd_loss": 0.5287767746266354,
+      "learning_rate": 3e-06,
+      "loss": 0.6987,
+      "masked_tokens": 113.725,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 113.725
+    },
+    {
+      "avg_mask_ratio": 0.565397203550674,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.565397203550674,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6109200767528364,
+      "epoch": 0.576,
+      "grad_norm": 1.0703125,
+      "kd_loss": 0.825409283463523,
+      "learning_rate": 3e-06,
+      "loss": 1.197,
+      "masked_tokens": 124.7125,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 124.7125
+    },
+    {
+      "avg_mask_ratio": 0.4814051762456074,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4814051762456074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48712714778754956,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.5236038082813025,
+      "learning_rate": 3e-06,
+      "loss": 0.8092,
+      "masked_tokens": 129.975,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 129.975
+    },
+    {
+      "avg_mask_ratio": 0.4164489531540312,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.4164489531540312,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1603455784160815,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.0849609375,
+      "kd_loss": 0.459246705797786,
+      "learning_rate": 3e-06,
+      "loss": 0.5704,
+      "masked_tokens": 100.625,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.625
+    },
+    {
+      "avg_mask_ratio": 0.47973727830685675,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.47973727830685675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4429863921482934,
+      "epoch": 0.64,
+      "grad_norm": 0.9609375,
+      "kd_loss": 0.6260649390371327,
+      "learning_rate": 3e-06,
+      "loss": 0.847,
+      "masked_tokens": 107.2375,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 107.2375
+    },
+    {
+      "avg_mask_ratio": 0.514206234831363,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.514206234831363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5035652598873412,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.51953125,
+      "kd_loss": 0.6732076569831407,
+      "learning_rate": 3e-06,
+      "loss": 0.9616,
+      "masked_tokens": 88.925,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 88.925
+    },
+    {
+      "avg_mask_ratio": 0.5238390378654003,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.5238390378654003,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.511856697372275,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.8046875,
+      "kd_loss": 0.7122418887767707,
+      "learning_rate": 3e-06,
+      "loss": 1.0805,
+      "masked_tokens": 123.25,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 123.25
+    },
+    {
+      "avg_mask_ratio": 0.5409158666618168,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.5409158666618168,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4557006843067654,
+      "epoch": 0.704,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.681670887517248,
+      "learning_rate": 3e-06,
+      "loss": 0.9853,
+      "masked_tokens": 132.2625,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 132.2625
+    },
+    {
+      "avg_mask_ratio": 0.47903697268920953,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.47903697268920953,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5934374530742617,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 1.265625,
+      "kd_loss": 0.7481572654819957,
+      "learning_rate": 3e-06,
+      "loss": 0.973,
+      "masked_tokens": 114.6375,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 114.6375
+    },
+    {
+      "avg_mask_ratio": 0.5254506973840762,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.5254506973840762,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6267570938103404,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 1.25,
+      "kd_loss": 0.7243690301205706,
+      "learning_rate": 3e-06,
+      "loss": 1.0663,
+      "masked_tokens": 129.7,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 129.7
+    },
+    {
+      "avg_mask_ratio": 0.5038800648180768,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.5038800648180768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37673365626956185,
+      "epoch": 0.768,
+      "grad_norm": 0.1943359375,
+      "kd_loss": 0.7419421046464777,
+      "learning_rate": 3e-06,
+      "loss": 0.8731,
+      "masked_tokens": 118.8375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 118.8375
+    },
+    {
+      "avg_mask_ratio": 0.5092529703164473,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.5092529703164473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5048238590722349,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.1875,
+      "kd_loss": 0.6675537923798401,
+      "learning_rate": 3e-06,
+      "loss": 0.8751,
+      "masked_tokens": 130.4375,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 130.4375
+    },
+    {
+      "avg_mask_ratio": 0.47521690553985535,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.47521690553985535,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3009114894711729,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.96875,
+      "kd_loss": 0.5576462655375508,
+      "learning_rate": 3e-06,
+      "loss": 0.7398,
+      "masked_tokens": 99.7125,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 99.7125
+    },
+    {
+      "avg_mask_ratio": 0.5089340912294574,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.5089340912294574,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42893290086574326,
+      "epoch": 0.832,
+      "grad_norm": 0.345703125,
+      "kd_loss": 0.6829758724064732,
+      "learning_rate": 3e-06,
+      "loss": 0.9627,
+      "masked_tokens": 115.925,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 115.925
+    },
+    {
+      "avg_mask_ratio": 0.5041010878514498,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.5041010878514498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5260561651751459,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 1.484375,
+      "kd_loss": 0.7314142643579544,
+      "learning_rate": 3e-06,
+      "loss": 1.0058,
+      "masked_tokens": 124.875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 124.875
+    },
+    {
+      "avg_mask_ratio": 0.5127229066158179,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.5127229066158179,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7525517523653207,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.7839959781571068,
+      "learning_rate": 3e-06,
+      "loss": 1.0779,
+      "masked_tokens": 123.575,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 123.575
+    },
+    {
+      "avg_mask_ratio": 0.47257317856419834,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.47257317856419834,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.27125484775717723,
+      "epoch": 0.896,
+      "grad_norm": 0.4609375,
+      "kd_loss": 0.5007113358164134,
+      "learning_rate": 3e-06,
+      "loss": 0.6359,
+      "masked_tokens": 90.175,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 90.175
+    },
+    {
+      "avg_mask_ratio": 0.49090774822980165,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.49090774822980165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4832683085802046,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.7578125,
+      "kd_loss": 0.579282811923622,
+      "learning_rate": 3e-06,
+      "loss": 0.8552,
+      "masked_tokens": 108.85,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4731982925441116,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.4731982925441116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5204018854022934,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.6328125,
+      "kd_loss": 0.7500040688079025,
+      "learning_rate": 3e-06,
+      "loss": 0.9602,
+      "masked_tokens": 111.5125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 111.5125
+    },
+    {
+      "avg_mask_ratio": 0.4575169428717345,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4575169428717345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.406052071983504,
+      "epoch": 0.96,
+      "grad_norm": 0.54296875,
+      "kd_loss": 0.6662414132848653,
+      "learning_rate": 3e-06,
+      "loss": 0.8269,
+      "masked_tokens": 107.8875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 107.8875
+    },
+    {
+      "avg_mask_ratio": 0.5099512930959463,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.5099512930959463,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37280656259052875,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.1357421875,
+      "kd_loss": 0.5824788354082557,
+      "learning_rate": 3e-06,
+      "loss": 0.8147,
+      "masked_tokens": 109.275,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 109.275
+    },
+    {
+      "avg_mask_ratio": 0.5108432768334058,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.5108432768334058,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4123959210827912,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.7391241128261429,
+      "learning_rate": 3e-06,
+      "loss": 0.9907,
+      "masked_tokens": 111.75,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 111.75
+    },
+    {
+      "avg_mask_ratio": 0.49879020540975033,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.49879020540975033,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4003649194146419,
+      "epoch": 1.0256,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.7427737473112416,
+      "learning_rate": 3e-06,
+      "loss": 0.9109,
+      "masked_tokens": 121.6875,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 121.6875
+    },
+    {
+      "avg_mask_ratio": 0.48607371354009954,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.48607371354009954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44578953055922454,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.7265625,
+      "kd_loss": 0.616808931784999,
+      "learning_rate": 3e-06,
+      "loss": 0.8475,
+      "masked_tokens": 102.1625,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 102.1625
+    },
+    {
+      "avg_mask_ratio": 0.5385718538891524,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5385718538891524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4503419364663387,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 2.3125,
+      "kd_loss": 0.7079652768573397,
+      "learning_rate": 3e-06,
+      "loss": 0.9016,
+      "masked_tokens": 127.6125,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.6125
+    },
+    {
+      "avg_mask_ratio": 0.5356179510476068,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5356179510476068,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5114419424304287,
+      "epoch": 1.0896,
+      "grad_norm": 2.421875,
+      "kd_loss": 0.8057545678497263,
+      "learning_rate": 3e-06,
+      "loss": 0.9692,
+      "masked_tokens": 136.725,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 136.725
+    },
+    {
+      "avg_mask_ratio": 0.4930020817089826,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4930020817089826,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45815266029704843,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.73046875,
+      "kd_loss": 0.6372618476128536,
+      "learning_rate": 3e-06,
+      "loss": 0.8813,
+      "masked_tokens": 91.975,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 91.975
+    },
+    {
+      "avg_mask_ratio": 0.4998604157241061,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4998604157241061,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5302904470116345,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.793973692091648,
+      "learning_rate": 3e-06,
+      "loss": 1.0187,
+      "masked_tokens": 103.4125,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 103.4125
+    },
+    {
+      "avg_mask_ratio": 0.5236943962518126,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.5236943962518126,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5931689006157967,
+      "epoch": 1.1536,
+      "grad_norm": 0.52734375,
+      "kd_loss": 0.6770309421569891,
+      "learning_rate": 3e-06,
+      "loss": 0.9785,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.4774137590778992,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4774137590778992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33368192758867965,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.552957196540001,
+      "learning_rate": 3e-06,
+      "loss": 0.7195,
+      "masked_tokens": 101.425,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 101.425
+    },
+    {
+      "avg_mask_ratio": 0.41173738130601123,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.41173738130601123,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37927565402829944,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.6514939159315418,
+      "learning_rate": 3e-06,
+      "loss": 0.8086,
+      "masked_tokens": 102.0375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 102.0375
+    },
+    {
+      "avg_mask_ratio": 0.4797614786075428,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.4797614786075428,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38391759186294505,
+      "epoch": 1.2176,
+      "grad_norm": 0.6875,
+      "kd_loss": 0.5477132451650505,
+      "learning_rate": 3e-06,
+      "loss": 0.7759,
+      "masked_tokens": 112.25,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 112.25
+    },
+    {
+      "avg_mask_ratio": 0.4974610014585778,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4974610014585778,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4613554118723869,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.8125,
+      "kd_loss": 0.7624371517440522,
+      "learning_rate": 3e-06,
+      "loss": 0.8935,
+      "masked_tokens": 134.025,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 134.025
+    },
+    {
+      "avg_mask_ratio": 0.5565119812032208,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5565119812032208,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45800332796356996,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.84765625,
+      "kd_loss": 0.7569947588504874,
+      "learning_rate": 3e-06,
+      "loss": 0.9895,
+      "masked_tokens": 126.1375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 126.1375
+    },
+    {
+      "avg_mask_ratio": 0.4784870075061917,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.4784870075061917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42934841778910593,
+      "epoch": 1.2816,
+      "grad_norm": 0.8203125,
+      "kd_loss": 0.6223680850909716,
+      "learning_rate": 3e-06,
+      "loss": 0.8811,
+      "masked_tokens": 113.875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 113.875
+    },
+    {
+      "avg_mask_ratio": 0.5459770569577813,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5459770569577813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46207297699419314,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.193359375,
+      "kd_loss": 0.7765240747485223,
+      "learning_rate": 3e-06,
+      "loss": 1.0488,
+      "masked_tokens": 115.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 115.85
+    },
+    {
+      "avg_mask_ratio": 0.5268841385375709,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.5268841385375709,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5177706574260241,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.42578125,
+      "kd_loss": 0.7318520774833814,
+      "learning_rate": 3e-06,
+      "loss": 1.029,
+      "masked_tokens": 114.6625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 114.6625
+    },
+    {
+      "avg_mask_ratio": 0.5629246362368576,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5629246362368576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5840083432511903,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.140625,
+      "kd_loss": 0.7630814963876361,
+      "learning_rate": 3e-06,
+      "loss": 1.2155,
+      "masked_tokens": 130.675,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.675
+    },
+    {
+      "avg_mask_ratio": 0.5119291188195347,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.5119291188195347,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3960001973351496,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.431640625,
+      "kd_loss": 0.5936955793389871,
+      "learning_rate": 3e-06,
+      "loss": 0.7667,
+      "masked_tokens": 125.9,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 125.9
+    },
+    {
+      "avg_mask_ratio": 0.5539714884362184,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5539714884362184,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6942642153400925,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.625,
+      "kd_loss": 0.8088986897017776,
+      "learning_rate": 3e-06,
+      "loss": 1.1162,
+      "masked_tokens": 131.7625,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 131.7625
+    },
+    {
+      "avg_mask_ratio": 0.5141558598377742,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.5141558598377742,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43362003258180265,
+      "epoch": 1.4096,
+      "grad_norm": 1.1875,
+      "kd_loss": 0.6850645583552251,
+      "learning_rate": 3e-06,
+      "loss": 0.9688,
+      "masked_tokens": 133.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 133.35
+    },
+    {
+      "avg_mask_ratio": 0.4895282822311856,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.4895282822311856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40907790855981147,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 1.1953125,
+      "kd_loss": 0.5458992913151632,
+      "learning_rate": 3e-06,
+      "loss": 0.7579,
+      "masked_tokens": 123.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 123.125
+    },
+    {
+      "avg_mask_ratio": 0.4951617428450845,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4951617428450845,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4818705332485024,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.45703125,
+      "kd_loss": 0.6341485966125283,
+      "learning_rate": 3e-06,
+      "loss": 0.8653,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.5100495176156983,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.5100495176156983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5174321293302568,
+      "epoch": 1.4736,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.692481756274401,
+      "learning_rate": 3e-06,
+      "loss": 0.8852,
+      "masked_tokens": 115.8875,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 115.8875
+    },
+    {
+      "avg_mask_ratio": 0.5639110118616373,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5639110118616373,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4771150118128048,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.46484375,
+      "kd_loss": 0.9550692923163296,
+      "learning_rate": 3e-06,
+      "loss": 1.1428,
+      "masked_tokens": 138.2,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 138.2
+    },
+    {
+      "avg_mask_ratio": 0.5292218026472255,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.5292218026472255,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.36868086418378426,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.283203125,
+      "kd_loss": 0.6208373298897982,
+      "learning_rate": 3e-06,
+      "loss": 0.782,
+      "masked_tokens": 113.375,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 113.375
+    },
+    {
+      "avg_mask_ratio": 0.5125403102487326,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.5125403102487326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5343423100649431,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 1.1796875,
+      "kd_loss": 0.8048794609300558,
+      "learning_rate": 3e-06,
+      "loss": 1.0332,
+      "masked_tokens": 115.675,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 115.675
+    },
+    {
+      "avg_mask_ratio": 0.48811948703369124,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.48811948703369124,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5609710454625769,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.6331229786760091,
+      "learning_rate": 3e-06,
+      "loss": 0.906,
+      "masked_tokens": 106.65,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.5547609420493245,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5547609420493245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6117596920593087,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7415648018846696,
+      "learning_rate": 3e-06,
+      "loss": 1.0796,
+      "masked_tokens": 100.775,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 100.775
+    },
+    {
+      "avg_mask_ratio": 0.44697874613921157,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.44697874613921157,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4524557980651025,
+      "epoch": 1.6016,
+      "grad_norm": 0.8125,
+      "kd_loss": 0.7427308069233959,
+      "learning_rate": 3e-06,
+      "loss": 0.8892,
+      "masked_tokens": 99.5,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 99.5
+    },
+    {
+      "avg_mask_ratio": 0.49905171967693607,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.49905171967693607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5331612331144584,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.869146594414876,
+      "learning_rate": 3e-06,
+      "loss": 1.024,
+      "masked_tokens": 111.525,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 111.525
+    },
+    {
+      "avg_mask_ratio": 0.4734679562970996,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4734679562970996,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3263147723814427,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.78515625,
+      "kd_loss": 0.5851220395212977,
+      "learning_rate": 3e-06,
+      "loss": 0.6745,
+      "masked_tokens": 124.1625,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 124.1625
+    },
+    {
+      "avg_mask_ratio": 0.45589545626135075,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.45589545626135075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40714966794125756,
+      "epoch": 1.6656,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.5390054064211469,
+      "learning_rate": 3e-06,
+      "loss": 0.6662,
+      "masked_tokens": 120.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 120.2
+    },
+    {
+      "avg_mask_ratio": 0.4975356309209019,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4975356309209019,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3924752975523688,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.125,
+      "kd_loss": 0.5501754963899657,
+      "learning_rate": 3e-06,
+      "loss": 0.7353,
+      "masked_tokens": 107.35,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 107.35
+    },
+    {
+      "avg_mask_ratio": 0.4948011673986912,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.4948011673986912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.32645424191614436,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 1.125,
+      "kd_loss": 0.5133585692653468,
+      "learning_rate": 3e-06,
+      "loss": 0.7248,
+      "masked_tokens": 109.1875,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 109.1875
+    },
+    {
+      "avg_mask_ratio": 0.5321399106411263,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5321399106411263,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5308010719869571,
+      "epoch": 1.7296,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.6684280144268996,
+      "learning_rate": 3e-06,
+      "loss": 0.9887,
+      "masked_tokens": 123.0375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 123.0375
+    },
+    {
+      "avg_mask_ratio": 0.5357416228158399,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5357416228158399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5018791363456672,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.8031356525036244,
+      "learning_rate": 3e-06,
+      "loss": 1.0867,
+      "masked_tokens": 121.5625,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 121.5625
+    },
+    {
+      "avg_mask_ratio": 0.5232944375369698,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.5232944375369698,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4925852923925049,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.9140625,
+      "kd_loss": 0.796732622892182,
+      "learning_rate": 3e-06,
+      "loss": 1.0096,
+      "masked_tokens": 138.675,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 138.675
+    },
+    {
+      "avg_mask_ratio": 0.5344608084415086,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5344608084415086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39800737908125255,
+      "epoch": 1.7936,
+      "grad_norm": 0.2041015625,
+      "kd_loss": 0.6124706103168371,
+      "learning_rate": 3e-06,
+      "loss": 0.806,
+      "masked_tokens": 119.225,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 119.225
+    },
+    {
+      "avg_mask_ratio": 0.5352845921181142,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5352845921181142,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.398157004356932,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.5720241727458018,
+      "learning_rate": 3e-06,
+      "loss": 0.7641,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4979630701942369,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4979630701942369,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4882415359192748,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.75,
+      "kd_loss": 0.6685384457871464,
+      "learning_rate": 3e-06,
+      "loss": 0.8746,
+      "masked_tokens": 111.275,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 111.275
+    },
+    {
+      "avg_mask_ratio": 0.5208624298567883,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.5208624298567883,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3722814081119509,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.90234375,
+      "kd_loss": 0.6259868353982938,
+      "learning_rate": 3e-06,
+      "loss": 0.931,
+      "masked_tokens": 119.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 119.8875
+    },
+    {
+      "avg_mask_ratio": 0.4884064760175534,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.4884064760175534,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3424368775197877,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.9453125,
+      "kd_loss": 0.7612382256288385,
+      "learning_rate": 3e-06,
+      "loss": 0.9297,
+      "masked_tokens": 97.0,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 97.0
+    },
+    {
+      "avg_mask_ratio": 0.47601241993543225,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.47601241993543225,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.29439299476740644,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.279296875,
+      "kd_loss": 0.5778021513873682,
+      "learning_rate": 3e-06,
+      "loss": 0.6641,
+      "masked_tokens": 107.1375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 107.1375
+    },
+    {
+      "avg_mask_ratio": 0.4820589871611446,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4820589871611446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4219410733627285,
+      "epoch": 1.9216,
+      "grad_norm": 0.51953125,
+      "kd_loss": 0.6610599886284263,
+      "learning_rate": 3e-06,
+      "loss": 0.8875,
+      "masked_tokens": 106.675,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 106.675
+    },
+    {
+      "avg_mask_ratio": 0.5009820312960074,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.5009820312960074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4484953544199811,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.423828125,
+      "kd_loss": 0.6393605162026915,
+      "learning_rate": 3e-06,
+      "loss": 0.8387,
+      "masked_tokens": 129.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 129.1625
+    },
+    {
+      "avg_mask_ratio": 0.4952817424898967,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4952817424898967,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4003621553950552,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 1.28125,
+      "kd_loss": 0.7165493822423741,
+      "learning_rate": 3e-06,
+      "loss": 0.87,
+      "masked_tokens": 114.2625,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 114.2625
+    },
+    {
+      "avg_mask_ratio": 0.5025755434762686,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.5025755434762686,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44728768827070553,
+      "epoch": 1.9856,
+      "grad_norm": 0.380859375,
+      "kd_loss": 0.5998596335327193,
+      "learning_rate": 3e-06,
+      "loss": 0.769,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5328231096001608,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5328231096001608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34552624383357916,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.6015625,
+      "kd_loss": 0.6963629411807882,
+      "learning_rate": 3e-06,
+      "loss": 0.9754,
+      "masked_tokens": 126.51190476190476,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 126.51190476190476
+    },
+    {
+      "avg_mask_ratio": 0.46634063599049114,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.46634063599049114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3470186025385317,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.6648836274001951,
+      "learning_rate": 3e-06,
+      "loss": 0.8308,
+      "masked_tokens": 111.15,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 111.15
+    },
+    {
+      "avg_mask_ratio": 0.5202614731155336,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.5202614731155336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39255777003332354,
+      "epoch": 2.0512,
+      "grad_norm": 0.9921875,
+      "kd_loss": 0.8403500239691084,
+      "learning_rate": 3e-06,
+      "loss": 1.0217,
+      "masked_tokens": 146.275,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 146.275
+    },
+    {
+      "avg_mask_ratio": 0.4732307325524744,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4732307325524744,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4136752946935417,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 1.6015625,
+      "kd_loss": 0.5665071224577105,
+      "learning_rate": 3e-06,
+      "loss": 0.7771,
+      "masked_tokens": 111.6375,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 111.6375
+    },
+    {
+      "avg_mask_ratio": 0.45015103057958183,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.45015103057958183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24581733565152036,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.515625,
+      "kd_loss": 0.5551535871100896,
+      "learning_rate": 3e-06,
+      "loss": 0.697,
+      "masked_tokens": 102.2875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 102.2875
+    },
+    {
+      "avg_mask_ratio": 0.48529006402241065,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.48529006402241065,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2608388370660464,
+      "epoch": 2.1152,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.5421745884898016,
+      "learning_rate": 3e-06,
+      "loss": 0.6504,
+      "masked_tokens": 123.075,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 123.075
+    },
+    {
+      "avg_mask_ratio": 0.4696127205621451,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4696127205621451,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3479914085991481,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.5546875,
+      "kd_loss": 0.5542144843038216,
+      "learning_rate": 3e-06,
+      "loss": 0.7371,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 103.0875
+    },
+    {
+      "avg_mask_ratio": 0.46368037317879496,
+      "avg_response_length": 213.175,
+      "avg_student_mask_ratio": 0.46368037317879496,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3293848424991552,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.2275390625,
+      "kd_loss": 0.5148144457874537,
+      "learning_rate": 3e-06,
+      "loss": 0.7009,
+      "masked_tokens": 93.1375,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 93.1375
+    },
+    {
+      "avg_mask_ratio": 0.5379365492146462,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5379365492146462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46470798885368936,
+      "epoch": 2.1792,
+      "grad_norm": 0.6484375,
+      "kd_loss": 0.7375291446910068,
+      "learning_rate": 3e-06,
+      "loss": 0.9784,
+      "masked_tokens": 114.975,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 114.975
+    },
+    {
+      "avg_mask_ratio": 0.5036081655998714,
+      "avg_response_length": 219.175,
+      "avg_student_mask_ratio": 0.5036081655998714,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45209096000687055,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 1.5,
+      "kd_loss": 0.7249787944556466,
+      "learning_rate": 3e-06,
+      "loss": 0.9227,
+      "masked_tokens": 109.9125,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 109.9125
+    },
+    {
+      "avg_mask_ratio": 0.529415801318828,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.529415801318828,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3901031159069248,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.58984375,
+      "kd_loss": 0.6996272807611149,
+      "learning_rate": 3e-06,
+      "loss": 0.8817,
+      "masked_tokens": 104.3125,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 104.3125
+    },
+    {
+      "avg_mask_ratio": 0.5512922222726047,
+      "avg_response_length": 237.875,
+      "avg_student_mask_ratio": 0.5512922222726047,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41538310063565403,
+      "epoch": 2.2432,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.7574320874500586,
+      "learning_rate": 3e-06,
+      "loss": 0.903,
+      "masked_tokens": 127.425,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 127.425
+    },
+    {
+      "avg_mask_ratio": 0.480971388152102,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.480971388152102,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3390387184460337,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.546875,
+      "kd_loss": 0.568807559109473,
+      "learning_rate": 3e-06,
+      "loss": 0.7216,
+      "masked_tokens": 125.65,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 125.65
+    },
+    {
+      "avg_mask_ratio": 0.5030692228931002,
+      "avg_response_length": 253.8375,
+      "avg_student_mask_ratio": 0.5030692228931002,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38280749060686503,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.5519341900614336,
+      "learning_rate": 3e-06,
+      "loss": 0.8228,
+      "masked_tokens": 139.2,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 139.2
+    },
+    {
+      "avg_mask_ratio": 0.4997857674607076,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4997857674607076,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.25629440465650644,
+      "epoch": 2.3072,
+      "grad_norm": 0.140625,
+      "kd_loss": 0.5524894027101936,
+      "learning_rate": 3e-06,
+      "loss": 0.7452,
+      "masked_tokens": 102.5125,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 102.5125
+    },
+    {
+      "avg_mask_ratio": 0.4668914210633375,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.4668914210633375,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28395035568828464,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.373046875,
+      "kd_loss": 0.5042674671490367,
+      "learning_rate": 3e-06,
+      "loss": 0.6619,
+      "masked_tokens": 94.85,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 94.85
+    },
+    {
+      "avg_mask_ratio": 0.561556038632989,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.561556038632989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.556549127957021,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.58203125,
+      "kd_loss": 0.6619636991126754,
+      "learning_rate": 3e-06,
+      "loss": 1.0218,
+      "masked_tokens": 139.1375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 139.1375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7d6e2353a4a011ee9d5626d3ff19f8d774bfd347bf01c9b971a0eaf1aee9d41
+size 7608

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a28623fd9d8542344478ea36feb36512410e50d7b8da9c80950898647a7bf00
+size 2406624648

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc185531a4d44da6e4fc57740d3d3954ba1f6ac569df6e2c2e0091689b44ebf6
+size 671304442

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbbd0e8c1bfbd7ba8c634ca07b1d8702578d8a5068f2536ae69c20a51bf959b7
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76f6abf5ed464ad05ce07fc3eaa3005f1e7bc064355635524d65b9082829c58d
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f8c95a6d9085dfcee1e6620c88ede526366d3a02c5018932b1bc04809c0e0c7
+size 1064

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/trainer_state.json

@@ -0,0 +1,2793 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.5632,
+  "eval_steps": 500,
+  "global_step": 1200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7686590318728121,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7938352275807403,
+      "learning_rate": 3e-06,
+      "loss": 1.1776,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5329017310026074,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.0625,
+      "kd_loss": 0.6616400777624165,
+      "learning_rate": 3e-06,
+      "loss": 0.881,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5311922653714646,
+      "epoch": 0.064,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.7425047810846863,
+      "learning_rate": 3e-06,
+      "loss": 0.8917,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46332676129907213,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.828125,
+      "kd_loss": 0.5697841891103053,
+      "learning_rate": 3e-06,
+      "loss": 0.7561,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6159080647946553,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.4999190036984601,
+      "learning_rate": 3e-06,
+      "loss": 0.8703,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6686879911097549,
+      "epoch": 0.128,
+      "grad_norm": 0.66796875,
+      "kd_loss": 0.6171865205901298,
+      "learning_rate": 3e-06,
+      "loss": 1.0246,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913988395589967,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 2.53125,
+      "kd_loss": 0.6814423620926391,
+      "learning_rate": 3e-06,
+      "loss": 1.1566,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5689800441559327,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.33984375,
+      "kd_loss": 0.49904182197715274,
+      "learning_rate": 3e-06,
+      "loss": 0.7152,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9389124085606454,
+      "epoch": 0.192,
+      "grad_norm": 0.6796875,
+      "kd_loss": 0.8731421671935546,
+      "learning_rate": 3e-06,
+      "loss": 1.2336,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5309442258150738,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.71875,
+      "kd_loss": 0.6812383219526226,
+      "learning_rate": 3e-06,
+      "loss": 1.035,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    },
+    {
+      "avg_mask_ratio": 0.5232905174256303,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.5232905174256303,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.54909596200132,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 1.1875,
+      "kd_loss": 0.7053092580745443,
+      "learning_rate": 3e-06,
+      "loss": 1.0537,
+      "masked_tokens": 106.4375,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 106.4375
+    },
+    {
+      "avg_mask_ratio": 0.4815562474541366,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4815562474541366,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5007095821860276,
+      "epoch": 0.256,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.6486993154838274,
+      "learning_rate": 3e-06,
+      "loss": 0.9178,
+      "masked_tokens": 102.2,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 102.2
+    },
+    {
+      "avg_mask_ratio": 0.47414465841138737,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.47414465841138737,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.47370488773524355,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.4394873990022063,
+      "learning_rate": 3e-06,
+      "loss": 0.6913,
+      "masked_tokens": 94.7875,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 94.7875
+    },
+    {
+      "avg_mask_ratio": 0.523321858420968,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.523321858420968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9210947082143207,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 1.4375,
+      "kd_loss": 0.8009161149782358,
+      "learning_rate": 3e-06,
+      "loss": 1.2177,
+      "masked_tokens": 135.4,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 135.4
+    },
+    {
+      "avg_mask_ratio": 0.4975809322553687,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4975809322553687,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6427831699944363,
+      "epoch": 0.32,
+      "grad_norm": 0.1591796875,
+      "kd_loss": 0.6877767233976414,
+      "learning_rate": 3e-06,
+      "loss": 1.0534,
+      "masked_tokens": 129.925,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 129.925
+    },
+    {
+      "avg_mask_ratio": 0.5385947977076284,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5385947977076284,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9350756403351624,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.86328125,
+      "kd_loss": 0.7191342834705636,
+      "learning_rate": 3e-06,
+      "loss": 1.0812,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.5177937666652724,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.5177937666652724,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7097591370865757,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.83984375,
+      "kd_loss": 0.698952960721499,
+      "learning_rate": 3e-06,
+      "loss": 1.0164,
+      "masked_tokens": 93.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 93.225
+    },
+    {
+      "avg_mask_ratio": 0.4772969324782025,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.4772969324782025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6852161650983248,
+      "epoch": 0.384,
+      "grad_norm": 0.90234375,
+      "kd_loss": 0.6290313734622742,
+      "learning_rate": 3e-06,
+      "loss": 0.9438,
+      "masked_tokens": 108.8375,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 108.8375
+    },
+    {
+      "avg_mask_ratio": 0.5173690344206989,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.5173690344206989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6193340622402956,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.6558441586004846,
+      "learning_rate": 3e-06,
+      "loss": 0.959,
+      "masked_tokens": 114.0625,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 114.0625
+    },
+    {
+      "avg_mask_ratio": 0.49981915440876035,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.49981915440876035,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5121921868557535,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.40625,
+      "kd_loss": 0.5239893537682292,
+      "learning_rate": 3e-06,
+      "loss": 0.7682,
+      "masked_tokens": 101.175,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.484982778178528,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.484982778178528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.487391425676833,
+      "epoch": 0.448,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.5454409581767322,
+      "learning_rate": 3e-06,
+      "loss": 0.7901,
+      "masked_tokens": 101.2,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.5744095016038046,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5744095016038046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7552260112240674,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 1.1015625,
+      "kd_loss": 0.7122204179657274,
+      "learning_rate": 3e-06,
+      "loss": 1.1559,
+      "masked_tokens": 129.4,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 129.4
+    },
+    {
+      "avg_mask_ratio": 0.4629370831884444,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4629370831884444,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3121085743832509,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.81640625,
+      "kd_loss": 0.5816853262786026,
+      "learning_rate": 3e-06,
+      "loss": 0.8351,
+      "masked_tokens": 110.5125,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 110.5125
+    },
+    {
+      "avg_mask_ratio": 0.499816512214602,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.499816512214602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45488183712676006,
+      "epoch": 0.512,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.621679913950502,
+      "learning_rate": 3e-06,
+      "loss": 0.9475,
+      "masked_tokens": 110.075,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.075
+    },
+    {
+      "avg_mask_ratio": 0.4744578254292719,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4744578254292719,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40257708527117303,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.181640625,
+      "kd_loss": 0.5342776940590724,
+      "learning_rate": 3e-06,
+      "loss": 0.7566,
+      "masked_tokens": 117.1125,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 117.1125
+    },
+    {
+      "avg_mask_ratio": 0.4852474880579393,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.4852474880579393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3536207471866419,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.73046875,
+      "kd_loss": 0.5287767746266354,
+      "learning_rate": 3e-06,
+      "loss": 0.6987,
+      "masked_tokens": 113.725,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 113.725
+    },
+    {
+      "avg_mask_ratio": 0.565397203550674,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.565397203550674,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6109200767528364,
+      "epoch": 0.576,
+      "grad_norm": 1.0703125,
+      "kd_loss": 0.825409283463523,
+      "learning_rate": 3e-06,
+      "loss": 1.197,
+      "masked_tokens": 124.7125,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 124.7125
+    },
+    {
+      "avg_mask_ratio": 0.4814051762456074,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4814051762456074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48712714778754956,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.5236038082813025,
+      "learning_rate": 3e-06,
+      "loss": 0.8092,
+      "masked_tokens": 129.975,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 129.975
+    },
+    {
+      "avg_mask_ratio": 0.4164489531540312,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.4164489531540312,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1603455784160815,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.0849609375,
+      "kd_loss": 0.459246705797786,
+      "learning_rate": 3e-06,
+      "loss": 0.5704,
+      "masked_tokens": 100.625,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.625
+    },
+    {
+      "avg_mask_ratio": 0.47973727830685675,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.47973727830685675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4429863921482934,
+      "epoch": 0.64,
+      "grad_norm": 0.9609375,
+      "kd_loss": 0.6260649390371327,
+      "learning_rate": 3e-06,
+      "loss": 0.847,
+      "masked_tokens": 107.2375,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 107.2375
+    },
+    {
+      "avg_mask_ratio": 0.514206234831363,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.514206234831363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5035652598873412,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.51953125,
+      "kd_loss": 0.6732076569831407,
+      "learning_rate": 3e-06,
+      "loss": 0.9616,
+      "masked_tokens": 88.925,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 88.925
+    },
+    {
+      "avg_mask_ratio": 0.5238390378654003,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.5238390378654003,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.511856697372275,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.8046875,
+      "kd_loss": 0.7122418887767707,
+      "learning_rate": 3e-06,
+      "loss": 1.0805,
+      "masked_tokens": 123.25,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 123.25
+    },
+    {
+      "avg_mask_ratio": 0.5409158666618168,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.5409158666618168,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4557006843067654,
+      "epoch": 0.704,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.681670887517248,
+      "learning_rate": 3e-06,
+      "loss": 0.9853,
+      "masked_tokens": 132.2625,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 132.2625
+    },
+    {
+      "avg_mask_ratio": 0.47903697268920953,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.47903697268920953,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5934374530742617,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 1.265625,
+      "kd_loss": 0.7481572654819957,
+      "learning_rate": 3e-06,
+      "loss": 0.973,
+      "masked_tokens": 114.6375,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 114.6375
+    },
+    {
+      "avg_mask_ratio": 0.5254506973840762,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.5254506973840762,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6267570938103404,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 1.25,
+      "kd_loss": 0.7243690301205706,
+      "learning_rate": 3e-06,
+      "loss": 1.0663,
+      "masked_tokens": 129.7,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 129.7
+    },
+    {
+      "avg_mask_ratio": 0.5038800648180768,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.5038800648180768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37673365626956185,
+      "epoch": 0.768,
+      "grad_norm": 0.1943359375,
+      "kd_loss": 0.7419421046464777,
+      "learning_rate": 3e-06,
+      "loss": 0.8731,
+      "masked_tokens": 118.8375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 118.8375
+    },
+    {
+      "avg_mask_ratio": 0.5092529703164473,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.5092529703164473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5048238590722349,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.1875,
+      "kd_loss": 0.6675537923798401,
+      "learning_rate": 3e-06,
+      "loss": 0.8751,
+      "masked_tokens": 130.4375,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 130.4375
+    },
+    {
+      "avg_mask_ratio": 0.47521690553985535,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.47521690553985535,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3009114894711729,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.96875,
+      "kd_loss": 0.5576462655375508,
+      "learning_rate": 3e-06,
+      "loss": 0.7398,
+      "masked_tokens": 99.7125,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 99.7125
+    },
+    {
+      "avg_mask_ratio": 0.5089340912294574,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.5089340912294574,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42893290086574326,
+      "epoch": 0.832,
+      "grad_norm": 0.345703125,
+      "kd_loss": 0.6829758724064732,
+      "learning_rate": 3e-06,
+      "loss": 0.9627,
+      "masked_tokens": 115.925,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 115.925
+    },
+    {
+      "avg_mask_ratio": 0.5041010878514498,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.5041010878514498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5260561651751459,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 1.484375,
+      "kd_loss": 0.7314142643579544,
+      "learning_rate": 3e-06,
+      "loss": 1.0058,
+      "masked_tokens": 124.875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 124.875
+    },
+    {
+      "avg_mask_ratio": 0.5127229066158179,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.5127229066158179,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7525517523653207,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.7839959781571068,
+      "learning_rate": 3e-06,
+      "loss": 1.0779,
+      "masked_tokens": 123.575,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 123.575
+    },
+    {
+      "avg_mask_ratio": 0.47257317856419834,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.47257317856419834,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.27125484775717723,
+      "epoch": 0.896,
+      "grad_norm": 0.4609375,
+      "kd_loss": 0.5007113358164134,
+      "learning_rate": 3e-06,
+      "loss": 0.6359,
+      "masked_tokens": 90.175,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 90.175
+    },
+    {
+      "avg_mask_ratio": 0.49090774822980165,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.49090774822980165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4832683085802046,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.7578125,
+      "kd_loss": 0.579282811923622,
+      "learning_rate": 3e-06,
+      "loss": 0.8552,
+      "masked_tokens": 108.85,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4731982925441116,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.4731982925441116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5204018854022934,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.6328125,
+      "kd_loss": 0.7500040688079025,
+      "learning_rate": 3e-06,
+      "loss": 0.9602,
+      "masked_tokens": 111.5125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 111.5125
+    },
+    {
+      "avg_mask_ratio": 0.4575169428717345,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4575169428717345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.406052071983504,
+      "epoch": 0.96,
+      "grad_norm": 0.54296875,
+      "kd_loss": 0.6662414132848653,
+      "learning_rate": 3e-06,
+      "loss": 0.8269,
+      "masked_tokens": 107.8875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 107.8875
+    },
+    {
+      "avg_mask_ratio": 0.5099512930959463,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.5099512930959463,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37280656259052875,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.1357421875,
+      "kd_loss": 0.5824788354082557,
+      "learning_rate": 3e-06,
+      "loss": 0.8147,
+      "masked_tokens": 109.275,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 109.275
+    },
+    {
+      "avg_mask_ratio": 0.5108432768334058,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.5108432768334058,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4123959210827912,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.7391241128261429,
+      "learning_rate": 3e-06,
+      "loss": 0.9907,
+      "masked_tokens": 111.75,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 111.75
+    },
+    {
+      "avg_mask_ratio": 0.49879020540975033,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.49879020540975033,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4003649194146419,
+      "epoch": 1.0256,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.7427737473112416,
+      "learning_rate": 3e-06,
+      "loss": 0.9109,
+      "masked_tokens": 121.6875,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 121.6875
+    },
+    {
+      "avg_mask_ratio": 0.48607371354009954,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.48607371354009954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44578953055922454,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.7265625,
+      "kd_loss": 0.616808931784999,
+      "learning_rate": 3e-06,
+      "loss": 0.8475,
+      "masked_tokens": 102.1625,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 102.1625
+    },
+    {
+      "avg_mask_ratio": 0.5385718538891524,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5385718538891524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4503419364663387,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 2.3125,
+      "kd_loss": 0.7079652768573397,
+      "learning_rate": 3e-06,
+      "loss": 0.9016,
+      "masked_tokens": 127.6125,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.6125
+    },
+    {
+      "avg_mask_ratio": 0.5356179510476068,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5356179510476068,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5114419424304287,
+      "epoch": 1.0896,
+      "grad_norm": 2.421875,
+      "kd_loss": 0.8057545678497263,
+      "learning_rate": 3e-06,
+      "loss": 0.9692,
+      "masked_tokens": 136.725,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 136.725
+    },
+    {
+      "avg_mask_ratio": 0.4930020817089826,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4930020817089826,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45815266029704843,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.73046875,
+      "kd_loss": 0.6372618476128536,
+      "learning_rate": 3e-06,
+      "loss": 0.8813,
+      "masked_tokens": 91.975,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 91.975
+    },
+    {
+      "avg_mask_ratio": 0.4998604157241061,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4998604157241061,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5302904470116345,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.793973692091648,
+      "learning_rate": 3e-06,
+      "loss": 1.0187,
+      "masked_tokens": 103.4125,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 103.4125
+    },
+    {
+      "avg_mask_ratio": 0.5236943962518126,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.5236943962518126,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5931689006157967,
+      "epoch": 1.1536,
+      "grad_norm": 0.52734375,
+      "kd_loss": 0.6770309421569891,
+      "learning_rate": 3e-06,
+      "loss": 0.9785,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.4774137590778992,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4774137590778992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33368192758867965,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.552957196540001,
+      "learning_rate": 3e-06,
+      "loss": 0.7195,
+      "masked_tokens": 101.425,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 101.425
+    },
+    {
+      "avg_mask_ratio": 0.41173738130601123,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.41173738130601123,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37927565402829944,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.6514939159315418,
+      "learning_rate": 3e-06,
+      "loss": 0.8086,
+      "masked_tokens": 102.0375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 102.0375
+    },
+    {
+      "avg_mask_ratio": 0.4797614786075428,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.4797614786075428,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38391759186294505,
+      "epoch": 1.2176,
+      "grad_norm": 0.6875,
+      "kd_loss": 0.5477132451650505,
+      "learning_rate": 3e-06,
+      "loss": 0.7759,
+      "masked_tokens": 112.25,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 112.25
+    },
+    {
+      "avg_mask_ratio": 0.4974610014585778,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4974610014585778,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4613554118723869,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.8125,
+      "kd_loss": 0.7624371517440522,
+      "learning_rate": 3e-06,
+      "loss": 0.8935,
+      "masked_tokens": 134.025,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 134.025
+    },
+    {
+      "avg_mask_ratio": 0.5565119812032208,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5565119812032208,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45800332796356996,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.84765625,
+      "kd_loss": 0.7569947588504874,
+      "learning_rate": 3e-06,
+      "loss": 0.9895,
+      "masked_tokens": 126.1375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 126.1375
+    },
+    {
+      "avg_mask_ratio": 0.4784870075061917,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.4784870075061917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42934841778910593,
+      "epoch": 1.2816,
+      "grad_norm": 0.8203125,
+      "kd_loss": 0.6223680850909716,
+      "learning_rate": 3e-06,
+      "loss": 0.8811,
+      "masked_tokens": 113.875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 113.875
+    },
+    {
+      "avg_mask_ratio": 0.5459770569577813,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5459770569577813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46207297699419314,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.193359375,
+      "kd_loss": 0.7765240747485223,
+      "learning_rate": 3e-06,
+      "loss": 1.0488,
+      "masked_tokens": 115.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 115.85
+    },
+    {
+      "avg_mask_ratio": 0.5268841385375709,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.5268841385375709,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5177706574260241,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.42578125,
+      "kd_loss": 0.7318520774833814,
+      "learning_rate": 3e-06,
+      "loss": 1.029,
+      "masked_tokens": 114.6625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 114.6625
+    },
+    {
+      "avg_mask_ratio": 0.5629246362368576,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5629246362368576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5840083432511903,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.140625,
+      "kd_loss": 0.7630814963876361,
+      "learning_rate": 3e-06,
+      "loss": 1.2155,
+      "masked_tokens": 130.675,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.675
+    },
+    {
+      "avg_mask_ratio": 0.5119291188195347,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.5119291188195347,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3960001973351496,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.431640625,
+      "kd_loss": 0.5936955793389871,
+      "learning_rate": 3e-06,
+      "loss": 0.7667,
+      "masked_tokens": 125.9,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 125.9
+    },
+    {
+      "avg_mask_ratio": 0.5539714884362184,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5539714884362184,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6942642153400925,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.625,
+      "kd_loss": 0.8088986897017776,
+      "learning_rate": 3e-06,
+      "loss": 1.1162,
+      "masked_tokens": 131.7625,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 131.7625
+    },
+    {
+      "avg_mask_ratio": 0.5141558598377742,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.5141558598377742,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43362003258180265,
+      "epoch": 1.4096,
+      "grad_norm": 1.1875,
+      "kd_loss": 0.6850645583552251,
+      "learning_rate": 3e-06,
+      "loss": 0.9688,
+      "masked_tokens": 133.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 133.35
+    },
+    {
+      "avg_mask_ratio": 0.4895282822311856,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.4895282822311856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40907790855981147,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 1.1953125,
+      "kd_loss": 0.5458992913151632,
+      "learning_rate": 3e-06,
+      "loss": 0.7579,
+      "masked_tokens": 123.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 123.125
+    },
+    {
+      "avg_mask_ratio": 0.4951617428450845,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4951617428450845,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4818705332485024,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.45703125,
+      "kd_loss": 0.6341485966125283,
+      "learning_rate": 3e-06,
+      "loss": 0.8653,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.5100495176156983,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.5100495176156983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5174321293302568,
+      "epoch": 1.4736,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.692481756274401,
+      "learning_rate": 3e-06,
+      "loss": 0.8852,
+      "masked_tokens": 115.8875,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 115.8875
+    },
+    {
+      "avg_mask_ratio": 0.5639110118616373,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5639110118616373,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4771150118128048,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.46484375,
+      "kd_loss": 0.9550692923163296,
+      "learning_rate": 3e-06,
+      "loss": 1.1428,
+      "masked_tokens": 138.2,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 138.2
+    },
+    {
+      "avg_mask_ratio": 0.5292218026472255,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.5292218026472255,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.36868086418378426,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.283203125,
+      "kd_loss": 0.6208373298897982,
+      "learning_rate": 3e-06,
+      "loss": 0.782,
+      "masked_tokens": 113.375,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 113.375
+    },
+    {
+      "avg_mask_ratio": 0.5125403102487326,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.5125403102487326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5343423100649431,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 1.1796875,
+      "kd_loss": 0.8048794609300558,
+      "learning_rate": 3e-06,
+      "loss": 1.0332,
+      "masked_tokens": 115.675,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 115.675
+    },
+    {
+      "avg_mask_ratio": 0.48811948703369124,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.48811948703369124,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5609710454625769,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.6331229786760091,
+      "learning_rate": 3e-06,
+      "loss": 0.906,
+      "masked_tokens": 106.65,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.5547609420493245,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5547609420493245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6117596920593087,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7415648018846696,
+      "learning_rate": 3e-06,
+      "loss": 1.0796,
+      "masked_tokens": 100.775,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 100.775
+    },
+    {
+      "avg_mask_ratio": 0.44697874613921157,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.44697874613921157,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4524557980651025,
+      "epoch": 1.6016,
+      "grad_norm": 0.8125,
+      "kd_loss": 0.7427308069233959,
+      "learning_rate": 3e-06,
+      "loss": 0.8892,
+      "masked_tokens": 99.5,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 99.5
+    },
+    {
+      "avg_mask_ratio": 0.49905171967693607,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.49905171967693607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5331612331144584,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.869146594414876,
+      "learning_rate": 3e-06,
+      "loss": 1.024,
+      "masked_tokens": 111.525,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 111.525
+    },
+    {
+      "avg_mask_ratio": 0.4734679562970996,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4734679562970996,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3263147723814427,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.78515625,
+      "kd_loss": 0.5851220395212977,
+      "learning_rate": 3e-06,
+      "loss": 0.6745,
+      "masked_tokens": 124.1625,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 124.1625
+    },
+    {
+      "avg_mask_ratio": 0.45589545626135075,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.45589545626135075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40714966794125756,
+      "epoch": 1.6656,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.5390054064211469,
+      "learning_rate": 3e-06,
+      "loss": 0.6662,
+      "masked_tokens": 120.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 120.2
+    },
+    {
+      "avg_mask_ratio": 0.4975356309209019,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4975356309209019,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3924752975523688,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.125,
+      "kd_loss": 0.5501754963899657,
+      "learning_rate": 3e-06,
+      "loss": 0.7353,
+      "masked_tokens": 107.35,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 107.35
+    },
+    {
+      "avg_mask_ratio": 0.4948011673986912,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.4948011673986912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.32645424191614436,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 1.125,
+      "kd_loss": 0.5133585692653468,
+      "learning_rate": 3e-06,
+      "loss": 0.7248,
+      "masked_tokens": 109.1875,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 109.1875
+    },
+    {
+      "avg_mask_ratio": 0.5321399106411263,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5321399106411263,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5308010719869571,
+      "epoch": 1.7296,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.6684280144268996,
+      "learning_rate": 3e-06,
+      "loss": 0.9887,
+      "masked_tokens": 123.0375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 123.0375
+    },
+    {
+      "avg_mask_ratio": 0.5357416228158399,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5357416228158399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5018791363456672,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.8031356525036244,
+      "learning_rate": 3e-06,
+      "loss": 1.0867,
+      "masked_tokens": 121.5625,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 121.5625
+    },
+    {
+      "avg_mask_ratio": 0.5232944375369698,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.5232944375369698,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4925852923925049,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.9140625,
+      "kd_loss": 0.796732622892182,
+      "learning_rate": 3e-06,
+      "loss": 1.0096,
+      "masked_tokens": 138.675,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 138.675
+    },
+    {
+      "avg_mask_ratio": 0.5344608084415086,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5344608084415086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39800737908125255,
+      "epoch": 1.7936,
+      "grad_norm": 0.2041015625,
+      "kd_loss": 0.6124706103168371,
+      "learning_rate": 3e-06,
+      "loss": 0.806,
+      "masked_tokens": 119.225,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 119.225
+    },
+    {
+      "avg_mask_ratio": 0.5352845921181142,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5352845921181142,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.398157004356932,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.5720241727458018,
+      "learning_rate": 3e-06,
+      "loss": 0.7641,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4979630701942369,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4979630701942369,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4882415359192748,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.75,
+      "kd_loss": 0.6685384457871464,
+      "learning_rate": 3e-06,
+      "loss": 0.8746,
+      "masked_tokens": 111.275,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 111.275
+    },
+    {
+      "avg_mask_ratio": 0.5208624298567883,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.5208624298567883,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3722814081119509,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.90234375,
+      "kd_loss": 0.6259868353982938,
+      "learning_rate": 3e-06,
+      "loss": 0.931,
+      "masked_tokens": 119.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 119.8875
+    },
+    {
+      "avg_mask_ratio": 0.4884064760175534,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.4884064760175534,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3424368775197877,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.9453125,
+      "kd_loss": 0.7612382256288385,
+      "learning_rate": 3e-06,
+      "loss": 0.9297,
+      "masked_tokens": 97.0,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 97.0
+    },
+    {
+      "avg_mask_ratio": 0.47601241993543225,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.47601241993543225,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.29439299476740644,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.279296875,
+      "kd_loss": 0.5778021513873682,
+      "learning_rate": 3e-06,
+      "loss": 0.6641,
+      "masked_tokens": 107.1375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 107.1375
+    },
+    {
+      "avg_mask_ratio": 0.4820589871611446,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4820589871611446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4219410733627285,
+      "epoch": 1.9216,
+      "grad_norm": 0.51953125,
+      "kd_loss": 0.6610599886284263,
+      "learning_rate": 3e-06,
+      "loss": 0.8875,
+      "masked_tokens": 106.675,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 106.675
+    },
+    {
+      "avg_mask_ratio": 0.5009820312960074,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.5009820312960074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4484953544199811,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.423828125,
+      "kd_loss": 0.6393605162026915,
+      "learning_rate": 3e-06,
+      "loss": 0.8387,
+      "masked_tokens": 129.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 129.1625
+    },
+    {
+      "avg_mask_ratio": 0.4952817424898967,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4952817424898967,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4003621553950552,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 1.28125,
+      "kd_loss": 0.7165493822423741,
+      "learning_rate": 3e-06,
+      "loss": 0.87,
+      "masked_tokens": 114.2625,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 114.2625
+    },
+    {
+      "avg_mask_ratio": 0.5025755434762686,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.5025755434762686,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44728768827070553,
+      "epoch": 1.9856,
+      "grad_norm": 0.380859375,
+      "kd_loss": 0.5998596335327193,
+      "learning_rate": 3e-06,
+      "loss": 0.769,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5328231096001608,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5328231096001608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34552624383357916,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.6015625,
+      "kd_loss": 0.6963629411807882,
+      "learning_rate": 3e-06,
+      "loss": 0.9754,
+      "masked_tokens": 126.51190476190476,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 126.51190476190476
+    },
+    {
+      "avg_mask_ratio": 0.46634063599049114,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.46634063599049114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3470186025385317,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.6648836274001951,
+      "learning_rate": 3e-06,
+      "loss": 0.8308,
+      "masked_tokens": 111.15,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 111.15
+    },
+    {
+      "avg_mask_ratio": 0.5202614731155336,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.5202614731155336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39255777003332354,
+      "epoch": 2.0512,
+      "grad_norm": 0.9921875,
+      "kd_loss": 0.8403500239691084,
+      "learning_rate": 3e-06,
+      "loss": 1.0217,
+      "masked_tokens": 146.275,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 146.275
+    },
+    {
+      "avg_mask_ratio": 0.4732307325524744,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4732307325524744,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4136752946935417,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 1.6015625,
+      "kd_loss": 0.5665071224577105,
+      "learning_rate": 3e-06,
+      "loss": 0.7771,
+      "masked_tokens": 111.6375,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 111.6375
+    },
+    {
+      "avg_mask_ratio": 0.45015103057958183,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.45015103057958183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24581733565152036,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.515625,
+      "kd_loss": 0.5551535871100896,
+      "learning_rate": 3e-06,
+      "loss": 0.697,
+      "masked_tokens": 102.2875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 102.2875
+    },
+    {
+      "avg_mask_ratio": 0.48529006402241065,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.48529006402241065,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2608388370660464,
+      "epoch": 2.1152,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.5421745884898016,
+      "learning_rate": 3e-06,
+      "loss": 0.6504,
+      "masked_tokens": 123.075,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 123.075
+    },
+    {
+      "avg_mask_ratio": 0.4696127205621451,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4696127205621451,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3479914085991481,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.5546875,
+      "kd_loss": 0.5542144843038216,
+      "learning_rate": 3e-06,
+      "loss": 0.7371,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 103.0875
+    },
+    {
+      "avg_mask_ratio": 0.46368037317879496,
+      "avg_response_length": 213.175,
+      "avg_student_mask_ratio": 0.46368037317879496,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3293848424991552,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.2275390625,
+      "kd_loss": 0.5148144457874537,
+      "learning_rate": 3e-06,
+      "loss": 0.7009,
+      "masked_tokens": 93.1375,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 93.1375
+    },
+    {
+      "avg_mask_ratio": 0.5379365492146462,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5379365492146462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46470798885368936,
+      "epoch": 2.1792,
+      "grad_norm": 0.6484375,
+      "kd_loss": 0.7375291446910068,
+      "learning_rate": 3e-06,
+      "loss": 0.9784,
+      "masked_tokens": 114.975,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 114.975
+    },
+    {
+      "avg_mask_ratio": 0.5036081655998714,
+      "avg_response_length": 219.175,
+      "avg_student_mask_ratio": 0.5036081655998714,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45209096000687055,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 1.5,
+      "kd_loss": 0.7249787944556466,
+      "learning_rate": 3e-06,
+      "loss": 0.9227,
+      "masked_tokens": 109.9125,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 109.9125
+    },
+    {
+      "avg_mask_ratio": 0.529415801318828,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.529415801318828,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3901031159069248,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.58984375,
+      "kd_loss": 0.6996272807611149,
+      "learning_rate": 3e-06,
+      "loss": 0.8817,
+      "masked_tokens": 104.3125,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 104.3125
+    },
+    {
+      "avg_mask_ratio": 0.5512922222726047,
+      "avg_response_length": 237.875,
+      "avg_student_mask_ratio": 0.5512922222726047,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41538310063565403,
+      "epoch": 2.2432,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.7574320874500586,
+      "learning_rate": 3e-06,
+      "loss": 0.903,
+      "masked_tokens": 127.425,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 127.425
+    },
+    {
+      "avg_mask_ratio": 0.480971388152102,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.480971388152102,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3390387184460337,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.546875,
+      "kd_loss": 0.568807559109473,
+      "learning_rate": 3e-06,
+      "loss": 0.7216,
+      "masked_tokens": 125.65,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 125.65
+    },
+    {
+      "avg_mask_ratio": 0.5030692228931002,
+      "avg_response_length": 253.8375,
+      "avg_student_mask_ratio": 0.5030692228931002,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38280749060686503,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.5519341900614336,
+      "learning_rate": 3e-06,
+      "loss": 0.8228,
+      "masked_tokens": 139.2,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 139.2
+    },
+    {
+      "avg_mask_ratio": 0.4997857674607076,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4997857674607076,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.25629440465650644,
+      "epoch": 2.3072,
+      "grad_norm": 0.140625,
+      "kd_loss": 0.5524894027101936,
+      "learning_rate": 3e-06,
+      "loss": 0.7452,
+      "masked_tokens": 102.5125,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 102.5125
+    },
+    {
+      "avg_mask_ratio": 0.4668914210633375,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.4668914210633375,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28395035568828464,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.373046875,
+      "kd_loss": 0.5042674671490367,
+      "learning_rate": 3e-06,
+      "loss": 0.6619,
+      "masked_tokens": 94.85,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 94.85
+    },
+    {
+      "avg_mask_ratio": 0.561556038632989,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.561556038632989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.556549127957021,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.58203125,
+      "kd_loss": 0.6619636991126754,
+      "learning_rate": 3e-06,
+      "loss": 1.0218,
+      "masked_tokens": 139.1375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 139.1375
+    },
+    {
+      "avg_mask_ratio": 0.47325256096664814,
+      "avg_response_length": 226.6375,
+      "avg_student_mask_ratio": 0.47325256096664814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3334150109870279,
+      "epoch": 2.3712,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.44958578083260364,
+      "learning_rate": 3e-06,
+      "loss": 0.696,
+      "masked_tokens": 107.3,
+      "mean_t": 0.4757364276825683,
+      "step": 1110,
+      "student_masked_tokens": 107.3
+    },
+    {
+      "avg_mask_ratio": 0.5052445781533607,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.5052445781533607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33756577983574515,
+      "epoch": 2.392533333333333,
+      "grad_norm": 2.234375,
+      "kd_loss": 0.47280435794045844,
+      "learning_rate": 3e-06,
+      "loss": 0.7787,
+      "masked_tokens": 117.325,
+      "mean_t": 0.5013068238971755,
+      "step": 1120,
+      "student_masked_tokens": 117.325
+    },
+    {
+      "avg_mask_ratio": 0.5352560582570731,
+      "avg_response_length": 262.25,
+      "avg_student_mask_ratio": 0.5352560582570731,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.49989761952609796,
+      "epoch": 2.413866666666667,
+      "grad_norm": 2.1875,
+      "kd_loss": 0.7383381549463138,
+      "learning_rate": 3e-06,
+      "loss": 0.97,
+      "masked_tokens": 140.5375,
+      "mean_t": 0.5303254407714121,
+      "step": 1130,
+      "student_masked_tokens": 140.5375
+    },
+    {
+      "avg_mask_ratio": 0.4938803721917793,
+      "avg_response_length": 217.05,
+      "avg_student_mask_ratio": 0.4938803721917793,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37043922989309974,
+      "epoch": 2.4352,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.5307485466641083,
+      "learning_rate": 3e-06,
+      "loss": 0.6962,
+      "masked_tokens": 107.6125,
+      "mean_t": 0.4845335395424627,
+      "step": 1140,
+      "student_masked_tokens": 107.6125
+    },
+    {
+      "avg_mask_ratio": 0.5653773612109945,
+      "avg_response_length": 212.5375,
+      "avg_student_mask_ratio": 0.5653773612109945,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7858076869456567,
+      "epoch": 2.4565333333333332,
+      "grad_norm": 1.84375,
+      "kd_loss": 0.7624981299486535,
+      "learning_rate": 3e-06,
+      "loss": 1.1469,
+      "masked_tokens": 114.6,
+      "mean_t": 0.5690932425903157,
+      "step": 1150,
+      "student_masked_tokens": 114.6
+    },
+    {
+      "avg_mask_ratio": 0.49925965811125933,
+      "avg_response_length": 225.6125,
+      "avg_student_mask_ratio": 0.49925965811125933,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46302462185976767,
+      "epoch": 2.4778666666666664,
+      "grad_norm": 3.25,
+      "kd_loss": 0.5881607823695049,
+      "learning_rate": 3e-06,
+      "loss": 0.7393,
+      "masked_tokens": 106.125,
+      "mean_t": 0.5040684466948733,
+      "step": 1160,
+      "student_masked_tokens": 106.125
+    },
+    {
+      "avg_mask_ratio": 0.5130727548676077,
+      "avg_response_length": 247.7625,
+      "avg_student_mask_ratio": 0.5130727548676077,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43388418721337985,
+      "epoch": 2.4992,
+      "grad_norm": 0.38671875,
+      "kd_loss": 0.593448537235713,
+      "learning_rate": 3e-06,
+      "loss": 0.7761,
+      "masked_tokens": 128.05,
+      "mean_t": 0.5114516971167177,
+      "step": 1170,
+      "student_masked_tokens": 128.05
+    },
+    {
+      "avg_mask_ratio": 0.4515186986711342,
+      "avg_response_length": 214.725,
+      "avg_student_mask_ratio": 0.4515186986711342,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24207257057115045,
+      "epoch": 2.5205333333333333,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.446691993195418,
+      "learning_rate": 3e-06,
+      "loss": 0.649,
+      "masked_tokens": 89.6375,
+      "mean_t": 0.4491677140351385,
+      "step": 1180,
+      "student_masked_tokens": 89.6375
+    },
+    {
+      "avg_mask_ratio": 0.5575610842439346,
+      "avg_response_length": 220.3375,
+      "avg_student_mask_ratio": 0.5575610842439346,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5483032605723451,
+      "epoch": 2.5418666666666665,
+      "grad_norm": 0.95703125,
+      "kd_loss": 0.8963939757952517,
+      "learning_rate": 3e-06,
+      "loss": 1.0849,
+      "masked_tokens": 129.425,
+      "mean_t": 0.5590635397238657,
+      "step": 1190,
+      "student_masked_tokens": 129.425
+    },
+    {
+      "avg_mask_ratio": 0.5073940187954576,
+      "avg_response_length": 215.675,
+      "avg_student_mask_ratio": 0.5073940187954576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3367691548969276,
+      "epoch": 2.5632,
+      "grad_norm": 1.4140625,
+      "kd_loss": 0.5662632359816826,
+      "learning_rate": 3e-06,
+      "loss": 0.7514,
+      "masked_tokens": 103.8375,
+      "mean_t": 0.506370971655997,
+      "step": 1200,
+      "student_masked_tokens": 103.8375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7d6e2353a4a011ee9d5626d3ff19f8d774bfd347bf01c9b971a0eaf1aee9d41
+size 7608

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a25da3efe8f0369bd48fda335a128bfbe5a1b35236f9c1b94c5f6250623ffc07
+size 2406624648

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e9ec042208237ec2e7ef1513c8f2f472c14da1919d144e0081f2fbd76828b4e
+size 671304442

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2de0073388a2b514a6f97ca23626cb552641aaab9fcf4308111c0ad94ee7e712
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f34ca8ba860d1ab11666737e7bfb3827b2c85a30d364d9cff93ea785a42427ce
+size 14512

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:368502bb8b2f2d6f452bda7249e88ca57b330ec2ba407ec248613b4300c99d0d
+size 1064

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/trainer_state.json

@@ -0,0 +1,3023 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.776533333333333,
+  "eval_steps": 500,
+  "global_step": 1300,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7686590318728121,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7938352275807403,
+      "learning_rate": 3e-06,
+      "loss": 1.1776,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5329017310026074,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.0625,
+      "kd_loss": 0.6616400777624165,
+      "learning_rate": 3e-06,
+      "loss": 0.881,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5311922653714646,
+      "epoch": 0.064,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.7425047810846863,
+      "learning_rate": 3e-06,
+      "loss": 0.8917,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46332676129907213,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.828125,
+      "kd_loss": 0.5697841891103053,
+      "learning_rate": 3e-06,
+      "loss": 0.7561,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6159080647946553,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.4999190036984601,
+      "learning_rate": 3e-06,
+      "loss": 0.8703,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6686879911097549,
+      "epoch": 0.128,
+      "grad_norm": 0.66796875,
+      "kd_loss": 0.6171865205901298,
+      "learning_rate": 3e-06,
+      "loss": 1.0246,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913988395589967,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 2.53125,
+      "kd_loss": 0.6814423620926391,
+      "learning_rate": 3e-06,
+      "loss": 1.1566,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5689800441559327,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.33984375,
+      "kd_loss": 0.49904182197715274,
+      "learning_rate": 3e-06,
+      "loss": 0.7152,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9389124085606454,
+      "epoch": 0.192,
+      "grad_norm": 0.6796875,
+      "kd_loss": 0.8731421671935546,
+      "learning_rate": 3e-06,
+      "loss": 1.2336,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5309442258150738,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.71875,
+      "kd_loss": 0.6812383219526226,
+      "learning_rate": 3e-06,
+      "loss": 1.035,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    },
+    {
+      "avg_mask_ratio": 0.5232905174256303,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.5232905174256303,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.54909596200132,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 1.1875,
+      "kd_loss": 0.7053092580745443,
+      "learning_rate": 3e-06,
+      "loss": 1.0537,
+      "masked_tokens": 106.4375,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 106.4375
+    },
+    {
+      "avg_mask_ratio": 0.4815562474541366,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4815562474541366,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5007095821860276,
+      "epoch": 0.256,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.6486993154838274,
+      "learning_rate": 3e-06,
+      "loss": 0.9178,
+      "masked_tokens": 102.2,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 102.2
+    },
+    {
+      "avg_mask_ratio": 0.47414465841138737,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.47414465841138737,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.47370488773524355,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.4394873990022063,
+      "learning_rate": 3e-06,
+      "loss": 0.6913,
+      "masked_tokens": 94.7875,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 94.7875
+    },
+    {
+      "avg_mask_ratio": 0.523321858420968,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.523321858420968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9210947082143207,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 1.4375,
+      "kd_loss": 0.8009161149782358,
+      "learning_rate": 3e-06,
+      "loss": 1.2177,
+      "masked_tokens": 135.4,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 135.4
+    },
+    {
+      "avg_mask_ratio": 0.4975809322553687,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4975809322553687,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6427831699944363,
+      "epoch": 0.32,
+      "grad_norm": 0.1591796875,
+      "kd_loss": 0.6877767233976414,
+      "learning_rate": 3e-06,
+      "loss": 1.0534,
+      "masked_tokens": 129.925,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 129.925
+    },
+    {
+      "avg_mask_ratio": 0.5385947977076284,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5385947977076284,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9350756403351624,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.86328125,
+      "kd_loss": 0.7191342834705636,
+      "learning_rate": 3e-06,
+      "loss": 1.0812,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.5177937666652724,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.5177937666652724,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7097591370865757,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.83984375,
+      "kd_loss": 0.698952960721499,
+      "learning_rate": 3e-06,
+      "loss": 1.0164,
+      "masked_tokens": 93.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 93.225
+    },
+    {
+      "avg_mask_ratio": 0.4772969324782025,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.4772969324782025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6852161650983248,
+      "epoch": 0.384,
+      "grad_norm": 0.90234375,
+      "kd_loss": 0.6290313734622742,
+      "learning_rate": 3e-06,
+      "loss": 0.9438,
+      "masked_tokens": 108.8375,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 108.8375
+    },
+    {
+      "avg_mask_ratio": 0.5173690344206989,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.5173690344206989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6193340622402956,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.6558441586004846,
+      "learning_rate": 3e-06,
+      "loss": 0.959,
+      "masked_tokens": 114.0625,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 114.0625
+    },
+    {
+      "avg_mask_ratio": 0.49981915440876035,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.49981915440876035,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5121921868557535,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.40625,
+      "kd_loss": 0.5239893537682292,
+      "learning_rate": 3e-06,
+      "loss": 0.7682,
+      "masked_tokens": 101.175,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.484982778178528,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.484982778178528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.487391425676833,
+      "epoch": 0.448,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.5454409581767322,
+      "learning_rate": 3e-06,
+      "loss": 0.7901,
+      "masked_tokens": 101.2,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.5744095016038046,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5744095016038046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7552260112240674,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 1.1015625,
+      "kd_loss": 0.7122204179657274,
+      "learning_rate": 3e-06,
+      "loss": 1.1559,
+      "masked_tokens": 129.4,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 129.4
+    },
+    {
+      "avg_mask_ratio": 0.4629370831884444,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4629370831884444,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3121085743832509,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.81640625,
+      "kd_loss": 0.5816853262786026,
+      "learning_rate": 3e-06,
+      "loss": 0.8351,
+      "masked_tokens": 110.5125,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 110.5125
+    },
+    {
+      "avg_mask_ratio": 0.499816512214602,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.499816512214602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45488183712676006,
+      "epoch": 0.512,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.621679913950502,
+      "learning_rate": 3e-06,
+      "loss": 0.9475,
+      "masked_tokens": 110.075,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.075
+    },
+    {
+      "avg_mask_ratio": 0.4744578254292719,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4744578254292719,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40257708527117303,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.181640625,
+      "kd_loss": 0.5342776940590724,
+      "learning_rate": 3e-06,
+      "loss": 0.7566,
+      "masked_tokens": 117.1125,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 117.1125
+    },
+    {
+      "avg_mask_ratio": 0.4852474880579393,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.4852474880579393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3536207471866419,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.73046875,
+      "kd_loss": 0.5287767746266354,
+      "learning_rate": 3e-06,
+      "loss": 0.6987,
+      "masked_tokens": 113.725,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 113.725
+    },
+    {
+      "avg_mask_ratio": 0.565397203550674,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.565397203550674,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6109200767528364,
+      "epoch": 0.576,
+      "grad_norm": 1.0703125,
+      "kd_loss": 0.825409283463523,
+      "learning_rate": 3e-06,
+      "loss": 1.197,
+      "masked_tokens": 124.7125,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 124.7125
+    },
+    {
+      "avg_mask_ratio": 0.4814051762456074,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4814051762456074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48712714778754956,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.5236038082813025,
+      "learning_rate": 3e-06,
+      "loss": 0.8092,
+      "masked_tokens": 129.975,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 129.975
+    },
+    {
+      "avg_mask_ratio": 0.4164489531540312,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.4164489531540312,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1603455784160815,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.0849609375,
+      "kd_loss": 0.459246705797786,
+      "learning_rate": 3e-06,
+      "loss": 0.5704,
+      "masked_tokens": 100.625,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.625
+    },
+    {
+      "avg_mask_ratio": 0.47973727830685675,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.47973727830685675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4429863921482934,
+      "epoch": 0.64,
+      "grad_norm": 0.9609375,
+      "kd_loss": 0.6260649390371327,
+      "learning_rate": 3e-06,
+      "loss": 0.847,
+      "masked_tokens": 107.2375,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 107.2375
+    },
+    {
+      "avg_mask_ratio": 0.514206234831363,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.514206234831363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5035652598873412,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.51953125,
+      "kd_loss": 0.6732076569831407,
+      "learning_rate": 3e-06,
+      "loss": 0.9616,
+      "masked_tokens": 88.925,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 88.925
+    },
+    {
+      "avg_mask_ratio": 0.5238390378654003,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.5238390378654003,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.511856697372275,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.8046875,
+      "kd_loss": 0.7122418887767707,
+      "learning_rate": 3e-06,
+      "loss": 1.0805,
+      "masked_tokens": 123.25,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 123.25
+    },
+    {
+      "avg_mask_ratio": 0.5409158666618168,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.5409158666618168,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4557006843067654,
+      "epoch": 0.704,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.681670887517248,
+      "learning_rate": 3e-06,
+      "loss": 0.9853,
+      "masked_tokens": 132.2625,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 132.2625
+    },
+    {
+      "avg_mask_ratio": 0.47903697268920953,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.47903697268920953,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5934374530742617,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 1.265625,
+      "kd_loss": 0.7481572654819957,
+      "learning_rate": 3e-06,
+      "loss": 0.973,
+      "masked_tokens": 114.6375,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 114.6375
+    },
+    {
+      "avg_mask_ratio": 0.5254506973840762,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.5254506973840762,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6267570938103404,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 1.25,
+      "kd_loss": 0.7243690301205706,
+      "learning_rate": 3e-06,
+      "loss": 1.0663,
+      "masked_tokens": 129.7,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 129.7
+    },
+    {
+      "avg_mask_ratio": 0.5038800648180768,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.5038800648180768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37673365626956185,
+      "epoch": 0.768,
+      "grad_norm": 0.1943359375,
+      "kd_loss": 0.7419421046464777,
+      "learning_rate": 3e-06,
+      "loss": 0.8731,
+      "masked_tokens": 118.8375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 118.8375
+    },
+    {
+      "avg_mask_ratio": 0.5092529703164473,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.5092529703164473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5048238590722349,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.1875,
+      "kd_loss": 0.6675537923798401,
+      "learning_rate": 3e-06,
+      "loss": 0.8751,
+      "masked_tokens": 130.4375,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 130.4375
+    },
+    {
+      "avg_mask_ratio": 0.47521690553985535,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.47521690553985535,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3009114894711729,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.96875,
+      "kd_loss": 0.5576462655375508,
+      "learning_rate": 3e-06,
+      "loss": 0.7398,
+      "masked_tokens": 99.7125,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 99.7125
+    },
+    {
+      "avg_mask_ratio": 0.5089340912294574,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.5089340912294574,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42893290086574326,
+      "epoch": 0.832,
+      "grad_norm": 0.345703125,
+      "kd_loss": 0.6829758724064732,
+      "learning_rate": 3e-06,
+      "loss": 0.9627,
+      "masked_tokens": 115.925,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 115.925
+    },
+    {
+      "avg_mask_ratio": 0.5041010878514498,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.5041010878514498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5260561651751459,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 1.484375,
+      "kd_loss": 0.7314142643579544,
+      "learning_rate": 3e-06,
+      "loss": 1.0058,
+      "masked_tokens": 124.875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 124.875
+    },
+    {
+      "avg_mask_ratio": 0.5127229066158179,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.5127229066158179,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7525517523653207,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.7839959781571068,
+      "learning_rate": 3e-06,
+      "loss": 1.0779,
+      "masked_tokens": 123.575,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 123.575
+    },
+    {
+      "avg_mask_ratio": 0.47257317856419834,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.47257317856419834,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.27125484775717723,
+      "epoch": 0.896,
+      "grad_norm": 0.4609375,
+      "kd_loss": 0.5007113358164134,
+      "learning_rate": 3e-06,
+      "loss": 0.6359,
+      "masked_tokens": 90.175,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 90.175
+    },
+    {
+      "avg_mask_ratio": 0.49090774822980165,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.49090774822980165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4832683085802046,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.7578125,
+      "kd_loss": 0.579282811923622,
+      "learning_rate": 3e-06,
+      "loss": 0.8552,
+      "masked_tokens": 108.85,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4731982925441116,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.4731982925441116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5204018854022934,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.6328125,
+      "kd_loss": 0.7500040688079025,
+      "learning_rate": 3e-06,
+      "loss": 0.9602,
+      "masked_tokens": 111.5125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 111.5125
+    },
+    {
+      "avg_mask_ratio": 0.4575169428717345,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4575169428717345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.406052071983504,
+      "epoch": 0.96,
+      "grad_norm": 0.54296875,
+      "kd_loss": 0.6662414132848653,
+      "learning_rate": 3e-06,
+      "loss": 0.8269,
+      "masked_tokens": 107.8875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 107.8875
+    },
+    {
+      "avg_mask_ratio": 0.5099512930959463,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.5099512930959463,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37280656259052875,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.1357421875,
+      "kd_loss": 0.5824788354082557,
+      "learning_rate": 3e-06,
+      "loss": 0.8147,
+      "masked_tokens": 109.275,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 109.275
+    },
+    {
+      "avg_mask_ratio": 0.5108432768334058,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.5108432768334058,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4123959210827912,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.7391241128261429,
+      "learning_rate": 3e-06,
+      "loss": 0.9907,
+      "masked_tokens": 111.75,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 111.75
+    },
+    {
+      "avg_mask_ratio": 0.49879020540975033,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.49879020540975033,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4003649194146419,
+      "epoch": 1.0256,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.7427737473112416,
+      "learning_rate": 3e-06,
+      "loss": 0.9109,
+      "masked_tokens": 121.6875,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 121.6875
+    },
+    {
+      "avg_mask_ratio": 0.48607371354009954,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.48607371354009954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44578953055922454,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.7265625,
+      "kd_loss": 0.616808931784999,
+      "learning_rate": 3e-06,
+      "loss": 0.8475,
+      "masked_tokens": 102.1625,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 102.1625
+    },
+    {
+      "avg_mask_ratio": 0.5385718538891524,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5385718538891524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4503419364663387,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 2.3125,
+      "kd_loss": 0.7079652768573397,
+      "learning_rate": 3e-06,
+      "loss": 0.9016,
+      "masked_tokens": 127.6125,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.6125
+    },
+    {
+      "avg_mask_ratio": 0.5356179510476068,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5356179510476068,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5114419424304287,
+      "epoch": 1.0896,
+      "grad_norm": 2.421875,
+      "kd_loss": 0.8057545678497263,
+      "learning_rate": 3e-06,
+      "loss": 0.9692,
+      "masked_tokens": 136.725,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 136.725
+    },
+    {
+      "avg_mask_ratio": 0.4930020817089826,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4930020817089826,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45815266029704843,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.73046875,
+      "kd_loss": 0.6372618476128536,
+      "learning_rate": 3e-06,
+      "loss": 0.8813,
+      "masked_tokens": 91.975,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 91.975
+    },
+    {
+      "avg_mask_ratio": 0.4998604157241061,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4998604157241061,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5302904470116345,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.793973692091648,
+      "learning_rate": 3e-06,
+      "loss": 1.0187,
+      "masked_tokens": 103.4125,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 103.4125
+    },
+    {
+      "avg_mask_ratio": 0.5236943962518126,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.5236943962518126,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5931689006157967,
+      "epoch": 1.1536,
+      "grad_norm": 0.52734375,
+      "kd_loss": 0.6770309421569891,
+      "learning_rate": 3e-06,
+      "loss": 0.9785,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.4774137590778992,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4774137590778992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33368192758867965,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.552957196540001,
+      "learning_rate": 3e-06,
+      "loss": 0.7195,
+      "masked_tokens": 101.425,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 101.425
+    },
+    {
+      "avg_mask_ratio": 0.41173738130601123,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.41173738130601123,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37927565402829944,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.6514939159315418,
+      "learning_rate": 3e-06,
+      "loss": 0.8086,
+      "masked_tokens": 102.0375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 102.0375
+    },
+    {
+      "avg_mask_ratio": 0.4797614786075428,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.4797614786075428,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38391759186294505,
+      "epoch": 1.2176,
+      "grad_norm": 0.6875,
+      "kd_loss": 0.5477132451650505,
+      "learning_rate": 3e-06,
+      "loss": 0.7759,
+      "masked_tokens": 112.25,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 112.25
+    },
+    {
+      "avg_mask_ratio": 0.4974610014585778,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4974610014585778,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4613554118723869,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.8125,
+      "kd_loss": 0.7624371517440522,
+      "learning_rate": 3e-06,
+      "loss": 0.8935,
+      "masked_tokens": 134.025,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 134.025
+    },
+    {
+      "avg_mask_ratio": 0.5565119812032208,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5565119812032208,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45800332796356996,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.84765625,
+      "kd_loss": 0.7569947588504874,
+      "learning_rate": 3e-06,
+      "loss": 0.9895,
+      "masked_tokens": 126.1375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 126.1375
+    },
+    {
+      "avg_mask_ratio": 0.4784870075061917,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.4784870075061917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42934841778910593,
+      "epoch": 1.2816,
+      "grad_norm": 0.8203125,
+      "kd_loss": 0.6223680850909716,
+      "learning_rate": 3e-06,
+      "loss": 0.8811,
+      "masked_tokens": 113.875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 113.875
+    },
+    {
+      "avg_mask_ratio": 0.5459770569577813,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5459770569577813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46207297699419314,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.193359375,
+      "kd_loss": 0.7765240747485223,
+      "learning_rate": 3e-06,
+      "loss": 1.0488,
+      "masked_tokens": 115.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 115.85
+    },
+    {
+      "avg_mask_ratio": 0.5268841385375709,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.5268841385375709,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5177706574260241,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.42578125,
+      "kd_loss": 0.7318520774833814,
+      "learning_rate": 3e-06,
+      "loss": 1.029,
+      "masked_tokens": 114.6625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 114.6625
+    },
+    {
+      "avg_mask_ratio": 0.5629246362368576,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5629246362368576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5840083432511903,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.140625,
+      "kd_loss": 0.7630814963876361,
+      "learning_rate": 3e-06,
+      "loss": 1.2155,
+      "masked_tokens": 130.675,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.675
+    },
+    {
+      "avg_mask_ratio": 0.5119291188195347,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.5119291188195347,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3960001973351496,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.431640625,
+      "kd_loss": 0.5936955793389871,
+      "learning_rate": 3e-06,
+      "loss": 0.7667,
+      "masked_tokens": 125.9,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 125.9
+    },
+    {
+      "avg_mask_ratio": 0.5539714884362184,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5539714884362184,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6942642153400925,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.625,
+      "kd_loss": 0.8088986897017776,
+      "learning_rate": 3e-06,
+      "loss": 1.1162,
+      "masked_tokens": 131.7625,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 131.7625
+    },
+    {
+      "avg_mask_ratio": 0.5141558598377742,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.5141558598377742,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43362003258180265,
+      "epoch": 1.4096,
+      "grad_norm": 1.1875,
+      "kd_loss": 0.6850645583552251,
+      "learning_rate": 3e-06,
+      "loss": 0.9688,
+      "masked_tokens": 133.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 133.35
+    },
+    {
+      "avg_mask_ratio": 0.4895282822311856,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.4895282822311856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40907790855981147,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 1.1953125,
+      "kd_loss": 0.5458992913151632,
+      "learning_rate": 3e-06,
+      "loss": 0.7579,
+      "masked_tokens": 123.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 123.125
+    },
+    {
+      "avg_mask_ratio": 0.4951617428450845,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4951617428450845,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4818705332485024,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.45703125,
+      "kd_loss": 0.6341485966125283,
+      "learning_rate": 3e-06,
+      "loss": 0.8653,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.5100495176156983,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.5100495176156983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5174321293302568,
+      "epoch": 1.4736,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.692481756274401,
+      "learning_rate": 3e-06,
+      "loss": 0.8852,
+      "masked_tokens": 115.8875,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 115.8875
+    },
+    {
+      "avg_mask_ratio": 0.5639110118616373,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5639110118616373,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4771150118128048,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.46484375,
+      "kd_loss": 0.9550692923163296,
+      "learning_rate": 3e-06,
+      "loss": 1.1428,
+      "masked_tokens": 138.2,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 138.2
+    },
+    {
+      "avg_mask_ratio": 0.5292218026472255,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.5292218026472255,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.36868086418378426,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.283203125,
+      "kd_loss": 0.6208373298897982,
+      "learning_rate": 3e-06,
+      "loss": 0.782,
+      "masked_tokens": 113.375,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 113.375
+    },
+    {
+      "avg_mask_ratio": 0.5125403102487326,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.5125403102487326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5343423100649431,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 1.1796875,
+      "kd_loss": 0.8048794609300558,
+      "learning_rate": 3e-06,
+      "loss": 1.0332,
+      "masked_tokens": 115.675,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 115.675
+    },
+    {
+      "avg_mask_ratio": 0.48811948703369124,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.48811948703369124,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5609710454625769,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.6331229786760091,
+      "learning_rate": 3e-06,
+      "loss": 0.906,
+      "masked_tokens": 106.65,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.5547609420493245,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5547609420493245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6117596920593087,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.7415648018846696,
+      "learning_rate": 3e-06,
+      "loss": 1.0796,
+      "masked_tokens": 100.775,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 100.775
+    },
+    {
+      "avg_mask_ratio": 0.44697874613921157,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.44697874613921157,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4524557980651025,
+      "epoch": 1.6016,
+      "grad_norm": 0.8125,
+      "kd_loss": 0.7427308069233959,
+      "learning_rate": 3e-06,
+      "loss": 0.8892,
+      "masked_tokens": 99.5,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 99.5
+    },
+    {
+      "avg_mask_ratio": 0.49905171967693607,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.49905171967693607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5331612331144584,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.869146594414876,
+      "learning_rate": 3e-06,
+      "loss": 1.024,
+      "masked_tokens": 111.525,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 111.525
+    },
+    {
+      "avg_mask_ratio": 0.4734679562970996,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4734679562970996,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3263147723814427,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.78515625,
+      "kd_loss": 0.5851220395212977,
+      "learning_rate": 3e-06,
+      "loss": 0.6745,
+      "masked_tokens": 124.1625,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 124.1625
+    },
+    {
+      "avg_mask_ratio": 0.45589545626135075,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.45589545626135075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40714966794125756,
+      "epoch": 1.6656,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.5390054064211469,
+      "learning_rate": 3e-06,
+      "loss": 0.6662,
+      "masked_tokens": 120.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 120.2
+    },
+    {
+      "avg_mask_ratio": 0.4975356309209019,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4975356309209019,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3924752975523688,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.125,
+      "kd_loss": 0.5501754963899657,
+      "learning_rate": 3e-06,
+      "loss": 0.7353,
+      "masked_tokens": 107.35,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 107.35
+    },
+    {
+      "avg_mask_ratio": 0.4948011673986912,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.4948011673986912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.32645424191614436,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 1.125,
+      "kd_loss": 0.5133585692653468,
+      "learning_rate": 3e-06,
+      "loss": 0.7248,
+      "masked_tokens": 109.1875,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 109.1875
+    },
+    {
+      "avg_mask_ratio": 0.5321399106411263,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5321399106411263,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5308010719869571,
+      "epoch": 1.7296,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.6684280144268996,
+      "learning_rate": 3e-06,
+      "loss": 0.9887,
+      "masked_tokens": 123.0375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 123.0375
+    },
+    {
+      "avg_mask_ratio": 0.5357416228158399,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5357416228158399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5018791363456672,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.8031356525036244,
+      "learning_rate": 3e-06,
+      "loss": 1.0867,
+      "masked_tokens": 121.5625,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 121.5625
+    },
+    {
+      "avg_mask_ratio": 0.5232944375369698,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.5232944375369698,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4925852923925049,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.9140625,
+      "kd_loss": 0.796732622892182,
+      "learning_rate": 3e-06,
+      "loss": 1.0096,
+      "masked_tokens": 138.675,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 138.675
+    },
+    {
+      "avg_mask_ratio": 0.5344608084415086,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5344608084415086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39800737908125255,
+      "epoch": 1.7936,
+      "grad_norm": 0.2041015625,
+      "kd_loss": 0.6124706103168371,
+      "learning_rate": 3e-06,
+      "loss": 0.806,
+      "masked_tokens": 119.225,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 119.225
+    },
+    {
+      "avg_mask_ratio": 0.5352845921181142,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5352845921181142,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.398157004356932,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.5720241727458018,
+      "learning_rate": 3e-06,
+      "loss": 0.7641,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4979630701942369,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4979630701942369,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4882415359192748,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.75,
+      "kd_loss": 0.6685384457871464,
+      "learning_rate": 3e-06,
+      "loss": 0.8746,
+      "masked_tokens": 111.275,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 111.275
+    },
+    {
+      "avg_mask_ratio": 0.5208624298567883,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.5208624298567883,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3722814081119509,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.90234375,
+      "kd_loss": 0.6259868353982938,
+      "learning_rate": 3e-06,
+      "loss": 0.931,
+      "masked_tokens": 119.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 119.8875
+    },
+    {
+      "avg_mask_ratio": 0.4884064760175534,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.4884064760175534,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3424368775197877,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.9453125,
+      "kd_loss": 0.7612382256288385,
+      "learning_rate": 3e-06,
+      "loss": 0.9297,
+      "masked_tokens": 97.0,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 97.0
+    },
+    {
+      "avg_mask_ratio": 0.47601241993543225,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.47601241993543225,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.29439299476740644,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.279296875,
+      "kd_loss": 0.5778021513873682,
+      "learning_rate": 3e-06,
+      "loss": 0.6641,
+      "masked_tokens": 107.1375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 107.1375
+    },
+    {
+      "avg_mask_ratio": 0.4820589871611446,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4820589871611446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4219410733627285,
+      "epoch": 1.9216,
+      "grad_norm": 0.51953125,
+      "kd_loss": 0.6610599886284263,
+      "learning_rate": 3e-06,
+      "loss": 0.8875,
+      "masked_tokens": 106.675,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 106.675
+    },
+    {
+      "avg_mask_ratio": 0.5009820312960074,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.5009820312960074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4484953544199811,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.423828125,
+      "kd_loss": 0.6393605162026915,
+      "learning_rate": 3e-06,
+      "loss": 0.8387,
+      "masked_tokens": 129.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 129.1625
+    },
+    {
+      "avg_mask_ratio": 0.4952817424898967,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4952817424898967,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4003621553950552,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 1.28125,
+      "kd_loss": 0.7165493822423741,
+      "learning_rate": 3e-06,
+      "loss": 0.87,
+      "masked_tokens": 114.2625,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 114.2625
+    },
+    {
+      "avg_mask_ratio": 0.5025755434762686,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.5025755434762686,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44728768827070553,
+      "epoch": 1.9856,
+      "grad_norm": 0.380859375,
+      "kd_loss": 0.5998596335327193,
+      "learning_rate": 3e-06,
+      "loss": 0.769,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5328231096001608,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5328231096001608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34552624383357916,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.6015625,
+      "kd_loss": 0.6963629411807882,
+      "learning_rate": 3e-06,
+      "loss": 0.9754,
+      "masked_tokens": 126.51190476190476,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 126.51190476190476
+    },
+    {
+      "avg_mask_ratio": 0.46634063599049114,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.46634063599049114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3470186025385317,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.6648836274001951,
+      "learning_rate": 3e-06,
+      "loss": 0.8308,
+      "masked_tokens": 111.15,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 111.15
+    },
+    {
+      "avg_mask_ratio": 0.5202614731155336,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.5202614731155336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39255777003332354,
+      "epoch": 2.0512,
+      "grad_norm": 0.9921875,
+      "kd_loss": 0.8403500239691084,
+      "learning_rate": 3e-06,
+      "loss": 1.0217,
+      "masked_tokens": 146.275,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 146.275
+    },
+    {
+      "avg_mask_ratio": 0.4732307325524744,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4732307325524744,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4136752946935417,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 1.6015625,
+      "kd_loss": 0.5665071224577105,
+      "learning_rate": 3e-06,
+      "loss": 0.7771,
+      "masked_tokens": 111.6375,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 111.6375
+    },
+    {
+      "avg_mask_ratio": 0.45015103057958183,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.45015103057958183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24581733565152036,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.515625,
+      "kd_loss": 0.5551535871100896,
+      "learning_rate": 3e-06,
+      "loss": 0.697,
+      "masked_tokens": 102.2875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 102.2875
+    },
+    {
+      "avg_mask_ratio": 0.48529006402241065,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.48529006402241065,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2608388370660464,
+      "epoch": 2.1152,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.5421745884898016,
+      "learning_rate": 3e-06,
+      "loss": 0.6504,
+      "masked_tokens": 123.075,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 123.075
+    },
+    {
+      "avg_mask_ratio": 0.4696127205621451,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4696127205621451,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3479914085991481,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.5546875,
+      "kd_loss": 0.5542144843038216,
+      "learning_rate": 3e-06,
+      "loss": 0.7371,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 103.0875
+    },
+    {
+      "avg_mask_ratio": 0.46368037317879496,
+      "avg_response_length": 213.175,
+      "avg_student_mask_ratio": 0.46368037317879496,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3293848424991552,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.2275390625,
+      "kd_loss": 0.5148144457874537,
+      "learning_rate": 3e-06,
+      "loss": 0.7009,
+      "masked_tokens": 93.1375,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 93.1375
+    },
+    {
+      "avg_mask_ratio": 0.5379365492146462,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5379365492146462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46470798885368936,
+      "epoch": 2.1792,
+      "grad_norm": 0.6484375,
+      "kd_loss": 0.7375291446910068,
+      "learning_rate": 3e-06,
+      "loss": 0.9784,
+      "masked_tokens": 114.975,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 114.975
+    },
+    {
+      "avg_mask_ratio": 0.5036081655998714,
+      "avg_response_length": 219.175,
+      "avg_student_mask_ratio": 0.5036081655998714,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45209096000687055,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 1.5,
+      "kd_loss": 0.7249787944556466,
+      "learning_rate": 3e-06,
+      "loss": 0.9227,
+      "masked_tokens": 109.9125,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 109.9125
+    },
+    {
+      "avg_mask_ratio": 0.529415801318828,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.529415801318828,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3901031159069248,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.58984375,
+      "kd_loss": 0.6996272807611149,
+      "learning_rate": 3e-06,
+      "loss": 0.8817,
+      "masked_tokens": 104.3125,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 104.3125
+    },
+    {
+      "avg_mask_ratio": 0.5512922222726047,
+      "avg_response_length": 237.875,
+      "avg_student_mask_ratio": 0.5512922222726047,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41538310063565403,
+      "epoch": 2.2432,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.7574320874500586,
+      "learning_rate": 3e-06,
+      "loss": 0.903,
+      "masked_tokens": 127.425,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 127.425
+    },
+    {
+      "avg_mask_ratio": 0.480971388152102,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.480971388152102,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3390387184460337,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.546875,
+      "kd_loss": 0.568807559109473,
+      "learning_rate": 3e-06,
+      "loss": 0.7216,
+      "masked_tokens": 125.65,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 125.65
+    },
+    {
+      "avg_mask_ratio": 0.5030692228931002,
+      "avg_response_length": 253.8375,
+      "avg_student_mask_ratio": 0.5030692228931002,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38280749060686503,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.5519341900614336,
+      "learning_rate": 3e-06,
+      "loss": 0.8228,
+      "masked_tokens": 139.2,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 139.2
+    },
+    {
+      "avg_mask_ratio": 0.4997857674607076,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4997857674607076,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.25629440465650644,
+      "epoch": 2.3072,
+      "grad_norm": 0.140625,
+      "kd_loss": 0.5524894027101936,
+      "learning_rate": 3e-06,
+      "loss": 0.7452,
+      "masked_tokens": 102.5125,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 102.5125
+    },
+    {
+      "avg_mask_ratio": 0.4668914210633375,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.4668914210633375,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28395035568828464,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.373046875,
+      "kd_loss": 0.5042674671490367,
+      "learning_rate": 3e-06,
+      "loss": 0.6619,
+      "masked_tokens": 94.85,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 94.85
+    },
+    {
+      "avg_mask_ratio": 0.561556038632989,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.561556038632989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.556549127957021,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.58203125,
+      "kd_loss": 0.6619636991126754,
+      "learning_rate": 3e-06,
+      "loss": 1.0218,
+      "masked_tokens": 139.1375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 139.1375
+    },
+    {
+      "avg_mask_ratio": 0.47325256096664814,
+      "avg_response_length": 226.6375,
+      "avg_student_mask_ratio": 0.47325256096664814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3334150109870279,
+      "epoch": 2.3712,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.44958578083260364,
+      "learning_rate": 3e-06,
+      "loss": 0.696,
+      "masked_tokens": 107.3,
+      "mean_t": 0.4757364276825683,
+      "step": 1110,
+      "student_masked_tokens": 107.3
+    },
+    {
+      "avg_mask_ratio": 0.5052445781533607,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.5052445781533607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33756577983574515,
+      "epoch": 2.392533333333333,
+      "grad_norm": 2.234375,
+      "kd_loss": 0.47280435794045844,
+      "learning_rate": 3e-06,
+      "loss": 0.7787,
+      "masked_tokens": 117.325,
+      "mean_t": 0.5013068238971755,
+      "step": 1120,
+      "student_masked_tokens": 117.325
+    },
+    {
+      "avg_mask_ratio": 0.5352560582570731,
+      "avg_response_length": 262.25,
+      "avg_student_mask_ratio": 0.5352560582570731,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.49989761952609796,
+      "epoch": 2.413866666666667,
+      "grad_norm": 2.1875,
+      "kd_loss": 0.7383381549463138,
+      "learning_rate": 3e-06,
+      "loss": 0.97,
+      "masked_tokens": 140.5375,
+      "mean_t": 0.5303254407714121,
+      "step": 1130,
+      "student_masked_tokens": 140.5375
+    },
+    {
+      "avg_mask_ratio": 0.4938803721917793,
+      "avg_response_length": 217.05,
+      "avg_student_mask_ratio": 0.4938803721917793,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37043922989309974,
+      "epoch": 2.4352,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.5307485466641083,
+      "learning_rate": 3e-06,
+      "loss": 0.6962,
+      "masked_tokens": 107.6125,
+      "mean_t": 0.4845335395424627,
+      "step": 1140,
+      "student_masked_tokens": 107.6125
+    },
+    {
+      "avg_mask_ratio": 0.5653773612109945,
+      "avg_response_length": 212.5375,
+      "avg_student_mask_ratio": 0.5653773612109945,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7858076869456567,
+      "epoch": 2.4565333333333332,
+      "grad_norm": 1.84375,
+      "kd_loss": 0.7624981299486535,
+      "learning_rate": 3e-06,
+      "loss": 1.1469,
+      "masked_tokens": 114.6,
+      "mean_t": 0.5690932425903157,
+      "step": 1150,
+      "student_masked_tokens": 114.6
+    },
+    {
+      "avg_mask_ratio": 0.49925965811125933,
+      "avg_response_length": 225.6125,
+      "avg_student_mask_ratio": 0.49925965811125933,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46302462185976767,
+      "epoch": 2.4778666666666664,
+      "grad_norm": 3.25,
+      "kd_loss": 0.5881607823695049,
+      "learning_rate": 3e-06,
+      "loss": 0.7393,
+      "masked_tokens": 106.125,
+      "mean_t": 0.5040684466948733,
+      "step": 1160,
+      "student_masked_tokens": 106.125
+    },
+    {
+      "avg_mask_ratio": 0.5130727548676077,
+      "avg_response_length": 247.7625,
+      "avg_student_mask_ratio": 0.5130727548676077,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43388418721337985,
+      "epoch": 2.4992,
+      "grad_norm": 0.38671875,
+      "kd_loss": 0.593448537235713,
+      "learning_rate": 3e-06,
+      "loss": 0.7761,
+      "masked_tokens": 128.05,
+      "mean_t": 0.5114516971167177,
+      "step": 1170,
+      "student_masked_tokens": 128.05
+    },
+    {
+      "avg_mask_ratio": 0.4515186986711342,
+      "avg_response_length": 214.725,
+      "avg_student_mask_ratio": 0.4515186986711342,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24207257057115045,
+      "epoch": 2.5205333333333333,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.446691993195418,
+      "learning_rate": 3e-06,
+      "loss": 0.649,
+      "masked_tokens": 89.6375,
+      "mean_t": 0.4491677140351385,
+      "step": 1180,
+      "student_masked_tokens": 89.6375
+    },
+    {
+      "avg_mask_ratio": 0.5575610842439346,
+      "avg_response_length": 220.3375,
+      "avg_student_mask_ratio": 0.5575610842439346,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5483032605723451,
+      "epoch": 2.5418666666666665,
+      "grad_norm": 0.95703125,
+      "kd_loss": 0.8963939757952517,
+      "learning_rate": 3e-06,
+      "loss": 1.0849,
+      "masked_tokens": 129.425,
+      "mean_t": 0.5590635397238657,
+      "step": 1190,
+      "student_masked_tokens": 129.425
+    },
+    {
+      "avg_mask_ratio": 0.5073940187954576,
+      "avg_response_length": 215.675,
+      "avg_student_mask_ratio": 0.5073940187954576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3367691548969276,
+      "epoch": 2.5632,
+      "grad_norm": 1.4140625,
+      "kd_loss": 0.5662632359816826,
+      "learning_rate": 3e-06,
+      "loss": 0.7514,
+      "masked_tokens": 103.8375,
+      "mean_t": 0.506370971655997,
+      "step": 1200,
+      "student_masked_tokens": 103.8375
+    },
+    {
+      "avg_mask_ratio": 0.49865787494927644,
+      "avg_response_length": 223.8875,
+      "avg_student_mask_ratio": 0.49865787494927644,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4686214710733111,
+      "epoch": 2.5845333333333333,
+      "grad_norm": 0.4453125,
+      "kd_loss": 0.693641352130814,
+      "learning_rate": 3e-06,
+      "loss": 0.9178,
+      "masked_tokens": 107.6,
+      "mean_t": 0.5015621181708412,
+      "step": 1210,
+      "student_masked_tokens": 107.6
+    },
+    {
+      "avg_mask_ratio": 0.49044962012558246,
+      "avg_response_length": 235.95,
+      "avg_student_mask_ratio": 0.49044962012558246,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43763301398147403,
+      "epoch": 2.6058666666666666,
+      "grad_norm": 0.263671875,
+      "kd_loss": 0.6695325696462533,
+      "learning_rate": 3e-06,
+      "loss": 0.8517,
+      "masked_tokens": 112.1375,
+      "mean_t": 0.48838805715786293,
+      "step": 1220,
+      "student_masked_tokens": 112.1375
+    },
+    {
+      "avg_mask_ratio": 0.5458357291528955,
+      "avg_response_length": 213.425,
+      "avg_student_mask_ratio": 0.5458357291528955,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5803454444972772,
+      "epoch": 2.6272,
+      "grad_norm": 0.8359375,
+      "kd_loss": 0.7648584355971252,
+      "learning_rate": 3e-06,
+      "loss": 1.071,
+      "masked_tokens": 119.2625,
+      "mean_t": 0.5434774641471449,
+      "step": 1230,
+      "student_masked_tokens": 119.2625
+    },
+    {
+      "avg_mask_ratio": 0.48120573440101,
+      "avg_response_length": 233.4875,
+      "avg_student_mask_ratio": 0.48120573440101,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2953461956545766,
+      "epoch": 2.6485333333333334,
+      "grad_norm": 0.5546875,
+      "kd_loss": 0.5628422546075598,
+      "learning_rate": 3e-06,
+      "loss": 0.759,
+      "masked_tokens": 102.775,
+      "mean_t": 0.48289845581166446,
+      "step": 1240,
+      "student_masked_tokens": 102.775
+    },
+    {
+      "avg_mask_ratio": 0.48337737330584785,
+      "avg_response_length": 250.5875,
+      "avg_student_mask_ratio": 0.48337737330584785,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5189007750311248,
+      "epoch": 2.6698666666666666,
+      "grad_norm": 1.7578125,
+      "kd_loss": 0.6163040083439683,
+      "learning_rate": 3e-06,
+      "loss": 0.8516,
+      "masked_tokens": 121.9875,
+      "mean_t": 0.4824625514564104,
+      "step": 1250,
+      "student_masked_tokens": 121.9875
+    },
+    {
+      "avg_mask_ratio": 0.451413996523479,
+      "avg_response_length": 237.1875,
+      "avg_student_mask_ratio": 0.451413996523479,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.351084196654574,
+      "epoch": 2.6912000000000003,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.45329858138342943,
+      "learning_rate": 3e-06,
+      "loss": 0.6288,
+      "masked_tokens": 100.075,
+      "mean_t": 0.44945613203162793,
+      "step": 1260,
+      "student_masked_tokens": 100.075
+    },
+    {
+      "avg_mask_ratio": 0.5235667617758736,
+      "avg_response_length": 226.925,
+      "avg_student_mask_ratio": 0.5235667617758736,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42535067965817175,
+      "epoch": 2.7125333333333335,
+      "grad_norm": 1.0703125,
+      "kd_loss": 0.7561996218850254,
+      "learning_rate": 3e-06,
+      "loss": 0.9996,
+      "masked_tokens": 125.1625,
+      "mean_t": 0.5232691798126325,
+      "step": 1270,
+      "student_masked_tokens": 125.1625
+    },
+    {
+      "avg_mask_ratio": 0.44069963712245225,
+      "avg_response_length": 236.0625,
+      "avg_student_mask_ratio": 0.44069963712245225,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3502078311451868,
+      "epoch": 2.7338666666666667,
+      "grad_norm": 1.3671875,
+      "kd_loss": 0.510219725947995,
+      "learning_rate": 3e-06,
+      "loss": 0.677,
+      "masked_tokens": 99.4625,
+      "mean_t": 0.4489673391333781,
+      "step": 1280,
+      "student_masked_tokens": 99.4625
+    },
+    {
+      "avg_mask_ratio": 0.4850222111213952,
+      "avg_response_length": 204.1625,
+      "avg_student_mask_ratio": 0.4850222111213952,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.30171046461458106,
+      "epoch": 2.7552,
+      "grad_norm": 1.125,
+      "kd_loss": 0.6329020506331972,
+      "learning_rate": 3e-06,
+      "loss": 0.8098,
+      "masked_tokens": 96.5,
+      "mean_t": 0.4806917709764093,
+      "step": 1290,
+      "student_masked_tokens": 96.5
+    },
+    {
+      "avg_mask_ratio": 0.45676524901064114,
+      "avg_response_length": 244.0375,
+      "avg_student_mask_ratio": 0.45676524901064114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4464949894013728,
+      "epoch": 2.776533333333333,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.6368680644820728,
+      "learning_rate": 3e-06,
+      "loss": 0.868,
+      "masked_tokens": 106.5125,
+      "mean_t": 0.4616577463573776,
+      "step": 1300,
+      "student_masked_tokens": 106.5125
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7d6e2353a4a011ee9d5626d3ff19f8d774bfd347bf01c9b971a0eaf1aee9d41
+size 7608

math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/checkpoint-1400/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1