Upload folder using huggingface_hub

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad4a9f6462cc9a90e52594f6bd32574ef3ef3eadaa8bb8b48cf62640ec1604e1
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a557c0caa06d6149244435a36e20fb0ff820012384501a96717af84d73b4ab7
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5527614ca942123c39a84d47990fc1d0173a78ef912e710e6b2e720027d420da
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50c81d5c0a6501b9a3dad1fd7e729e9c5beb647ab7f6cf96d4b9a494ef0702d6
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c49496ce00eb78c814fcb6b8085fb3bdf31dcf5e4152f99090d12a4b280f9538
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/trainer_state.json

@@ -0,0 +1,911 @@
+{
+  "best_metric": 1.3188884258270264,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-407",
+  "epoch": 2.9987730061349693,
+  "eval_steps": 10,
+  "global_step": 1222,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.024539877300613498,
+      "grad_norm": 1.0104172229766846,
+      "learning_rate": 0.0002,
+      "loss": 1.7717,
+      "step": 10
+    },
+    {
+      "epoch": 0.049079754601226995,
+      "grad_norm": 0.8800041079521179,
+      "learning_rate": 0.0002,
+      "loss": 1.6041,
+      "step": 20
+    },
+    {
+      "epoch": 0.0736196319018405,
+      "grad_norm": 1.337620496749878,
+      "learning_rate": 0.0002,
+      "loss": 1.4421,
+      "step": 30
+    },
+    {
+      "epoch": 0.09815950920245399,
+      "grad_norm": 0.6757020354270935,
+      "learning_rate": 0.0002,
+      "loss": 1.5542,
+      "step": 40
+    },
+    {
+      "epoch": 0.12269938650306748,
+      "grad_norm": 42.27006149291992,
+      "learning_rate": 0.0002,
+      "loss": 1.4365,
+      "step": 50
+    },
+    {
+      "epoch": 0.147239263803681,
+      "grad_norm": 0.8157640099525452,
+      "learning_rate": 0.0002,
+      "loss": 1.1226,
+      "step": 60
+    },
+    {
+      "epoch": 0.17177914110429449,
+      "grad_norm": 1.0711787939071655,
+      "learning_rate": 0.0002,
+      "loss": 1.1886,
+      "step": 70
+    },
+    {
+      "epoch": 0.19631901840490798,
+      "grad_norm": 0.5880025029182434,
+      "learning_rate": 0.0002,
+      "loss": 1.1239,
+      "step": 80
+    },
+    {
+      "epoch": 0.22085889570552147,
+      "grad_norm": 0.827833354473114,
+      "learning_rate": 0.0002,
+      "loss": 1.0696,
+      "step": 90
+    },
+    {
+      "epoch": 0.24539877300613497,
+      "grad_norm": 0.579414963722229,
+      "learning_rate": 0.0002,
+      "loss": 1.0468,
+      "step": 100
+    },
+    {
+      "epoch": 0.26993865030674846,
+      "grad_norm": 0.6620142459869385,
+      "learning_rate": 0.0002,
+      "loss": 1.0914,
+      "step": 110
+    },
+    {
+      "epoch": 0.294478527607362,
+      "grad_norm": 0.5601297616958618,
+      "learning_rate": 0.0002,
+      "loss": 1.0873,
+      "step": 120
+    },
+    {
+      "epoch": 0.31901840490797545,
+      "grad_norm": 0.5814566612243652,
+      "learning_rate": 0.0002,
+      "loss": 1.1652,
+      "step": 130
+    },
+    {
+      "epoch": 0.34355828220858897,
+      "grad_norm": 0.6171417236328125,
+      "learning_rate": 0.0002,
+      "loss": 1.0915,
+      "step": 140
+    },
+    {
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.6176294088363647,
+      "learning_rate": 0.0002,
+      "loss": 1.1572,
+      "step": 150
+    },
+    {
+      "epoch": 0.39263803680981596,
+      "grad_norm": 0.8398241400718689,
+      "learning_rate": 0.0002,
+      "loss": 1.1135,
+      "step": 160
+    },
+    {
+      "epoch": 0.4171779141104294,
+      "grad_norm": 0.5397658944129944,
+      "learning_rate": 0.0002,
+      "loss": 1.1235,
+      "step": 170
+    },
+    {
+      "epoch": 0.44171779141104295,
+      "grad_norm": 0.7487576603889465,
+      "learning_rate": 0.0002,
+      "loss": 1.1438,
+      "step": 180
+    },
+    {
+      "epoch": 0.4662576687116564,
+      "grad_norm": 0.4767085015773773,
+      "learning_rate": 0.0002,
+      "loss": 1.1178,
+      "step": 190
+    },
+    {
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.6086363196372986,
+      "learning_rate": 0.0002,
+      "loss": 1.1222,
+      "step": 200
+    },
+    {
+      "epoch": 0.5153374233128835,
+      "grad_norm": 0.6940106749534607,
+      "learning_rate": 0.0002,
+      "loss": 1.0346,
+      "step": 210
+    },
+    {
+      "epoch": 0.5398773006134969,
+      "grad_norm": 0.775067150592804,
+      "learning_rate": 0.0002,
+      "loss": 1.1162,
+      "step": 220
+    },
+    {
+      "epoch": 0.5644171779141104,
+      "grad_norm": 0.7273200154304504,
+      "learning_rate": 0.0002,
+      "loss": 1.1335,
+      "step": 230
+    },
+    {
+      "epoch": 0.588957055214724,
+      "grad_norm": 0.5168078541755676,
+      "learning_rate": 0.0002,
+      "loss": 1.0606,
+      "step": 240
+    },
+    {
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6487783193588257,
+      "learning_rate": 0.0002,
+      "loss": 1.0792,
+      "step": 250
+    },
+    {
+      "epoch": 0.6380368098159509,
+      "grad_norm": 1.0914227962493896,
+      "learning_rate": 0.0002,
+      "loss": 1.1642,
+      "step": 260
+    },
+    {
+      "epoch": 0.6625766871165644,
+      "grad_norm": 1.0182702541351318,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 270
+    },
+    {
+      "epoch": 0.6871165644171779,
+      "grad_norm": 0.7045943140983582,
+      "learning_rate": 0.0002,
+      "loss": 0.9816,
+      "step": 280
+    },
+    {
+      "epoch": 0.7116564417177914,
+      "grad_norm": 0.6228044033050537,
+      "learning_rate": 0.0002,
+      "loss": 0.9675,
+      "step": 290
+    },
+    {
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.6161119341850281,
+      "learning_rate": 0.0002,
+      "loss": 1.076,
+      "step": 300
+    },
+    {
+      "epoch": 0.7607361963190185,
+      "grad_norm": 0.8310935497283936,
+      "learning_rate": 0.0002,
+      "loss": 1.0597,
+      "step": 310
+    },
+    {
+      "epoch": 0.7852760736196319,
+      "grad_norm": 0.6500546336174011,
+      "learning_rate": 0.0002,
+      "loss": 1.0817,
+      "step": 320
+    },
+    {
+      "epoch": 0.8098159509202454,
+      "grad_norm": 0.9254736304283142,
+      "learning_rate": 0.0002,
+      "loss": 1.109,
+      "step": 330
+    },
+    {
+      "epoch": 0.8343558282208589,
+      "grad_norm": 0.5892964005470276,
+      "learning_rate": 0.0002,
+      "loss": 0.9929,
+      "step": 340
+    },
+    {
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.5500181913375854,
+      "learning_rate": 0.0002,
+      "loss": 1.0637,
+      "step": 350
+    },
+    {
+      "epoch": 0.8834355828220859,
+      "grad_norm": 0.7814139127731323,
+      "learning_rate": 0.0002,
+      "loss": 1.0693,
+      "step": 360
+    },
+    {
+      "epoch": 0.9079754601226994,
+      "grad_norm": 0.6288005113601685,
+      "learning_rate": 0.0002,
+      "loss": 1.059,
+      "step": 370
+    },
+    {
+      "epoch": 0.9325153374233128,
+      "grad_norm": 0.5031183958053589,
+      "learning_rate": 0.0002,
+      "loss": 0.9904,
+      "step": 380
+    },
+    {
+      "epoch": 0.9570552147239264,
+      "grad_norm": 0.8656964898109436,
+      "learning_rate": 0.0002,
+      "loss": 1.1183,
+      "step": 390
+    },
+    {
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6746202111244202,
+      "learning_rate": 0.0002,
+      "loss": 1.0296,
+      "step": 400
+    },
+    {
+      "epoch": 0.9987730061349693,
+      "eval_loss": 1.3188884258270264,
+      "eval_runtime": 124.2158,
+      "eval_samples_per_second": 3.671,
+      "eval_steps_per_second": 0.459,
+      "step": 407
+    },
+    {
+      "epoch": 1.0061349693251533,
+      "grad_norm": 0.5244082808494568,
+      "learning_rate": 0.0002,
+      "loss": 0.9679,
+      "step": 410
+    },
+    {
+      "epoch": 1.030674846625767,
+      "grad_norm": 0.6792967319488525,
+      "learning_rate": 0.0002,
+      "loss": 0.97,
+      "step": 420
+    },
+    {
+      "epoch": 1.0552147239263803,
+      "grad_norm": 0.6114474534988403,
+      "learning_rate": 0.0002,
+      "loss": 0.8935,
+      "step": 430
+    },
+    {
+      "epoch": 1.0797546012269938,
+      "grad_norm": 0.6367565393447876,
+      "learning_rate": 0.0002,
+      "loss": 0.9469,
+      "step": 440
+    },
+    {
+      "epoch": 1.1042944785276074,
+      "grad_norm": 0.7007757425308228,
+      "learning_rate": 0.0002,
+      "loss": 0.9536,
+      "step": 450
+    },
+    {
+      "epoch": 1.1288343558282208,
+      "grad_norm": 0.9737136960029602,
+      "learning_rate": 0.0002,
+      "loss": 0.8806,
+      "step": 460
+    },
+    {
+      "epoch": 1.1533742331288344,
+      "grad_norm": 1.1624044179916382,
+      "learning_rate": 0.0002,
+      "loss": 0.9217,
+      "step": 470
+    },
+    {
+      "epoch": 1.177914110429448,
+      "grad_norm": 2.1017873287200928,
+      "learning_rate": 0.0002,
+      "loss": 0.9056,
+      "step": 480
+    },
+    {
+      "epoch": 1.2024539877300613,
+      "grad_norm": 0.7352198958396912,
+      "learning_rate": 0.0002,
+      "loss": 0.9945,
+      "step": 490
+    },
+    {
+      "epoch": 1.2269938650306749,
+      "grad_norm": 0.9491982460021973,
+      "learning_rate": 0.0002,
+      "loss": 0.9224,
+      "step": 500
+    },
+    {
+      "epoch": 1.2515337423312882,
+      "grad_norm": 1.0124034881591797,
+      "learning_rate": 0.0002,
+      "loss": 0.9079,
+      "step": 510
+    },
+    {
+      "epoch": 1.2760736196319018,
+      "grad_norm": 0.7620294094085693,
+      "learning_rate": 0.0002,
+      "loss": 0.9596,
+      "step": 520
+    },
+    {
+      "epoch": 1.3006134969325154,
+      "grad_norm": 0.7207106351852417,
+      "learning_rate": 0.0002,
+      "loss": 0.9095,
+      "step": 530
+    },
+    {
+      "epoch": 1.3251533742331287,
+      "grad_norm": 0.586559534072876,
+      "learning_rate": 0.0002,
+      "loss": 0.8588,
+      "step": 540
+    },
+    {
+      "epoch": 1.3496932515337423,
+      "grad_norm": 0.7113742828369141,
+      "learning_rate": 0.0002,
+      "loss": 0.9146,
+      "step": 550
+    },
+    {
+      "epoch": 1.3742331288343559,
+      "grad_norm": 0.9574087262153625,
+      "learning_rate": 0.0002,
+      "loss": 0.9504,
+      "step": 560
+    },
+    {
+      "epoch": 1.3987730061349692,
+      "grad_norm": 0.71551913022995,
+      "learning_rate": 0.0002,
+      "loss": 0.9124,
+      "step": 570
+    },
+    {
+      "epoch": 1.4233128834355828,
+      "grad_norm": 1.0000630617141724,
+      "learning_rate": 0.0002,
+      "loss": 0.8944,
+      "step": 580
+    },
+    {
+      "epoch": 1.4478527607361964,
+      "grad_norm": 0.6878040432929993,
+      "learning_rate": 0.0002,
+      "loss": 0.8707,
+      "step": 590
+    },
+    {
+      "epoch": 1.4723926380368098,
+      "grad_norm": 0.770113468170166,
+      "learning_rate": 0.0002,
+      "loss": 0.953,
+      "step": 600
+    },
+    {
+      "epoch": 1.4969325153374233,
+      "grad_norm": 0.7941587567329407,
+      "learning_rate": 0.0002,
+      "loss": 0.9293,
+      "step": 610
+    },
+    {
+      "epoch": 1.521472392638037,
+      "grad_norm": 0.971196711063385,
+      "learning_rate": 0.0002,
+      "loss": 0.9177,
+      "step": 620
+    },
+    {
+      "epoch": 1.5460122699386503,
+      "grad_norm": 0.6944937109947205,
+      "learning_rate": 0.0002,
+      "loss": 0.8605,
+      "step": 630
+    },
+    {
+      "epoch": 1.5705521472392638,
+      "grad_norm": 1.104216456413269,
+      "learning_rate": 0.0002,
+      "loss": 0.8922,
+      "step": 640
+    },
+    {
+      "epoch": 1.5950920245398774,
+      "grad_norm": 0.786145806312561,
+      "learning_rate": 0.0002,
+      "loss": 0.9264,
+      "step": 650
+    },
+    {
+      "epoch": 1.6196319018404908,
+      "grad_norm": 0.8748652935028076,
+      "learning_rate": 0.0002,
+      "loss": 0.8515,
+      "step": 660
+    },
+    {
+      "epoch": 1.6441717791411041,
+      "grad_norm": 0.6184355020523071,
+      "learning_rate": 0.0002,
+      "loss": 0.8818,
+      "step": 670
+    },
+    {
+      "epoch": 1.668711656441718,
+      "grad_norm": 1.0236579179763794,
+      "learning_rate": 0.0002,
+      "loss": 0.9484,
+      "step": 680
+    },
+    {
+      "epoch": 1.6932515337423313,
+      "grad_norm": 0.9815388917922974,
+      "learning_rate": 0.0002,
+      "loss": 0.9078,
+      "step": 690
+    },
+    {
+      "epoch": 1.7177914110429446,
+      "grad_norm": 0.6398373246192932,
+      "learning_rate": 0.0002,
+      "loss": 0.9036,
+      "step": 700
+    },
+    {
+      "epoch": 1.7423312883435584,
+      "grad_norm": 0.6157359480857849,
+      "learning_rate": 0.0002,
+      "loss": 0.8702,
+      "step": 710
+    },
+    {
+      "epoch": 1.7668711656441718,
+      "grad_norm": 0.7352675795555115,
+      "learning_rate": 0.0002,
+      "loss": 0.8712,
+      "step": 720
+    },
+    {
+      "epoch": 1.7914110429447851,
+      "grad_norm": 0.6840626001358032,
+      "learning_rate": 0.0002,
+      "loss": 0.8737,
+      "step": 730
+    },
+    {
+      "epoch": 1.8159509202453987,
+      "grad_norm": 0.8363635540008545,
+      "learning_rate": 0.0002,
+      "loss": 0.9113,
+      "step": 740
+    },
+    {
+      "epoch": 1.8404907975460123,
+      "grad_norm": 1.7173194885253906,
+      "learning_rate": 0.0002,
+      "loss": 0.973,
+      "step": 750
+    },
+    {
+      "epoch": 1.8650306748466257,
+      "grad_norm": 0.8277300000190735,
+      "learning_rate": 0.0002,
+      "loss": 0.9438,
+      "step": 760
+    },
+    {
+      "epoch": 1.8895705521472392,
+      "grad_norm": 0.6714297533035278,
+      "learning_rate": 0.0002,
+      "loss": 0.919,
+      "step": 770
+    },
+    {
+      "epoch": 1.9141104294478528,
+      "grad_norm": 0.7646296620368958,
+      "learning_rate": 0.0002,
+      "loss": 0.889,
+      "step": 780
+    },
+    {
+      "epoch": 1.9386503067484662,
+      "grad_norm": 0.9777507781982422,
+      "learning_rate": 0.0002,
+      "loss": 0.8433,
+      "step": 790
+    },
+    {
+      "epoch": 1.9631901840490797,
+      "grad_norm": 0.7732912302017212,
+      "learning_rate": 0.0002,
+      "loss": 0.9255,
+      "step": 800
+    },
+    {
+      "epoch": 1.9877300613496933,
+      "grad_norm": 0.7485368847846985,
+      "learning_rate": 0.0002,
+      "loss": 0.9556,
+      "step": 810
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.335689663887024,
+      "eval_runtime": 80.1256,
+      "eval_samples_per_second": 5.691,
+      "eval_steps_per_second": 0.711,
+      "step": 815
+    },
+    {
+      "epoch": 2.0122699386503067,
+      "grad_norm": 0.7061107158660889,
+      "learning_rate": 0.0002,
+      "loss": 0.9555,
+      "step": 820
+    },
+    {
+      "epoch": 2.03680981595092,
+      "grad_norm": 1.101184368133545,
+      "learning_rate": 0.0002,
+      "loss": 0.7363,
+      "step": 830
+    },
+    {
+      "epoch": 2.061349693251534,
+      "grad_norm": 0.6724491119384766,
+      "learning_rate": 0.0002,
+      "loss": 0.7269,
+      "step": 840
+    },
+    {
+      "epoch": 2.085889570552147,
+      "grad_norm": 0.8159838318824768,
+      "learning_rate": 0.0002,
+      "loss": 0.6836,
+      "step": 850
+    },
+    {
+      "epoch": 2.1104294478527605,
+      "grad_norm": 0.9286916255950928,
+      "learning_rate": 0.0002,
+      "loss": 0.7237,
+      "step": 860
+    },
+    {
+      "epoch": 2.1349693251533743,
+      "grad_norm": 0.9122375845909119,
+      "learning_rate": 0.0002,
+      "loss": 0.7258,
+      "step": 870
+    },
+    {
+      "epoch": 2.1595092024539877,
+      "grad_norm": 0.9655355215072632,
+      "learning_rate": 0.0002,
+      "loss": 0.7164,
+      "step": 880
+    },
+    {
+      "epoch": 2.184049079754601,
+      "grad_norm": 1.1539593935012817,
+      "learning_rate": 0.0002,
+      "loss": 0.741,
+      "step": 890
+    },
+    {
+      "epoch": 2.208588957055215,
+      "grad_norm": 1.0535199642181396,
+      "learning_rate": 0.0002,
+      "loss": 0.7368,
+      "step": 900
+    },
+    {
+      "epoch": 2.233128834355828,
+      "grad_norm": 1.2841371297836304,
+      "learning_rate": 0.0002,
+      "loss": 0.6615,
+      "step": 910
+    },
+    {
+      "epoch": 2.2576687116564416,
+      "grad_norm": 0.8669798970222473,
+      "learning_rate": 0.0002,
+      "loss": 0.767,
+      "step": 920
+    },
+    {
+      "epoch": 2.2822085889570554,
+      "grad_norm": 0.8732201457023621,
+      "learning_rate": 0.0002,
+      "loss": 0.7303,
+      "step": 930
+    },
+    {
+      "epoch": 2.3067484662576687,
+      "grad_norm": 0.9955021739006042,
+      "learning_rate": 0.0002,
+      "loss": 0.7205,
+      "step": 940
+    },
+    {
+      "epoch": 2.331288343558282,
+      "grad_norm": 1.0610932111740112,
+      "learning_rate": 0.0002,
+      "loss": 0.6781,
+      "step": 950
+    },
+    {
+      "epoch": 2.355828220858896,
+      "grad_norm": 0.9506151080131531,
+      "learning_rate": 0.0002,
+      "loss": 0.729,
+      "step": 960
+    },
+    {
+      "epoch": 2.3803680981595092,
+      "grad_norm": 1.259052038192749,
+      "learning_rate": 0.0002,
+      "loss": 0.6784,
+      "step": 970
+    },
+    {
+      "epoch": 2.4049079754601226,
+      "grad_norm": 1.1822247505187988,
+      "learning_rate": 0.0002,
+      "loss": 0.6668,
+      "step": 980
+    },
+    {
+      "epoch": 2.4294478527607364,
+      "grad_norm": 0.6920015811920166,
+      "learning_rate": 0.0002,
+      "loss": 0.6623,
+      "step": 990
+    },
+    {
+      "epoch": 2.4539877300613497,
+      "grad_norm": 0.7560105323791504,
+      "learning_rate": 0.0002,
+      "loss": 0.6802,
+      "step": 1000
+    },
+    {
+      "epoch": 2.478527607361963,
+      "grad_norm": 0.8672356009483337,
+      "learning_rate": 0.0002,
+      "loss": 0.7685,
+      "step": 1010
+    },
+    {
+      "epoch": 2.5030674846625764,
+      "grad_norm": 1.1074872016906738,
+      "learning_rate": 0.0002,
+      "loss": 0.7058,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5276073619631902,
+      "grad_norm": 1.1430137157440186,
+      "learning_rate": 0.0002,
+      "loss": 0.7746,
+      "step": 1030
+    },
+    {
+      "epoch": 2.5521472392638036,
+      "grad_norm": 1.6926707029342651,
+      "learning_rate": 0.0002,
+      "loss": 0.7444,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5766871165644174,
+      "grad_norm": 1.0170048475265503,
+      "learning_rate": 0.0002,
+      "loss": 0.7131,
+      "step": 1050
+    },
+    {
+      "epoch": 2.6012269938650308,
+      "grad_norm": 1.1462562084197998,
+      "learning_rate": 0.0002,
+      "loss": 0.7275,
+      "step": 1060
+    },
+    {
+      "epoch": 2.625766871165644,
+      "grad_norm": 1.127669334411621,
+      "learning_rate": 0.0002,
+      "loss": 0.6736,
+      "step": 1070
+    },
+    {
+      "epoch": 2.6503067484662575,
+      "grad_norm": 0.9649022221565247,
+      "learning_rate": 0.0002,
+      "loss": 0.7218,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6748466257668713,
+      "grad_norm": 0.9426548480987549,
+      "learning_rate": 0.0002,
+      "loss": 0.7651,
+      "step": 1090
+    },
+    {
+      "epoch": 2.6993865030674846,
+      "grad_norm": 1.1191051006317139,
+      "learning_rate": 0.0002,
+      "loss": 0.7829,
+      "step": 1100
+    },
+    {
+      "epoch": 2.7239263803680984,
+      "grad_norm": 2.229809284210205,
+      "learning_rate": 0.0002,
+      "loss": 0.7238,
+      "step": 1110
+    },
+    {
+      "epoch": 2.7484662576687118,
+      "grad_norm": 1.2478930950164795,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 1120
+    },
+    {
+      "epoch": 2.773006134969325,
+      "grad_norm": 0.9907709360122681,
+      "learning_rate": 0.0002,
+      "loss": 0.7194,
+      "step": 1130
+    },
+    {
+      "epoch": 2.7975460122699385,
+      "grad_norm": 1.1670643091201782,
+      "learning_rate": 0.0002,
+      "loss": 0.6531,
+      "step": 1140
+    },
+    {
+      "epoch": 2.8220858895705523,
+      "grad_norm": 1.1675913333892822,
+      "learning_rate": 0.0002,
+      "loss": 0.7076,
+      "step": 1150
+    },
+    {
+      "epoch": 2.8466257668711656,
+      "grad_norm": 0.9909353256225586,
+      "learning_rate": 0.0002,
+      "loss": 0.6915,
+      "step": 1160
+    },
+    {
+      "epoch": 2.871165644171779,
+      "grad_norm": 0.8759778141975403,
+      "learning_rate": 0.0002,
+      "loss": 0.7294,
+      "step": 1170
+    },
+    {
+      "epoch": 2.895705521472393,
+      "grad_norm": 0.8080666661262512,
+      "learning_rate": 0.0002,
+      "loss": 0.6451,
+      "step": 1180
+    },
+    {
+      "epoch": 2.920245398773006,
+      "grad_norm": 0.9743189811706543,
+      "learning_rate": 0.0002,
+      "loss": 0.7331,
+      "step": 1190
+    },
+    {
+      "epoch": 2.9447852760736195,
+      "grad_norm": 0.8573821187019348,
+      "learning_rate": 0.0002,
+      "loss": 0.7471,
+      "step": 1200
+    },
+    {
+      "epoch": 2.969325153374233,
+      "grad_norm": 1.0196788311004639,
+      "learning_rate": 0.0002,
+      "loss": 0.7396,
+      "step": 1210
+    },
+    {
+      "epoch": 2.9938650306748467,
+      "grad_norm": 0.8840402960777283,
+      "learning_rate": 0.0002,
+      "loss": 0.7288,
+      "step": 1220
+    },
+    {
+      "epoch": 2.9987730061349693,
+      "eval_loss": 1.4348119497299194,
+      "eval_runtime": 80.7583,
+      "eval_samples_per_second": 5.646,
+      "eval_steps_per_second": 0.706,
+      "step": 1222
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 3256,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.3636853399552e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1222/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0e586c733c860e8234b028525d14453b4a2a52ad2c815f3775cbbbef35878fb
+size 5624

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bae1c53ba1462780acdb93d2a60c6abd0cfbc0157a080916a2e4121129b7d003
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9aaaf9208b8ee8828f2ef2d1eb396ea5dc14d5fa5bfc02759a0a15d9ec4ac3b
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20d5d7f3cd1f1e2df2c979a0255d46f1f4d1561d1d1012f4d8c7001cf8c51381
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6325287366e061d5bbdcfa31d6e731688ff21db98e00579126330afff60ececf
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/trainer_state.json

@@ -0,0 +1,1206 @@
+{
+  "best_metric": 1.3188884258270264,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-407",
+  "epoch": 4.0,
+  "eval_steps": 10,
+  "global_step": 1630,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.024539877300613498,
+      "grad_norm": 1.0104172229766846,
+      "learning_rate": 0.0002,
+      "loss": 1.7717,
+      "step": 10
+    },
+    {
+      "epoch": 0.049079754601226995,
+      "grad_norm": 0.8800041079521179,
+      "learning_rate": 0.0002,
+      "loss": 1.6041,
+      "step": 20
+    },
+    {
+      "epoch": 0.0736196319018405,
+      "grad_norm": 1.337620496749878,
+      "learning_rate": 0.0002,
+      "loss": 1.4421,
+      "step": 30
+    },
+    {
+      "epoch": 0.09815950920245399,
+      "grad_norm": 0.6757020354270935,
+      "learning_rate": 0.0002,
+      "loss": 1.5542,
+      "step": 40
+    },
+    {
+      "epoch": 0.12269938650306748,
+      "grad_norm": 42.27006149291992,
+      "learning_rate": 0.0002,
+      "loss": 1.4365,
+      "step": 50
+    },
+    {
+      "epoch": 0.147239263803681,
+      "grad_norm": 0.8157640099525452,
+      "learning_rate": 0.0002,
+      "loss": 1.1226,
+      "step": 60
+    },
+    {
+      "epoch": 0.17177914110429449,
+      "grad_norm": 1.0711787939071655,
+      "learning_rate": 0.0002,
+      "loss": 1.1886,
+      "step": 70
+    },
+    {
+      "epoch": 0.19631901840490798,
+      "grad_norm": 0.5880025029182434,
+      "learning_rate": 0.0002,
+      "loss": 1.1239,
+      "step": 80
+    },
+    {
+      "epoch": 0.22085889570552147,
+      "grad_norm": 0.827833354473114,
+      "learning_rate": 0.0002,
+      "loss": 1.0696,
+      "step": 90
+    },
+    {
+      "epoch": 0.24539877300613497,
+      "grad_norm": 0.579414963722229,
+      "learning_rate": 0.0002,
+      "loss": 1.0468,
+      "step": 100
+    },
+    {
+      "epoch": 0.26993865030674846,
+      "grad_norm": 0.6620142459869385,
+      "learning_rate": 0.0002,
+      "loss": 1.0914,
+      "step": 110
+    },
+    {
+      "epoch": 0.294478527607362,
+      "grad_norm": 0.5601297616958618,
+      "learning_rate": 0.0002,
+      "loss": 1.0873,
+      "step": 120
+    },
+    {
+      "epoch": 0.31901840490797545,
+      "grad_norm": 0.5814566612243652,
+      "learning_rate": 0.0002,
+      "loss": 1.1652,
+      "step": 130
+    },
+    {
+      "epoch": 0.34355828220858897,
+      "grad_norm": 0.6171417236328125,
+      "learning_rate": 0.0002,
+      "loss": 1.0915,
+      "step": 140
+    },
+    {
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.6176294088363647,
+      "learning_rate": 0.0002,
+      "loss": 1.1572,
+      "step": 150
+    },
+    {
+      "epoch": 0.39263803680981596,
+      "grad_norm": 0.8398241400718689,
+      "learning_rate": 0.0002,
+      "loss": 1.1135,
+      "step": 160
+    },
+    {
+      "epoch": 0.4171779141104294,
+      "grad_norm": 0.5397658944129944,
+      "learning_rate": 0.0002,
+      "loss": 1.1235,
+      "step": 170
+    },
+    {
+      "epoch": 0.44171779141104295,
+      "grad_norm": 0.7487576603889465,
+      "learning_rate": 0.0002,
+      "loss": 1.1438,
+      "step": 180
+    },
+    {
+      "epoch": 0.4662576687116564,
+      "grad_norm": 0.4767085015773773,
+      "learning_rate": 0.0002,
+      "loss": 1.1178,
+      "step": 190
+    },
+    {
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.6086363196372986,
+      "learning_rate": 0.0002,
+      "loss": 1.1222,
+      "step": 200
+    },
+    {
+      "epoch": 0.5153374233128835,
+      "grad_norm": 0.6940106749534607,
+      "learning_rate": 0.0002,
+      "loss": 1.0346,
+      "step": 210
+    },
+    {
+      "epoch": 0.5398773006134969,
+      "grad_norm": 0.775067150592804,
+      "learning_rate": 0.0002,
+      "loss": 1.1162,
+      "step": 220
+    },
+    {
+      "epoch": 0.5644171779141104,
+      "grad_norm": 0.7273200154304504,
+      "learning_rate": 0.0002,
+      "loss": 1.1335,
+      "step": 230
+    },
+    {
+      "epoch": 0.588957055214724,
+      "grad_norm": 0.5168078541755676,
+      "learning_rate": 0.0002,
+      "loss": 1.0606,
+      "step": 240
+    },
+    {
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6487783193588257,
+      "learning_rate": 0.0002,
+      "loss": 1.0792,
+      "step": 250
+    },
+    {
+      "epoch": 0.6380368098159509,
+      "grad_norm": 1.0914227962493896,
+      "learning_rate": 0.0002,
+      "loss": 1.1642,
+      "step": 260
+    },
+    {
+      "epoch": 0.6625766871165644,
+      "grad_norm": 1.0182702541351318,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 270
+    },
+    {
+      "epoch": 0.6871165644171779,
+      "grad_norm": 0.7045943140983582,
+      "learning_rate": 0.0002,
+      "loss": 0.9816,
+      "step": 280
+    },
+    {
+      "epoch": 0.7116564417177914,
+      "grad_norm": 0.6228044033050537,
+      "learning_rate": 0.0002,
+      "loss": 0.9675,
+      "step": 290
+    },
+    {
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.6161119341850281,
+      "learning_rate": 0.0002,
+      "loss": 1.076,
+      "step": 300
+    },
+    {
+      "epoch": 0.7607361963190185,
+      "grad_norm": 0.8310935497283936,
+      "learning_rate": 0.0002,
+      "loss": 1.0597,
+      "step": 310
+    },
+    {
+      "epoch": 0.7852760736196319,
+      "grad_norm": 0.6500546336174011,
+      "learning_rate": 0.0002,
+      "loss": 1.0817,
+      "step": 320
+    },
+    {
+      "epoch": 0.8098159509202454,
+      "grad_norm": 0.9254736304283142,
+      "learning_rate": 0.0002,
+      "loss": 1.109,
+      "step": 330
+    },
+    {
+      "epoch": 0.8343558282208589,
+      "grad_norm": 0.5892964005470276,
+      "learning_rate": 0.0002,
+      "loss": 0.9929,
+      "step": 340
+    },
+    {
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.5500181913375854,
+      "learning_rate": 0.0002,
+      "loss": 1.0637,
+      "step": 350
+    },
+    {
+      "epoch": 0.8834355828220859,
+      "grad_norm": 0.7814139127731323,
+      "learning_rate": 0.0002,
+      "loss": 1.0693,
+      "step": 360
+    },
+    {
+      "epoch": 0.9079754601226994,
+      "grad_norm": 0.6288005113601685,
+      "learning_rate": 0.0002,
+      "loss": 1.059,
+      "step": 370
+    },
+    {
+      "epoch": 0.9325153374233128,
+      "grad_norm": 0.5031183958053589,
+      "learning_rate": 0.0002,
+      "loss": 0.9904,
+      "step": 380
+    },
+    {
+      "epoch": 0.9570552147239264,
+      "grad_norm": 0.8656964898109436,
+      "learning_rate": 0.0002,
+      "loss": 1.1183,
+      "step": 390
+    },
+    {
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6746202111244202,
+      "learning_rate": 0.0002,
+      "loss": 1.0296,
+      "step": 400
+    },
+    {
+      "epoch": 0.9987730061349693,
+      "eval_loss": 1.3188884258270264,
+      "eval_runtime": 124.2158,
+      "eval_samples_per_second": 3.671,
+      "eval_steps_per_second": 0.459,
+      "step": 407
+    },
+    {
+      "epoch": 1.0061349693251533,
+      "grad_norm": 0.5244082808494568,
+      "learning_rate": 0.0002,
+      "loss": 0.9679,
+      "step": 410
+    },
+    {
+      "epoch": 1.030674846625767,
+      "grad_norm": 0.6792967319488525,
+      "learning_rate": 0.0002,
+      "loss": 0.97,
+      "step": 420
+    },
+    {
+      "epoch": 1.0552147239263803,
+      "grad_norm": 0.6114474534988403,
+      "learning_rate": 0.0002,
+      "loss": 0.8935,
+      "step": 430
+    },
+    {
+      "epoch": 1.0797546012269938,
+      "grad_norm": 0.6367565393447876,
+      "learning_rate": 0.0002,
+      "loss": 0.9469,
+      "step": 440
+    },
+    {
+      "epoch": 1.1042944785276074,
+      "grad_norm": 0.7007757425308228,
+      "learning_rate": 0.0002,
+      "loss": 0.9536,
+      "step": 450
+    },
+    {
+      "epoch": 1.1288343558282208,
+      "grad_norm": 0.9737136960029602,
+      "learning_rate": 0.0002,
+      "loss": 0.8806,
+      "step": 460
+    },
+    {
+      "epoch": 1.1533742331288344,
+      "grad_norm": 1.1624044179916382,
+      "learning_rate": 0.0002,
+      "loss": 0.9217,
+      "step": 470
+    },
+    {
+      "epoch": 1.177914110429448,
+      "grad_norm": 2.1017873287200928,
+      "learning_rate": 0.0002,
+      "loss": 0.9056,
+      "step": 480
+    },
+    {
+      "epoch": 1.2024539877300613,
+      "grad_norm": 0.7352198958396912,
+      "learning_rate": 0.0002,
+      "loss": 0.9945,
+      "step": 490
+    },
+    {
+      "epoch": 1.2269938650306749,
+      "grad_norm": 0.9491982460021973,
+      "learning_rate": 0.0002,
+      "loss": 0.9224,
+      "step": 500
+    },
+    {
+      "epoch": 1.2515337423312882,
+      "grad_norm": 1.0124034881591797,
+      "learning_rate": 0.0002,
+      "loss": 0.9079,
+      "step": 510
+    },
+    {
+      "epoch": 1.2760736196319018,
+      "grad_norm": 0.7620294094085693,
+      "learning_rate": 0.0002,
+      "loss": 0.9596,
+      "step": 520
+    },
+    {
+      "epoch": 1.3006134969325154,
+      "grad_norm": 0.7207106351852417,
+      "learning_rate": 0.0002,
+      "loss": 0.9095,
+      "step": 530
+    },
+    {
+      "epoch": 1.3251533742331287,
+      "grad_norm": 0.586559534072876,
+      "learning_rate": 0.0002,
+      "loss": 0.8588,
+      "step": 540
+    },
+    {
+      "epoch": 1.3496932515337423,
+      "grad_norm": 0.7113742828369141,
+      "learning_rate": 0.0002,
+      "loss": 0.9146,
+      "step": 550
+    },
+    {
+      "epoch": 1.3742331288343559,
+      "grad_norm": 0.9574087262153625,
+      "learning_rate": 0.0002,
+      "loss": 0.9504,
+      "step": 560
+    },
+    {
+      "epoch": 1.3987730061349692,
+      "grad_norm": 0.71551913022995,
+      "learning_rate": 0.0002,
+      "loss": 0.9124,
+      "step": 570
+    },
+    {
+      "epoch": 1.4233128834355828,
+      "grad_norm": 1.0000630617141724,
+      "learning_rate": 0.0002,
+      "loss": 0.8944,
+      "step": 580
+    },
+    {
+      "epoch": 1.4478527607361964,
+      "grad_norm": 0.6878040432929993,
+      "learning_rate": 0.0002,
+      "loss": 0.8707,
+      "step": 590
+    },
+    {
+      "epoch": 1.4723926380368098,
+      "grad_norm": 0.770113468170166,
+      "learning_rate": 0.0002,
+      "loss": 0.953,
+      "step": 600
+    },
+    {
+      "epoch": 1.4969325153374233,
+      "grad_norm": 0.7941587567329407,
+      "learning_rate": 0.0002,
+      "loss": 0.9293,
+      "step": 610
+    },
+    {
+      "epoch": 1.521472392638037,
+      "grad_norm": 0.971196711063385,
+      "learning_rate": 0.0002,
+      "loss": 0.9177,
+      "step": 620
+    },
+    {
+      "epoch": 1.5460122699386503,
+      "grad_norm": 0.6944937109947205,
+      "learning_rate": 0.0002,
+      "loss": 0.8605,
+      "step": 630
+    },
+    {
+      "epoch": 1.5705521472392638,
+      "grad_norm": 1.104216456413269,
+      "learning_rate": 0.0002,
+      "loss": 0.8922,
+      "step": 640
+    },
+    {
+      "epoch": 1.5950920245398774,
+      "grad_norm": 0.786145806312561,
+      "learning_rate": 0.0002,
+      "loss": 0.9264,
+      "step": 650
+    },
+    {
+      "epoch": 1.6196319018404908,
+      "grad_norm": 0.8748652935028076,
+      "learning_rate": 0.0002,
+      "loss": 0.8515,
+      "step": 660
+    },
+    {
+      "epoch": 1.6441717791411041,
+      "grad_norm": 0.6184355020523071,
+      "learning_rate": 0.0002,
+      "loss": 0.8818,
+      "step": 670
+    },
+    {
+      "epoch": 1.668711656441718,
+      "grad_norm": 1.0236579179763794,
+      "learning_rate": 0.0002,
+      "loss": 0.9484,
+      "step": 680
+    },
+    {
+      "epoch": 1.6932515337423313,
+      "grad_norm": 0.9815388917922974,
+      "learning_rate": 0.0002,
+      "loss": 0.9078,
+      "step": 690
+    },
+    {
+      "epoch": 1.7177914110429446,
+      "grad_norm": 0.6398373246192932,
+      "learning_rate": 0.0002,
+      "loss": 0.9036,
+      "step": 700
+    },
+    {
+      "epoch": 1.7423312883435584,
+      "grad_norm": 0.6157359480857849,
+      "learning_rate": 0.0002,
+      "loss": 0.8702,
+      "step": 710
+    },
+    {
+      "epoch": 1.7668711656441718,
+      "grad_norm": 0.7352675795555115,
+      "learning_rate": 0.0002,
+      "loss": 0.8712,
+      "step": 720
+    },
+    {
+      "epoch": 1.7914110429447851,
+      "grad_norm": 0.6840626001358032,
+      "learning_rate": 0.0002,
+      "loss": 0.8737,
+      "step": 730
+    },
+    {
+      "epoch": 1.8159509202453987,
+      "grad_norm": 0.8363635540008545,
+      "learning_rate": 0.0002,
+      "loss": 0.9113,
+      "step": 740
+    },
+    {
+      "epoch": 1.8404907975460123,
+      "grad_norm": 1.7173194885253906,
+      "learning_rate": 0.0002,
+      "loss": 0.973,
+      "step": 750
+    },
+    {
+      "epoch": 1.8650306748466257,
+      "grad_norm": 0.8277300000190735,
+      "learning_rate": 0.0002,
+      "loss": 0.9438,
+      "step": 760
+    },
+    {
+      "epoch": 1.8895705521472392,
+      "grad_norm": 0.6714297533035278,
+      "learning_rate": 0.0002,
+      "loss": 0.919,
+      "step": 770
+    },
+    {
+      "epoch": 1.9141104294478528,
+      "grad_norm": 0.7646296620368958,
+      "learning_rate": 0.0002,
+      "loss": 0.889,
+      "step": 780
+    },
+    {
+      "epoch": 1.9386503067484662,
+      "grad_norm": 0.9777507781982422,
+      "learning_rate": 0.0002,
+      "loss": 0.8433,
+      "step": 790
+    },
+    {
+      "epoch": 1.9631901840490797,
+      "grad_norm": 0.7732912302017212,
+      "learning_rate": 0.0002,
+      "loss": 0.9255,
+      "step": 800
+    },
+    {
+      "epoch": 1.9877300613496933,
+      "grad_norm": 0.7485368847846985,
+      "learning_rate": 0.0002,
+      "loss": 0.9556,
+      "step": 810
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.335689663887024,
+      "eval_runtime": 80.1256,
+      "eval_samples_per_second": 5.691,
+      "eval_steps_per_second": 0.711,
+      "step": 815
+    },
+    {
+      "epoch": 2.0122699386503067,
+      "grad_norm": 0.7061107158660889,
+      "learning_rate": 0.0002,
+      "loss": 0.9555,
+      "step": 820
+    },
+    {
+      "epoch": 2.03680981595092,
+      "grad_norm": 1.101184368133545,
+      "learning_rate": 0.0002,
+      "loss": 0.7363,
+      "step": 830
+    },
+    {
+      "epoch": 2.061349693251534,
+      "grad_norm": 0.6724491119384766,
+      "learning_rate": 0.0002,
+      "loss": 0.7269,
+      "step": 840
+    },
+    {
+      "epoch": 2.085889570552147,
+      "grad_norm": 0.8159838318824768,
+      "learning_rate": 0.0002,
+      "loss": 0.6836,
+      "step": 850
+    },
+    {
+      "epoch": 2.1104294478527605,
+      "grad_norm": 0.9286916255950928,
+      "learning_rate": 0.0002,
+      "loss": 0.7237,
+      "step": 860
+    },
+    {
+      "epoch": 2.1349693251533743,
+      "grad_norm": 0.9122375845909119,
+      "learning_rate": 0.0002,
+      "loss": 0.7258,
+      "step": 870
+    },
+    {
+      "epoch": 2.1595092024539877,
+      "grad_norm": 0.9655355215072632,
+      "learning_rate": 0.0002,
+      "loss": 0.7164,
+      "step": 880
+    },
+    {
+      "epoch": 2.184049079754601,
+      "grad_norm": 1.1539593935012817,
+      "learning_rate": 0.0002,
+      "loss": 0.741,
+      "step": 890
+    },
+    {
+      "epoch": 2.208588957055215,
+      "grad_norm": 1.0535199642181396,
+      "learning_rate": 0.0002,
+      "loss": 0.7368,
+      "step": 900
+    },
+    {
+      "epoch": 2.233128834355828,
+      "grad_norm": 1.2841371297836304,
+      "learning_rate": 0.0002,
+      "loss": 0.6615,
+      "step": 910
+    },
+    {
+      "epoch": 2.2576687116564416,
+      "grad_norm": 0.8669798970222473,
+      "learning_rate": 0.0002,
+      "loss": 0.767,
+      "step": 920
+    },
+    {
+      "epoch": 2.2822085889570554,
+      "grad_norm": 0.8732201457023621,
+      "learning_rate": 0.0002,
+      "loss": 0.7303,
+      "step": 930
+    },
+    {
+      "epoch": 2.3067484662576687,
+      "grad_norm": 0.9955021739006042,
+      "learning_rate": 0.0002,
+      "loss": 0.7205,
+      "step": 940
+    },
+    {
+      "epoch": 2.331288343558282,
+      "grad_norm": 1.0610932111740112,
+      "learning_rate": 0.0002,
+      "loss": 0.6781,
+      "step": 950
+    },
+    {
+      "epoch": 2.355828220858896,
+      "grad_norm": 0.9506151080131531,
+      "learning_rate": 0.0002,
+      "loss": 0.729,
+      "step": 960
+    },
+    {
+      "epoch": 2.3803680981595092,
+      "grad_norm": 1.259052038192749,
+      "learning_rate": 0.0002,
+      "loss": 0.6784,
+      "step": 970
+    },
+    {
+      "epoch": 2.4049079754601226,
+      "grad_norm": 1.1822247505187988,
+      "learning_rate": 0.0002,
+      "loss": 0.6668,
+      "step": 980
+    },
+    {
+      "epoch": 2.4294478527607364,
+      "grad_norm": 0.6920015811920166,
+      "learning_rate": 0.0002,
+      "loss": 0.6623,
+      "step": 990
+    },
+    {
+      "epoch": 2.4539877300613497,
+      "grad_norm": 0.7560105323791504,
+      "learning_rate": 0.0002,
+      "loss": 0.6802,
+      "step": 1000
+    },
+    {
+      "epoch": 2.478527607361963,
+      "grad_norm": 0.8672356009483337,
+      "learning_rate": 0.0002,
+      "loss": 0.7685,
+      "step": 1010
+    },
+    {
+      "epoch": 2.5030674846625764,
+      "grad_norm": 1.1074872016906738,
+      "learning_rate": 0.0002,
+      "loss": 0.7058,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5276073619631902,
+      "grad_norm": 1.1430137157440186,
+      "learning_rate": 0.0002,
+      "loss": 0.7746,
+      "step": 1030
+    },
+    {
+      "epoch": 2.5521472392638036,
+      "grad_norm": 1.6926707029342651,
+      "learning_rate": 0.0002,
+      "loss": 0.7444,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5766871165644174,
+      "grad_norm": 1.0170048475265503,
+      "learning_rate": 0.0002,
+      "loss": 0.7131,
+      "step": 1050
+    },
+    {
+      "epoch": 2.6012269938650308,
+      "grad_norm": 1.1462562084197998,
+      "learning_rate": 0.0002,
+      "loss": 0.7275,
+      "step": 1060
+    },
+    {
+      "epoch": 2.625766871165644,
+      "grad_norm": 1.127669334411621,
+      "learning_rate": 0.0002,
+      "loss": 0.6736,
+      "step": 1070
+    },
+    {
+      "epoch": 2.6503067484662575,
+      "grad_norm": 0.9649022221565247,
+      "learning_rate": 0.0002,
+      "loss": 0.7218,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6748466257668713,
+      "grad_norm": 0.9426548480987549,
+      "learning_rate": 0.0002,
+      "loss": 0.7651,
+      "step": 1090
+    },
+    {
+      "epoch": 2.6993865030674846,
+      "grad_norm": 1.1191051006317139,
+      "learning_rate": 0.0002,
+      "loss": 0.7829,
+      "step": 1100
+    },
+    {
+      "epoch": 2.7239263803680984,
+      "grad_norm": 2.229809284210205,
+      "learning_rate": 0.0002,
+      "loss": 0.7238,
+      "step": 1110
+    },
+    {
+      "epoch": 2.7484662576687118,
+      "grad_norm": 1.2478930950164795,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 1120
+    },
+    {
+      "epoch": 2.773006134969325,
+      "grad_norm": 0.9907709360122681,
+      "learning_rate": 0.0002,
+      "loss": 0.7194,
+      "step": 1130
+    },
+    {
+      "epoch": 2.7975460122699385,
+      "grad_norm": 1.1670643091201782,
+      "learning_rate": 0.0002,
+      "loss": 0.6531,
+      "step": 1140
+    },
+    {
+      "epoch": 2.8220858895705523,
+      "grad_norm": 1.1675913333892822,
+      "learning_rate": 0.0002,
+      "loss": 0.7076,
+      "step": 1150
+    },
+    {
+      "epoch": 2.8466257668711656,
+      "grad_norm": 0.9909353256225586,
+      "learning_rate": 0.0002,
+      "loss": 0.6915,
+      "step": 1160
+    },
+    {
+      "epoch": 2.871165644171779,
+      "grad_norm": 0.8759778141975403,
+      "learning_rate": 0.0002,
+      "loss": 0.7294,
+      "step": 1170
+    },
+    {
+      "epoch": 2.895705521472393,
+      "grad_norm": 0.8080666661262512,
+      "learning_rate": 0.0002,
+      "loss": 0.6451,
+      "step": 1180
+    },
+    {
+      "epoch": 2.920245398773006,
+      "grad_norm": 0.9743189811706543,
+      "learning_rate": 0.0002,
+      "loss": 0.7331,
+      "step": 1190
+    },
+    {
+      "epoch": 2.9447852760736195,
+      "grad_norm": 0.8573821187019348,
+      "learning_rate": 0.0002,
+      "loss": 0.7471,
+      "step": 1200
+    },
+    {
+      "epoch": 2.969325153374233,
+      "grad_norm": 1.0196788311004639,
+      "learning_rate": 0.0002,
+      "loss": 0.7396,
+      "step": 1210
+    },
+    {
+      "epoch": 2.9938650306748467,
+      "grad_norm": 0.8840402960777283,
+      "learning_rate": 0.0002,
+      "loss": 0.7288,
+      "step": 1220
+    },
+    {
+      "epoch": 2.9987730061349693,
+      "eval_loss": 1.4348119497299194,
+      "eval_runtime": 80.7583,
+      "eval_samples_per_second": 5.646,
+      "eval_steps_per_second": 0.706,
+      "step": 1222
+    },
+    {
+      "epoch": 3.01840490797546,
+      "grad_norm": 1.2376960515975952,
+      "learning_rate": 0.0002,
+      "loss": 0.5671,
+      "step": 1230
+    },
+    {
+      "epoch": 3.042944785276074,
+      "grad_norm": 0.9861388206481934,
+      "learning_rate": 0.0002,
+      "loss": 0.5176,
+      "step": 1240
+    },
+    {
+      "epoch": 3.067484662576687,
+      "grad_norm": 1.2193198204040527,
+      "learning_rate": 0.0002,
+      "loss": 0.5623,
+      "step": 1250
+    },
+    {
+      "epoch": 3.0920245398773005,
+      "grad_norm": 1.1927645206451416,
+      "learning_rate": 0.0002,
+      "loss": 0.5727,
+      "step": 1260
+    },
+    {
+      "epoch": 3.116564417177914,
+      "grad_norm": 1.0420559644699097,
+      "learning_rate": 0.0002,
+      "loss": 0.5296,
+      "step": 1270
+    },
+    {
+      "epoch": 3.1411042944785277,
+      "grad_norm": 1.014664649963379,
+      "learning_rate": 0.0002,
+      "loss": 0.5322,
+      "step": 1280
+    },
+    {
+      "epoch": 3.165644171779141,
+      "grad_norm": 1.3103076219558716,
+      "learning_rate": 0.0002,
+      "loss": 0.5519,
+      "step": 1290
+    },
+    {
+      "epoch": 3.190184049079755,
+      "grad_norm": 1.2735213041305542,
+      "learning_rate": 0.0002,
+      "loss": 0.5133,
+      "step": 1300
+    },
+    {
+      "epoch": 3.214723926380368,
+      "grad_norm": 1.147608995437622,
+      "learning_rate": 0.0002,
+      "loss": 0.514,
+      "step": 1310
+    },
+    {
+      "epoch": 3.2392638036809815,
+      "grad_norm": 0.922386884689331,
+      "learning_rate": 0.0002,
+      "loss": 0.58,
+      "step": 1320
+    },
+    {
+      "epoch": 3.263803680981595,
+      "grad_norm": 1.1271566152572632,
+      "learning_rate": 0.0002,
+      "loss": 0.5425,
+      "step": 1330
+    },
+    {
+      "epoch": 3.2883435582822087,
+      "grad_norm": 1.2994354963302612,
+      "learning_rate": 0.0002,
+      "loss": 0.519,
+      "step": 1340
+    },
+    {
+      "epoch": 3.312883435582822,
+      "grad_norm": 1.0001686811447144,
+      "learning_rate": 0.0002,
+      "loss": 0.5521,
+      "step": 1350
+    },
+    {
+      "epoch": 3.3374233128834354,
+      "grad_norm": 1.6737695932388306,
+      "learning_rate": 0.0002,
+      "loss": 0.564,
+      "step": 1360
+    },
+    {
+      "epoch": 3.361963190184049,
+      "grad_norm": 1.1842162609100342,
+      "learning_rate": 0.0002,
+      "loss": 0.5529,
+      "step": 1370
+    },
+    {
+      "epoch": 3.3865030674846626,
+      "grad_norm": 1.1873128414154053,
+      "learning_rate": 0.0002,
+      "loss": 0.569,
+      "step": 1380
+    },
+    {
+      "epoch": 3.411042944785276,
+      "grad_norm": 1.4505162239074707,
+      "learning_rate": 0.0002,
+      "loss": 0.5199,
+      "step": 1390
+    },
+    {
+      "epoch": 3.4355828220858897,
+      "grad_norm": 1.254621982574463,
+      "learning_rate": 0.0002,
+      "loss": 0.5536,
+      "step": 1400
+    },
+    {
+      "epoch": 3.460122699386503,
+      "grad_norm": 1.5493544340133667,
+      "learning_rate": 0.0002,
+      "loss": 0.5547,
+      "step": 1410
+    },
+    {
+      "epoch": 3.4846625766871164,
+      "grad_norm": 1.4435759782791138,
+      "learning_rate": 0.0002,
+      "loss": 0.542,
+      "step": 1420
+    },
+    {
+      "epoch": 3.5092024539877302,
+      "grad_norm": 1.5885447263717651,
+      "learning_rate": 0.0002,
+      "loss": 0.6124,
+      "step": 1430
+    },
+    {
+      "epoch": 3.5337423312883436,
+      "grad_norm": 1.3357222080230713,
+      "learning_rate": 0.0002,
+      "loss": 0.5737,
+      "step": 1440
+    },
+    {
+      "epoch": 3.558282208588957,
+      "grad_norm": 1.8510551452636719,
+      "learning_rate": 0.0002,
+      "loss": 0.5903,
+      "step": 1450
+    },
+    {
+      "epoch": 3.5828220858895703,
+      "grad_norm": 1.1081656217575073,
+      "learning_rate": 0.0002,
+      "loss": 0.5214,
+      "step": 1460
+    },
+    {
+      "epoch": 3.607361963190184,
+      "grad_norm": 1.2849210500717163,
+      "learning_rate": 0.0002,
+      "loss": 0.5351,
+      "step": 1470
+    },
+    {
+      "epoch": 3.6319018404907975,
+      "grad_norm": 1.2472909688949585,
+      "learning_rate": 0.0002,
+      "loss": 0.5364,
+      "step": 1480
+    },
+    {
+      "epoch": 3.6564417177914113,
+      "grad_norm": 1.1466434001922607,
+      "learning_rate": 0.0002,
+      "loss": 0.5371,
+      "step": 1490
+    },
+    {
+      "epoch": 3.6809815950920246,
+      "grad_norm": 1.325461506843567,
+      "learning_rate": 0.0002,
+      "loss": 0.565,
+      "step": 1500
+    },
+    {
+      "epoch": 3.705521472392638,
+      "grad_norm": 1.4214563369750977,
+      "learning_rate": 0.0002,
+      "loss": 0.6154,
+      "step": 1510
+    },
+    {
+      "epoch": 3.7300613496932513,
+      "grad_norm": 1.1496703624725342,
+      "learning_rate": 0.0002,
+      "loss": 0.5726,
+      "step": 1520
+    },
+    {
+      "epoch": 3.754601226993865,
+      "grad_norm": 1.205261468887329,
+      "learning_rate": 0.0002,
+      "loss": 0.5687,
+      "step": 1530
+    },
+    {
+      "epoch": 3.7791411042944785,
+      "grad_norm": 1.6516913175582886,
+      "learning_rate": 0.0002,
+      "loss": 0.5593,
+      "step": 1540
+    },
+    {
+      "epoch": 3.8036809815950923,
+      "grad_norm": 1.7035053968429565,
+      "learning_rate": 0.0002,
+      "loss": 0.5767,
+      "step": 1550
+    },
+    {
+      "epoch": 3.8282208588957056,
+      "grad_norm": 1.463699460029602,
+      "learning_rate": 0.0002,
+      "loss": 0.5948,
+      "step": 1560
+    },
+    {
+      "epoch": 3.852760736196319,
+      "grad_norm": 1.7970601320266724,
+      "learning_rate": 0.0002,
+      "loss": 0.5326,
+      "step": 1570
+    },
+    {
+      "epoch": 3.8773006134969323,
+      "grad_norm": 1.2967402935028076,
+      "learning_rate": 0.0002,
+      "loss": 0.5919,
+      "step": 1580
+    },
+    {
+      "epoch": 3.901840490797546,
+      "grad_norm": 1.0639182329177856,
+      "learning_rate": 0.0002,
+      "loss": 0.5502,
+      "step": 1590
+    },
+    {
+      "epoch": 3.9263803680981595,
+      "grad_norm": 1.4388368129730225,
+      "learning_rate": 0.0002,
+      "loss": 0.5281,
+      "step": 1600
+    },
+    {
+      "epoch": 3.950920245398773,
+      "grad_norm": 1.2425099611282349,
+      "learning_rate": 0.0002,
+      "loss": 0.6305,
+      "step": 1610
+    },
+    {
+      "epoch": 3.9754601226993866,
+      "grad_norm": 1.3989970684051514,
+      "learning_rate": 0.0002,
+      "loss": 0.5548,
+      "step": 1620
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.9850553274154663,
+      "learning_rate": 0.0002,
+      "loss": 0.5661,
+      "step": 1630
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.5340977907180786,
+      "eval_runtime": 79.7233,
+      "eval_samples_per_second": 5.72,
+      "eval_steps_per_second": 0.715,
+      "step": 1630
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 3256,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.1515804532736e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-1630/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0e586c733c860e8234b028525d14453b4a2a52ad2c815f3775cbbbef35878fb
+size 5624

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84fb720fe1b2949bab1c77875553a8b4113c41aadf21a58eba32891e4c934e7e
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fca324ccfd1000d178f88ec3c67a9d8fca7a508effb3a8e59b728e9dbbe0751
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6eae64b0137634cf7c08ef754094f7efde2030ee6f7b894972e6e9d8c43d734b
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:587a5e5a97615ccb59addd37c4b5047258f7e6fb8e0a0f3019f257ace6f89800
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/trainer_state.json

@@ -0,0 +1,1494 @@
+{
+  "best_metric": 1.3188884258270264,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-407",
+  "epoch": 4.99877300613497,
+  "eval_steps": 10,
+  "global_step": 2037,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.024539877300613498,
+      "grad_norm": 1.0104172229766846,
+      "learning_rate": 0.0002,
+      "loss": 1.7717,
+      "step": 10
+    },
+    {
+      "epoch": 0.049079754601226995,
+      "grad_norm": 0.8800041079521179,
+      "learning_rate": 0.0002,
+      "loss": 1.6041,
+      "step": 20
+    },
+    {
+      "epoch": 0.0736196319018405,
+      "grad_norm": 1.337620496749878,
+      "learning_rate": 0.0002,
+      "loss": 1.4421,
+      "step": 30
+    },
+    {
+      "epoch": 0.09815950920245399,
+      "grad_norm": 0.6757020354270935,
+      "learning_rate": 0.0002,
+      "loss": 1.5542,
+      "step": 40
+    },
+    {
+      "epoch": 0.12269938650306748,
+      "grad_norm": 42.27006149291992,
+      "learning_rate": 0.0002,
+      "loss": 1.4365,
+      "step": 50
+    },
+    {
+      "epoch": 0.147239263803681,
+      "grad_norm": 0.8157640099525452,
+      "learning_rate": 0.0002,
+      "loss": 1.1226,
+      "step": 60
+    },
+    {
+      "epoch": 0.17177914110429449,
+      "grad_norm": 1.0711787939071655,
+      "learning_rate": 0.0002,
+      "loss": 1.1886,
+      "step": 70
+    },
+    {
+      "epoch": 0.19631901840490798,
+      "grad_norm": 0.5880025029182434,
+      "learning_rate": 0.0002,
+      "loss": 1.1239,
+      "step": 80
+    },
+    {
+      "epoch": 0.22085889570552147,
+      "grad_norm": 0.827833354473114,
+      "learning_rate": 0.0002,
+      "loss": 1.0696,
+      "step": 90
+    },
+    {
+      "epoch": 0.24539877300613497,
+      "grad_norm": 0.579414963722229,
+      "learning_rate": 0.0002,
+      "loss": 1.0468,
+      "step": 100
+    },
+    {
+      "epoch": 0.26993865030674846,
+      "grad_norm": 0.6620142459869385,
+      "learning_rate": 0.0002,
+      "loss": 1.0914,
+      "step": 110
+    },
+    {
+      "epoch": 0.294478527607362,
+      "grad_norm": 0.5601297616958618,
+      "learning_rate": 0.0002,
+      "loss": 1.0873,
+      "step": 120
+    },
+    {
+      "epoch": 0.31901840490797545,
+      "grad_norm": 0.5814566612243652,
+      "learning_rate": 0.0002,
+      "loss": 1.1652,
+      "step": 130
+    },
+    {
+      "epoch": 0.34355828220858897,
+      "grad_norm": 0.6171417236328125,
+      "learning_rate": 0.0002,
+      "loss": 1.0915,
+      "step": 140
+    },
+    {
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.6176294088363647,
+      "learning_rate": 0.0002,
+      "loss": 1.1572,
+      "step": 150
+    },
+    {
+      "epoch": 0.39263803680981596,
+      "grad_norm": 0.8398241400718689,
+      "learning_rate": 0.0002,
+      "loss": 1.1135,
+      "step": 160
+    },
+    {
+      "epoch": 0.4171779141104294,
+      "grad_norm": 0.5397658944129944,
+      "learning_rate": 0.0002,
+      "loss": 1.1235,
+      "step": 170
+    },
+    {
+      "epoch": 0.44171779141104295,
+      "grad_norm": 0.7487576603889465,
+      "learning_rate": 0.0002,
+      "loss": 1.1438,
+      "step": 180
+    },
+    {
+      "epoch": 0.4662576687116564,
+      "grad_norm": 0.4767085015773773,
+      "learning_rate": 0.0002,
+      "loss": 1.1178,
+      "step": 190
+    },
+    {
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.6086363196372986,
+      "learning_rate": 0.0002,
+      "loss": 1.1222,
+      "step": 200
+    },
+    {
+      "epoch": 0.5153374233128835,
+      "grad_norm": 0.6940106749534607,
+      "learning_rate": 0.0002,
+      "loss": 1.0346,
+      "step": 210
+    },
+    {
+      "epoch": 0.5398773006134969,
+      "grad_norm": 0.775067150592804,
+      "learning_rate": 0.0002,
+      "loss": 1.1162,
+      "step": 220
+    },
+    {
+      "epoch": 0.5644171779141104,
+      "grad_norm": 0.7273200154304504,
+      "learning_rate": 0.0002,
+      "loss": 1.1335,
+      "step": 230
+    },
+    {
+      "epoch": 0.588957055214724,
+      "grad_norm": 0.5168078541755676,
+      "learning_rate": 0.0002,
+      "loss": 1.0606,
+      "step": 240
+    },
+    {
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6487783193588257,
+      "learning_rate": 0.0002,
+      "loss": 1.0792,
+      "step": 250
+    },
+    {
+      "epoch": 0.6380368098159509,
+      "grad_norm": 1.0914227962493896,
+      "learning_rate": 0.0002,
+      "loss": 1.1642,
+      "step": 260
+    },
+    {
+      "epoch": 0.6625766871165644,
+      "grad_norm": 1.0182702541351318,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 270
+    },
+    {
+      "epoch": 0.6871165644171779,
+      "grad_norm": 0.7045943140983582,
+      "learning_rate": 0.0002,
+      "loss": 0.9816,
+      "step": 280
+    },
+    {
+      "epoch": 0.7116564417177914,
+      "grad_norm": 0.6228044033050537,
+      "learning_rate": 0.0002,
+      "loss": 0.9675,
+      "step": 290
+    },
+    {
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.6161119341850281,
+      "learning_rate": 0.0002,
+      "loss": 1.076,
+      "step": 300
+    },
+    {
+      "epoch": 0.7607361963190185,
+      "grad_norm": 0.8310935497283936,
+      "learning_rate": 0.0002,
+      "loss": 1.0597,
+      "step": 310
+    },
+    {
+      "epoch": 0.7852760736196319,
+      "grad_norm": 0.6500546336174011,
+      "learning_rate": 0.0002,
+      "loss": 1.0817,
+      "step": 320
+    },
+    {
+      "epoch": 0.8098159509202454,
+      "grad_norm": 0.9254736304283142,
+      "learning_rate": 0.0002,
+      "loss": 1.109,
+      "step": 330
+    },
+    {
+      "epoch": 0.8343558282208589,
+      "grad_norm": 0.5892964005470276,
+      "learning_rate": 0.0002,
+      "loss": 0.9929,
+      "step": 340
+    },
+    {
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.5500181913375854,
+      "learning_rate": 0.0002,
+      "loss": 1.0637,
+      "step": 350
+    },
+    {
+      "epoch": 0.8834355828220859,
+      "grad_norm": 0.7814139127731323,
+      "learning_rate": 0.0002,
+      "loss": 1.0693,
+      "step": 360
+    },
+    {
+      "epoch": 0.9079754601226994,
+      "grad_norm": 0.6288005113601685,
+      "learning_rate": 0.0002,
+      "loss": 1.059,
+      "step": 370
+    },
+    {
+      "epoch": 0.9325153374233128,
+      "grad_norm": 0.5031183958053589,
+      "learning_rate": 0.0002,
+      "loss": 0.9904,
+      "step": 380
+    },
+    {
+      "epoch": 0.9570552147239264,
+      "grad_norm": 0.8656964898109436,
+      "learning_rate": 0.0002,
+      "loss": 1.1183,
+      "step": 390
+    },
+    {
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6746202111244202,
+      "learning_rate": 0.0002,
+      "loss": 1.0296,
+      "step": 400
+    },
+    {
+      "epoch": 0.9987730061349693,
+      "eval_loss": 1.3188884258270264,
+      "eval_runtime": 124.2158,
+      "eval_samples_per_second": 3.671,
+      "eval_steps_per_second": 0.459,
+      "step": 407
+    },
+    {
+      "epoch": 1.0061349693251533,
+      "grad_norm": 0.5244082808494568,
+      "learning_rate": 0.0002,
+      "loss": 0.9679,
+      "step": 410
+    },
+    {
+      "epoch": 1.030674846625767,
+      "grad_norm": 0.6792967319488525,
+      "learning_rate": 0.0002,
+      "loss": 0.97,
+      "step": 420
+    },
+    {
+      "epoch": 1.0552147239263803,
+      "grad_norm": 0.6114474534988403,
+      "learning_rate": 0.0002,
+      "loss": 0.8935,
+      "step": 430
+    },
+    {
+      "epoch": 1.0797546012269938,
+      "grad_norm": 0.6367565393447876,
+      "learning_rate": 0.0002,
+      "loss": 0.9469,
+      "step": 440
+    },
+    {
+      "epoch": 1.1042944785276074,
+      "grad_norm": 0.7007757425308228,
+      "learning_rate": 0.0002,
+      "loss": 0.9536,
+      "step": 450
+    },
+    {
+      "epoch": 1.1288343558282208,
+      "grad_norm": 0.9737136960029602,
+      "learning_rate": 0.0002,
+      "loss": 0.8806,
+      "step": 460
+    },
+    {
+      "epoch": 1.1533742331288344,
+      "grad_norm": 1.1624044179916382,
+      "learning_rate": 0.0002,
+      "loss": 0.9217,
+      "step": 470
+    },
+    {
+      "epoch": 1.177914110429448,
+      "grad_norm": 2.1017873287200928,
+      "learning_rate": 0.0002,
+      "loss": 0.9056,
+      "step": 480
+    },
+    {
+      "epoch": 1.2024539877300613,
+      "grad_norm": 0.7352198958396912,
+      "learning_rate": 0.0002,
+      "loss": 0.9945,
+      "step": 490
+    },
+    {
+      "epoch": 1.2269938650306749,
+      "grad_norm": 0.9491982460021973,
+      "learning_rate": 0.0002,
+      "loss": 0.9224,
+      "step": 500
+    },
+    {
+      "epoch": 1.2515337423312882,
+      "grad_norm": 1.0124034881591797,
+      "learning_rate": 0.0002,
+      "loss": 0.9079,
+      "step": 510
+    },
+    {
+      "epoch": 1.2760736196319018,
+      "grad_norm": 0.7620294094085693,
+      "learning_rate": 0.0002,
+      "loss": 0.9596,
+      "step": 520
+    },
+    {
+      "epoch": 1.3006134969325154,
+      "grad_norm": 0.7207106351852417,
+      "learning_rate": 0.0002,
+      "loss": 0.9095,
+      "step": 530
+    },
+    {
+      "epoch": 1.3251533742331287,
+      "grad_norm": 0.586559534072876,
+      "learning_rate": 0.0002,
+      "loss": 0.8588,
+      "step": 540
+    },
+    {
+      "epoch": 1.3496932515337423,
+      "grad_norm": 0.7113742828369141,
+      "learning_rate": 0.0002,
+      "loss": 0.9146,
+      "step": 550
+    },
+    {
+      "epoch": 1.3742331288343559,
+      "grad_norm": 0.9574087262153625,
+      "learning_rate": 0.0002,
+      "loss": 0.9504,
+      "step": 560
+    },
+    {
+      "epoch": 1.3987730061349692,
+      "grad_norm": 0.71551913022995,
+      "learning_rate": 0.0002,
+      "loss": 0.9124,
+      "step": 570
+    },
+    {
+      "epoch": 1.4233128834355828,
+      "grad_norm": 1.0000630617141724,
+      "learning_rate": 0.0002,
+      "loss": 0.8944,
+      "step": 580
+    },
+    {
+      "epoch": 1.4478527607361964,
+      "grad_norm": 0.6878040432929993,
+      "learning_rate": 0.0002,
+      "loss": 0.8707,
+      "step": 590
+    },
+    {
+      "epoch": 1.4723926380368098,
+      "grad_norm": 0.770113468170166,
+      "learning_rate": 0.0002,
+      "loss": 0.953,
+      "step": 600
+    },
+    {
+      "epoch": 1.4969325153374233,
+      "grad_norm": 0.7941587567329407,
+      "learning_rate": 0.0002,
+      "loss": 0.9293,
+      "step": 610
+    },
+    {
+      "epoch": 1.521472392638037,
+      "grad_norm": 0.971196711063385,
+      "learning_rate": 0.0002,
+      "loss": 0.9177,
+      "step": 620
+    },
+    {
+      "epoch": 1.5460122699386503,
+      "grad_norm": 0.6944937109947205,
+      "learning_rate": 0.0002,
+      "loss": 0.8605,
+      "step": 630
+    },
+    {
+      "epoch": 1.5705521472392638,
+      "grad_norm": 1.104216456413269,
+      "learning_rate": 0.0002,
+      "loss": 0.8922,
+      "step": 640
+    },
+    {
+      "epoch": 1.5950920245398774,
+      "grad_norm": 0.786145806312561,
+      "learning_rate": 0.0002,
+      "loss": 0.9264,
+      "step": 650
+    },
+    {
+      "epoch": 1.6196319018404908,
+      "grad_norm": 0.8748652935028076,
+      "learning_rate": 0.0002,
+      "loss": 0.8515,
+      "step": 660
+    },
+    {
+      "epoch": 1.6441717791411041,
+      "grad_norm": 0.6184355020523071,
+      "learning_rate": 0.0002,
+      "loss": 0.8818,
+      "step": 670
+    },
+    {
+      "epoch": 1.668711656441718,
+      "grad_norm": 1.0236579179763794,
+      "learning_rate": 0.0002,
+      "loss": 0.9484,
+      "step": 680
+    },
+    {
+      "epoch": 1.6932515337423313,
+      "grad_norm": 0.9815388917922974,
+      "learning_rate": 0.0002,
+      "loss": 0.9078,
+      "step": 690
+    },
+    {
+      "epoch": 1.7177914110429446,
+      "grad_norm": 0.6398373246192932,
+      "learning_rate": 0.0002,
+      "loss": 0.9036,
+      "step": 700
+    },
+    {
+      "epoch": 1.7423312883435584,
+      "grad_norm": 0.6157359480857849,
+      "learning_rate": 0.0002,
+      "loss": 0.8702,
+      "step": 710
+    },
+    {
+      "epoch": 1.7668711656441718,
+      "grad_norm": 0.7352675795555115,
+      "learning_rate": 0.0002,
+      "loss": 0.8712,
+      "step": 720
+    },
+    {
+      "epoch": 1.7914110429447851,
+      "grad_norm": 0.6840626001358032,
+      "learning_rate": 0.0002,
+      "loss": 0.8737,
+      "step": 730
+    },
+    {
+      "epoch": 1.8159509202453987,
+      "grad_norm": 0.8363635540008545,
+      "learning_rate": 0.0002,
+      "loss": 0.9113,
+      "step": 740
+    },
+    {
+      "epoch": 1.8404907975460123,
+      "grad_norm": 1.7173194885253906,
+      "learning_rate": 0.0002,
+      "loss": 0.973,
+      "step": 750
+    },
+    {
+      "epoch": 1.8650306748466257,
+      "grad_norm": 0.8277300000190735,
+      "learning_rate": 0.0002,
+      "loss": 0.9438,
+      "step": 760
+    },
+    {
+      "epoch": 1.8895705521472392,
+      "grad_norm": 0.6714297533035278,
+      "learning_rate": 0.0002,
+      "loss": 0.919,
+      "step": 770
+    },
+    {
+      "epoch": 1.9141104294478528,
+      "grad_norm": 0.7646296620368958,
+      "learning_rate": 0.0002,
+      "loss": 0.889,
+      "step": 780
+    },
+    {
+      "epoch": 1.9386503067484662,
+      "grad_norm": 0.9777507781982422,
+      "learning_rate": 0.0002,
+      "loss": 0.8433,
+      "step": 790
+    },
+    {
+      "epoch": 1.9631901840490797,
+      "grad_norm": 0.7732912302017212,
+      "learning_rate": 0.0002,
+      "loss": 0.9255,
+      "step": 800
+    },
+    {
+      "epoch": 1.9877300613496933,
+      "grad_norm": 0.7485368847846985,
+      "learning_rate": 0.0002,
+      "loss": 0.9556,
+      "step": 810
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.335689663887024,
+      "eval_runtime": 80.1256,
+      "eval_samples_per_second": 5.691,
+      "eval_steps_per_second": 0.711,
+      "step": 815
+    },
+    {
+      "epoch": 2.0122699386503067,
+      "grad_norm": 0.7061107158660889,
+      "learning_rate": 0.0002,
+      "loss": 0.9555,
+      "step": 820
+    },
+    {
+      "epoch": 2.03680981595092,
+      "grad_norm": 1.101184368133545,
+      "learning_rate": 0.0002,
+      "loss": 0.7363,
+      "step": 830
+    },
+    {
+      "epoch": 2.061349693251534,
+      "grad_norm": 0.6724491119384766,
+      "learning_rate": 0.0002,
+      "loss": 0.7269,
+      "step": 840
+    },
+    {
+      "epoch": 2.085889570552147,
+      "grad_norm": 0.8159838318824768,
+      "learning_rate": 0.0002,
+      "loss": 0.6836,
+      "step": 850
+    },
+    {
+      "epoch": 2.1104294478527605,
+      "grad_norm": 0.9286916255950928,
+      "learning_rate": 0.0002,
+      "loss": 0.7237,
+      "step": 860
+    },
+    {
+      "epoch": 2.1349693251533743,
+      "grad_norm": 0.9122375845909119,
+      "learning_rate": 0.0002,
+      "loss": 0.7258,
+      "step": 870
+    },
+    {
+      "epoch": 2.1595092024539877,
+      "grad_norm": 0.9655355215072632,
+      "learning_rate": 0.0002,
+      "loss": 0.7164,
+      "step": 880
+    },
+    {
+      "epoch": 2.184049079754601,
+      "grad_norm": 1.1539593935012817,
+      "learning_rate": 0.0002,
+      "loss": 0.741,
+      "step": 890
+    },
+    {
+      "epoch": 2.208588957055215,
+      "grad_norm": 1.0535199642181396,
+      "learning_rate": 0.0002,
+      "loss": 0.7368,
+      "step": 900
+    },
+    {
+      "epoch": 2.233128834355828,
+      "grad_norm": 1.2841371297836304,
+      "learning_rate": 0.0002,
+      "loss": 0.6615,
+      "step": 910
+    },
+    {
+      "epoch": 2.2576687116564416,
+      "grad_norm": 0.8669798970222473,
+      "learning_rate": 0.0002,
+      "loss": 0.767,
+      "step": 920
+    },
+    {
+      "epoch": 2.2822085889570554,
+      "grad_norm": 0.8732201457023621,
+      "learning_rate": 0.0002,
+      "loss": 0.7303,
+      "step": 930
+    },
+    {
+      "epoch": 2.3067484662576687,
+      "grad_norm": 0.9955021739006042,
+      "learning_rate": 0.0002,
+      "loss": 0.7205,
+      "step": 940
+    },
+    {
+      "epoch": 2.331288343558282,
+      "grad_norm": 1.0610932111740112,
+      "learning_rate": 0.0002,
+      "loss": 0.6781,
+      "step": 950
+    },
+    {
+      "epoch": 2.355828220858896,
+      "grad_norm": 0.9506151080131531,
+      "learning_rate": 0.0002,
+      "loss": 0.729,
+      "step": 960
+    },
+    {
+      "epoch": 2.3803680981595092,
+      "grad_norm": 1.259052038192749,
+      "learning_rate": 0.0002,
+      "loss": 0.6784,
+      "step": 970
+    },
+    {
+      "epoch": 2.4049079754601226,
+      "grad_norm": 1.1822247505187988,
+      "learning_rate": 0.0002,
+      "loss": 0.6668,
+      "step": 980
+    },
+    {
+      "epoch": 2.4294478527607364,
+      "grad_norm": 0.6920015811920166,
+      "learning_rate": 0.0002,
+      "loss": 0.6623,
+      "step": 990
+    },
+    {
+      "epoch": 2.4539877300613497,
+      "grad_norm": 0.7560105323791504,
+      "learning_rate": 0.0002,
+      "loss": 0.6802,
+      "step": 1000
+    },
+    {
+      "epoch": 2.478527607361963,
+      "grad_norm": 0.8672356009483337,
+      "learning_rate": 0.0002,
+      "loss": 0.7685,
+      "step": 1010
+    },
+    {
+      "epoch": 2.5030674846625764,
+      "grad_norm": 1.1074872016906738,
+      "learning_rate": 0.0002,
+      "loss": 0.7058,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5276073619631902,
+      "grad_norm": 1.1430137157440186,
+      "learning_rate": 0.0002,
+      "loss": 0.7746,
+      "step": 1030
+    },
+    {
+      "epoch": 2.5521472392638036,
+      "grad_norm": 1.6926707029342651,
+      "learning_rate": 0.0002,
+      "loss": 0.7444,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5766871165644174,
+      "grad_norm": 1.0170048475265503,
+      "learning_rate": 0.0002,
+      "loss": 0.7131,
+      "step": 1050
+    },
+    {
+      "epoch": 2.6012269938650308,
+      "grad_norm": 1.1462562084197998,
+      "learning_rate": 0.0002,
+      "loss": 0.7275,
+      "step": 1060
+    },
+    {
+      "epoch": 2.625766871165644,
+      "grad_norm": 1.127669334411621,
+      "learning_rate": 0.0002,
+      "loss": 0.6736,
+      "step": 1070
+    },
+    {
+      "epoch": 2.6503067484662575,
+      "grad_norm": 0.9649022221565247,
+      "learning_rate": 0.0002,
+      "loss": 0.7218,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6748466257668713,
+      "grad_norm": 0.9426548480987549,
+      "learning_rate": 0.0002,
+      "loss": 0.7651,
+      "step": 1090
+    },
+    {
+      "epoch": 2.6993865030674846,
+      "grad_norm": 1.1191051006317139,
+      "learning_rate": 0.0002,
+      "loss": 0.7829,
+      "step": 1100
+    },
+    {
+      "epoch": 2.7239263803680984,
+      "grad_norm": 2.229809284210205,
+      "learning_rate": 0.0002,
+      "loss": 0.7238,
+      "step": 1110
+    },
+    {
+      "epoch": 2.7484662576687118,
+      "grad_norm": 1.2478930950164795,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 1120
+    },
+    {
+      "epoch": 2.773006134969325,
+      "grad_norm": 0.9907709360122681,
+      "learning_rate": 0.0002,
+      "loss": 0.7194,
+      "step": 1130
+    },
+    {
+      "epoch": 2.7975460122699385,
+      "grad_norm": 1.1670643091201782,
+      "learning_rate": 0.0002,
+      "loss": 0.6531,
+      "step": 1140
+    },
+    {
+      "epoch": 2.8220858895705523,
+      "grad_norm": 1.1675913333892822,
+      "learning_rate": 0.0002,
+      "loss": 0.7076,
+      "step": 1150
+    },
+    {
+      "epoch": 2.8466257668711656,
+      "grad_norm": 0.9909353256225586,
+      "learning_rate": 0.0002,
+      "loss": 0.6915,
+      "step": 1160
+    },
+    {
+      "epoch": 2.871165644171779,
+      "grad_norm": 0.8759778141975403,
+      "learning_rate": 0.0002,
+      "loss": 0.7294,
+      "step": 1170
+    },
+    {
+      "epoch": 2.895705521472393,
+      "grad_norm": 0.8080666661262512,
+      "learning_rate": 0.0002,
+      "loss": 0.6451,
+      "step": 1180
+    },
+    {
+      "epoch": 2.920245398773006,
+      "grad_norm": 0.9743189811706543,
+      "learning_rate": 0.0002,
+      "loss": 0.7331,
+      "step": 1190
+    },
+    {
+      "epoch": 2.9447852760736195,
+      "grad_norm": 0.8573821187019348,
+      "learning_rate": 0.0002,
+      "loss": 0.7471,
+      "step": 1200
+    },
+    {
+      "epoch": 2.969325153374233,
+      "grad_norm": 1.0196788311004639,
+      "learning_rate": 0.0002,
+      "loss": 0.7396,
+      "step": 1210
+    },
+    {
+      "epoch": 2.9938650306748467,
+      "grad_norm": 0.8840402960777283,
+      "learning_rate": 0.0002,
+      "loss": 0.7288,
+      "step": 1220
+    },
+    {
+      "epoch": 2.9987730061349693,
+      "eval_loss": 1.4348119497299194,
+      "eval_runtime": 80.7583,
+      "eval_samples_per_second": 5.646,
+      "eval_steps_per_second": 0.706,
+      "step": 1222
+    },
+    {
+      "epoch": 3.01840490797546,
+      "grad_norm": 1.2376960515975952,
+      "learning_rate": 0.0002,
+      "loss": 0.5671,
+      "step": 1230
+    },
+    {
+      "epoch": 3.042944785276074,
+      "grad_norm": 0.9861388206481934,
+      "learning_rate": 0.0002,
+      "loss": 0.5176,
+      "step": 1240
+    },
+    {
+      "epoch": 3.067484662576687,
+      "grad_norm": 1.2193198204040527,
+      "learning_rate": 0.0002,
+      "loss": 0.5623,
+      "step": 1250
+    },
+    {
+      "epoch": 3.0920245398773005,
+      "grad_norm": 1.1927645206451416,
+      "learning_rate": 0.0002,
+      "loss": 0.5727,
+      "step": 1260
+    },
+    {
+      "epoch": 3.116564417177914,
+      "grad_norm": 1.0420559644699097,
+      "learning_rate": 0.0002,
+      "loss": 0.5296,
+      "step": 1270
+    },
+    {
+      "epoch": 3.1411042944785277,
+      "grad_norm": 1.014664649963379,
+      "learning_rate": 0.0002,
+      "loss": 0.5322,
+      "step": 1280
+    },
+    {
+      "epoch": 3.165644171779141,
+      "grad_norm": 1.3103076219558716,
+      "learning_rate": 0.0002,
+      "loss": 0.5519,
+      "step": 1290
+    },
+    {
+      "epoch": 3.190184049079755,
+      "grad_norm": 1.2735213041305542,
+      "learning_rate": 0.0002,
+      "loss": 0.5133,
+      "step": 1300
+    },
+    {
+      "epoch": 3.214723926380368,
+      "grad_norm": 1.147608995437622,
+      "learning_rate": 0.0002,
+      "loss": 0.514,
+      "step": 1310
+    },
+    {
+      "epoch": 3.2392638036809815,
+      "grad_norm": 0.922386884689331,
+      "learning_rate": 0.0002,
+      "loss": 0.58,
+      "step": 1320
+    },
+    {
+      "epoch": 3.263803680981595,
+      "grad_norm": 1.1271566152572632,
+      "learning_rate": 0.0002,
+      "loss": 0.5425,
+      "step": 1330
+    },
+    {
+      "epoch": 3.2883435582822087,
+      "grad_norm": 1.2994354963302612,
+      "learning_rate": 0.0002,
+      "loss": 0.519,
+      "step": 1340
+    },
+    {
+      "epoch": 3.312883435582822,
+      "grad_norm": 1.0001686811447144,
+      "learning_rate": 0.0002,
+      "loss": 0.5521,
+      "step": 1350
+    },
+    {
+      "epoch": 3.3374233128834354,
+      "grad_norm": 1.6737695932388306,
+      "learning_rate": 0.0002,
+      "loss": 0.564,
+      "step": 1360
+    },
+    {
+      "epoch": 3.361963190184049,
+      "grad_norm": 1.1842162609100342,
+      "learning_rate": 0.0002,
+      "loss": 0.5529,
+      "step": 1370
+    },
+    {
+      "epoch": 3.3865030674846626,
+      "grad_norm": 1.1873128414154053,
+      "learning_rate": 0.0002,
+      "loss": 0.569,
+      "step": 1380
+    },
+    {
+      "epoch": 3.411042944785276,
+      "grad_norm": 1.4505162239074707,
+      "learning_rate": 0.0002,
+      "loss": 0.5199,
+      "step": 1390
+    },
+    {
+      "epoch": 3.4355828220858897,
+      "grad_norm": 1.254621982574463,
+      "learning_rate": 0.0002,
+      "loss": 0.5536,
+      "step": 1400
+    },
+    {
+      "epoch": 3.460122699386503,
+      "grad_norm": 1.5493544340133667,
+      "learning_rate": 0.0002,
+      "loss": 0.5547,
+      "step": 1410
+    },
+    {
+      "epoch": 3.4846625766871164,
+      "grad_norm": 1.4435759782791138,
+      "learning_rate": 0.0002,
+      "loss": 0.542,
+      "step": 1420
+    },
+    {
+      "epoch": 3.5092024539877302,
+      "grad_norm": 1.5885447263717651,
+      "learning_rate": 0.0002,
+      "loss": 0.6124,
+      "step": 1430
+    },
+    {
+      "epoch": 3.5337423312883436,
+      "grad_norm": 1.3357222080230713,
+      "learning_rate": 0.0002,
+      "loss": 0.5737,
+      "step": 1440
+    },
+    {
+      "epoch": 3.558282208588957,
+      "grad_norm": 1.8510551452636719,
+      "learning_rate": 0.0002,
+      "loss": 0.5903,
+      "step": 1450
+    },
+    {
+      "epoch": 3.5828220858895703,
+      "grad_norm": 1.1081656217575073,
+      "learning_rate": 0.0002,
+      "loss": 0.5214,
+      "step": 1460
+    },
+    {
+      "epoch": 3.607361963190184,
+      "grad_norm": 1.2849210500717163,
+      "learning_rate": 0.0002,
+      "loss": 0.5351,
+      "step": 1470
+    },
+    {
+      "epoch": 3.6319018404907975,
+      "grad_norm": 1.2472909688949585,
+      "learning_rate": 0.0002,
+      "loss": 0.5364,
+      "step": 1480
+    },
+    {
+      "epoch": 3.6564417177914113,
+      "grad_norm": 1.1466434001922607,
+      "learning_rate": 0.0002,
+      "loss": 0.5371,
+      "step": 1490
+    },
+    {
+      "epoch": 3.6809815950920246,
+      "grad_norm": 1.325461506843567,
+      "learning_rate": 0.0002,
+      "loss": 0.565,
+      "step": 1500
+    },
+    {
+      "epoch": 3.705521472392638,
+      "grad_norm": 1.4214563369750977,
+      "learning_rate": 0.0002,
+      "loss": 0.6154,
+      "step": 1510
+    },
+    {
+      "epoch": 3.7300613496932513,
+      "grad_norm": 1.1496703624725342,
+      "learning_rate": 0.0002,
+      "loss": 0.5726,
+      "step": 1520
+    },
+    {
+      "epoch": 3.754601226993865,
+      "grad_norm": 1.205261468887329,
+      "learning_rate": 0.0002,
+      "loss": 0.5687,
+      "step": 1530
+    },
+    {
+      "epoch": 3.7791411042944785,
+      "grad_norm": 1.6516913175582886,
+      "learning_rate": 0.0002,
+      "loss": 0.5593,
+      "step": 1540
+    },
+    {
+      "epoch": 3.8036809815950923,
+      "grad_norm": 1.7035053968429565,
+      "learning_rate": 0.0002,
+      "loss": 0.5767,
+      "step": 1550
+    },
+    {
+      "epoch": 3.8282208588957056,
+      "grad_norm": 1.463699460029602,
+      "learning_rate": 0.0002,
+      "loss": 0.5948,
+      "step": 1560
+    },
+    {
+      "epoch": 3.852760736196319,
+      "grad_norm": 1.7970601320266724,
+      "learning_rate": 0.0002,
+      "loss": 0.5326,
+      "step": 1570
+    },
+    {
+      "epoch": 3.8773006134969323,
+      "grad_norm": 1.2967402935028076,
+      "learning_rate": 0.0002,
+      "loss": 0.5919,
+      "step": 1580
+    },
+    {
+      "epoch": 3.901840490797546,
+      "grad_norm": 1.0639182329177856,
+      "learning_rate": 0.0002,
+      "loss": 0.5502,
+      "step": 1590
+    },
+    {
+      "epoch": 3.9263803680981595,
+      "grad_norm": 1.4388368129730225,
+      "learning_rate": 0.0002,
+      "loss": 0.5281,
+      "step": 1600
+    },
+    {
+      "epoch": 3.950920245398773,
+      "grad_norm": 1.2425099611282349,
+      "learning_rate": 0.0002,
+      "loss": 0.6305,
+      "step": 1610
+    },
+    {
+      "epoch": 3.9754601226993866,
+      "grad_norm": 1.3989970684051514,
+      "learning_rate": 0.0002,
+      "loss": 0.5548,
+      "step": 1620
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.9850553274154663,
+      "learning_rate": 0.0002,
+      "loss": 0.5661,
+      "step": 1630
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.5340977907180786,
+      "eval_runtime": 79.7233,
+      "eval_samples_per_second": 5.72,
+      "eval_steps_per_second": 0.715,
+      "step": 1630
+    },
+    {
+      "epoch": 4.024539877300613,
+      "grad_norm": 2.1861188411712646,
+      "learning_rate": 0.0002,
+      "loss": 0.4365,
+      "step": 1640
+    },
+    {
+      "epoch": 4.049079754601227,
+      "grad_norm": 1.4574151039123535,
+      "learning_rate": 0.0002,
+      "loss": 0.3987,
+      "step": 1650
+    },
+    {
+      "epoch": 4.07361963190184,
+      "grad_norm": 1.093583345413208,
+      "learning_rate": 0.0002,
+      "loss": 0.404,
+      "step": 1660
+    },
+    {
+      "epoch": 4.098159509202454,
+      "grad_norm": 1.4852519035339355,
+      "learning_rate": 0.0002,
+      "loss": 0.4493,
+      "step": 1670
+    },
+    {
+      "epoch": 4.122699386503068,
+      "grad_norm": 1.9289690256118774,
+      "learning_rate": 0.0002,
+      "loss": 0.3941,
+      "step": 1680
+    },
+    {
+      "epoch": 4.147239263803681,
+      "grad_norm": 1.286780834197998,
+      "learning_rate": 0.0002,
+      "loss": 0.4015,
+      "step": 1690
+    },
+    {
+      "epoch": 4.171779141104294,
+      "grad_norm": 1.1915839910507202,
+      "learning_rate": 0.0002,
+      "loss": 0.396,
+      "step": 1700
+    },
+    {
+      "epoch": 4.196319018404908,
+      "grad_norm": 0.9827993512153625,
+      "learning_rate": 0.0002,
+      "loss": 0.4053,
+      "step": 1710
+    },
+    {
+      "epoch": 4.220858895705521,
+      "grad_norm": 1.314813494682312,
+      "learning_rate": 0.0002,
+      "loss": 0.4341,
+      "step": 1720
+    },
+    {
+      "epoch": 4.245398773006135,
+      "grad_norm": 1.6890170574188232,
+      "learning_rate": 0.0002,
+      "loss": 0.4187,
+      "step": 1730
+    },
+    {
+      "epoch": 4.269938650306749,
+      "grad_norm": 1.5559035539627075,
+      "learning_rate": 0.0002,
+      "loss": 0.4047,
+      "step": 1740
+    },
+    {
+      "epoch": 4.294478527607362,
+      "grad_norm": 1.8186208009719849,
+      "learning_rate": 0.0002,
+      "loss": 0.4398,
+      "step": 1750
+    },
+    {
+      "epoch": 4.319018404907975,
+      "grad_norm": 0.9572193026542664,
+      "learning_rate": 0.0002,
+      "loss": 0.4223,
+      "step": 1760
+    },
+    {
+      "epoch": 4.343558282208589,
+      "grad_norm": 2.1470448970794678,
+      "learning_rate": 0.0002,
+      "loss": 0.5141,
+      "step": 1770
+    },
+    {
+      "epoch": 4.368098159509202,
+      "grad_norm": 1.2788262367248535,
+      "learning_rate": 0.0002,
+      "loss": 0.4051,
+      "step": 1780
+    },
+    {
+      "epoch": 4.392638036809816,
+      "grad_norm": 1.8074915409088135,
+      "learning_rate": 0.0002,
+      "loss": 0.4574,
+      "step": 1790
+    },
+    {
+      "epoch": 4.41717791411043,
+      "grad_norm": 0.9725378751754761,
+      "learning_rate": 0.0002,
+      "loss": 0.4195,
+      "step": 1800
+    },
+    {
+      "epoch": 4.441717791411043,
+      "grad_norm": 1.7154884338378906,
+      "learning_rate": 0.0002,
+      "loss": 0.4267,
+      "step": 1810
+    },
+    {
+      "epoch": 4.466257668711656,
+      "grad_norm": 1.07939612865448,
+      "learning_rate": 0.0002,
+      "loss": 0.4059,
+      "step": 1820
+    },
+    {
+      "epoch": 4.49079754601227,
+      "grad_norm": 1.1510107517242432,
+      "learning_rate": 0.0002,
+      "loss": 0.4572,
+      "step": 1830
+    },
+    {
+      "epoch": 4.515337423312883,
+      "grad_norm": 1.2309428453445435,
+      "learning_rate": 0.0002,
+      "loss": 0.4053,
+      "step": 1840
+    },
+    {
+      "epoch": 4.539877300613497,
+      "grad_norm": 1.31099271774292,
+      "learning_rate": 0.0002,
+      "loss": 0.4275,
+      "step": 1850
+    },
+    {
+      "epoch": 4.564417177914111,
+      "grad_norm": 1.2938907146453857,
+      "learning_rate": 0.0002,
+      "loss": 0.4755,
+      "step": 1860
+    },
+    {
+      "epoch": 4.588957055214724,
+      "grad_norm": 1.6019647121429443,
+      "learning_rate": 0.0002,
+      "loss": 0.478,
+      "step": 1870
+    },
+    {
+      "epoch": 4.613496932515337,
+      "grad_norm": 1.9093713760375977,
+      "learning_rate": 0.0002,
+      "loss": 0.4492,
+      "step": 1880
+    },
+    {
+      "epoch": 4.638036809815951,
+      "grad_norm": 1.4761801958084106,
+      "learning_rate": 0.0002,
+      "loss": 0.4522,
+      "step": 1890
+    },
+    {
+      "epoch": 4.662576687116564,
+      "grad_norm": 1.2435506582260132,
+      "learning_rate": 0.0002,
+      "loss": 0.5078,
+      "step": 1900
+    },
+    {
+      "epoch": 4.6871165644171775,
+      "grad_norm": 1.5027297735214233,
+      "learning_rate": 0.0002,
+      "loss": 0.4656,
+      "step": 1910
+    },
+    {
+      "epoch": 4.711656441717792,
+      "grad_norm": 1.323907494544983,
+      "learning_rate": 0.0002,
+      "loss": 0.4428,
+      "step": 1920
+    },
+    {
+      "epoch": 4.736196319018405,
+      "grad_norm": 1.1643304824829102,
+      "learning_rate": 0.0002,
+      "loss": 0.4113,
+      "step": 1930
+    },
+    {
+      "epoch": 4.7607361963190185,
+      "grad_norm": 1.4745224714279175,
+      "learning_rate": 0.0002,
+      "loss": 0.4548,
+      "step": 1940
+    },
+    {
+      "epoch": 4.785276073619632,
+      "grad_norm": 1.6811035871505737,
+      "learning_rate": 0.0002,
+      "loss": 0.4447,
+      "step": 1950
+    },
+    {
+      "epoch": 4.809815950920245,
+      "grad_norm": 1.0878573656082153,
+      "learning_rate": 0.0002,
+      "loss": 0.4669,
+      "step": 1960
+    },
+    {
+      "epoch": 4.8343558282208585,
+      "grad_norm": 1.640516996383667,
+      "learning_rate": 0.0002,
+      "loss": 0.4589,
+      "step": 1970
+    },
+    {
+      "epoch": 4.858895705521473,
+      "grad_norm": 1.305409550666809,
+      "learning_rate": 0.0002,
+      "loss": 0.4534,
+      "step": 1980
+    },
+    {
+      "epoch": 4.883435582822086,
+      "grad_norm": 1.5608975887298584,
+      "learning_rate": 0.0002,
+      "loss": 0.4512,
+      "step": 1990
+    },
+    {
+      "epoch": 4.9079754601226995,
+      "grad_norm": 1.3681490421295166,
+      "learning_rate": 0.0002,
+      "loss": 0.4602,
+      "step": 2000
+    },
+    {
+      "epoch": 4.932515337423313,
+      "grad_norm": 1.241847276687622,
+      "learning_rate": 0.0002,
+      "loss": 0.4445,
+      "step": 2010
+    },
+    {
+      "epoch": 4.957055214723926,
+      "grad_norm": 1.1561694145202637,
+      "learning_rate": 0.0002,
+      "loss": 0.502,
+      "step": 2020
+    },
+    {
+      "epoch": 4.9815950920245395,
+      "grad_norm": 1.8284252882003784,
+      "learning_rate": 0.0002,
+      "loss": 0.4586,
+      "step": 2030
+    },
+    {
+      "epoch": 4.99877300613497,
+      "eval_loss": 1.702454686164856,
+      "eval_runtime": 79.3705,
+      "eval_samples_per_second": 5.745,
+      "eval_steps_per_second": 0.718,
+      "step": 2037
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 3256,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.939475566592e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2037/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0e586c733c860e8234b028525d14453b4a2a52ad2c815f3775cbbbef35878fb
+size 5624

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/README.md

@@ -0,0 +1,202 @@
+---
+base_model: mistralai/Mistral-7B-Instruct-v0.3
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3704a46f5053240d2273bfe6baeeae9fdb453b030065d89745894429a406be7
+size 109069176

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04f1b8c8fdb1dbd30c49bb9d505ebd78d47da0e13cb2ebd77caeb79424e086ec
+size 55532666

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09e18aecde89fbee1ed3f353f366a65d54be677d8ca152a01a71b721102d7a3c
+size 14244

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d1d0eb44bec42a7e4c96e25038586ef00bd6f82a0b4415f637c7063413fbe73
+size 1064

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "</s>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37f00374dea48658ee8f5d0f21895b9bc55cb0103939607c8185bfd1c6ca1f89
+size 587404

Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-2445/trainer_state.json

@@ -0,0 +1,1789 @@
+{
+  "best_metric": 1.3188884258270264,
+  "best_model_checkpoint": "outputs-001/Mistral-7B-Instruct-v0.3_int4_mmlu-routerbench-0shot-full-by-task_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.3-num-2942-sd-10000/checkpoint-407",
+  "epoch": 6.0,
+  "eval_steps": 10,
+  "global_step": 2445,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.024539877300613498,
+      "grad_norm": 1.0104172229766846,
+      "learning_rate": 0.0002,
+      "loss": 1.7717,
+      "step": 10
+    },
+    {
+      "epoch": 0.049079754601226995,
+      "grad_norm": 0.8800041079521179,
+      "learning_rate": 0.0002,
+      "loss": 1.6041,
+      "step": 20
+    },
+    {
+      "epoch": 0.0736196319018405,
+      "grad_norm": 1.337620496749878,
+      "learning_rate": 0.0002,
+      "loss": 1.4421,
+      "step": 30
+    },
+    {
+      "epoch": 0.09815950920245399,
+      "grad_norm": 0.6757020354270935,
+      "learning_rate": 0.0002,
+      "loss": 1.5542,
+      "step": 40
+    },
+    {
+      "epoch": 0.12269938650306748,
+      "grad_norm": 42.27006149291992,
+      "learning_rate": 0.0002,
+      "loss": 1.4365,
+      "step": 50
+    },
+    {
+      "epoch": 0.147239263803681,
+      "grad_norm": 0.8157640099525452,
+      "learning_rate": 0.0002,
+      "loss": 1.1226,
+      "step": 60
+    },
+    {
+      "epoch": 0.17177914110429449,
+      "grad_norm": 1.0711787939071655,
+      "learning_rate": 0.0002,
+      "loss": 1.1886,
+      "step": 70
+    },
+    {
+      "epoch": 0.19631901840490798,
+      "grad_norm": 0.5880025029182434,
+      "learning_rate": 0.0002,
+      "loss": 1.1239,
+      "step": 80
+    },
+    {
+      "epoch": 0.22085889570552147,
+      "grad_norm": 0.827833354473114,
+      "learning_rate": 0.0002,
+      "loss": 1.0696,
+      "step": 90
+    },
+    {
+      "epoch": 0.24539877300613497,
+      "grad_norm": 0.579414963722229,
+      "learning_rate": 0.0002,
+      "loss": 1.0468,
+      "step": 100
+    },
+    {
+      "epoch": 0.26993865030674846,
+      "grad_norm": 0.6620142459869385,
+      "learning_rate": 0.0002,
+      "loss": 1.0914,
+      "step": 110
+    },
+    {
+      "epoch": 0.294478527607362,
+      "grad_norm": 0.5601297616958618,
+      "learning_rate": 0.0002,
+      "loss": 1.0873,
+      "step": 120
+    },
+    {
+      "epoch": 0.31901840490797545,
+      "grad_norm": 0.5814566612243652,
+      "learning_rate": 0.0002,
+      "loss": 1.1652,
+      "step": 130
+    },
+    {
+      "epoch": 0.34355828220858897,
+      "grad_norm": 0.6171417236328125,
+      "learning_rate": 0.0002,
+      "loss": 1.0915,
+      "step": 140
+    },
+    {
+      "epoch": 0.36809815950920244,
+      "grad_norm": 0.6176294088363647,
+      "learning_rate": 0.0002,
+      "loss": 1.1572,
+      "step": 150
+    },
+    {
+      "epoch": 0.39263803680981596,
+      "grad_norm": 0.8398241400718689,
+      "learning_rate": 0.0002,
+      "loss": 1.1135,
+      "step": 160
+    },
+    {
+      "epoch": 0.4171779141104294,
+      "grad_norm": 0.5397658944129944,
+      "learning_rate": 0.0002,
+      "loss": 1.1235,
+      "step": 170
+    },
+    {
+      "epoch": 0.44171779141104295,
+      "grad_norm": 0.7487576603889465,
+      "learning_rate": 0.0002,
+      "loss": 1.1438,
+      "step": 180
+    },
+    {
+      "epoch": 0.4662576687116564,
+      "grad_norm": 0.4767085015773773,
+      "learning_rate": 0.0002,
+      "loss": 1.1178,
+      "step": 190
+    },
+    {
+      "epoch": 0.49079754601226994,
+      "grad_norm": 0.6086363196372986,
+      "learning_rate": 0.0002,
+      "loss": 1.1222,
+      "step": 200
+    },
+    {
+      "epoch": 0.5153374233128835,
+      "grad_norm": 0.6940106749534607,
+      "learning_rate": 0.0002,
+      "loss": 1.0346,
+      "step": 210
+    },
+    {
+      "epoch": 0.5398773006134969,
+      "grad_norm": 0.775067150592804,
+      "learning_rate": 0.0002,
+      "loss": 1.1162,
+      "step": 220
+    },
+    {
+      "epoch": 0.5644171779141104,
+      "grad_norm": 0.7273200154304504,
+      "learning_rate": 0.0002,
+      "loss": 1.1335,
+      "step": 230
+    },
+    {
+      "epoch": 0.588957055214724,
+      "grad_norm": 0.5168078541755676,
+      "learning_rate": 0.0002,
+      "loss": 1.0606,
+      "step": 240
+    },
+    {
+      "epoch": 0.6134969325153374,
+      "grad_norm": 0.6487783193588257,
+      "learning_rate": 0.0002,
+      "loss": 1.0792,
+      "step": 250
+    },
+    {
+      "epoch": 0.6380368098159509,
+      "grad_norm": 1.0914227962493896,
+      "learning_rate": 0.0002,
+      "loss": 1.1642,
+      "step": 260
+    },
+    {
+      "epoch": 0.6625766871165644,
+      "grad_norm": 1.0182702541351318,
+      "learning_rate": 0.0002,
+      "loss": 1.065,
+      "step": 270
+    },
+    {
+      "epoch": 0.6871165644171779,
+      "grad_norm": 0.7045943140983582,
+      "learning_rate": 0.0002,
+      "loss": 0.9816,
+      "step": 280
+    },
+    {
+      "epoch": 0.7116564417177914,
+      "grad_norm": 0.6228044033050537,
+      "learning_rate": 0.0002,
+      "loss": 0.9675,
+      "step": 290
+    },
+    {
+      "epoch": 0.7361963190184049,
+      "grad_norm": 0.6161119341850281,
+      "learning_rate": 0.0002,
+      "loss": 1.076,
+      "step": 300
+    },
+    {
+      "epoch": 0.7607361963190185,
+      "grad_norm": 0.8310935497283936,
+      "learning_rate": 0.0002,
+      "loss": 1.0597,
+      "step": 310
+    },
+    {
+      "epoch": 0.7852760736196319,
+      "grad_norm": 0.6500546336174011,
+      "learning_rate": 0.0002,
+      "loss": 1.0817,
+      "step": 320
+    },
+    {
+      "epoch": 0.8098159509202454,
+      "grad_norm": 0.9254736304283142,
+      "learning_rate": 0.0002,
+      "loss": 1.109,
+      "step": 330
+    },
+    {
+      "epoch": 0.8343558282208589,
+      "grad_norm": 0.5892964005470276,
+      "learning_rate": 0.0002,
+      "loss": 0.9929,
+      "step": 340
+    },
+    {
+      "epoch": 0.8588957055214724,
+      "grad_norm": 0.5500181913375854,
+      "learning_rate": 0.0002,
+      "loss": 1.0637,
+      "step": 350
+    },
+    {
+      "epoch": 0.8834355828220859,
+      "grad_norm": 0.7814139127731323,
+      "learning_rate": 0.0002,
+      "loss": 1.0693,
+      "step": 360
+    },
+    {
+      "epoch": 0.9079754601226994,
+      "grad_norm": 0.6288005113601685,
+      "learning_rate": 0.0002,
+      "loss": 1.059,
+      "step": 370
+    },
+    {
+      "epoch": 0.9325153374233128,
+      "grad_norm": 0.5031183958053589,
+      "learning_rate": 0.0002,
+      "loss": 0.9904,
+      "step": 380
+    },
+    {
+      "epoch": 0.9570552147239264,
+      "grad_norm": 0.8656964898109436,
+      "learning_rate": 0.0002,
+      "loss": 1.1183,
+      "step": 390
+    },
+    {
+      "epoch": 0.9815950920245399,
+      "grad_norm": 0.6746202111244202,
+      "learning_rate": 0.0002,
+      "loss": 1.0296,
+      "step": 400
+    },
+    {
+      "epoch": 0.9987730061349693,
+      "eval_loss": 1.3188884258270264,
+      "eval_runtime": 124.2158,
+      "eval_samples_per_second": 3.671,
+      "eval_steps_per_second": 0.459,
+      "step": 407
+    },
+    {
+      "epoch": 1.0061349693251533,
+      "grad_norm": 0.5244082808494568,
+      "learning_rate": 0.0002,
+      "loss": 0.9679,
+      "step": 410
+    },
+    {
+      "epoch": 1.030674846625767,
+      "grad_norm": 0.6792967319488525,
+      "learning_rate": 0.0002,
+      "loss": 0.97,
+      "step": 420
+    },
+    {
+      "epoch": 1.0552147239263803,
+      "grad_norm": 0.6114474534988403,
+      "learning_rate": 0.0002,
+      "loss": 0.8935,
+      "step": 430
+    },
+    {
+      "epoch": 1.0797546012269938,
+      "grad_norm": 0.6367565393447876,
+      "learning_rate": 0.0002,
+      "loss": 0.9469,
+      "step": 440
+    },
+    {
+      "epoch": 1.1042944785276074,
+      "grad_norm": 0.7007757425308228,
+      "learning_rate": 0.0002,
+      "loss": 0.9536,
+      "step": 450
+    },
+    {
+      "epoch": 1.1288343558282208,
+      "grad_norm": 0.9737136960029602,
+      "learning_rate": 0.0002,
+      "loss": 0.8806,
+      "step": 460
+    },
+    {
+      "epoch": 1.1533742331288344,
+      "grad_norm": 1.1624044179916382,
+      "learning_rate": 0.0002,
+      "loss": 0.9217,
+      "step": 470
+    },
+    {
+      "epoch": 1.177914110429448,
+      "grad_norm": 2.1017873287200928,
+      "learning_rate": 0.0002,
+      "loss": 0.9056,
+      "step": 480
+    },
+    {
+      "epoch": 1.2024539877300613,
+      "grad_norm": 0.7352198958396912,
+      "learning_rate": 0.0002,
+      "loss": 0.9945,
+      "step": 490
+    },
+    {
+      "epoch": 1.2269938650306749,
+      "grad_norm": 0.9491982460021973,
+      "learning_rate": 0.0002,
+      "loss": 0.9224,
+      "step": 500
+    },
+    {
+      "epoch": 1.2515337423312882,
+      "grad_norm": 1.0124034881591797,
+      "learning_rate": 0.0002,
+      "loss": 0.9079,
+      "step": 510
+    },
+    {
+      "epoch": 1.2760736196319018,
+      "grad_norm": 0.7620294094085693,
+      "learning_rate": 0.0002,
+      "loss": 0.9596,
+      "step": 520
+    },
+    {
+      "epoch": 1.3006134969325154,
+      "grad_norm": 0.7207106351852417,
+      "learning_rate": 0.0002,
+      "loss": 0.9095,
+      "step": 530
+    },
+    {
+      "epoch": 1.3251533742331287,
+      "grad_norm": 0.586559534072876,
+      "learning_rate": 0.0002,
+      "loss": 0.8588,
+      "step": 540
+    },
+    {
+      "epoch": 1.3496932515337423,
+      "grad_norm": 0.7113742828369141,
+      "learning_rate": 0.0002,
+      "loss": 0.9146,
+      "step": 550
+    },
+    {
+      "epoch": 1.3742331288343559,
+      "grad_norm": 0.9574087262153625,
+      "learning_rate": 0.0002,
+      "loss": 0.9504,
+      "step": 560
+    },
+    {
+      "epoch": 1.3987730061349692,
+      "grad_norm": 0.71551913022995,
+      "learning_rate": 0.0002,
+      "loss": 0.9124,
+      "step": 570
+    },
+    {
+      "epoch": 1.4233128834355828,
+      "grad_norm": 1.0000630617141724,
+      "learning_rate": 0.0002,
+      "loss": 0.8944,
+      "step": 580
+    },
+    {
+      "epoch": 1.4478527607361964,
+      "grad_norm": 0.6878040432929993,
+      "learning_rate": 0.0002,
+      "loss": 0.8707,
+      "step": 590
+    },
+    {
+      "epoch": 1.4723926380368098,
+      "grad_norm": 0.770113468170166,
+      "learning_rate": 0.0002,
+      "loss": 0.953,
+      "step": 600
+    },
+    {
+      "epoch": 1.4969325153374233,
+      "grad_norm": 0.7941587567329407,
+      "learning_rate": 0.0002,
+      "loss": 0.9293,
+      "step": 610
+    },
+    {
+      "epoch": 1.521472392638037,
+      "grad_norm": 0.971196711063385,
+      "learning_rate": 0.0002,
+      "loss": 0.9177,
+      "step": 620
+    },
+    {
+      "epoch": 1.5460122699386503,
+      "grad_norm": 0.6944937109947205,
+      "learning_rate": 0.0002,
+      "loss": 0.8605,
+      "step": 630
+    },
+    {
+      "epoch": 1.5705521472392638,
+      "grad_norm": 1.104216456413269,
+      "learning_rate": 0.0002,
+      "loss": 0.8922,
+      "step": 640
+    },
+    {
+      "epoch": 1.5950920245398774,
+      "grad_norm": 0.786145806312561,
+      "learning_rate": 0.0002,
+      "loss": 0.9264,
+      "step": 650
+    },
+    {
+      "epoch": 1.6196319018404908,
+      "grad_norm": 0.8748652935028076,
+      "learning_rate": 0.0002,
+      "loss": 0.8515,
+      "step": 660
+    },
+    {
+      "epoch": 1.6441717791411041,
+      "grad_norm": 0.6184355020523071,
+      "learning_rate": 0.0002,
+      "loss": 0.8818,
+      "step": 670
+    },
+    {
+      "epoch": 1.668711656441718,
+      "grad_norm": 1.0236579179763794,
+      "learning_rate": 0.0002,
+      "loss": 0.9484,
+      "step": 680
+    },
+    {
+      "epoch": 1.6932515337423313,
+      "grad_norm": 0.9815388917922974,
+      "learning_rate": 0.0002,
+      "loss": 0.9078,
+      "step": 690
+    },
+    {
+      "epoch": 1.7177914110429446,
+      "grad_norm": 0.6398373246192932,
+      "learning_rate": 0.0002,
+      "loss": 0.9036,
+      "step": 700
+    },
+    {
+      "epoch": 1.7423312883435584,
+      "grad_norm": 0.6157359480857849,
+      "learning_rate": 0.0002,
+      "loss": 0.8702,
+      "step": 710
+    },
+    {
+      "epoch": 1.7668711656441718,
+      "grad_norm": 0.7352675795555115,
+      "learning_rate": 0.0002,
+      "loss": 0.8712,
+      "step": 720
+    },
+    {
+      "epoch": 1.7914110429447851,
+      "grad_norm": 0.6840626001358032,
+      "learning_rate": 0.0002,
+      "loss": 0.8737,
+      "step": 730
+    },
+    {
+      "epoch": 1.8159509202453987,
+      "grad_norm": 0.8363635540008545,
+      "learning_rate": 0.0002,
+      "loss": 0.9113,
+      "step": 740
+    },
+    {
+      "epoch": 1.8404907975460123,
+      "grad_norm": 1.7173194885253906,
+      "learning_rate": 0.0002,
+      "loss": 0.973,
+      "step": 750
+    },
+    {
+      "epoch": 1.8650306748466257,
+      "grad_norm": 0.8277300000190735,
+      "learning_rate": 0.0002,
+      "loss": 0.9438,
+      "step": 760
+    },
+    {
+      "epoch": 1.8895705521472392,
+      "grad_norm": 0.6714297533035278,
+      "learning_rate": 0.0002,
+      "loss": 0.919,
+      "step": 770
+    },
+    {
+      "epoch": 1.9141104294478528,
+      "grad_norm": 0.7646296620368958,
+      "learning_rate": 0.0002,
+      "loss": 0.889,
+      "step": 780
+    },
+    {
+      "epoch": 1.9386503067484662,
+      "grad_norm": 0.9777507781982422,
+      "learning_rate": 0.0002,
+      "loss": 0.8433,
+      "step": 790
+    },
+    {
+      "epoch": 1.9631901840490797,
+      "grad_norm": 0.7732912302017212,
+      "learning_rate": 0.0002,
+      "loss": 0.9255,
+      "step": 800
+    },
+    {
+      "epoch": 1.9877300613496933,
+      "grad_norm": 0.7485368847846985,
+      "learning_rate": 0.0002,
+      "loss": 0.9556,
+      "step": 810
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 1.335689663887024,
+      "eval_runtime": 80.1256,
+      "eval_samples_per_second": 5.691,
+      "eval_steps_per_second": 0.711,
+      "step": 815
+    },
+    {
+      "epoch": 2.0122699386503067,
+      "grad_norm": 0.7061107158660889,
+      "learning_rate": 0.0002,
+      "loss": 0.9555,
+      "step": 820
+    },
+    {
+      "epoch": 2.03680981595092,
+      "grad_norm": 1.101184368133545,
+      "learning_rate": 0.0002,
+      "loss": 0.7363,
+      "step": 830
+    },
+    {
+      "epoch": 2.061349693251534,
+      "grad_norm": 0.6724491119384766,
+      "learning_rate": 0.0002,
+      "loss": 0.7269,
+      "step": 840
+    },
+    {
+      "epoch": 2.085889570552147,
+      "grad_norm": 0.8159838318824768,
+      "learning_rate": 0.0002,
+      "loss": 0.6836,
+      "step": 850
+    },
+    {
+      "epoch": 2.1104294478527605,
+      "grad_norm": 0.9286916255950928,
+      "learning_rate": 0.0002,
+      "loss": 0.7237,
+      "step": 860
+    },
+    {
+      "epoch": 2.1349693251533743,
+      "grad_norm": 0.9122375845909119,
+      "learning_rate": 0.0002,
+      "loss": 0.7258,
+      "step": 870
+    },
+    {
+      "epoch": 2.1595092024539877,
+      "grad_norm": 0.9655355215072632,
+      "learning_rate": 0.0002,
+      "loss": 0.7164,
+      "step": 880
+    },
+    {
+      "epoch": 2.184049079754601,
+      "grad_norm": 1.1539593935012817,
+      "learning_rate": 0.0002,
+      "loss": 0.741,
+      "step": 890
+    },
+    {
+      "epoch": 2.208588957055215,
+      "grad_norm": 1.0535199642181396,
+      "learning_rate": 0.0002,
+      "loss": 0.7368,
+      "step": 900
+    },
+    {
+      "epoch": 2.233128834355828,
+      "grad_norm": 1.2841371297836304,
+      "learning_rate": 0.0002,
+      "loss": 0.6615,
+      "step": 910
+    },
+    {
+      "epoch": 2.2576687116564416,
+      "grad_norm": 0.8669798970222473,
+      "learning_rate": 0.0002,
+      "loss": 0.767,
+      "step": 920
+    },
+    {
+      "epoch": 2.2822085889570554,
+      "grad_norm": 0.8732201457023621,
+      "learning_rate": 0.0002,
+      "loss": 0.7303,
+      "step": 930
+    },
+    {
+      "epoch": 2.3067484662576687,
+      "grad_norm": 0.9955021739006042,
+      "learning_rate": 0.0002,
+      "loss": 0.7205,
+      "step": 940
+    },
+    {
+      "epoch": 2.331288343558282,
+      "grad_norm": 1.0610932111740112,
+      "learning_rate": 0.0002,
+      "loss": 0.6781,
+      "step": 950
+    },
+    {
+      "epoch": 2.355828220858896,
+      "grad_norm": 0.9506151080131531,
+      "learning_rate": 0.0002,
+      "loss": 0.729,
+      "step": 960
+    },
+    {
+      "epoch": 2.3803680981595092,
+      "grad_norm": 1.259052038192749,
+      "learning_rate": 0.0002,
+      "loss": 0.6784,
+      "step": 970
+    },
+    {
+      "epoch": 2.4049079754601226,
+      "grad_norm": 1.1822247505187988,
+      "learning_rate": 0.0002,
+      "loss": 0.6668,
+      "step": 980
+    },
+    {
+      "epoch": 2.4294478527607364,
+      "grad_norm": 0.6920015811920166,
+      "learning_rate": 0.0002,
+      "loss": 0.6623,
+      "step": 990
+    },
+    {
+      "epoch": 2.4539877300613497,
+      "grad_norm": 0.7560105323791504,
+      "learning_rate": 0.0002,
+      "loss": 0.6802,
+      "step": 1000
+    },
+    {
+      "epoch": 2.478527607361963,
+      "grad_norm": 0.8672356009483337,
+      "learning_rate": 0.0002,
+      "loss": 0.7685,
+      "step": 1010
+    },
+    {
+      "epoch": 2.5030674846625764,
+      "grad_norm": 1.1074872016906738,
+      "learning_rate": 0.0002,
+      "loss": 0.7058,
+      "step": 1020
+    },
+    {
+      "epoch": 2.5276073619631902,
+      "grad_norm": 1.1430137157440186,
+      "learning_rate": 0.0002,
+      "loss": 0.7746,
+      "step": 1030
+    },
+    {
+      "epoch": 2.5521472392638036,
+      "grad_norm": 1.6926707029342651,
+      "learning_rate": 0.0002,
+      "loss": 0.7444,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5766871165644174,
+      "grad_norm": 1.0170048475265503,
+      "learning_rate": 0.0002,
+      "loss": 0.7131,
+      "step": 1050
+    },
+    {
+      "epoch": 2.6012269938650308,
+      "grad_norm": 1.1462562084197998,
+      "learning_rate": 0.0002,
+      "loss": 0.7275,
+      "step": 1060
+    },
+    {
+      "epoch": 2.625766871165644,
+      "grad_norm": 1.127669334411621,
+      "learning_rate": 0.0002,
+      "loss": 0.6736,
+      "step": 1070
+    },
+    {
+      "epoch": 2.6503067484662575,
+      "grad_norm": 0.9649022221565247,
+      "learning_rate": 0.0002,
+      "loss": 0.7218,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6748466257668713,
+      "grad_norm": 0.9426548480987549,
+      "learning_rate": 0.0002,
+      "loss": 0.7651,
+      "step": 1090
+    },
+    {
+      "epoch": 2.6993865030674846,
+      "grad_norm": 1.1191051006317139,
+      "learning_rate": 0.0002,
+      "loss": 0.7829,
+      "step": 1100
+    },
+    {
+      "epoch": 2.7239263803680984,
+      "grad_norm": 2.229809284210205,
+      "learning_rate": 0.0002,
+      "loss": 0.7238,
+      "step": 1110
+    },
+    {
+      "epoch": 2.7484662576687118,
+      "grad_norm": 1.2478930950164795,
+      "learning_rate": 0.0002,
+      "loss": 0.6828,
+      "step": 1120
+    },
+    {
+      "epoch": 2.773006134969325,
+      "grad_norm": 0.9907709360122681,
+      "learning_rate": 0.0002,
+      "loss": 0.7194,
+      "step": 1130
+    },
+    {
+      "epoch": 2.7975460122699385,
+      "grad_norm": 1.1670643091201782,
+      "learning_rate": 0.0002,
+      "loss": 0.6531,
+      "step": 1140
+    },
+    {
+      "epoch": 2.8220858895705523,
+      "grad_norm": 1.1675913333892822,
+      "learning_rate": 0.0002,
+      "loss": 0.7076,
+      "step": 1150
+    },
+    {
+      "epoch": 2.8466257668711656,
+      "grad_norm": 0.9909353256225586,
+      "learning_rate": 0.0002,
+      "loss": 0.6915,
+      "step": 1160
+    },
+    {
+      "epoch": 2.871165644171779,
+      "grad_norm": 0.8759778141975403,
+      "learning_rate": 0.0002,
+      "loss": 0.7294,
+      "step": 1170
+    },
+    {
+      "epoch": 2.895705521472393,
+      "grad_norm": 0.8080666661262512,
+      "learning_rate": 0.0002,
+      "loss": 0.6451,
+      "step": 1180
+    },
+    {
+      "epoch": 2.920245398773006,
+      "grad_norm": 0.9743189811706543,
+      "learning_rate": 0.0002,
+      "loss": 0.7331,
+      "step": 1190
+    },
+    {
+      "epoch": 2.9447852760736195,
+      "grad_norm": 0.8573821187019348,
+      "learning_rate": 0.0002,
+      "loss": 0.7471,
+      "step": 1200
+    },
+    {
+      "epoch": 2.969325153374233,
+      "grad_norm": 1.0196788311004639,
+      "learning_rate": 0.0002,
+      "loss": 0.7396,
+      "step": 1210
+    },
+    {
+      "epoch": 2.9938650306748467,
+      "grad_norm": 0.8840402960777283,
+      "learning_rate": 0.0002,
+      "loss": 0.7288,
+      "step": 1220
+    },
+    {
+      "epoch": 2.9987730061349693,
+      "eval_loss": 1.4348119497299194,
+      "eval_runtime": 80.7583,
+      "eval_samples_per_second": 5.646,
+      "eval_steps_per_second": 0.706,
+      "step": 1222
+    },
+    {
+      "epoch": 3.01840490797546,
+      "grad_norm": 1.2376960515975952,
+      "learning_rate": 0.0002,
+      "loss": 0.5671,
+      "step": 1230
+    },
+    {
+      "epoch": 3.042944785276074,
+      "grad_norm": 0.9861388206481934,
+      "learning_rate": 0.0002,
+      "loss": 0.5176,
+      "step": 1240
+    },
+    {
+      "epoch": 3.067484662576687,
+      "grad_norm": 1.2193198204040527,
+      "learning_rate": 0.0002,
+      "loss": 0.5623,
+      "step": 1250
+    },
+    {
+      "epoch": 3.0920245398773005,
+      "grad_norm": 1.1927645206451416,
+      "learning_rate": 0.0002,
+      "loss": 0.5727,
+      "step": 1260
+    },
+    {
+      "epoch": 3.116564417177914,
+      "grad_norm": 1.0420559644699097,
+      "learning_rate": 0.0002,
+      "loss": 0.5296,
+      "step": 1270
+    },
+    {
+      "epoch": 3.1411042944785277,
+      "grad_norm": 1.014664649963379,
+      "learning_rate": 0.0002,
+      "loss": 0.5322,
+      "step": 1280
+    },
+    {
+      "epoch": 3.165644171779141,
+      "grad_norm": 1.3103076219558716,
+      "learning_rate": 0.0002,
+      "loss": 0.5519,
+      "step": 1290
+    },
+    {
+      "epoch": 3.190184049079755,
+      "grad_norm": 1.2735213041305542,
+      "learning_rate": 0.0002,
+      "loss": 0.5133,
+      "step": 1300
+    },
+    {
+      "epoch": 3.214723926380368,
+      "grad_norm": 1.147608995437622,
+      "learning_rate": 0.0002,
+      "loss": 0.514,
+      "step": 1310
+    },
+    {
+      "epoch": 3.2392638036809815,
+      "grad_norm": 0.922386884689331,
+      "learning_rate": 0.0002,
+      "loss": 0.58,
+      "step": 1320
+    },
+    {
+      "epoch": 3.263803680981595,
+      "grad_norm": 1.1271566152572632,
+      "learning_rate": 0.0002,
+      "loss": 0.5425,
+      "step": 1330
+    },
+    {
+      "epoch": 3.2883435582822087,
+      "grad_norm": 1.2994354963302612,
+      "learning_rate": 0.0002,
+      "loss": 0.519,
+      "step": 1340
+    },
+    {
+      "epoch": 3.312883435582822,
+      "grad_norm": 1.0001686811447144,
+      "learning_rate": 0.0002,
+      "loss": 0.5521,
+      "step": 1350
+    },
+    {
+      "epoch": 3.3374233128834354,
+      "grad_norm": 1.6737695932388306,
+      "learning_rate": 0.0002,
+      "loss": 0.564,
+      "step": 1360
+    },
+    {
+      "epoch": 3.361963190184049,
+      "grad_norm": 1.1842162609100342,
+      "learning_rate": 0.0002,
+      "loss": 0.5529,
+      "step": 1370
+    },
+    {
+      "epoch": 3.3865030674846626,
+      "grad_norm": 1.1873128414154053,
+      "learning_rate": 0.0002,
+      "loss": 0.569,
+      "step": 1380
+    },
+    {
+      "epoch": 3.411042944785276,
+      "grad_norm": 1.4505162239074707,
+      "learning_rate": 0.0002,
+      "loss": 0.5199,
+      "step": 1390
+    },
+    {
+      "epoch": 3.4355828220858897,
+      "grad_norm": 1.254621982574463,
+      "learning_rate": 0.0002,
+      "loss": 0.5536,
+      "step": 1400
+    },
+    {
+      "epoch": 3.460122699386503,
+      "grad_norm": 1.5493544340133667,
+      "learning_rate": 0.0002,
+      "loss": 0.5547,
+      "step": 1410
+    },
+    {
+      "epoch": 3.4846625766871164,
+      "grad_norm": 1.4435759782791138,
+      "learning_rate": 0.0002,
+      "loss": 0.542,
+      "step": 1420
+    },
+    {
+      "epoch": 3.5092024539877302,
+      "grad_norm": 1.5885447263717651,
+      "learning_rate": 0.0002,
+      "loss": 0.6124,
+      "step": 1430
+    },
+    {
+      "epoch": 3.5337423312883436,
+      "grad_norm": 1.3357222080230713,
+      "learning_rate": 0.0002,
+      "loss": 0.5737,
+      "step": 1440
+    },
+    {
+      "epoch": 3.558282208588957,
+      "grad_norm": 1.8510551452636719,
+      "learning_rate": 0.0002,
+      "loss": 0.5903,
+      "step": 1450
+    },
+    {
+      "epoch": 3.5828220858895703,
+      "grad_norm": 1.1081656217575073,
+      "learning_rate": 0.0002,
+      "loss": 0.5214,
+      "step": 1460
+    },
+    {
+      "epoch": 3.607361963190184,
+      "grad_norm": 1.2849210500717163,
+      "learning_rate": 0.0002,
+      "loss": 0.5351,
+      "step": 1470
+    },
+    {
+      "epoch": 3.6319018404907975,
+      "grad_norm": 1.2472909688949585,
+      "learning_rate": 0.0002,
+      "loss": 0.5364,
+      "step": 1480
+    },
+    {
+      "epoch": 3.6564417177914113,
+      "grad_norm": 1.1466434001922607,
+      "learning_rate": 0.0002,
+      "loss": 0.5371,
+      "step": 1490
+    },
+    {
+      "epoch": 3.6809815950920246,
+      "grad_norm": 1.325461506843567,
+      "learning_rate": 0.0002,
+      "loss": 0.565,
+      "step": 1500
+    },
+    {
+      "epoch": 3.705521472392638,
+      "grad_norm": 1.4214563369750977,
+      "learning_rate": 0.0002,
+      "loss": 0.6154,
+      "step": 1510
+    },
+    {
+      "epoch": 3.7300613496932513,
+      "grad_norm": 1.1496703624725342,
+      "learning_rate": 0.0002,
+      "loss": 0.5726,
+      "step": 1520
+    },
+    {
+      "epoch": 3.754601226993865,
+      "grad_norm": 1.205261468887329,
+      "learning_rate": 0.0002,
+      "loss": 0.5687,
+      "step": 1530
+    },
+    {
+      "epoch": 3.7791411042944785,
+      "grad_norm": 1.6516913175582886,
+      "learning_rate": 0.0002,
+      "loss": 0.5593,
+      "step": 1540
+    },
+    {
+      "epoch": 3.8036809815950923,
+      "grad_norm": 1.7035053968429565,
+      "learning_rate": 0.0002,
+      "loss": 0.5767,
+      "step": 1550
+    },
+    {
+      "epoch": 3.8282208588957056,
+      "grad_norm": 1.463699460029602,
+      "learning_rate": 0.0002,
+      "loss": 0.5948,
+      "step": 1560
+    },
+    {
+      "epoch": 3.852760736196319,
+      "grad_norm": 1.7970601320266724,
+      "learning_rate": 0.0002,
+      "loss": 0.5326,
+      "step": 1570
+    },
+    {
+      "epoch": 3.8773006134969323,
+      "grad_norm": 1.2967402935028076,
+      "learning_rate": 0.0002,
+      "loss": 0.5919,
+      "step": 1580
+    },
+    {
+      "epoch": 3.901840490797546,
+      "grad_norm": 1.0639182329177856,
+      "learning_rate": 0.0002,
+      "loss": 0.5502,
+      "step": 1590
+    },
+    {
+      "epoch": 3.9263803680981595,
+      "grad_norm": 1.4388368129730225,
+      "learning_rate": 0.0002,
+      "loss": 0.5281,
+      "step": 1600
+    },
+    {
+      "epoch": 3.950920245398773,
+      "grad_norm": 1.2425099611282349,
+      "learning_rate": 0.0002,
+      "loss": 0.6305,
+      "step": 1610
+    },
+    {
+      "epoch": 3.9754601226993866,
+      "grad_norm": 1.3989970684051514,
+      "learning_rate": 0.0002,
+      "loss": 0.5548,
+      "step": 1620
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.9850553274154663,
+      "learning_rate": 0.0002,
+      "loss": 0.5661,
+      "step": 1630
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.5340977907180786,
+      "eval_runtime": 79.7233,
+      "eval_samples_per_second": 5.72,
+      "eval_steps_per_second": 0.715,
+      "step": 1630
+    },
+    {
+      "epoch": 4.024539877300613,
+      "grad_norm": 2.1861188411712646,
+      "learning_rate": 0.0002,
+      "loss": 0.4365,
+      "step": 1640
+    },
+    {
+      "epoch": 4.049079754601227,
+      "grad_norm": 1.4574151039123535,
+      "learning_rate": 0.0002,
+      "loss": 0.3987,
+      "step": 1650
+    },
+    {
+      "epoch": 4.07361963190184,
+      "grad_norm": 1.093583345413208,
+      "learning_rate": 0.0002,
+      "loss": 0.404,
+      "step": 1660
+    },
+    {
+      "epoch": 4.098159509202454,
+      "grad_norm": 1.4852519035339355,
+      "learning_rate": 0.0002,
+      "loss": 0.4493,
+      "step": 1670
+    },
+    {
+      "epoch": 4.122699386503068,
+      "grad_norm": 1.9289690256118774,
+      "learning_rate": 0.0002,
+      "loss": 0.3941,
+      "step": 1680
+    },
+    {
+      "epoch": 4.147239263803681,
+      "grad_norm": 1.286780834197998,
+      "learning_rate": 0.0002,
+      "loss": 0.4015,
+      "step": 1690
+    },
+    {
+      "epoch": 4.171779141104294,
+      "grad_norm": 1.1915839910507202,
+      "learning_rate": 0.0002,
+      "loss": 0.396,
+      "step": 1700
+    },
+    {
+      "epoch": 4.196319018404908,
+      "grad_norm": 0.9827993512153625,
+      "learning_rate": 0.0002,
+      "loss": 0.4053,
+      "step": 1710
+    },
+    {
+      "epoch": 4.220858895705521,
+      "grad_norm": 1.314813494682312,
+      "learning_rate": 0.0002,
+      "loss": 0.4341,
+      "step": 1720
+    },
+    {
+      "epoch": 4.245398773006135,
+      "grad_norm": 1.6890170574188232,
+      "learning_rate": 0.0002,
+      "loss": 0.4187,
+      "step": 1730
+    },
+    {
+      "epoch": 4.269938650306749,
+      "grad_norm": 1.5559035539627075,
+      "learning_rate": 0.0002,
+      "loss": 0.4047,
+      "step": 1740
+    },
+    {
+      "epoch": 4.294478527607362,
+      "grad_norm": 1.8186208009719849,
+      "learning_rate": 0.0002,
+      "loss": 0.4398,
+      "step": 1750
+    },
+    {
+      "epoch": 4.319018404907975,
+      "grad_norm": 0.9572193026542664,
+      "learning_rate": 0.0002,
+      "loss": 0.4223,
+      "step": 1760
+    },
+    {
+      "epoch": 4.343558282208589,
+      "grad_norm": 2.1470448970794678,
+      "learning_rate": 0.0002,
+      "loss": 0.5141,
+      "step": 1770
+    },
+    {
+      "epoch": 4.368098159509202,
+      "grad_norm": 1.2788262367248535,
+      "learning_rate": 0.0002,
+      "loss": 0.4051,
+      "step": 1780
+    },
+    {
+      "epoch": 4.392638036809816,
+      "grad_norm": 1.8074915409088135,
+      "learning_rate": 0.0002,
+      "loss": 0.4574,
+      "step": 1790
+    },
+    {
+      "epoch": 4.41717791411043,
+      "grad_norm": 0.9725378751754761,
+      "learning_rate": 0.0002,
+      "loss": 0.4195,
+      "step": 1800
+    },
+    {
+      "epoch": 4.441717791411043,
+      "grad_norm": 1.7154884338378906,
+      "learning_rate": 0.0002,
+      "loss": 0.4267,
+      "step": 1810
+    },
+    {
+      "epoch": 4.466257668711656,
+      "grad_norm": 1.07939612865448,
+      "learning_rate": 0.0002,
+      "loss": 0.4059,
+      "step": 1820
+    },
+    {
+      "epoch": 4.49079754601227,
+      "grad_norm": 1.1510107517242432,
+      "learning_rate": 0.0002,
+      "loss": 0.4572,
+      "step": 1830
+    },
+    {
+      "epoch": 4.515337423312883,
+      "grad_norm": 1.2309428453445435,
+      "learning_rate": 0.0002,
+      "loss": 0.4053,
+      "step": 1840
+    },
+    {
+      "epoch": 4.539877300613497,
+      "grad_norm": 1.31099271774292,
+      "learning_rate": 0.0002,
+      "loss": 0.4275,
+      "step": 1850
+    },
+    {
+      "epoch": 4.564417177914111,
+      "grad_norm": 1.2938907146453857,
+      "learning_rate": 0.0002,
+      "loss": 0.4755,
+      "step": 1860
+    },
+    {
+      "epoch": 4.588957055214724,
+      "grad_norm": 1.6019647121429443,
+      "learning_rate": 0.0002,
+      "loss": 0.478,
+      "step": 1870
+    },
+    {
+      "epoch": 4.613496932515337,
+      "grad_norm": 1.9093713760375977,
+      "learning_rate": 0.0002,
+      "loss": 0.4492,
+      "step": 1880
+    },
+    {
+      "epoch": 4.638036809815951,
+      "grad_norm": 1.4761801958084106,
+      "learning_rate": 0.0002,
+      "loss": 0.4522,
+      "step": 1890
+    },
+    {
+      "epoch": 4.662576687116564,
+      "grad_norm": 1.2435506582260132,
+      "learning_rate": 0.0002,
+      "loss": 0.5078,
+      "step": 1900
+    },
+    {
+      "epoch": 4.6871165644171775,
+      "grad_norm": 1.5027297735214233,
+      "learning_rate": 0.0002,
+      "loss": 0.4656,
+      "step": 1910
+    },
+    {
+      "epoch": 4.711656441717792,
+      "grad_norm": 1.323907494544983,
+      "learning_rate": 0.0002,
+      "loss": 0.4428,
+      "step": 1920
+    },
+    {
+      "epoch": 4.736196319018405,
+      "grad_norm": 1.1643304824829102,
+      "learning_rate": 0.0002,
+      "loss": 0.4113,
+      "step": 1930
+    },
+    {
+      "epoch": 4.7607361963190185,
+      "grad_norm": 1.4745224714279175,
+      "learning_rate": 0.0002,
+      "loss": 0.4548,
+      "step": 1940
+    },
+    {
+      "epoch": 4.785276073619632,
+      "grad_norm": 1.6811035871505737,
+      "learning_rate": 0.0002,
+      "loss": 0.4447,
+      "step": 1950
+    },
+    {
+      "epoch": 4.809815950920245,
+      "grad_norm": 1.0878573656082153,
+      "learning_rate": 0.0002,
+      "loss": 0.4669,
+      "step": 1960
+    },
+    {
+      "epoch": 4.8343558282208585,
+      "grad_norm": 1.640516996383667,
+      "learning_rate": 0.0002,
+      "loss": 0.4589,
+      "step": 1970
+    },
+    {
+      "epoch": 4.858895705521473,
+      "grad_norm": 1.305409550666809,
+      "learning_rate": 0.0002,
+      "loss": 0.4534,
+      "step": 1980
+    },
+    {
+      "epoch": 4.883435582822086,
+      "grad_norm": 1.5608975887298584,
+      "learning_rate": 0.0002,
+      "loss": 0.4512,
+      "step": 1990
+    },
+    {
+      "epoch": 4.9079754601226995,
+      "grad_norm": 1.3681490421295166,
+      "learning_rate": 0.0002,
+      "loss": 0.4602,
+      "step": 2000
+    },
+    {
+      "epoch": 4.932515337423313,
+      "grad_norm": 1.241847276687622,
+      "learning_rate": 0.0002,
+      "loss": 0.4445,
+      "step": 2010
+    },
+    {
+      "epoch": 4.957055214723926,
+      "grad_norm": 1.1561694145202637,
+      "learning_rate": 0.0002,
+      "loss": 0.502,
+      "step": 2020
+    },
+    {
+      "epoch": 4.9815950920245395,
+      "grad_norm": 1.8284252882003784,
+      "learning_rate": 0.0002,
+      "loss": 0.4586,
+      "step": 2030
+    },
+    {
+      "epoch": 4.99877300613497,
+      "eval_loss": 1.702454686164856,
+      "eval_runtime": 79.3705,
+      "eval_samples_per_second": 5.745,
+      "eval_steps_per_second": 0.718,
+      "step": 2037
+    },
+    {
+      "epoch": 5.006134969325154,
+      "grad_norm": 0.8960777521133423,
+      "learning_rate": 0.0002,
+      "loss": 0.4039,
+      "step": 2040
+    },
+    {
+      "epoch": 5.030674846625767,
+      "grad_norm": 2.3825552463531494,
+      "learning_rate": 0.0002,
+      "loss": 0.3437,
+      "step": 2050
+    },
+    {
+      "epoch": 5.0552147239263805,
+      "grad_norm": 1.1818524599075317,
+      "learning_rate": 0.0002,
+      "loss": 0.34,
+      "step": 2060
+    },
+    {
+      "epoch": 5.079754601226994,
+      "grad_norm": 1.327019214630127,
+      "learning_rate": 0.0002,
+      "loss": 0.3574,
+      "step": 2070
+    },
+    {
+      "epoch": 5.104294478527607,
+      "grad_norm": 1.3264802694320679,
+      "learning_rate": 0.0002,
+      "loss": 0.3471,
+      "step": 2080
+    },
+    {
+      "epoch": 5.128834355828221,
+      "grad_norm": 1.3847426176071167,
+      "learning_rate": 0.0002,
+      "loss": 0.3311,
+      "step": 2090
+    },
+    {
+      "epoch": 5.153374233128835,
+      "grad_norm": 2.4007370471954346,
+      "learning_rate": 0.0002,
+      "loss": 0.3588,
+      "step": 2100
+    },
+    {
+      "epoch": 5.177914110429448,
+      "grad_norm": 1.1784768104553223,
+      "learning_rate": 0.0002,
+      "loss": 0.3359,
+      "step": 2110
+    },
+    {
+      "epoch": 5.2024539877300615,
+      "grad_norm": 1.4660961627960205,
+      "learning_rate": 0.0002,
+      "loss": 0.3955,
+      "step": 2120
+    },
+    {
+      "epoch": 5.226993865030675,
+      "grad_norm": 2.423863410949707,
+      "learning_rate": 0.0002,
+      "loss": 0.3947,
+      "step": 2130
+    },
+    {
+      "epoch": 5.251533742331288,
+      "grad_norm": 1.2838112115859985,
+      "learning_rate": 0.0002,
+      "loss": 0.3396,
+      "step": 2140
+    },
+    {
+      "epoch": 5.276073619631902,
+      "grad_norm": 1.5166656970977783,
+      "learning_rate": 0.0002,
+      "loss": 0.3538,
+      "step": 2150
+    },
+    {
+      "epoch": 5.300613496932515,
+      "grad_norm": 1.9416661262512207,
+      "learning_rate": 0.0002,
+      "loss": 0.3618,
+      "step": 2160
+    },
+    {
+      "epoch": 5.325153374233129,
+      "grad_norm": 1.56295645236969,
+      "learning_rate": 0.0002,
+      "loss": 0.3592,
+      "step": 2170
+    },
+    {
+      "epoch": 5.3496932515337425,
+      "grad_norm": 1.5476324558258057,
+      "learning_rate": 0.0002,
+      "loss": 0.3571,
+      "step": 2180
+    },
+    {
+      "epoch": 5.374233128834356,
+      "grad_norm": 1.5130436420440674,
+      "learning_rate": 0.0002,
+      "loss": 0.3448,
+      "step": 2190
+    },
+    {
+      "epoch": 5.398773006134969,
+      "grad_norm": 1.7863224744796753,
+      "learning_rate": 0.0002,
+      "loss": 0.3683,
+      "step": 2200
+    },
+    {
+      "epoch": 5.423312883435583,
+      "grad_norm": 1.837112545967102,
+      "learning_rate": 0.0002,
+      "loss": 0.348,
+      "step": 2210
+    },
+    {
+      "epoch": 5.447852760736196,
+      "grad_norm": 1.7759106159210205,
+      "learning_rate": 0.0002,
+      "loss": 0.3835,
+      "step": 2220
+    },
+    {
+      "epoch": 5.47239263803681,
+      "grad_norm": 1.3154255151748657,
+      "learning_rate": 0.0002,
+      "loss": 0.3909,
+      "step": 2230
+    },
+    {
+      "epoch": 5.4969325153374236,
+      "grad_norm": 1.8282175064086914,
+      "learning_rate": 0.0002,
+      "loss": 0.3792,
+      "step": 2240
+    },
+    {
+      "epoch": 5.521472392638037,
+      "grad_norm": 1.486496090888977,
+      "learning_rate": 0.0002,
+      "loss": 0.3751,
+      "step": 2250
+    },
+    {
+      "epoch": 5.54601226993865,
+      "grad_norm": 1.2023154497146606,
+      "learning_rate": 0.0002,
+      "loss": 0.3616,
+      "step": 2260
+    },
+    {
+      "epoch": 5.570552147239264,
+      "grad_norm": 1.6753804683685303,
+      "learning_rate": 0.0002,
+      "loss": 0.3804,
+      "step": 2270
+    },
+    {
+      "epoch": 5.595092024539877,
+      "grad_norm": 1.472532868385315,
+      "learning_rate": 0.0002,
+      "loss": 0.386,
+      "step": 2280
+    },
+    {
+      "epoch": 5.61963190184049,
+      "grad_norm": 1.6524516344070435,
+      "learning_rate": 0.0002,
+      "loss": 0.3882,
+      "step": 2290
+    },
+    {
+      "epoch": 5.644171779141105,
+      "grad_norm": 1.5600810050964355,
+      "learning_rate": 0.0002,
+      "loss": 0.3757,
+      "step": 2300
+    },
+    {
+      "epoch": 5.668711656441718,
+      "grad_norm": 1.521317958831787,
+      "learning_rate": 0.0002,
+      "loss": 0.3974,
+      "step": 2310
+    },
+    {
+      "epoch": 5.693251533742331,
+      "grad_norm": 1.3662383556365967,
+      "learning_rate": 0.0002,
+      "loss": 0.3625,
+      "step": 2320
+    },
+    {
+      "epoch": 5.717791411042945,
+      "grad_norm": 1.0605283975601196,
+      "learning_rate": 0.0002,
+      "loss": 0.3192,
+      "step": 2330
+    },
+    {
+      "epoch": 5.742331288343558,
+      "grad_norm": 1.3059141635894775,
+      "learning_rate": 0.0002,
+      "loss": 0.3721,
+      "step": 2340
+    },
+    {
+      "epoch": 5.766871165644172,
+      "grad_norm": 1.697109341621399,
+      "learning_rate": 0.0002,
+      "loss": 0.3798,
+      "step": 2350
+    },
+    {
+      "epoch": 5.791411042944786,
+      "grad_norm": 1.7123175859451294,
+      "learning_rate": 0.0002,
+      "loss": 0.3648,
+      "step": 2360
+    },
+    {
+      "epoch": 5.815950920245399,
+      "grad_norm": 1.9613343477249146,
+      "learning_rate": 0.0002,
+      "loss": 0.3891,
+      "step": 2370
+    },
+    {
+      "epoch": 5.840490797546012,
+      "grad_norm": 1.3213013410568237,
+      "learning_rate": 0.0002,
+      "loss": 0.3307,
+      "step": 2380
+    },
+    {
+      "epoch": 5.865030674846626,
+      "grad_norm": 1.56651771068573,
+      "learning_rate": 0.0002,
+      "loss": 0.3987,
+      "step": 2390
+    },
+    {
+      "epoch": 5.889570552147239,
+      "grad_norm": 1.8884061574935913,
+      "learning_rate": 0.0002,
+      "loss": 0.3714,
+      "step": 2400
+    },
+    {
+      "epoch": 5.914110429447852,
+      "grad_norm": 1.8811849355697632,
+      "learning_rate": 0.0002,
+      "loss": 0.3654,
+      "step": 2410
+    },
+    {
+      "epoch": 5.938650306748467,
+      "grad_norm": 1.3935836553573608,
+      "learning_rate": 0.0002,
+      "loss": 0.3823,
+      "step": 2420
+    },
+    {
+      "epoch": 5.96319018404908,
+      "grad_norm": 1.6891478300094604,
+      "learning_rate": 0.0002,
+      "loss": 0.3814,
+      "step": 2430
+    },
+    {
+      "epoch": 5.987730061349693,
+      "grad_norm": 1.5192315578460693,
+      "learning_rate": 0.0002,
+      "loss": 0.3738,
+      "step": 2440
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 1.783948540687561,
+      "eval_runtime": 79.3389,
+      "eval_samples_per_second": 5.747,
+      "eval_steps_per_second": 0.718,
+      "step": 2445
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 3256,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.07273706799104e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}