Upload folder using huggingface_hub

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bca64cc84ff73c12e7cef4b0e172cb7a1f9e426c15879ebb84296fd66907ea53
+size 80755416

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7428340d784699e741dcd69e0578e373472a5d385bcfc5a392f746b79381d28d
+size 80755416

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:017ddd47da9630bba466f8751b922c5ef78f17bdb7dc371e18159a97f7af0882
+size 41136570

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1eb30cbef2b7e1873f8935c9a22f37e4fbe0ac8ecb54a648bad0297a42bf99b
+size 14244

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21878273a735f044e83414a39a919be2969bece88ceb8432a73dc0683c795cc0
+size 1064

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/trainer_state.json

@@ -0,0 +1,857 @@
+{
+  "best_metric": 0.5916207432746887,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125",
+  "epoch": 5.0,
+  "eval_steps": 10,
+  "global_step": 1125,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.044444444444444446,
+      "grad_norm": 0.2940807342529297,
+      "learning_rate": 0.0002,
+      "loss": 1.8338,
+      "step": 10
+    },
+    {
+      "epoch": 0.08888888888888889,
+      "grad_norm": 0.36792996525764465,
+      "learning_rate": 0.0002,
+      "loss": 1.5844,
+      "step": 20
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "grad_norm": 0.44259828329086304,
+      "learning_rate": 0.0002,
+      "loss": 1.3603,
+      "step": 30
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.760607898235321,
+      "learning_rate": 0.0002,
+      "loss": 1.1408,
+      "step": 40
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 0.608131468296051,
+      "learning_rate": 0.0002,
+      "loss": 0.9766,
+      "step": 50
+    },
+    {
+      "epoch": 0.26666666666666666,
+      "grad_norm": 0.5941349864006042,
+      "learning_rate": 0.0002,
+      "loss": 0.9176,
+      "step": 60
+    },
+    {
+      "epoch": 0.3111111111111111,
+      "grad_norm": 0.5939444899559021,
+      "learning_rate": 0.0002,
+      "loss": 0.9081,
+      "step": 70
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 0.6093971729278564,
+      "learning_rate": 0.0002,
+      "loss": 0.8406,
+      "step": 80
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.6158391237258911,
+      "learning_rate": 0.0002,
+      "loss": 0.8562,
+      "step": 90
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 0.4985930621623993,
+      "learning_rate": 0.0002,
+      "loss": 0.8111,
+      "step": 100
+    },
+    {
+      "epoch": 0.4888888888888889,
+      "grad_norm": 0.6661780476570129,
+      "learning_rate": 0.0002,
+      "loss": 0.7963,
+      "step": 110
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.5717976689338684,
+      "learning_rate": 0.0002,
+      "loss": 0.8114,
+      "step": 120
+    },
+    {
+      "epoch": 0.5777777777777777,
+      "grad_norm": 0.5666863918304443,
+      "learning_rate": 0.0002,
+      "loss": 0.7397,
+      "step": 130
+    },
+    {
+      "epoch": 0.6222222222222222,
+      "grad_norm": 0.49229782819747925,
+      "learning_rate": 0.0002,
+      "loss": 0.7671,
+      "step": 140
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 0.48167455196380615,
+      "learning_rate": 0.0002,
+      "loss": 0.7851,
+      "step": 150
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 0.4657461643218994,
+      "learning_rate": 0.0002,
+      "loss": 0.7786,
+      "step": 160
+    },
+    {
+      "epoch": 0.7555555555555555,
+      "grad_norm": 0.39121416211128235,
+      "learning_rate": 0.0002,
+      "loss": 0.7691,
+      "step": 170
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.441007524728775,
+      "learning_rate": 0.0002,
+      "loss": 0.7323,
+      "step": 180
+    },
+    {
+      "epoch": 0.8444444444444444,
+      "grad_norm": 0.4213836193084717,
+      "learning_rate": 0.0002,
+      "loss": 0.7418,
+      "step": 190
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.5080695152282715,
+      "learning_rate": 0.0002,
+      "loss": 0.7451,
+      "step": 200
+    },
+    {
+      "epoch": 0.9333333333333333,
+      "grad_norm": 0.4761652648448944,
+      "learning_rate": 0.0002,
+      "loss": 0.757,
+      "step": 210
+    },
+    {
+      "epoch": 0.9777777777777777,
+      "grad_norm": 0.5242546796798706,
+      "learning_rate": 0.0002,
+      "loss": 0.741,
+      "step": 220
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.7312084436416626,
+      "eval_runtime": 86.046,
+      "eval_samples_per_second": 3.707,
+      "eval_steps_per_second": 0.465,
+      "step": 225
+    },
+    {
+      "epoch": 1.0222222222222221,
+      "grad_norm": 0.4389289915561676,
+      "learning_rate": 0.0002,
+      "loss": 0.7365,
+      "step": 230
+    },
+    {
+      "epoch": 1.0666666666666667,
+      "grad_norm": 0.4293370246887207,
+      "learning_rate": 0.0002,
+      "loss": 0.7223,
+      "step": 240
+    },
+    {
+      "epoch": 1.1111111111111112,
+      "grad_norm": 0.45308825373649597,
+      "learning_rate": 0.0002,
+      "loss": 0.6934,
+      "step": 250
+    },
+    {
+      "epoch": 1.1555555555555554,
+      "grad_norm": 0.4458293318748474,
+      "learning_rate": 0.0002,
+      "loss": 0.7058,
+      "step": 260
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.46963292360305786,
+      "learning_rate": 0.0002,
+      "loss": 0.6956,
+      "step": 270
+    },
+    {
+      "epoch": 1.2444444444444445,
+      "grad_norm": 0.39571475982666016,
+      "learning_rate": 0.0002,
+      "loss": 0.6882,
+      "step": 280
+    },
+    {
+      "epoch": 1.2888888888888888,
+      "grad_norm": 0.4619075655937195,
+      "learning_rate": 0.0002,
+      "loss": 0.7241,
+      "step": 290
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 0.457443505525589,
+      "learning_rate": 0.0002,
+      "loss": 0.7281,
+      "step": 300
+    },
+    {
+      "epoch": 1.3777777777777778,
+      "grad_norm": 0.5545842051506042,
+      "learning_rate": 0.0002,
+      "loss": 0.7138,
+      "step": 310
+    },
+    {
+      "epoch": 1.4222222222222223,
+      "grad_norm": 0.4492949843406677,
+      "learning_rate": 0.0002,
+      "loss": 0.7082,
+      "step": 320
+    },
+    {
+      "epoch": 1.4666666666666668,
+      "grad_norm": 0.46642452478408813,
+      "learning_rate": 0.0002,
+      "loss": 0.6733,
+      "step": 330
+    },
+    {
+      "epoch": 1.511111111111111,
+      "grad_norm": 0.4598081707954407,
+      "learning_rate": 0.0002,
+      "loss": 0.6846,
+      "step": 340
+    },
+    {
+      "epoch": 1.5555555555555556,
+      "grad_norm": 0.34535378217697144,
+      "learning_rate": 0.0002,
+      "loss": 0.6981,
+      "step": 350
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.4226590096950531,
+      "learning_rate": 0.0002,
+      "loss": 0.672,
+      "step": 360
+    },
+    {
+      "epoch": 1.6444444444444444,
+      "grad_norm": 0.47323980927467346,
+      "learning_rate": 0.0002,
+      "loss": 0.6638,
+      "step": 370
+    },
+    {
+      "epoch": 1.6888888888888889,
+      "grad_norm": 0.39699724316596985,
+      "learning_rate": 0.0002,
+      "loss": 0.6813,
+      "step": 380
+    },
+    {
+      "epoch": 1.7333333333333334,
+      "grad_norm": 0.5325330495834351,
+      "learning_rate": 0.0002,
+      "loss": 0.6852,
+      "step": 390
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 0.4336804449558258,
+      "learning_rate": 0.0002,
+      "loss": 0.6716,
+      "step": 400
+    },
+    {
+      "epoch": 1.8222222222222222,
+      "grad_norm": 0.44034498929977417,
+      "learning_rate": 0.0002,
+      "loss": 0.6538,
+      "step": 410
+    },
+    {
+      "epoch": 1.8666666666666667,
+      "grad_norm": 0.4527396261692047,
+      "learning_rate": 0.0002,
+      "loss": 0.6706,
+      "step": 420
+    },
+    {
+      "epoch": 1.911111111111111,
+      "grad_norm": 0.40166375041007996,
+      "learning_rate": 0.0002,
+      "loss": 0.6507,
+      "step": 430
+    },
+    {
+      "epoch": 1.9555555555555557,
+      "grad_norm": 0.49753332138061523,
+      "learning_rate": 0.0002,
+      "loss": 0.6599,
+      "step": 440
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.5019579529762268,
+      "learning_rate": 0.0002,
+      "loss": 0.6482,
+      "step": 450
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.6728096604347229,
+      "eval_runtime": 85.827,
+      "eval_samples_per_second": 3.717,
+      "eval_steps_per_second": 0.466,
+      "step": 450
+    },
+    {
+      "epoch": 2.0444444444444443,
+      "grad_norm": 0.5162734985351562,
+      "learning_rate": 0.0002,
+      "loss": 0.626,
+      "step": 460
+    },
+    {
+      "epoch": 2.088888888888889,
+      "grad_norm": 0.4973997175693512,
+      "learning_rate": 0.0002,
+      "loss": 0.6363,
+      "step": 470
+    },
+    {
+      "epoch": 2.1333333333333333,
+      "grad_norm": 0.6232216358184814,
+      "learning_rate": 0.0002,
+      "loss": 0.6225,
+      "step": 480
+    },
+    {
+      "epoch": 2.1777777777777776,
+      "grad_norm": 0.5598695278167725,
+      "learning_rate": 0.0002,
+      "loss": 0.6038,
+      "step": 490
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 0.5359365344047546,
+      "learning_rate": 0.0002,
+      "loss": 0.6247,
+      "step": 500
+    },
+    {
+      "epoch": 2.2666666666666666,
+      "grad_norm": 0.5992209315299988,
+      "learning_rate": 0.0002,
+      "loss": 0.6203,
+      "step": 510
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 0.4921131730079651,
+      "learning_rate": 0.0002,
+      "loss": 0.6416,
+      "step": 520
+    },
+    {
+      "epoch": 2.3555555555555556,
+      "grad_norm": 0.5449638366699219,
+      "learning_rate": 0.0002,
+      "loss": 0.6157,
+      "step": 530
+    },
+    {
+      "epoch": 2.4,
+      "grad_norm": 0.45068296790122986,
+      "learning_rate": 0.0002,
+      "loss": 0.6244,
+      "step": 540
+    },
+    {
+      "epoch": 2.4444444444444446,
+      "grad_norm": 0.5800191164016724,
+      "learning_rate": 0.0002,
+      "loss": 0.6368,
+      "step": 550
+    },
+    {
+      "epoch": 2.488888888888889,
+      "grad_norm": 0.4825330674648285,
+      "learning_rate": 0.0002,
+      "loss": 0.6437,
+      "step": 560
+    },
+    {
+      "epoch": 2.533333333333333,
+      "grad_norm": 0.6174261569976807,
+      "learning_rate": 0.0002,
+      "loss": 0.606,
+      "step": 570
+    },
+    {
+      "epoch": 2.5777777777777775,
+      "grad_norm": 0.5854967832565308,
+      "learning_rate": 0.0002,
+      "loss": 0.5666,
+      "step": 580
+    },
+    {
+      "epoch": 2.6222222222222222,
+      "grad_norm": 0.44053414463996887,
+      "learning_rate": 0.0002,
+      "loss": 0.6017,
+      "step": 590
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 0.4803224503993988,
+      "learning_rate": 0.0002,
+      "loss": 0.6032,
+      "step": 600
+    },
+    {
+      "epoch": 2.7111111111111112,
+      "grad_norm": 0.5700278282165527,
+      "learning_rate": 0.0002,
+      "loss": 0.6238,
+      "step": 610
+    },
+    {
+      "epoch": 2.7555555555555555,
+      "grad_norm": 0.5318542122840881,
+      "learning_rate": 0.0002,
+      "loss": 0.5767,
+      "step": 620
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 0.5594431161880493,
+      "learning_rate": 0.0002,
+      "loss": 0.5849,
+      "step": 630
+    },
+    {
+      "epoch": 2.8444444444444446,
+      "grad_norm": 0.5598340630531311,
+      "learning_rate": 0.0002,
+      "loss": 0.6094,
+      "step": 640
+    },
+    {
+      "epoch": 2.888888888888889,
+      "grad_norm": 0.5017251372337341,
+      "learning_rate": 0.0002,
+      "loss": 0.6059,
+      "step": 650
+    },
+    {
+      "epoch": 2.9333333333333336,
+      "grad_norm": 0.46027806401252747,
+      "learning_rate": 0.0002,
+      "loss": 0.6047,
+      "step": 660
+    },
+    {
+      "epoch": 2.977777777777778,
+      "grad_norm": 0.5706299543380737,
+      "learning_rate": 0.0002,
+      "loss": 0.5912,
+      "step": 670
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 0.6362627148628235,
+      "eval_runtime": 85.1312,
+      "eval_samples_per_second": 3.747,
+      "eval_steps_per_second": 0.47,
+      "step": 675
+    },
+    {
+      "epoch": 3.022222222222222,
+      "grad_norm": 0.6016985774040222,
+      "learning_rate": 0.0002,
+      "loss": 0.5507,
+      "step": 680
+    },
+    {
+      "epoch": 3.066666666666667,
+      "grad_norm": 0.544710099697113,
+      "learning_rate": 0.0002,
+      "loss": 0.538,
+      "step": 690
+    },
+    {
+      "epoch": 3.111111111111111,
+      "grad_norm": 0.5868740081787109,
+      "learning_rate": 0.0002,
+      "loss": 0.5446,
+      "step": 700
+    },
+    {
+      "epoch": 3.1555555555555554,
+      "grad_norm": 0.4717068672180176,
+      "learning_rate": 0.0002,
+      "loss": 0.5337,
+      "step": 710
+    },
+    {
+      "epoch": 3.2,
+      "grad_norm": 0.5336525440216064,
+      "learning_rate": 0.0002,
+      "loss": 0.5561,
+      "step": 720
+    },
+    {
+      "epoch": 3.2444444444444445,
+      "grad_norm": 0.7006786465644836,
+      "learning_rate": 0.0002,
+      "loss": 0.5565,
+      "step": 730
+    },
+    {
+      "epoch": 3.2888888888888888,
+      "grad_norm": 0.5565499663352966,
+      "learning_rate": 0.0002,
+      "loss": 0.5312,
+      "step": 740
+    },
+    {
+      "epoch": 3.3333333333333335,
+      "grad_norm": 0.568071722984314,
+      "learning_rate": 0.0002,
+      "loss": 0.554,
+      "step": 750
+    },
+    {
+      "epoch": 3.3777777777777778,
+      "grad_norm": 0.6389057040214539,
+      "learning_rate": 0.0002,
+      "loss": 0.5297,
+      "step": 760
+    },
+    {
+      "epoch": 3.422222222222222,
+      "grad_norm": 0.5239511132240295,
+      "learning_rate": 0.0002,
+      "loss": 0.5191,
+      "step": 770
+    },
+    {
+      "epoch": 3.466666666666667,
+      "grad_norm": 0.6089216470718384,
+      "learning_rate": 0.0002,
+      "loss": 0.559,
+      "step": 780
+    },
+    {
+      "epoch": 3.511111111111111,
+      "grad_norm": 0.5546727776527405,
+      "learning_rate": 0.0002,
+      "loss": 0.5399,
+      "step": 790
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "grad_norm": 0.5639946460723877,
+      "learning_rate": 0.0002,
+      "loss": 0.5558,
+      "step": 800
+    },
+    {
+      "epoch": 3.6,
+      "grad_norm": 0.5576934218406677,
+      "learning_rate": 0.0002,
+      "loss": 0.5476,
+      "step": 810
+    },
+    {
+      "epoch": 3.6444444444444444,
+      "grad_norm": 0.6561392545700073,
+      "learning_rate": 0.0002,
+      "loss": 0.5361,
+      "step": 820
+    },
+    {
+      "epoch": 3.688888888888889,
+      "grad_norm": 0.7699626088142395,
+      "learning_rate": 0.0002,
+      "loss": 0.5324,
+      "step": 830
+    },
+    {
+      "epoch": 3.7333333333333334,
+      "grad_norm": 0.47137776017189026,
+      "learning_rate": 0.0002,
+      "loss": 0.551,
+      "step": 840
+    },
+    {
+      "epoch": 3.7777777777777777,
+      "grad_norm": 0.7258023023605347,
+      "learning_rate": 0.0002,
+      "loss": 0.5581,
+      "step": 850
+    },
+    {
+      "epoch": 3.822222222222222,
+      "grad_norm": 0.510877251625061,
+      "learning_rate": 0.0002,
+      "loss": 0.5586,
+      "step": 860
+    },
+    {
+      "epoch": 3.8666666666666667,
+      "grad_norm": 0.6691411733627319,
+      "learning_rate": 0.0002,
+      "loss": 0.564,
+      "step": 870
+    },
+    {
+      "epoch": 3.911111111111111,
+      "grad_norm": 0.5722544193267822,
+      "learning_rate": 0.0002,
+      "loss": 0.5361,
+      "step": 880
+    },
+    {
+      "epoch": 3.9555555555555557,
+      "grad_norm": 0.6555589437484741,
+      "learning_rate": 0.0002,
+      "loss": 0.5307,
+      "step": 890
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.5689453482627869,
+      "learning_rate": 0.0002,
+      "loss": 0.5251,
+      "step": 900
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.6063342690467834,
+      "eval_runtime": 62.7917,
+      "eval_samples_per_second": 5.08,
+      "eval_steps_per_second": 0.637,
+      "step": 900
+    },
+    {
+      "epoch": 4.044444444444444,
+      "grad_norm": 0.8715280294418335,
+      "learning_rate": 0.0002,
+      "loss": 0.4847,
+      "step": 910
+    },
+    {
+      "epoch": 4.088888888888889,
+      "grad_norm": 0.5597540736198425,
+      "learning_rate": 0.0002,
+      "loss": 0.4756,
+      "step": 920
+    },
+    {
+      "epoch": 4.133333333333334,
+      "grad_norm": 0.7538669109344482,
+      "learning_rate": 0.0002,
+      "loss": 0.4673,
+      "step": 930
+    },
+    {
+      "epoch": 4.177777777777778,
+      "grad_norm": 0.7434868812561035,
+      "learning_rate": 0.0002,
+      "loss": 0.4835,
+      "step": 940
+    },
+    {
+      "epoch": 4.222222222222222,
+      "grad_norm": 0.6148455142974854,
+      "learning_rate": 0.0002,
+      "loss": 0.486,
+      "step": 950
+    },
+    {
+      "epoch": 4.266666666666667,
+      "grad_norm": 0.6513713598251343,
+      "learning_rate": 0.0002,
+      "loss": 0.4812,
+      "step": 960
+    },
+    {
+      "epoch": 4.311111111111111,
+      "grad_norm": 0.7177010774612427,
+      "learning_rate": 0.0002,
+      "loss": 0.4823,
+      "step": 970
+    },
+    {
+      "epoch": 4.355555555555555,
+      "grad_norm": 0.710017204284668,
+      "learning_rate": 0.0002,
+      "loss": 0.4934,
+      "step": 980
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 0.7506688833236694,
+      "learning_rate": 0.0002,
+      "loss": 0.4682,
+      "step": 990
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 0.7618675231933594,
+      "learning_rate": 0.0002,
+      "loss": 0.4782,
+      "step": 1000
+    },
+    {
+      "epoch": 4.488888888888889,
+      "grad_norm": 0.6317481994628906,
+      "learning_rate": 0.0002,
+      "loss": 0.5075,
+      "step": 1010
+    },
+    {
+      "epoch": 4.533333333333333,
+      "grad_norm": 0.5700439214706421,
+      "learning_rate": 0.0002,
+      "loss": 0.4822,
+      "step": 1020
+    },
+    {
+      "epoch": 4.5777777777777775,
+      "grad_norm": 0.7100785374641418,
+      "learning_rate": 0.0002,
+      "loss": 0.4734,
+      "step": 1030
+    },
+    {
+      "epoch": 4.622222222222222,
+      "grad_norm": 0.6751446723937988,
+      "learning_rate": 0.0002,
+      "loss": 0.4764,
+      "step": 1040
+    },
+    {
+      "epoch": 4.666666666666667,
+      "grad_norm": 0.7100826501846313,
+      "learning_rate": 0.0002,
+      "loss": 0.4843,
+      "step": 1050
+    },
+    {
+      "epoch": 4.711111111111111,
+      "grad_norm": 0.6513156890869141,
+      "learning_rate": 0.0002,
+      "loss": 0.5018,
+      "step": 1060
+    },
+    {
+      "epoch": 4.7555555555555555,
+      "grad_norm": 0.8129924535751343,
+      "learning_rate": 0.0002,
+      "loss": 0.495,
+      "step": 1070
+    },
+    {
+      "epoch": 4.8,
+      "grad_norm": 0.6378636956214905,
+      "learning_rate": 0.0002,
+      "loss": 0.4819,
+      "step": 1080
+    },
+    {
+      "epoch": 4.844444444444444,
+      "grad_norm": 0.8045517802238464,
+      "learning_rate": 0.0002,
+      "loss": 0.4809,
+      "step": 1090
+    },
+    {
+      "epoch": 4.888888888888889,
+      "grad_norm": 0.712690532207489,
+      "learning_rate": 0.0002,
+      "loss": 0.4844,
+      "step": 1100
+    },
+    {
+      "epoch": 4.933333333333334,
+      "grad_norm": 0.7078566551208496,
+      "learning_rate": 0.0002,
+      "loss": 0.4732,
+      "step": 1110
+    },
+    {
+      "epoch": 4.977777777777778,
+      "grad_norm": 0.5979776382446289,
+      "learning_rate": 0.0002,
+      "loss": 0.4727,
+      "step": 1120
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 0.5916207432746887,
+      "eval_runtime": 62.8273,
+      "eval_samples_per_second": 5.077,
+      "eval_steps_per_second": 0.637,
+      "step": 1125
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1800,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.9011638796288e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1125/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e959dbea71d2fa39c568a9d5bd6f17054bc6f0bf1101d40e99f8f6178453e4e
+size 5560

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a78e49d90d8f168b3cb2b9baa4b2c51650661dcd9d5003c109e758ab8932f3d8
+size 80755416

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ea7a3b63e85af11ad4f5b05b8c2ee8745eda162409e7d491702465d861152049
+size 41136570

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2f5996f6467411d42e1fd6a96691600fb8b26e60054989e8a870a5b9f6d3a8a
+size 14244

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7629012de9a2f741f71ead0fcc6bf0b49375818c4f4b47e0a48313b9a209ae4a
+size 1064

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/trainer_state.json

@@ -0,0 +1,1026 @@
+{
+  "best_metric": 0.5706757307052612,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350",
+  "epoch": 6.0,
+  "eval_steps": 10,
+  "global_step": 1350,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.044444444444444446,
+      "grad_norm": 0.2940807342529297,
+      "learning_rate": 0.0002,
+      "loss": 1.8338,
+      "step": 10
+    },
+    {
+      "epoch": 0.08888888888888889,
+      "grad_norm": 0.36792996525764465,
+      "learning_rate": 0.0002,
+      "loss": 1.5844,
+      "step": 20
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "grad_norm": 0.44259828329086304,
+      "learning_rate": 0.0002,
+      "loss": 1.3603,
+      "step": 30
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.760607898235321,
+      "learning_rate": 0.0002,
+      "loss": 1.1408,
+      "step": 40
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 0.608131468296051,
+      "learning_rate": 0.0002,
+      "loss": 0.9766,
+      "step": 50
+    },
+    {
+      "epoch": 0.26666666666666666,
+      "grad_norm": 0.5941349864006042,
+      "learning_rate": 0.0002,
+      "loss": 0.9176,
+      "step": 60
+    },
+    {
+      "epoch": 0.3111111111111111,
+      "grad_norm": 0.5939444899559021,
+      "learning_rate": 0.0002,
+      "loss": 0.9081,
+      "step": 70
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 0.6093971729278564,
+      "learning_rate": 0.0002,
+      "loss": 0.8406,
+      "step": 80
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.6158391237258911,
+      "learning_rate": 0.0002,
+      "loss": 0.8562,
+      "step": 90
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 0.4985930621623993,
+      "learning_rate": 0.0002,
+      "loss": 0.8111,
+      "step": 100
+    },
+    {
+      "epoch": 0.4888888888888889,
+      "grad_norm": 0.6661780476570129,
+      "learning_rate": 0.0002,
+      "loss": 0.7963,
+      "step": 110
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.5717976689338684,
+      "learning_rate": 0.0002,
+      "loss": 0.8114,
+      "step": 120
+    },
+    {
+      "epoch": 0.5777777777777777,
+      "grad_norm": 0.5666863918304443,
+      "learning_rate": 0.0002,
+      "loss": 0.7397,
+      "step": 130
+    },
+    {
+      "epoch": 0.6222222222222222,
+      "grad_norm": 0.49229782819747925,
+      "learning_rate": 0.0002,
+      "loss": 0.7671,
+      "step": 140
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 0.48167455196380615,
+      "learning_rate": 0.0002,
+      "loss": 0.7851,
+      "step": 150
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 0.4657461643218994,
+      "learning_rate": 0.0002,
+      "loss": 0.7786,
+      "step": 160
+    },
+    {
+      "epoch": 0.7555555555555555,
+      "grad_norm": 0.39121416211128235,
+      "learning_rate": 0.0002,
+      "loss": 0.7691,
+      "step": 170
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.441007524728775,
+      "learning_rate": 0.0002,
+      "loss": 0.7323,
+      "step": 180
+    },
+    {
+      "epoch": 0.8444444444444444,
+      "grad_norm": 0.4213836193084717,
+      "learning_rate": 0.0002,
+      "loss": 0.7418,
+      "step": 190
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.5080695152282715,
+      "learning_rate": 0.0002,
+      "loss": 0.7451,
+      "step": 200
+    },
+    {
+      "epoch": 0.9333333333333333,
+      "grad_norm": 0.4761652648448944,
+      "learning_rate": 0.0002,
+      "loss": 0.757,
+      "step": 210
+    },
+    {
+      "epoch": 0.9777777777777777,
+      "grad_norm": 0.5242546796798706,
+      "learning_rate": 0.0002,
+      "loss": 0.741,
+      "step": 220
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.7312084436416626,
+      "eval_runtime": 86.046,
+      "eval_samples_per_second": 3.707,
+      "eval_steps_per_second": 0.465,
+      "step": 225
+    },
+    {
+      "epoch": 1.0222222222222221,
+      "grad_norm": 0.4389289915561676,
+      "learning_rate": 0.0002,
+      "loss": 0.7365,
+      "step": 230
+    },
+    {
+      "epoch": 1.0666666666666667,
+      "grad_norm": 0.4293370246887207,
+      "learning_rate": 0.0002,
+      "loss": 0.7223,
+      "step": 240
+    },
+    {
+      "epoch": 1.1111111111111112,
+      "grad_norm": 0.45308825373649597,
+      "learning_rate": 0.0002,
+      "loss": 0.6934,
+      "step": 250
+    },
+    {
+      "epoch": 1.1555555555555554,
+      "grad_norm": 0.4458293318748474,
+      "learning_rate": 0.0002,
+      "loss": 0.7058,
+      "step": 260
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.46963292360305786,
+      "learning_rate": 0.0002,
+      "loss": 0.6956,
+      "step": 270
+    },
+    {
+      "epoch": 1.2444444444444445,
+      "grad_norm": 0.39571475982666016,
+      "learning_rate": 0.0002,
+      "loss": 0.6882,
+      "step": 280
+    },
+    {
+      "epoch": 1.2888888888888888,
+      "grad_norm": 0.4619075655937195,
+      "learning_rate": 0.0002,
+      "loss": 0.7241,
+      "step": 290
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 0.457443505525589,
+      "learning_rate": 0.0002,
+      "loss": 0.7281,
+      "step": 300
+    },
+    {
+      "epoch": 1.3777777777777778,
+      "grad_norm": 0.5545842051506042,
+      "learning_rate": 0.0002,
+      "loss": 0.7138,
+      "step": 310
+    },
+    {
+      "epoch": 1.4222222222222223,
+      "grad_norm": 0.4492949843406677,
+      "learning_rate": 0.0002,
+      "loss": 0.7082,
+      "step": 320
+    },
+    {
+      "epoch": 1.4666666666666668,
+      "grad_norm": 0.46642452478408813,
+      "learning_rate": 0.0002,
+      "loss": 0.6733,
+      "step": 330
+    },
+    {
+      "epoch": 1.511111111111111,
+      "grad_norm": 0.4598081707954407,
+      "learning_rate": 0.0002,
+      "loss": 0.6846,
+      "step": 340
+    },
+    {
+      "epoch": 1.5555555555555556,
+      "grad_norm": 0.34535378217697144,
+      "learning_rate": 0.0002,
+      "loss": 0.6981,
+      "step": 350
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.4226590096950531,
+      "learning_rate": 0.0002,
+      "loss": 0.672,
+      "step": 360
+    },
+    {
+      "epoch": 1.6444444444444444,
+      "grad_norm": 0.47323980927467346,
+      "learning_rate": 0.0002,
+      "loss": 0.6638,
+      "step": 370
+    },
+    {
+      "epoch": 1.6888888888888889,
+      "grad_norm": 0.39699724316596985,
+      "learning_rate": 0.0002,
+      "loss": 0.6813,
+      "step": 380
+    },
+    {
+      "epoch": 1.7333333333333334,
+      "grad_norm": 0.5325330495834351,
+      "learning_rate": 0.0002,
+      "loss": 0.6852,
+      "step": 390
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 0.4336804449558258,
+      "learning_rate": 0.0002,
+      "loss": 0.6716,
+      "step": 400
+    },
+    {
+      "epoch": 1.8222222222222222,
+      "grad_norm": 0.44034498929977417,
+      "learning_rate": 0.0002,
+      "loss": 0.6538,
+      "step": 410
+    },
+    {
+      "epoch": 1.8666666666666667,
+      "grad_norm": 0.4527396261692047,
+      "learning_rate": 0.0002,
+      "loss": 0.6706,
+      "step": 420
+    },
+    {
+      "epoch": 1.911111111111111,
+      "grad_norm": 0.40166375041007996,
+      "learning_rate": 0.0002,
+      "loss": 0.6507,
+      "step": 430
+    },
+    {
+      "epoch": 1.9555555555555557,
+      "grad_norm": 0.49753332138061523,
+      "learning_rate": 0.0002,
+      "loss": 0.6599,
+      "step": 440
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.5019579529762268,
+      "learning_rate": 0.0002,
+      "loss": 0.6482,
+      "step": 450
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.6728096604347229,
+      "eval_runtime": 85.827,
+      "eval_samples_per_second": 3.717,
+      "eval_steps_per_second": 0.466,
+      "step": 450
+    },
+    {
+      "epoch": 2.0444444444444443,
+      "grad_norm": 0.5162734985351562,
+      "learning_rate": 0.0002,
+      "loss": 0.626,
+      "step": 460
+    },
+    {
+      "epoch": 2.088888888888889,
+      "grad_norm": 0.4973997175693512,
+      "learning_rate": 0.0002,
+      "loss": 0.6363,
+      "step": 470
+    },
+    {
+      "epoch": 2.1333333333333333,
+      "grad_norm": 0.6232216358184814,
+      "learning_rate": 0.0002,
+      "loss": 0.6225,
+      "step": 480
+    },
+    {
+      "epoch": 2.1777777777777776,
+      "grad_norm": 0.5598695278167725,
+      "learning_rate": 0.0002,
+      "loss": 0.6038,
+      "step": 490
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 0.5359365344047546,
+      "learning_rate": 0.0002,
+      "loss": 0.6247,
+      "step": 500
+    },
+    {
+      "epoch": 2.2666666666666666,
+      "grad_norm": 0.5992209315299988,
+      "learning_rate": 0.0002,
+      "loss": 0.6203,
+      "step": 510
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 0.4921131730079651,
+      "learning_rate": 0.0002,
+      "loss": 0.6416,
+      "step": 520
+    },
+    {
+      "epoch": 2.3555555555555556,
+      "grad_norm": 0.5449638366699219,
+      "learning_rate": 0.0002,
+      "loss": 0.6157,
+      "step": 530
+    },
+    {
+      "epoch": 2.4,
+      "grad_norm": 0.45068296790122986,
+      "learning_rate": 0.0002,
+      "loss": 0.6244,
+      "step": 540
+    },
+    {
+      "epoch": 2.4444444444444446,
+      "grad_norm": 0.5800191164016724,
+      "learning_rate": 0.0002,
+      "loss": 0.6368,
+      "step": 550
+    },
+    {
+      "epoch": 2.488888888888889,
+      "grad_norm": 0.4825330674648285,
+      "learning_rate": 0.0002,
+      "loss": 0.6437,
+      "step": 560
+    },
+    {
+      "epoch": 2.533333333333333,
+      "grad_norm": 0.6174261569976807,
+      "learning_rate": 0.0002,
+      "loss": 0.606,
+      "step": 570
+    },
+    {
+      "epoch": 2.5777777777777775,
+      "grad_norm": 0.5854967832565308,
+      "learning_rate": 0.0002,
+      "loss": 0.5666,
+      "step": 580
+    },
+    {
+      "epoch": 2.6222222222222222,
+      "grad_norm": 0.44053414463996887,
+      "learning_rate": 0.0002,
+      "loss": 0.6017,
+      "step": 590
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 0.4803224503993988,
+      "learning_rate": 0.0002,
+      "loss": 0.6032,
+      "step": 600
+    },
+    {
+      "epoch": 2.7111111111111112,
+      "grad_norm": 0.5700278282165527,
+      "learning_rate": 0.0002,
+      "loss": 0.6238,
+      "step": 610
+    },
+    {
+      "epoch": 2.7555555555555555,
+      "grad_norm": 0.5318542122840881,
+      "learning_rate": 0.0002,
+      "loss": 0.5767,
+      "step": 620
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 0.5594431161880493,
+      "learning_rate": 0.0002,
+      "loss": 0.5849,
+      "step": 630
+    },
+    {
+      "epoch": 2.8444444444444446,
+      "grad_norm": 0.5598340630531311,
+      "learning_rate": 0.0002,
+      "loss": 0.6094,
+      "step": 640
+    },
+    {
+      "epoch": 2.888888888888889,
+      "grad_norm": 0.5017251372337341,
+      "learning_rate": 0.0002,
+      "loss": 0.6059,
+      "step": 650
+    },
+    {
+      "epoch": 2.9333333333333336,
+      "grad_norm": 0.46027806401252747,
+      "learning_rate": 0.0002,
+      "loss": 0.6047,
+      "step": 660
+    },
+    {
+      "epoch": 2.977777777777778,
+      "grad_norm": 0.5706299543380737,
+      "learning_rate": 0.0002,
+      "loss": 0.5912,
+      "step": 670
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 0.6362627148628235,
+      "eval_runtime": 85.1312,
+      "eval_samples_per_second": 3.747,
+      "eval_steps_per_second": 0.47,
+      "step": 675
+    },
+    {
+      "epoch": 3.022222222222222,
+      "grad_norm": 0.6016985774040222,
+      "learning_rate": 0.0002,
+      "loss": 0.5507,
+      "step": 680
+    },
+    {
+      "epoch": 3.066666666666667,
+      "grad_norm": 0.544710099697113,
+      "learning_rate": 0.0002,
+      "loss": 0.538,
+      "step": 690
+    },
+    {
+      "epoch": 3.111111111111111,
+      "grad_norm": 0.5868740081787109,
+      "learning_rate": 0.0002,
+      "loss": 0.5446,
+      "step": 700
+    },
+    {
+      "epoch": 3.1555555555555554,
+      "grad_norm": 0.4717068672180176,
+      "learning_rate": 0.0002,
+      "loss": 0.5337,
+      "step": 710
+    },
+    {
+      "epoch": 3.2,
+      "grad_norm": 0.5336525440216064,
+      "learning_rate": 0.0002,
+      "loss": 0.5561,
+      "step": 720
+    },
+    {
+      "epoch": 3.2444444444444445,
+      "grad_norm": 0.7006786465644836,
+      "learning_rate": 0.0002,
+      "loss": 0.5565,
+      "step": 730
+    },
+    {
+      "epoch": 3.2888888888888888,
+      "grad_norm": 0.5565499663352966,
+      "learning_rate": 0.0002,
+      "loss": 0.5312,
+      "step": 740
+    },
+    {
+      "epoch": 3.3333333333333335,
+      "grad_norm": 0.568071722984314,
+      "learning_rate": 0.0002,
+      "loss": 0.554,
+      "step": 750
+    },
+    {
+      "epoch": 3.3777777777777778,
+      "grad_norm": 0.6389057040214539,
+      "learning_rate": 0.0002,
+      "loss": 0.5297,
+      "step": 760
+    },
+    {
+      "epoch": 3.422222222222222,
+      "grad_norm": 0.5239511132240295,
+      "learning_rate": 0.0002,
+      "loss": 0.5191,
+      "step": 770
+    },
+    {
+      "epoch": 3.466666666666667,
+      "grad_norm": 0.6089216470718384,
+      "learning_rate": 0.0002,
+      "loss": 0.559,
+      "step": 780
+    },
+    {
+      "epoch": 3.511111111111111,
+      "grad_norm": 0.5546727776527405,
+      "learning_rate": 0.0002,
+      "loss": 0.5399,
+      "step": 790
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "grad_norm": 0.5639946460723877,
+      "learning_rate": 0.0002,
+      "loss": 0.5558,
+      "step": 800
+    },
+    {
+      "epoch": 3.6,
+      "grad_norm": 0.5576934218406677,
+      "learning_rate": 0.0002,
+      "loss": 0.5476,
+      "step": 810
+    },
+    {
+      "epoch": 3.6444444444444444,
+      "grad_norm": 0.6561392545700073,
+      "learning_rate": 0.0002,
+      "loss": 0.5361,
+      "step": 820
+    },
+    {
+      "epoch": 3.688888888888889,
+      "grad_norm": 0.7699626088142395,
+      "learning_rate": 0.0002,
+      "loss": 0.5324,
+      "step": 830
+    },
+    {
+      "epoch": 3.7333333333333334,
+      "grad_norm": 0.47137776017189026,
+      "learning_rate": 0.0002,
+      "loss": 0.551,
+      "step": 840
+    },
+    {
+      "epoch": 3.7777777777777777,
+      "grad_norm": 0.7258023023605347,
+      "learning_rate": 0.0002,
+      "loss": 0.5581,
+      "step": 850
+    },
+    {
+      "epoch": 3.822222222222222,
+      "grad_norm": 0.510877251625061,
+      "learning_rate": 0.0002,
+      "loss": 0.5586,
+      "step": 860
+    },
+    {
+      "epoch": 3.8666666666666667,
+      "grad_norm": 0.6691411733627319,
+      "learning_rate": 0.0002,
+      "loss": 0.564,
+      "step": 870
+    },
+    {
+      "epoch": 3.911111111111111,
+      "grad_norm": 0.5722544193267822,
+      "learning_rate": 0.0002,
+      "loss": 0.5361,
+      "step": 880
+    },
+    {
+      "epoch": 3.9555555555555557,
+      "grad_norm": 0.6555589437484741,
+      "learning_rate": 0.0002,
+      "loss": 0.5307,
+      "step": 890
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.5689453482627869,
+      "learning_rate": 0.0002,
+      "loss": 0.5251,
+      "step": 900
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.6063342690467834,
+      "eval_runtime": 62.7917,
+      "eval_samples_per_second": 5.08,
+      "eval_steps_per_second": 0.637,
+      "step": 900
+    },
+    {
+      "epoch": 4.044444444444444,
+      "grad_norm": 0.8715280294418335,
+      "learning_rate": 0.0002,
+      "loss": 0.4847,
+      "step": 910
+    },
+    {
+      "epoch": 4.088888888888889,
+      "grad_norm": 0.5597540736198425,
+      "learning_rate": 0.0002,
+      "loss": 0.4756,
+      "step": 920
+    },
+    {
+      "epoch": 4.133333333333334,
+      "grad_norm": 0.7538669109344482,
+      "learning_rate": 0.0002,
+      "loss": 0.4673,
+      "step": 930
+    },
+    {
+      "epoch": 4.177777777777778,
+      "grad_norm": 0.7434868812561035,
+      "learning_rate": 0.0002,
+      "loss": 0.4835,
+      "step": 940
+    },
+    {
+      "epoch": 4.222222222222222,
+      "grad_norm": 0.6148455142974854,
+      "learning_rate": 0.0002,
+      "loss": 0.486,
+      "step": 950
+    },
+    {
+      "epoch": 4.266666666666667,
+      "grad_norm": 0.6513713598251343,
+      "learning_rate": 0.0002,
+      "loss": 0.4812,
+      "step": 960
+    },
+    {
+      "epoch": 4.311111111111111,
+      "grad_norm": 0.7177010774612427,
+      "learning_rate": 0.0002,
+      "loss": 0.4823,
+      "step": 970
+    },
+    {
+      "epoch": 4.355555555555555,
+      "grad_norm": 0.710017204284668,
+      "learning_rate": 0.0002,
+      "loss": 0.4934,
+      "step": 980
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 0.7506688833236694,
+      "learning_rate": 0.0002,
+      "loss": 0.4682,
+      "step": 990
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 0.7618675231933594,
+      "learning_rate": 0.0002,
+      "loss": 0.4782,
+      "step": 1000
+    },
+    {
+      "epoch": 4.488888888888889,
+      "grad_norm": 0.6317481994628906,
+      "learning_rate": 0.0002,
+      "loss": 0.5075,
+      "step": 1010
+    },
+    {
+      "epoch": 4.533333333333333,
+      "grad_norm": 0.5700439214706421,
+      "learning_rate": 0.0002,
+      "loss": 0.4822,
+      "step": 1020
+    },
+    {
+      "epoch": 4.5777777777777775,
+      "grad_norm": 0.7100785374641418,
+      "learning_rate": 0.0002,
+      "loss": 0.4734,
+      "step": 1030
+    },
+    {
+      "epoch": 4.622222222222222,
+      "grad_norm": 0.6751446723937988,
+      "learning_rate": 0.0002,
+      "loss": 0.4764,
+      "step": 1040
+    },
+    {
+      "epoch": 4.666666666666667,
+      "grad_norm": 0.7100826501846313,
+      "learning_rate": 0.0002,
+      "loss": 0.4843,
+      "step": 1050
+    },
+    {
+      "epoch": 4.711111111111111,
+      "grad_norm": 0.6513156890869141,
+      "learning_rate": 0.0002,
+      "loss": 0.5018,
+      "step": 1060
+    },
+    {
+      "epoch": 4.7555555555555555,
+      "grad_norm": 0.8129924535751343,
+      "learning_rate": 0.0002,
+      "loss": 0.495,
+      "step": 1070
+    },
+    {
+      "epoch": 4.8,
+      "grad_norm": 0.6378636956214905,
+      "learning_rate": 0.0002,
+      "loss": 0.4819,
+      "step": 1080
+    },
+    {
+      "epoch": 4.844444444444444,
+      "grad_norm": 0.8045517802238464,
+      "learning_rate": 0.0002,
+      "loss": 0.4809,
+      "step": 1090
+    },
+    {
+      "epoch": 4.888888888888889,
+      "grad_norm": 0.712690532207489,
+      "learning_rate": 0.0002,
+      "loss": 0.4844,
+      "step": 1100
+    },
+    {
+      "epoch": 4.933333333333334,
+      "grad_norm": 0.7078566551208496,
+      "learning_rate": 0.0002,
+      "loss": 0.4732,
+      "step": 1110
+    },
+    {
+      "epoch": 4.977777777777778,
+      "grad_norm": 0.5979776382446289,
+      "learning_rate": 0.0002,
+      "loss": 0.4727,
+      "step": 1120
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 0.5916207432746887,
+      "eval_runtime": 62.8273,
+      "eval_samples_per_second": 5.077,
+      "eval_steps_per_second": 0.637,
+      "step": 1125
+    },
+    {
+      "epoch": 5.022222222222222,
+      "grad_norm": 0.6213210225105286,
+      "learning_rate": 0.0002,
+      "loss": 0.4418,
+      "step": 1130
+    },
+    {
+      "epoch": 5.066666666666666,
+      "grad_norm": 0.8804424405097961,
+      "learning_rate": 0.0002,
+      "loss": 0.4213,
+      "step": 1140
+    },
+    {
+      "epoch": 5.111111111111111,
+      "grad_norm": 0.5710713267326355,
+      "learning_rate": 0.0002,
+      "loss": 0.4231,
+      "step": 1150
+    },
+    {
+      "epoch": 5.155555555555556,
+      "grad_norm": 0.6581844687461853,
+      "learning_rate": 0.0002,
+      "loss": 0.4279,
+      "step": 1160
+    },
+    {
+      "epoch": 5.2,
+      "grad_norm": 1.4485387802124023,
+      "learning_rate": 0.0002,
+      "loss": 0.4227,
+      "step": 1170
+    },
+    {
+      "epoch": 5.2444444444444445,
+      "grad_norm": 0.9506292343139648,
+      "learning_rate": 0.0002,
+      "loss": 0.4413,
+      "step": 1180
+    },
+    {
+      "epoch": 5.288888888888889,
+      "grad_norm": 0.6971795558929443,
+      "learning_rate": 0.0002,
+      "loss": 0.4129,
+      "step": 1190
+    },
+    {
+      "epoch": 5.333333333333333,
+      "grad_norm": 0.7639512419700623,
+      "learning_rate": 0.0002,
+      "loss": 0.4283,
+      "step": 1200
+    },
+    {
+      "epoch": 5.377777777777778,
+      "grad_norm": 0.7847402691841125,
+      "learning_rate": 0.0002,
+      "loss": 0.4581,
+      "step": 1210
+    },
+    {
+      "epoch": 5.4222222222222225,
+      "grad_norm": 0.6164460182189941,
+      "learning_rate": 0.0002,
+      "loss": 0.4311,
+      "step": 1220
+    },
+    {
+      "epoch": 5.466666666666667,
+      "grad_norm": 0.748383104801178,
+      "learning_rate": 0.0002,
+      "loss": 0.4304,
+      "step": 1230
+    },
+    {
+      "epoch": 5.511111111111111,
+      "grad_norm": 0.5674386620521545,
+      "learning_rate": 0.0002,
+      "loss": 0.4405,
+      "step": 1240
+    },
+    {
+      "epoch": 5.555555555555555,
+      "grad_norm": 0.9450201392173767,
+      "learning_rate": 0.0002,
+      "loss": 0.4396,
+      "step": 1250
+    },
+    {
+      "epoch": 5.6,
+      "grad_norm": 0.6066147089004517,
+      "learning_rate": 0.0002,
+      "loss": 0.4456,
+      "step": 1260
+    },
+    {
+      "epoch": 5.644444444444445,
+      "grad_norm": 1.1018579006195068,
+      "learning_rate": 0.0002,
+      "loss": 0.4154,
+      "step": 1270
+    },
+    {
+      "epoch": 5.688888888888889,
+      "grad_norm": 0.7561473846435547,
+      "learning_rate": 0.0002,
+      "loss": 0.4379,
+      "step": 1280
+    },
+    {
+      "epoch": 5.733333333333333,
+      "grad_norm": 0.8838764429092407,
+      "learning_rate": 0.0002,
+      "loss": 0.4421,
+      "step": 1290
+    },
+    {
+      "epoch": 5.777777777777778,
+      "grad_norm": 0.7195517420768738,
+      "learning_rate": 0.0002,
+      "loss": 0.439,
+      "step": 1300
+    },
+    {
+      "epoch": 5.822222222222222,
+      "grad_norm": 0.8123605847358704,
+      "learning_rate": 0.0002,
+      "loss": 0.4377,
+      "step": 1310
+    },
+    {
+      "epoch": 5.866666666666667,
+      "grad_norm": 0.6891103386878967,
+      "learning_rate": 0.0002,
+      "loss": 0.435,
+      "step": 1320
+    },
+    {
+      "epoch": 5.911111111111111,
+      "grad_norm": 1.0846312046051025,
+      "learning_rate": 0.0002,
+      "loss": 0.4276,
+      "step": 1330
+    },
+    {
+      "epoch": 5.955555555555556,
+      "grad_norm": 0.6821148991584778,
+      "learning_rate": 0.0002,
+      "loss": 0.4343,
+      "step": 1340
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 1.0653820037841797,
+      "learning_rate": 0.0002,
+      "loss": 0.4324,
+      "step": 1350
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 0.5706757307052612,
+      "eval_runtime": 62.688,
+      "eval_samples_per_second": 5.089,
+      "eval_steps_per_second": 0.638,
+      "step": 1350
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1800,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.88139665555456e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1350/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e959dbea71d2fa39c568a9d5bd6f17054bc6f0bf1101d40e99f8f6178453e4e
+size 5560

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bca64cc84ff73c12e7cef4b0e172cb7a1f9e426c15879ebb84296fd66907ea53
+size 80755416

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01779cd0b328fb9cba8d23086afe3652d4229dd04af6e879466d94e053a19d06
+size 41136570

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bb59cf38366d8dcd1c2124d54cc40a7fe30cb57aab4d93e47137ebb1c16ba3d
+size 14244

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6aad2b88bf1ffb4dc72f057d70b0e9ef4ef2ebf2f09db62fe672a9338a0c473d
+size 1064

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/special_tokens_map.json

@@ -0,0 +1,14 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|im_end|>"
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/tokenizer_config.json

@@ -0,0 +1,43 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/trainer_state.json

@@ -0,0 +1,1188 @@
+{
+  "best_metric": 0.5527341961860657,
+  "best_model_checkpoint": "outputs-001/Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575",
+  "epoch": 7.0,
+  "eval_steps": 10,
+  "global_step": 1575,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.044444444444444446,
+      "grad_norm": 0.2940807342529297,
+      "learning_rate": 0.0002,
+      "loss": 1.8338,
+      "step": 10
+    },
+    {
+      "epoch": 0.08888888888888889,
+      "grad_norm": 0.36792996525764465,
+      "learning_rate": 0.0002,
+      "loss": 1.5844,
+      "step": 20
+    },
+    {
+      "epoch": 0.13333333333333333,
+      "grad_norm": 0.44259828329086304,
+      "learning_rate": 0.0002,
+      "loss": 1.3603,
+      "step": 30
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 0.760607898235321,
+      "learning_rate": 0.0002,
+      "loss": 1.1408,
+      "step": 40
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 0.608131468296051,
+      "learning_rate": 0.0002,
+      "loss": 0.9766,
+      "step": 50
+    },
+    {
+      "epoch": 0.26666666666666666,
+      "grad_norm": 0.5941349864006042,
+      "learning_rate": 0.0002,
+      "loss": 0.9176,
+      "step": 60
+    },
+    {
+      "epoch": 0.3111111111111111,
+      "grad_norm": 0.5939444899559021,
+      "learning_rate": 0.0002,
+      "loss": 0.9081,
+      "step": 70
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 0.6093971729278564,
+      "learning_rate": 0.0002,
+      "loss": 0.8406,
+      "step": 80
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 0.6158391237258911,
+      "learning_rate": 0.0002,
+      "loss": 0.8562,
+      "step": 90
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 0.4985930621623993,
+      "learning_rate": 0.0002,
+      "loss": 0.8111,
+      "step": 100
+    },
+    {
+      "epoch": 0.4888888888888889,
+      "grad_norm": 0.6661780476570129,
+      "learning_rate": 0.0002,
+      "loss": 0.7963,
+      "step": 110
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.5717976689338684,
+      "learning_rate": 0.0002,
+      "loss": 0.8114,
+      "step": 120
+    },
+    {
+      "epoch": 0.5777777777777777,
+      "grad_norm": 0.5666863918304443,
+      "learning_rate": 0.0002,
+      "loss": 0.7397,
+      "step": 130
+    },
+    {
+      "epoch": 0.6222222222222222,
+      "grad_norm": 0.49229782819747925,
+      "learning_rate": 0.0002,
+      "loss": 0.7671,
+      "step": 140
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 0.48167455196380615,
+      "learning_rate": 0.0002,
+      "loss": 0.7851,
+      "step": 150
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 0.4657461643218994,
+      "learning_rate": 0.0002,
+      "loss": 0.7786,
+      "step": 160
+    },
+    {
+      "epoch": 0.7555555555555555,
+      "grad_norm": 0.39121416211128235,
+      "learning_rate": 0.0002,
+      "loss": 0.7691,
+      "step": 170
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.441007524728775,
+      "learning_rate": 0.0002,
+      "loss": 0.7323,
+      "step": 180
+    },
+    {
+      "epoch": 0.8444444444444444,
+      "grad_norm": 0.4213836193084717,
+      "learning_rate": 0.0002,
+      "loss": 0.7418,
+      "step": 190
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 0.5080695152282715,
+      "learning_rate": 0.0002,
+      "loss": 0.7451,
+      "step": 200
+    },
+    {
+      "epoch": 0.9333333333333333,
+      "grad_norm": 0.4761652648448944,
+      "learning_rate": 0.0002,
+      "loss": 0.757,
+      "step": 210
+    },
+    {
+      "epoch": 0.9777777777777777,
+      "grad_norm": 0.5242546796798706,
+      "learning_rate": 0.0002,
+      "loss": 0.741,
+      "step": 220
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.7312084436416626,
+      "eval_runtime": 86.046,
+      "eval_samples_per_second": 3.707,
+      "eval_steps_per_second": 0.465,
+      "step": 225
+    },
+    {
+      "epoch": 1.0222222222222221,
+      "grad_norm": 0.4389289915561676,
+      "learning_rate": 0.0002,
+      "loss": 0.7365,
+      "step": 230
+    },
+    {
+      "epoch": 1.0666666666666667,
+      "grad_norm": 0.4293370246887207,
+      "learning_rate": 0.0002,
+      "loss": 0.7223,
+      "step": 240
+    },
+    {
+      "epoch": 1.1111111111111112,
+      "grad_norm": 0.45308825373649597,
+      "learning_rate": 0.0002,
+      "loss": 0.6934,
+      "step": 250
+    },
+    {
+      "epoch": 1.1555555555555554,
+      "grad_norm": 0.4458293318748474,
+      "learning_rate": 0.0002,
+      "loss": 0.7058,
+      "step": 260
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.46963292360305786,
+      "learning_rate": 0.0002,
+      "loss": 0.6956,
+      "step": 270
+    },
+    {
+      "epoch": 1.2444444444444445,
+      "grad_norm": 0.39571475982666016,
+      "learning_rate": 0.0002,
+      "loss": 0.6882,
+      "step": 280
+    },
+    {
+      "epoch": 1.2888888888888888,
+      "grad_norm": 0.4619075655937195,
+      "learning_rate": 0.0002,
+      "loss": 0.7241,
+      "step": 290
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 0.457443505525589,
+      "learning_rate": 0.0002,
+      "loss": 0.7281,
+      "step": 300
+    },
+    {
+      "epoch": 1.3777777777777778,
+      "grad_norm": 0.5545842051506042,
+      "learning_rate": 0.0002,
+      "loss": 0.7138,
+      "step": 310
+    },
+    {
+      "epoch": 1.4222222222222223,
+      "grad_norm": 0.4492949843406677,
+      "learning_rate": 0.0002,
+      "loss": 0.7082,
+      "step": 320
+    },
+    {
+      "epoch": 1.4666666666666668,
+      "grad_norm": 0.46642452478408813,
+      "learning_rate": 0.0002,
+      "loss": 0.6733,
+      "step": 330
+    },
+    {
+      "epoch": 1.511111111111111,
+      "grad_norm": 0.4598081707954407,
+      "learning_rate": 0.0002,
+      "loss": 0.6846,
+      "step": 340
+    },
+    {
+      "epoch": 1.5555555555555556,
+      "grad_norm": 0.34535378217697144,
+      "learning_rate": 0.0002,
+      "loss": 0.6981,
+      "step": 350
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 0.4226590096950531,
+      "learning_rate": 0.0002,
+      "loss": 0.672,
+      "step": 360
+    },
+    {
+      "epoch": 1.6444444444444444,
+      "grad_norm": 0.47323980927467346,
+      "learning_rate": 0.0002,
+      "loss": 0.6638,
+      "step": 370
+    },
+    {
+      "epoch": 1.6888888888888889,
+      "grad_norm": 0.39699724316596985,
+      "learning_rate": 0.0002,
+      "loss": 0.6813,
+      "step": 380
+    },
+    {
+      "epoch": 1.7333333333333334,
+      "grad_norm": 0.5325330495834351,
+      "learning_rate": 0.0002,
+      "loss": 0.6852,
+      "step": 390
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 0.4336804449558258,
+      "learning_rate": 0.0002,
+      "loss": 0.6716,
+      "step": 400
+    },
+    {
+      "epoch": 1.8222222222222222,
+      "grad_norm": 0.44034498929977417,
+      "learning_rate": 0.0002,
+      "loss": 0.6538,
+      "step": 410
+    },
+    {
+      "epoch": 1.8666666666666667,
+      "grad_norm": 0.4527396261692047,
+      "learning_rate": 0.0002,
+      "loss": 0.6706,
+      "step": 420
+    },
+    {
+      "epoch": 1.911111111111111,
+      "grad_norm": 0.40166375041007996,
+      "learning_rate": 0.0002,
+      "loss": 0.6507,
+      "step": 430
+    },
+    {
+      "epoch": 1.9555555555555557,
+      "grad_norm": 0.49753332138061523,
+      "learning_rate": 0.0002,
+      "loss": 0.6599,
+      "step": 440
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.5019579529762268,
+      "learning_rate": 0.0002,
+      "loss": 0.6482,
+      "step": 450
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.6728096604347229,
+      "eval_runtime": 85.827,
+      "eval_samples_per_second": 3.717,
+      "eval_steps_per_second": 0.466,
+      "step": 450
+    },
+    {
+      "epoch": 2.0444444444444443,
+      "grad_norm": 0.5162734985351562,
+      "learning_rate": 0.0002,
+      "loss": 0.626,
+      "step": 460
+    },
+    {
+      "epoch": 2.088888888888889,
+      "grad_norm": 0.4973997175693512,
+      "learning_rate": 0.0002,
+      "loss": 0.6363,
+      "step": 470
+    },
+    {
+      "epoch": 2.1333333333333333,
+      "grad_norm": 0.6232216358184814,
+      "learning_rate": 0.0002,
+      "loss": 0.6225,
+      "step": 480
+    },
+    {
+      "epoch": 2.1777777777777776,
+      "grad_norm": 0.5598695278167725,
+      "learning_rate": 0.0002,
+      "loss": 0.6038,
+      "step": 490
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 0.5359365344047546,
+      "learning_rate": 0.0002,
+      "loss": 0.6247,
+      "step": 500
+    },
+    {
+      "epoch": 2.2666666666666666,
+      "grad_norm": 0.5992209315299988,
+      "learning_rate": 0.0002,
+      "loss": 0.6203,
+      "step": 510
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 0.4921131730079651,
+      "learning_rate": 0.0002,
+      "loss": 0.6416,
+      "step": 520
+    },
+    {
+      "epoch": 2.3555555555555556,
+      "grad_norm": 0.5449638366699219,
+      "learning_rate": 0.0002,
+      "loss": 0.6157,
+      "step": 530
+    },
+    {
+      "epoch": 2.4,
+      "grad_norm": 0.45068296790122986,
+      "learning_rate": 0.0002,
+      "loss": 0.6244,
+      "step": 540
+    },
+    {
+      "epoch": 2.4444444444444446,
+      "grad_norm": 0.5800191164016724,
+      "learning_rate": 0.0002,
+      "loss": 0.6368,
+      "step": 550
+    },
+    {
+      "epoch": 2.488888888888889,
+      "grad_norm": 0.4825330674648285,
+      "learning_rate": 0.0002,
+      "loss": 0.6437,
+      "step": 560
+    },
+    {
+      "epoch": 2.533333333333333,
+      "grad_norm": 0.6174261569976807,
+      "learning_rate": 0.0002,
+      "loss": 0.606,
+      "step": 570
+    },
+    {
+      "epoch": 2.5777777777777775,
+      "grad_norm": 0.5854967832565308,
+      "learning_rate": 0.0002,
+      "loss": 0.5666,
+      "step": 580
+    },
+    {
+      "epoch": 2.6222222222222222,
+      "grad_norm": 0.44053414463996887,
+      "learning_rate": 0.0002,
+      "loss": 0.6017,
+      "step": 590
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 0.4803224503993988,
+      "learning_rate": 0.0002,
+      "loss": 0.6032,
+      "step": 600
+    },
+    {
+      "epoch": 2.7111111111111112,
+      "grad_norm": 0.5700278282165527,
+      "learning_rate": 0.0002,
+      "loss": 0.6238,
+      "step": 610
+    },
+    {
+      "epoch": 2.7555555555555555,
+      "grad_norm": 0.5318542122840881,
+      "learning_rate": 0.0002,
+      "loss": 0.5767,
+      "step": 620
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 0.5594431161880493,
+      "learning_rate": 0.0002,
+      "loss": 0.5849,
+      "step": 630
+    },
+    {
+      "epoch": 2.8444444444444446,
+      "grad_norm": 0.5598340630531311,
+      "learning_rate": 0.0002,
+      "loss": 0.6094,
+      "step": 640
+    },
+    {
+      "epoch": 2.888888888888889,
+      "grad_norm": 0.5017251372337341,
+      "learning_rate": 0.0002,
+      "loss": 0.6059,
+      "step": 650
+    },
+    {
+      "epoch": 2.9333333333333336,
+      "grad_norm": 0.46027806401252747,
+      "learning_rate": 0.0002,
+      "loss": 0.6047,
+      "step": 660
+    },
+    {
+      "epoch": 2.977777777777778,
+      "grad_norm": 0.5706299543380737,
+      "learning_rate": 0.0002,
+      "loss": 0.5912,
+      "step": 670
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 0.6362627148628235,
+      "eval_runtime": 85.1312,
+      "eval_samples_per_second": 3.747,
+      "eval_steps_per_second": 0.47,
+      "step": 675
+    },
+    {
+      "epoch": 3.022222222222222,
+      "grad_norm": 0.6016985774040222,
+      "learning_rate": 0.0002,
+      "loss": 0.5507,
+      "step": 680
+    },
+    {
+      "epoch": 3.066666666666667,
+      "grad_norm": 0.544710099697113,
+      "learning_rate": 0.0002,
+      "loss": 0.538,
+      "step": 690
+    },
+    {
+      "epoch": 3.111111111111111,
+      "grad_norm": 0.5868740081787109,
+      "learning_rate": 0.0002,
+      "loss": 0.5446,
+      "step": 700
+    },
+    {
+      "epoch": 3.1555555555555554,
+      "grad_norm": 0.4717068672180176,
+      "learning_rate": 0.0002,
+      "loss": 0.5337,
+      "step": 710
+    },
+    {
+      "epoch": 3.2,
+      "grad_norm": 0.5336525440216064,
+      "learning_rate": 0.0002,
+      "loss": 0.5561,
+      "step": 720
+    },
+    {
+      "epoch": 3.2444444444444445,
+      "grad_norm": 0.7006786465644836,
+      "learning_rate": 0.0002,
+      "loss": 0.5565,
+      "step": 730
+    },
+    {
+      "epoch": 3.2888888888888888,
+      "grad_norm": 0.5565499663352966,
+      "learning_rate": 0.0002,
+      "loss": 0.5312,
+      "step": 740
+    },
+    {
+      "epoch": 3.3333333333333335,
+      "grad_norm": 0.568071722984314,
+      "learning_rate": 0.0002,
+      "loss": 0.554,
+      "step": 750
+    },
+    {
+      "epoch": 3.3777777777777778,
+      "grad_norm": 0.6389057040214539,
+      "learning_rate": 0.0002,
+      "loss": 0.5297,
+      "step": 760
+    },
+    {
+      "epoch": 3.422222222222222,
+      "grad_norm": 0.5239511132240295,
+      "learning_rate": 0.0002,
+      "loss": 0.5191,
+      "step": 770
+    },
+    {
+      "epoch": 3.466666666666667,
+      "grad_norm": 0.6089216470718384,
+      "learning_rate": 0.0002,
+      "loss": 0.559,
+      "step": 780
+    },
+    {
+      "epoch": 3.511111111111111,
+      "grad_norm": 0.5546727776527405,
+      "learning_rate": 0.0002,
+      "loss": 0.5399,
+      "step": 790
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "grad_norm": 0.5639946460723877,
+      "learning_rate": 0.0002,
+      "loss": 0.5558,
+      "step": 800
+    },
+    {
+      "epoch": 3.6,
+      "grad_norm": 0.5576934218406677,
+      "learning_rate": 0.0002,
+      "loss": 0.5476,
+      "step": 810
+    },
+    {
+      "epoch": 3.6444444444444444,
+      "grad_norm": 0.6561392545700073,
+      "learning_rate": 0.0002,
+      "loss": 0.5361,
+      "step": 820
+    },
+    {
+      "epoch": 3.688888888888889,
+      "grad_norm": 0.7699626088142395,
+      "learning_rate": 0.0002,
+      "loss": 0.5324,
+      "step": 830
+    },
+    {
+      "epoch": 3.7333333333333334,
+      "grad_norm": 0.47137776017189026,
+      "learning_rate": 0.0002,
+      "loss": 0.551,
+      "step": 840
+    },
+    {
+      "epoch": 3.7777777777777777,
+      "grad_norm": 0.7258023023605347,
+      "learning_rate": 0.0002,
+      "loss": 0.5581,
+      "step": 850
+    },
+    {
+      "epoch": 3.822222222222222,
+      "grad_norm": 0.510877251625061,
+      "learning_rate": 0.0002,
+      "loss": 0.5586,
+      "step": 860
+    },
+    {
+      "epoch": 3.8666666666666667,
+      "grad_norm": 0.6691411733627319,
+      "learning_rate": 0.0002,
+      "loss": 0.564,
+      "step": 870
+    },
+    {
+      "epoch": 3.911111111111111,
+      "grad_norm": 0.5722544193267822,
+      "learning_rate": 0.0002,
+      "loss": 0.5361,
+      "step": 880
+    },
+    {
+      "epoch": 3.9555555555555557,
+      "grad_norm": 0.6555589437484741,
+      "learning_rate": 0.0002,
+      "loss": 0.5307,
+      "step": 890
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.5689453482627869,
+      "learning_rate": 0.0002,
+      "loss": 0.5251,
+      "step": 900
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 0.6063342690467834,
+      "eval_runtime": 62.7917,
+      "eval_samples_per_second": 5.08,
+      "eval_steps_per_second": 0.637,
+      "step": 900
+    },
+    {
+      "epoch": 4.044444444444444,
+      "grad_norm": 0.8715280294418335,
+      "learning_rate": 0.0002,
+      "loss": 0.4847,
+      "step": 910
+    },
+    {
+      "epoch": 4.088888888888889,
+      "grad_norm": 0.5597540736198425,
+      "learning_rate": 0.0002,
+      "loss": 0.4756,
+      "step": 920
+    },
+    {
+      "epoch": 4.133333333333334,
+      "grad_norm": 0.7538669109344482,
+      "learning_rate": 0.0002,
+      "loss": 0.4673,
+      "step": 930
+    },
+    {
+      "epoch": 4.177777777777778,
+      "grad_norm": 0.7434868812561035,
+      "learning_rate": 0.0002,
+      "loss": 0.4835,
+      "step": 940
+    },
+    {
+      "epoch": 4.222222222222222,
+      "grad_norm": 0.6148455142974854,
+      "learning_rate": 0.0002,
+      "loss": 0.486,
+      "step": 950
+    },
+    {
+      "epoch": 4.266666666666667,
+      "grad_norm": 0.6513713598251343,
+      "learning_rate": 0.0002,
+      "loss": 0.4812,
+      "step": 960
+    },
+    {
+      "epoch": 4.311111111111111,
+      "grad_norm": 0.7177010774612427,
+      "learning_rate": 0.0002,
+      "loss": 0.4823,
+      "step": 970
+    },
+    {
+      "epoch": 4.355555555555555,
+      "grad_norm": 0.710017204284668,
+      "learning_rate": 0.0002,
+      "loss": 0.4934,
+      "step": 980
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 0.7506688833236694,
+      "learning_rate": 0.0002,
+      "loss": 0.4682,
+      "step": 990
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 0.7618675231933594,
+      "learning_rate": 0.0002,
+      "loss": 0.4782,
+      "step": 1000
+    },
+    {
+      "epoch": 4.488888888888889,
+      "grad_norm": 0.6317481994628906,
+      "learning_rate": 0.0002,
+      "loss": 0.5075,
+      "step": 1010
+    },
+    {
+      "epoch": 4.533333333333333,
+      "grad_norm": 0.5700439214706421,
+      "learning_rate": 0.0002,
+      "loss": 0.4822,
+      "step": 1020
+    },
+    {
+      "epoch": 4.5777777777777775,
+      "grad_norm": 0.7100785374641418,
+      "learning_rate": 0.0002,
+      "loss": 0.4734,
+      "step": 1030
+    },
+    {
+      "epoch": 4.622222222222222,
+      "grad_norm": 0.6751446723937988,
+      "learning_rate": 0.0002,
+      "loss": 0.4764,
+      "step": 1040
+    },
+    {
+      "epoch": 4.666666666666667,
+      "grad_norm": 0.7100826501846313,
+      "learning_rate": 0.0002,
+      "loss": 0.4843,
+      "step": 1050
+    },
+    {
+      "epoch": 4.711111111111111,
+      "grad_norm": 0.6513156890869141,
+      "learning_rate": 0.0002,
+      "loss": 0.5018,
+      "step": 1060
+    },
+    {
+      "epoch": 4.7555555555555555,
+      "grad_norm": 0.8129924535751343,
+      "learning_rate": 0.0002,
+      "loss": 0.495,
+      "step": 1070
+    },
+    {
+      "epoch": 4.8,
+      "grad_norm": 0.6378636956214905,
+      "learning_rate": 0.0002,
+      "loss": 0.4819,
+      "step": 1080
+    },
+    {
+      "epoch": 4.844444444444444,
+      "grad_norm": 0.8045517802238464,
+      "learning_rate": 0.0002,
+      "loss": 0.4809,
+      "step": 1090
+    },
+    {
+      "epoch": 4.888888888888889,
+      "grad_norm": 0.712690532207489,
+      "learning_rate": 0.0002,
+      "loss": 0.4844,
+      "step": 1100
+    },
+    {
+      "epoch": 4.933333333333334,
+      "grad_norm": 0.7078566551208496,
+      "learning_rate": 0.0002,
+      "loss": 0.4732,
+      "step": 1110
+    },
+    {
+      "epoch": 4.977777777777778,
+      "grad_norm": 0.5979776382446289,
+      "learning_rate": 0.0002,
+      "loss": 0.4727,
+      "step": 1120
+    },
+    {
+      "epoch": 5.0,
+      "eval_loss": 0.5916207432746887,
+      "eval_runtime": 62.8273,
+      "eval_samples_per_second": 5.077,
+      "eval_steps_per_second": 0.637,
+      "step": 1125
+    },
+    {
+      "epoch": 5.022222222222222,
+      "grad_norm": 0.6213210225105286,
+      "learning_rate": 0.0002,
+      "loss": 0.4418,
+      "step": 1130
+    },
+    {
+      "epoch": 5.066666666666666,
+      "grad_norm": 0.8804424405097961,
+      "learning_rate": 0.0002,
+      "loss": 0.4213,
+      "step": 1140
+    },
+    {
+      "epoch": 5.111111111111111,
+      "grad_norm": 0.5710713267326355,
+      "learning_rate": 0.0002,
+      "loss": 0.4231,
+      "step": 1150
+    },
+    {
+      "epoch": 5.155555555555556,
+      "grad_norm": 0.6581844687461853,
+      "learning_rate": 0.0002,
+      "loss": 0.4279,
+      "step": 1160
+    },
+    {
+      "epoch": 5.2,
+      "grad_norm": 1.4485387802124023,
+      "learning_rate": 0.0002,
+      "loss": 0.4227,
+      "step": 1170
+    },
+    {
+      "epoch": 5.2444444444444445,
+      "grad_norm": 0.9506292343139648,
+      "learning_rate": 0.0002,
+      "loss": 0.4413,
+      "step": 1180
+    },
+    {
+      "epoch": 5.288888888888889,
+      "grad_norm": 0.6971795558929443,
+      "learning_rate": 0.0002,
+      "loss": 0.4129,
+      "step": 1190
+    },
+    {
+      "epoch": 5.333333333333333,
+      "grad_norm": 0.7639512419700623,
+      "learning_rate": 0.0002,
+      "loss": 0.4283,
+      "step": 1200
+    },
+    {
+      "epoch": 5.377777777777778,
+      "grad_norm": 0.7847402691841125,
+      "learning_rate": 0.0002,
+      "loss": 0.4581,
+      "step": 1210
+    },
+    {
+      "epoch": 5.4222222222222225,
+      "grad_norm": 0.6164460182189941,
+      "learning_rate": 0.0002,
+      "loss": 0.4311,
+      "step": 1220
+    },
+    {
+      "epoch": 5.466666666666667,
+      "grad_norm": 0.748383104801178,
+      "learning_rate": 0.0002,
+      "loss": 0.4304,
+      "step": 1230
+    },
+    {
+      "epoch": 5.511111111111111,
+      "grad_norm": 0.5674386620521545,
+      "learning_rate": 0.0002,
+      "loss": 0.4405,
+      "step": 1240
+    },
+    {
+      "epoch": 5.555555555555555,
+      "grad_norm": 0.9450201392173767,
+      "learning_rate": 0.0002,
+      "loss": 0.4396,
+      "step": 1250
+    },
+    {
+      "epoch": 5.6,
+      "grad_norm": 0.6066147089004517,
+      "learning_rate": 0.0002,
+      "loss": 0.4456,
+      "step": 1260
+    },
+    {
+      "epoch": 5.644444444444445,
+      "grad_norm": 1.1018579006195068,
+      "learning_rate": 0.0002,
+      "loss": 0.4154,
+      "step": 1270
+    },
+    {
+      "epoch": 5.688888888888889,
+      "grad_norm": 0.7561473846435547,
+      "learning_rate": 0.0002,
+      "loss": 0.4379,
+      "step": 1280
+    },
+    {
+      "epoch": 5.733333333333333,
+      "grad_norm": 0.8838764429092407,
+      "learning_rate": 0.0002,
+      "loss": 0.4421,
+      "step": 1290
+    },
+    {
+      "epoch": 5.777777777777778,
+      "grad_norm": 0.7195517420768738,
+      "learning_rate": 0.0002,
+      "loss": 0.439,
+      "step": 1300
+    },
+    {
+      "epoch": 5.822222222222222,
+      "grad_norm": 0.8123605847358704,
+      "learning_rate": 0.0002,
+      "loss": 0.4377,
+      "step": 1310
+    },
+    {
+      "epoch": 5.866666666666667,
+      "grad_norm": 0.6891103386878967,
+      "learning_rate": 0.0002,
+      "loss": 0.435,
+      "step": 1320
+    },
+    {
+      "epoch": 5.911111111111111,
+      "grad_norm": 1.0846312046051025,
+      "learning_rate": 0.0002,
+      "loss": 0.4276,
+      "step": 1330
+    },
+    {
+      "epoch": 5.955555555555556,
+      "grad_norm": 0.6821148991584778,
+      "learning_rate": 0.0002,
+      "loss": 0.4343,
+      "step": 1340
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 1.0653820037841797,
+      "learning_rate": 0.0002,
+      "loss": 0.4324,
+      "step": 1350
+    },
+    {
+      "epoch": 6.0,
+      "eval_loss": 0.5706757307052612,
+      "eval_runtime": 62.688,
+      "eval_samples_per_second": 5.089,
+      "eval_steps_per_second": 0.638,
+      "step": 1350
+    },
+    {
+      "epoch": 6.044444444444444,
+      "grad_norm": 0.9802659749984741,
+      "learning_rate": 0.0002,
+      "loss": 0.3798,
+      "step": 1360
+    },
+    {
+      "epoch": 6.088888888888889,
+      "grad_norm": 0.6135040521621704,
+      "learning_rate": 0.0002,
+      "loss": 0.3922,
+      "step": 1370
+    },
+    {
+      "epoch": 6.133333333333334,
+      "grad_norm": 0.7262284755706787,
+      "learning_rate": 0.0002,
+      "loss": 0.3775,
+      "step": 1380
+    },
+    {
+      "epoch": 6.177777777777778,
+      "grad_norm": 1.0442302227020264,
+      "learning_rate": 0.0002,
+      "loss": 0.3938,
+      "step": 1390
+    },
+    {
+      "epoch": 6.222222222222222,
+      "grad_norm": 0.6385959982872009,
+      "learning_rate": 0.0002,
+      "loss": 0.3955,
+      "step": 1400
+    },
+    {
+      "epoch": 6.266666666666667,
+      "grad_norm": 0.6713771224021912,
+      "learning_rate": 0.0002,
+      "loss": 0.3784,
+      "step": 1410
+    },
+    {
+      "epoch": 6.311111111111111,
+      "grad_norm": 0.5622318983078003,
+      "learning_rate": 0.0002,
+      "loss": 0.4045,
+      "step": 1420
+    },
+    {
+      "epoch": 6.355555555555555,
+      "grad_norm": 0.807933509349823,
+      "learning_rate": 0.0002,
+      "loss": 0.378,
+      "step": 1430
+    },
+    {
+      "epoch": 6.4,
+      "grad_norm": 0.7370631694793701,
+      "learning_rate": 0.0002,
+      "loss": 0.4065,
+      "step": 1440
+    },
+    {
+      "epoch": 6.444444444444445,
+      "grad_norm": 0.8557499647140503,
+      "learning_rate": 0.0002,
+      "loss": 0.397,
+      "step": 1450
+    },
+    {
+      "epoch": 6.488888888888889,
+      "grad_norm": 0.703018069267273,
+      "learning_rate": 0.0002,
+      "loss": 0.3878,
+      "step": 1460
+    },
+    {
+      "epoch": 6.533333333333333,
+      "grad_norm": 0.7124331593513489,
+      "learning_rate": 0.0002,
+      "loss": 0.3885,
+      "step": 1470
+    },
+    {
+      "epoch": 6.5777777777777775,
+      "grad_norm": 0.6987470388412476,
+      "learning_rate": 0.0002,
+      "loss": 0.397,
+      "step": 1480
+    },
+    {
+      "epoch": 6.622222222222222,
+      "grad_norm": 0.8872422575950623,
+      "learning_rate": 0.0002,
+      "loss": 0.3895,
+      "step": 1490
+    },
+    {
+      "epoch": 6.666666666666667,
+      "grad_norm": 0.7840065956115723,
+      "learning_rate": 0.0002,
+      "loss": 0.4046,
+      "step": 1500
+    },
+    {
+      "epoch": 6.711111111111111,
+      "grad_norm": 0.6836118698120117,
+      "learning_rate": 0.0002,
+      "loss": 0.3932,
+      "step": 1510
+    },
+    {
+      "epoch": 6.7555555555555555,
+      "grad_norm": 0.7144377827644348,
+      "learning_rate": 0.0002,
+      "loss": 0.4101,
+      "step": 1520
+    },
+    {
+      "epoch": 6.8,
+      "grad_norm": 0.7409266829490662,
+      "learning_rate": 0.0002,
+      "loss": 0.3932,
+      "step": 1530
+    },
+    {
+      "epoch": 6.844444444444444,
+      "grad_norm": 0.7913723587989807,
+      "learning_rate": 0.0002,
+      "loss": 0.3989,
+      "step": 1540
+    },
+    {
+      "epoch": 6.888888888888889,
+      "grad_norm": 0.8343528509140015,
+      "learning_rate": 0.0002,
+      "loss": 0.3987,
+      "step": 1550
+    },
+    {
+      "epoch": 6.933333333333334,
+      "grad_norm": 0.7788136601448059,
+      "learning_rate": 0.0002,
+      "loss": 0.3945,
+      "step": 1560
+    },
+    {
+      "epoch": 6.977777777777778,
+      "grad_norm": 0.913601279258728,
+      "learning_rate": 0.0002,
+      "loss": 0.4042,
+      "step": 1570
+    },
+    {
+      "epoch": 7.0,
+      "eval_loss": 0.5527341961860657,
+      "eval_runtime": 46.4414,
+      "eval_samples_per_second": 6.869,
+      "eval_steps_per_second": 0.861,
+      "step": 1575
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1800,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 8,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.86162943148032e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1575/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e959dbea71d2fa39c568a9d5bd6f17054bc6f0bf1101d40e99f8f6178453e4e
+size 5560

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1800/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.1

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1800/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1800/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa07c21653694aa072db05e651373f2135ba91389f8905d3b6a6d3162d99dfe0
+size 80755416

Qwen2-7B-Instruct_int4_flare-headlines_lr-0.0002_e-8_seq-512_lora-a-32-d-0.05-r-64_bs-1_gas-2_tf32-True_tunedata-portion-p-0.2-num-4314-sd-1/checkpoint-1800/added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}