Upload 15 files

README.md

@@ -1,3 +1,202 @@
----
-license: mit
----
+---
+library_name: peft
+base_model: Qwen2/Qwen2-7B-Instruct
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/root/LLM_Data_Engineer/LLaMA-Factory/models/Qwen2-7B-Instruct-lora-06072000",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "k_proj",
+    "gate_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4b55800b868058f3ead65a3d587b3da3dc74f88cabe633a7e67341879cb3e15
+size 80792096

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

all_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 2.9528795811518327,
+    "total_flos": 5.744674604829082e+16,
+    "train_loss": 0.7884446775659602,
+    "train_runtime": 510.788,
+    "train_samples_per_second": 8.957,
+    "train_steps_per_second": 0.276
+}

special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% set system_message = 'You are a helpful assistant.' %}{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{% if system_message is defined %}{{ '<|im_start|>system\n' + system_message + '<|im_end|>\n' }}{% endif %}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ '<|im_start|>user\n' + content + '<|im_end|>\n<|im_start|>assistant\n' }}{% elif message['role'] == 'assistant' %}{{ content + '<|im_end|>' + '\n' }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 2.9528795811518327,
+    "total_flos": 5.744674604829082e+16,
+    "train_loss": 0.7884446775659602,
+    "train_runtime": 510.788,
+    "train_samples_per_second": 8.957,
+    "train_steps_per_second": 0.276
+}

trainer_log.jsonl

@@ -0,0 +1,71 @@
+{"current_steps": 2, "total_steps": 141, "loss": 1.0015, "learning_rate": 1.3333333333333333e-05, "epoch": 0.041884816753926704, "percentage": 1.42, "elapsed_time": "0:00:07", "remaining_time": "0:08:23", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 4, "total_steps": 141, "loss": 0.9283, "learning_rate": 2.6666666666666667e-05, "epoch": 0.08376963350785341, "percentage": 2.84, "elapsed_time": "0:00:15", "remaining_time": "0:08:48", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 6, "total_steps": 141, "loss": 0.8643, "learning_rate": 4e-05, "epoch": 0.1256544502617801, "percentage": 4.26, "elapsed_time": "0:00:23", "remaining_time": "0:08:42", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 8, "total_steps": 141, "loss": 1.0013, "learning_rate": 5.333333333333333e-05, "epoch": 0.16753926701570682, "percentage": 5.67, "elapsed_time": "0:00:30", "remaining_time": "0:08:24", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 10, "total_steps": 141, "loss": 0.9341, "learning_rate": 6.666666666666667e-05, "epoch": 0.2094240837696335, "percentage": 7.09, "elapsed_time": "0:00:38", "remaining_time": "0:08:24", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 12, "total_steps": 141, "loss": 0.9068, "learning_rate": 8e-05, "epoch": 0.2513089005235602, "percentage": 8.51, "elapsed_time": "0:00:45", "remaining_time": "0:08:13", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 14, "total_steps": 141, "loss": 0.8083, "learning_rate": 9.333333333333334e-05, "epoch": 0.2931937172774869, "percentage": 9.93, "elapsed_time": "0:00:53", "remaining_time": "0:08:08", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 16, "total_steps": 141, "loss": 0.8985, "learning_rate": 9.998445910004082e-05, "epoch": 0.33507853403141363, "percentage": 11.35, "elapsed_time": "0:01:00", "remaining_time": "0:07:54", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 18, "total_steps": 141, "loss": 0.9362, "learning_rate": 9.986018985905901e-05, "epoch": 0.3769633507853403, "percentage": 12.77, "elapsed_time": "0:01:07", "remaining_time": "0:07:41", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 20, "total_steps": 141, "loss": 0.9174, "learning_rate": 9.961196033000861e-05, "epoch": 0.418848167539267, "percentage": 14.18, "elapsed_time": "0:01:14", "remaining_time": "0:07:30", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 22, "total_steps": 141, "loss": 0.8732, "learning_rate": 9.924038765061042e-05, "epoch": 0.4607329842931937, "percentage": 15.6, "elapsed_time": "0:01:22", "remaining_time": "0:07:23", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 24, "total_steps": 141, "loss": 0.8653, "learning_rate": 9.874639560909117e-05, "epoch": 0.5026178010471204, "percentage": 17.02, "elapsed_time": "0:01:30", "remaining_time": "0:07:20", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 26, "total_steps": 141, "loss": 0.8592, "learning_rate": 9.81312123475006e-05, "epoch": 0.5445026178010471, "percentage": 18.44, "elapsed_time": "0:01:36", "remaining_time": "0:07:08", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 28, "total_steps": 141, "loss": 0.805, "learning_rate": 9.73963673083566e-05, "epoch": 0.5863874345549738, "percentage": 19.86, "elapsed_time": "0:01:45", "remaining_time": "0:07:04", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 30, "total_steps": 141, "loss": 0.7464, "learning_rate": 9.654368743221022e-05, "epoch": 0.6282722513089005, "percentage": 21.28, "elapsed_time": "0:01:53", "remaining_time": "0:07:00", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 32, "total_steps": 141, "loss": 0.7796, "learning_rate": 9.557529261558367e-05, "epoch": 0.6701570680628273, "percentage": 22.7, "elapsed_time": "0:02:00", "remaining_time": "0:06:51", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 34, "total_steps": 141, "loss": 0.7768, "learning_rate": 9.449359044057345e-05, "epoch": 0.7120418848167539, "percentage": 24.11, "elapsed_time": "0:02:08", "remaining_time": "0:06:44", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 36, "total_steps": 141, "loss": 0.8751, "learning_rate": 9.330127018922194e-05, "epoch": 0.7539267015706806, "percentage": 25.53, "elapsed_time": "0:02:15", "remaining_time": "0:06:35", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 38, "total_steps": 141, "loss": 0.7966, "learning_rate": 9.200129615753859e-05, "epoch": 0.7958115183246073, "percentage": 26.95, "elapsed_time": "0:02:22", "remaining_time": "0:06:26", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 40, "total_steps": 141, "loss": 0.7963, "learning_rate": 9.059690028579284e-05, "epoch": 0.837696335078534, "percentage": 28.37, "elapsed_time": "0:02:29", "remaining_time": "0:06:17", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 42, "total_steps": 141, "loss": 0.8332, "learning_rate": 8.90915741234015e-05, "epoch": 0.8795811518324608, "percentage": 29.79, "elapsed_time": "0:02:35", "remaining_time": "0:06:07", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 44, "total_steps": 141, "loss": 0.7869, "learning_rate": 8.748906014838672e-05, "epoch": 0.9214659685863874, "percentage": 31.21, "elapsed_time": "0:02:42", "remaining_time": "0:05:59", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 46, "total_steps": 141, "loss": 0.8517, "learning_rate": 8.579334246298593e-05, "epoch": 0.9633507853403142, "percentage": 32.62, "elapsed_time": "0:02:48", "remaining_time": "0:05:48", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 48, "total_steps": 141, "loss": 0.8694, "learning_rate": 8.400863688854597e-05, "epoch": 1.0052356020942408, "percentage": 34.04, "elapsed_time": "0:02:55", "remaining_time": "0:05:39", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 50, "total_steps": 141, "loss": 0.8002, "learning_rate": 8.213938048432697e-05, "epoch": 1.0471204188481675, "percentage": 35.46, "elapsed_time": "0:03:02", "remaining_time": "0:05:31", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 52, "total_steps": 141, "loss": 0.7798, "learning_rate": 8.019022051627388e-05, "epoch": 1.0890052356020943, "percentage": 36.88, "elapsed_time": "0:03:09", "remaining_time": "0:05:23", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 54, "total_steps": 141, "loss": 0.8114, "learning_rate": 7.81660029031811e-05, "epoch": 1.130890052356021, "percentage": 38.3, "elapsed_time": "0:03:16", "remaining_time": "0:05:16", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 56, "total_steps": 141, "loss": 0.8206, "learning_rate": 7.60717601689749e-05, "epoch": 1.1727748691099475, "percentage": 39.72, "elapsed_time": "0:03:24", "remaining_time": "0:05:09", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 58, "total_steps": 141, "loss": 0.8477, "learning_rate": 7.391269893106592e-05, "epoch": 1.2146596858638743, "percentage": 41.13, "elapsed_time": "0:03:30", "remaining_time": "0:05:01", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 60, "total_steps": 141, "loss": 0.8213, "learning_rate": 7.169418695587791e-05, "epoch": 1.256544502617801, "percentage": 42.55, "elapsed_time": "0:03:37", "remaining_time": "0:04:53", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 62, "total_steps": 141, "loss": 0.8287, "learning_rate": 6.942173981373474e-05, "epoch": 1.2984293193717278, "percentage": 43.97, "elapsed_time": "0:03:43", "remaining_time": "0:04:45", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 64, "total_steps": 141, "loss": 0.7643, "learning_rate": 6.710100716628344e-05, "epoch": 1.3403141361256545, "percentage": 45.39, "elapsed_time": "0:03:51", "remaining_time": "0:04:39", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 66, "total_steps": 141, "loss": 0.7315, "learning_rate": 6.473775872054521e-05, "epoch": 1.3821989528795813, "percentage": 46.81, "elapsed_time": "0:03:59", "remaining_time": "0:04:31", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 68, "total_steps": 141, "loss": 0.7811, "learning_rate": 6.233786988451468e-05, "epoch": 1.4240837696335078, "percentage": 48.23, "elapsed_time": "0:04:07", "remaining_time": "0:04:25", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 70, "total_steps": 141, "loss": 0.754, "learning_rate": 5.9907307159969884e-05, "epoch": 1.4659685863874345, "percentage": 49.65, "elapsed_time": "0:04:15", "remaining_time": "0:04:18", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 72, "total_steps": 141, "loss": 0.7809, "learning_rate": 5.745211330880872e-05, "epoch": 1.5078534031413613, "percentage": 51.06, "elapsed_time": "0:04:22", "remaining_time": "0:04:11", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 74, "total_steps": 141, "loss": 0.8319, "learning_rate": 5.497839232979084e-05, "epoch": 1.5497382198952878, "percentage": 52.48, "elapsed_time": "0:04:30", "remaining_time": "0:04:04", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 76, "total_steps": 141, "loss": 0.7745, "learning_rate": 5.249229428303486e-05, "epoch": 1.5916230366492146, "percentage": 53.9, "elapsed_time": "0:04:37", "remaining_time": "0:03:57", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 78, "total_steps": 141, "loss": 0.8002, "learning_rate": 5e-05, "epoch": 1.6335078534031413, "percentage": 55.32, "elapsed_time": "0:04:44", "remaining_time": "0:03:50", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 80, "total_steps": 141, "loss": 0.7718, "learning_rate": 4.750770571696514e-05, "epoch": 1.675392670157068, "percentage": 56.74, "elapsed_time": "0:04:51", "remaining_time": "0:03:42", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 82, "total_steps": 141, "loss": 0.742, "learning_rate": 4.502160767020918e-05, "epoch": 1.7172774869109948, "percentage": 58.16, "elapsed_time": "0:04:58", "remaining_time": "0:03:34", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 84, "total_steps": 141, "loss": 0.7687, "learning_rate": 4.254788669119127e-05, "epoch": 1.7591623036649215, "percentage": 59.57, "elapsed_time": "0:05:05", "remaining_time": "0:03:27", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 86, "total_steps": 141, "loss": 0.7066, "learning_rate": 4.0092692840030134e-05, "epoch": 1.8010471204188483, "percentage": 60.99, "elapsed_time": "0:05:12", "remaining_time": "0:03:19", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 88, "total_steps": 141, "loss": 0.7098, "learning_rate": 3.7662130115485314e-05, "epoch": 1.8429319371727748, "percentage": 62.41, "elapsed_time": "0:05:20", "remaining_time": "0:03:12", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 90, "total_steps": 141, "loss": 0.6961, "learning_rate": 3.5262241279454785e-05, "epoch": 1.8848167539267016, "percentage": 63.83, "elapsed_time": "0:05:28", "remaining_time": "0:03:06", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 92, "total_steps": 141, "loss": 0.7948, "learning_rate": 3.289899283371657e-05, "epoch": 1.9267015706806283, "percentage": 65.25, "elapsed_time": "0:05:34", "remaining_time": "0:02:58", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 94, "total_steps": 141, "loss": 0.7264, "learning_rate": 3.0578260186265265e-05, "epoch": 1.9685863874345548, "percentage": 66.67, "elapsed_time": "0:05:41", "remaining_time": "0:02:50", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 96, "total_steps": 141, "loss": 0.6696, "learning_rate": 2.8305813044122097e-05, "epoch": 2.0104712041884816, "percentage": 68.09, "elapsed_time": "0:05:49", "remaining_time": "0:02:43", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 98, "total_steps": 141, "loss": 0.7135, "learning_rate": 2.6087301068934106e-05, "epoch": 2.0523560209424083, "percentage": 69.5, "elapsed_time": "0:05:56", "remaining_time": "0:02:36", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 100, "total_steps": 141, "loss": 0.7865, "learning_rate": 2.39282398310251e-05, "epoch": 2.094240837696335, "percentage": 70.92, "elapsed_time": "0:06:03", "remaining_time": "0:02:28", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 102, "total_steps": 141, "loss": 0.7411, "learning_rate": 2.1833997096818898e-05, "epoch": 2.136125654450262, "percentage": 72.34, "elapsed_time": "0:06:11", "remaining_time": "0:02:21", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 104, "total_steps": 141, "loss": 0.772, "learning_rate": 1.980977948372612e-05, "epoch": 2.1780104712041886, "percentage": 73.76, "elapsed_time": "0:06:17", "remaining_time": "0:02:14", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 106, "total_steps": 141, "loss": 0.7096, "learning_rate": 1.7860619515673033e-05, "epoch": 2.2198952879581153, "percentage": 75.18, "elapsed_time": "0:06:25", "remaining_time": "0:02:07", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 108, "total_steps": 141, "loss": 0.7378, "learning_rate": 1.599136311145402e-05, "epoch": 2.261780104712042, "percentage": 76.6, "elapsed_time": "0:06:32", "remaining_time": "0:01:59", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 110, "total_steps": 141, "loss": 0.6471, "learning_rate": 1.4206657537014079e-05, "epoch": 2.303664921465969, "percentage": 78.01, "elapsed_time": "0:06:39", "remaining_time": "0:01:52", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 112, "total_steps": 141, "loss": 0.7359, "learning_rate": 1.2510939851613285e-05, "epoch": 2.345549738219895, "percentage": 79.43, "elapsed_time": "0:06:47", "remaining_time": "0:01:45", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 114, "total_steps": 141, "loss": 0.7572, "learning_rate": 1.090842587659851e-05, "epoch": 2.387434554973822, "percentage": 80.85, "elapsed_time": "0:06:54", "remaining_time": "0:01:38", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 116, "total_steps": 141, "loss": 0.7411, "learning_rate": 9.403099714207175e-06, "epoch": 2.4293193717277486, "percentage": 82.27, "elapsed_time": "0:07:01", "remaining_time": "0:01:30", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 118, "total_steps": 141, "loss": 0.6692, "learning_rate": 7.998703842461431e-06, "epoch": 2.4712041884816753, "percentage": 83.69, "elapsed_time": "0:07:08", "remaining_time": "0:01:23", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 120, "total_steps": 141, "loss": 0.7619, "learning_rate": 6.698729810778065e-06, "epoch": 2.513089005235602, "percentage": 85.11, "elapsed_time": "0:07:15", "remaining_time": "0:01:16", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 122, "total_steps": 141, "loss": 0.708, "learning_rate": 5.506409559426573e-06, "epoch": 2.554973821989529, "percentage": 86.52, "elapsed_time": "0:07:22", "remaining_time": "0:01:08", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 124, "total_steps": 141, "loss": 0.7279, "learning_rate": 4.424707384416344e-06, "epoch": 2.5968586387434556, "percentage": 87.94, "elapsed_time": "0:07:30", "remaining_time": "0:01:01", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 126, "total_steps": 141, "loss": 0.7629, "learning_rate": 3.4563125677897932e-06, "epoch": 2.6387434554973823, "percentage": 89.36, "elapsed_time": "0:07:37", "remaining_time": "0:00:54", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 128, "total_steps": 141, "loss": 0.7334, "learning_rate": 2.603632691643415e-06, "epoch": 2.680628272251309, "percentage": 90.78, "elapsed_time": "0:07:44", "remaining_time": "0:00:47", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 130, "total_steps": 141, "loss": 0.6643, "learning_rate": 1.8687876524993987e-06, "epoch": 2.7225130890052354, "percentage": 92.2, "elapsed_time": "0:07:52", "remaining_time": "0:00:39", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 132, "total_steps": 141, "loss": 0.7363, "learning_rate": 1.2536043909088191e-06, "epoch": 2.7643979057591626, "percentage": 93.62, "elapsed_time": "0:07:58", "remaining_time": "0:00:32", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 134, "total_steps": 141, "loss": 0.7018, "learning_rate": 7.596123493895991e-07, "epoch": 2.806282722513089, "percentage": 95.04, "elapsed_time": "0:08:06", "remaining_time": "0:00:25", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 136, "total_steps": 141, "loss": 0.8282, "learning_rate": 3.8803966999139684e-07, "epoch": 2.8481675392670156, "percentage": 96.45, "elapsed_time": "0:08:14", "remaining_time": "0:00:18", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 138, "total_steps": 141, "loss": 0.6387, "learning_rate": 1.3981014094099353e-07, "epoch": 2.8900523560209423, "percentage": 97.87, "elapsed_time": "0:08:21", "remaining_time": "0:00:10", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 140, "total_steps": 141, "loss": 0.7076, "learning_rate": 1.5540899959187727e-08, "epoch": 2.931937172774869, "percentage": 99.29, "elapsed_time": "0:08:27", "remaining_time": "0:00:03", "throughput": "0.00", "total_tokens": 0}
+{"current_steps": 141, "total_steps": 141, "epoch": 2.9528795811518327, "percentage": 100.0, "elapsed_time": "0:08:30", "remaining_time": "0:00:00", "throughput": "0.00", "total_tokens": 0}

trainer_state.json

@@ -0,0 +1,532 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.9528795811518327,
+  "eval_steps": 500,
+  "global_step": 141,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.041884816753926704,
+      "grad_norm": 0.21733999252319336,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.0015,
+      "step": 2
+    },
+    {
+      "epoch": 0.08376963350785341,
+      "grad_norm": 0.21422453224658966,
+      "learning_rate": 2.6666666666666667e-05,
+      "loss": 0.9283,
+      "step": 4
+    },
+    {
+      "epoch": 0.1256544502617801,
+      "grad_norm": 0.18293505907058716,
+      "learning_rate": 4e-05,
+      "loss": 0.8643,
+      "step": 6
+    },
+    {
+      "epoch": 0.16753926701570682,
+      "grad_norm": 0.2287076711654663,
+      "learning_rate": 5.333333333333333e-05,
+      "loss": 1.0013,
+      "step": 8
+    },
+    {
+      "epoch": 0.2094240837696335,
+      "grad_norm": 0.2509159445762634,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 0.9341,
+      "step": 10
+    },
+    {
+      "epoch": 0.2513089005235602,
+      "grad_norm": 0.21400025486946106,
+      "learning_rate": 8e-05,
+      "loss": 0.9068,
+      "step": 12
+    },
+    {
+      "epoch": 0.2931937172774869,
+      "grad_norm": 0.20343424379825592,
+      "learning_rate": 9.333333333333334e-05,
+      "loss": 0.8083,
+      "step": 14
+    },
+    {
+      "epoch": 0.33507853403141363,
+      "grad_norm": 0.25129085779190063,
+      "learning_rate": 9.998445910004082e-05,
+      "loss": 0.8985,
+      "step": 16
+    },
+    {
+      "epoch": 0.3769633507853403,
+      "grad_norm": 0.28799620270729065,
+      "learning_rate": 9.986018985905901e-05,
+      "loss": 0.9362,
+      "step": 18
+    },
+    {
+      "epoch": 0.418848167539267,
+      "grad_norm": 0.29551103711128235,
+      "learning_rate": 9.961196033000861e-05,
+      "loss": 0.9174,
+      "step": 20
+    },
+    {
+      "epoch": 0.4607329842931937,
+      "grad_norm": 0.31457847356796265,
+      "learning_rate": 9.924038765061042e-05,
+      "loss": 0.8732,
+      "step": 22
+    },
+    {
+      "epoch": 0.5026178010471204,
+      "grad_norm": 0.2715758979320526,
+      "learning_rate": 9.874639560909117e-05,
+      "loss": 0.8653,
+      "step": 24
+    },
+    {
+      "epoch": 0.5445026178010471,
+      "grad_norm": 0.25824904441833496,
+      "learning_rate": 9.81312123475006e-05,
+      "loss": 0.8592,
+      "step": 26
+    },
+    {
+      "epoch": 0.5863874345549738,
+      "grad_norm": 0.22251802682876587,
+      "learning_rate": 9.73963673083566e-05,
+      "loss": 0.805,
+      "step": 28
+    },
+    {
+      "epoch": 0.6282722513089005,
+      "grad_norm": 0.22290602326393127,
+      "learning_rate": 9.654368743221022e-05,
+      "loss": 0.7464,
+      "step": 30
+    },
+    {
+      "epoch": 0.6701570680628273,
+      "grad_norm": 0.2132934331893921,
+      "learning_rate": 9.557529261558367e-05,
+      "loss": 0.7796,
+      "step": 32
+    },
+    {
+      "epoch": 0.7120418848167539,
+      "grad_norm": 0.19897978007793427,
+      "learning_rate": 9.449359044057345e-05,
+      "loss": 0.7768,
+      "step": 34
+    },
+    {
+      "epoch": 0.7539267015706806,
+      "grad_norm": 0.21434754133224487,
+      "learning_rate": 9.330127018922194e-05,
+      "loss": 0.8751,
+      "step": 36
+    },
+    {
+      "epoch": 0.7958115183246073,
+      "grad_norm": 0.2214685082435608,
+      "learning_rate": 9.200129615753859e-05,
+      "loss": 0.7966,
+      "step": 38
+    },
+    {
+      "epoch": 0.837696335078534,
+      "grad_norm": 0.20432740449905396,
+      "learning_rate": 9.059690028579284e-05,
+      "loss": 0.7963,
+      "step": 40
+    },
+    {
+      "epoch": 0.8795811518324608,
+      "grad_norm": 0.23610389232635498,
+      "learning_rate": 8.90915741234015e-05,
+      "loss": 0.8332,
+      "step": 42
+    },
+    {
+      "epoch": 0.9214659685863874,
+      "grad_norm": 0.23847244679927826,
+      "learning_rate": 8.748906014838672e-05,
+      "loss": 0.7869,
+      "step": 44
+    },
+    {
+      "epoch": 0.9633507853403142,
+      "grad_norm": 0.23536360263824463,
+      "learning_rate": 8.579334246298593e-05,
+      "loss": 0.8517,
+      "step": 46
+    },
+    {
+      "epoch": 1.0052356020942408,
+      "grad_norm": 0.24705368280410767,
+      "learning_rate": 8.400863688854597e-05,
+      "loss": 0.8694,
+      "step": 48
+    },
+    {
+      "epoch": 1.0471204188481675,
+      "grad_norm": 0.22067400813102722,
+      "learning_rate": 8.213938048432697e-05,
+      "loss": 0.8002,
+      "step": 50
+    },
+    {
+      "epoch": 1.0890052356020943,
+      "grad_norm": 0.21238695085048676,
+      "learning_rate": 8.019022051627388e-05,
+      "loss": 0.7798,
+      "step": 52
+    },
+    {
+      "epoch": 1.130890052356021,
+      "grad_norm": 0.2280968427658081,
+      "learning_rate": 7.81660029031811e-05,
+      "loss": 0.8114,
+      "step": 54
+    },
+    {
+      "epoch": 1.1727748691099475,
+      "grad_norm": 0.2327934205532074,
+      "learning_rate": 7.60717601689749e-05,
+      "loss": 0.8206,
+      "step": 56
+    },
+    {
+      "epoch": 1.2146596858638743,
+      "grad_norm": 0.2437446266412735,
+      "learning_rate": 7.391269893106592e-05,
+      "loss": 0.8477,
+      "step": 58
+    },
+    {
+      "epoch": 1.256544502617801,
+      "grad_norm": 0.2429206371307373,
+      "learning_rate": 7.169418695587791e-05,
+      "loss": 0.8213,
+      "step": 60
+    },
+    {
+      "epoch": 1.2984293193717278,
+      "grad_norm": 0.23616257309913635,
+      "learning_rate": 6.942173981373474e-05,
+      "loss": 0.8287,
+      "step": 62
+    },
+    {
+      "epoch": 1.3403141361256545,
+      "grad_norm": 0.2597900629043579,
+      "learning_rate": 6.710100716628344e-05,
+      "loss": 0.7643,
+      "step": 64
+    },
+    {
+      "epoch": 1.3821989528795813,
+      "grad_norm": 0.2668094038963318,
+      "learning_rate": 6.473775872054521e-05,
+      "loss": 0.7315,
+      "step": 66
+    },
+    {
+      "epoch": 1.4240837696335078,
+      "grad_norm": 0.2583785355091095,
+      "learning_rate": 6.233786988451468e-05,
+      "loss": 0.7811,
+      "step": 68
+    },
+    {
+      "epoch": 1.4659685863874345,
+      "grad_norm": 0.2756960391998291,
+      "learning_rate": 5.9907307159969884e-05,
+      "loss": 0.754,
+      "step": 70
+    },
+    {
+      "epoch": 1.5078534031413613,
+      "grad_norm": 0.28554120659828186,
+      "learning_rate": 5.745211330880872e-05,
+      "loss": 0.7809,
+      "step": 72
+    },
+    {
+      "epoch": 1.5497382198952878,
+      "grad_norm": 0.3146248459815979,
+      "learning_rate": 5.497839232979084e-05,
+      "loss": 0.8319,
+      "step": 74
+    },
+    {
+      "epoch": 1.5916230366492146,
+      "grad_norm": 0.2715282142162323,
+      "learning_rate": 5.249229428303486e-05,
+      "loss": 0.7745,
+      "step": 76
+    },
+    {
+      "epoch": 1.6335078534031413,
+      "grad_norm": 0.28730642795562744,
+      "learning_rate": 5e-05,
+      "loss": 0.8002,
+      "step": 78
+    },
+    {
+      "epoch": 1.675392670157068,
+      "grad_norm": 0.3208574652671814,
+      "learning_rate": 4.750770571696514e-05,
+      "loss": 0.7718,
+      "step": 80
+    },
+    {
+      "epoch": 1.7172774869109948,
+      "grad_norm": 0.307959645986557,
+      "learning_rate": 4.502160767020918e-05,
+      "loss": 0.742,
+      "step": 82
+    },
+    {
+      "epoch": 1.7591623036649215,
+      "grad_norm": 0.34215685725212097,
+      "learning_rate": 4.254788669119127e-05,
+      "loss": 0.7687,
+      "step": 84
+    },
+    {
+      "epoch": 1.8010471204188483,
+      "grad_norm": 0.28537240624427795,
+      "learning_rate": 4.0092692840030134e-05,
+      "loss": 0.7066,
+      "step": 86
+    },
+    {
+      "epoch": 1.8429319371727748,
+      "grad_norm": 0.31295880675315857,
+      "learning_rate": 3.7662130115485314e-05,
+      "loss": 0.7098,
+      "step": 88
+    },
+    {
+      "epoch": 1.8848167539267016,
+      "grad_norm": 0.2994559705257416,
+      "learning_rate": 3.5262241279454785e-05,
+      "loss": 0.6961,
+      "step": 90
+    },
+    {
+      "epoch": 1.9267015706806283,
+      "grad_norm": 0.3853859603404999,
+      "learning_rate": 3.289899283371657e-05,
+      "loss": 0.7948,
+      "step": 92
+    },
+    {
+      "epoch": 1.9685863874345548,
+      "grad_norm": 0.3592422306537628,
+      "learning_rate": 3.0578260186265265e-05,
+      "loss": 0.7264,
+      "step": 94
+    },
+    {
+      "epoch": 2.0104712041884816,
+      "grad_norm": 0.42988094687461853,
+      "learning_rate": 2.8305813044122097e-05,
+      "loss": 0.6696,
+      "step": 96
+    },
+    {
+      "epoch": 2.0523560209424083,
+      "grad_norm": 0.3221912086009979,
+      "learning_rate": 2.6087301068934106e-05,
+      "loss": 0.7135,
+      "step": 98
+    },
+    {
+      "epoch": 2.094240837696335,
+      "grad_norm": 0.3392656743526459,
+      "learning_rate": 2.39282398310251e-05,
+      "loss": 0.7865,
+      "step": 100
+    },
+    {
+      "epoch": 2.136125654450262,
+      "grad_norm": 0.32586315274238586,
+      "learning_rate": 2.1833997096818898e-05,
+      "loss": 0.7411,
+      "step": 102
+    },
+    {
+      "epoch": 2.1780104712041886,
+      "grad_norm": 0.36464083194732666,
+      "learning_rate": 1.980977948372612e-05,
+      "loss": 0.772,
+      "step": 104
+    },
+    {
+      "epoch": 2.2198952879581153,
+      "grad_norm": 0.3270440697669983,
+      "learning_rate": 1.7860619515673033e-05,
+      "loss": 0.7096,
+      "step": 106
+    },
+    {
+      "epoch": 2.261780104712042,
+      "grad_norm": 0.3391498923301697,
+      "learning_rate": 1.599136311145402e-05,
+      "loss": 0.7378,
+      "step": 108
+    },
+    {
+      "epoch": 2.303664921465969,
+      "grad_norm": 0.3948444128036499,
+      "learning_rate": 1.4206657537014079e-05,
+      "loss": 0.6471,
+      "step": 110
+    },
+    {
+      "epoch": 2.345549738219895,
+      "grad_norm": 0.38464757800102234,
+      "learning_rate": 1.2510939851613285e-05,
+      "loss": 0.7359,
+      "step": 112
+    },
+    {
+      "epoch": 2.387434554973822,
+      "grad_norm": 0.38830411434173584,
+      "learning_rate": 1.090842587659851e-05,
+      "loss": 0.7572,
+      "step": 114
+    },
+    {
+      "epoch": 2.4293193717277486,
+      "grad_norm": 0.37749606370925903,
+      "learning_rate": 9.403099714207175e-06,
+      "loss": 0.7411,
+      "step": 116
+    },
+    {
+      "epoch": 2.4712041884816753,
+      "grad_norm": 0.3769814968109131,
+      "learning_rate": 7.998703842461431e-06,
+      "loss": 0.6692,
+      "step": 118
+    },
+    {
+      "epoch": 2.513089005235602,
+      "grad_norm": 0.4309244453907013,
+      "learning_rate": 6.698729810778065e-06,
+      "loss": 0.7619,
+      "step": 120
+    },
+    {
+      "epoch": 2.554973821989529,
+      "grad_norm": 0.37709513306617737,
+      "learning_rate": 5.506409559426573e-06,
+      "loss": 0.708,
+      "step": 122
+    },
+    {
+      "epoch": 2.5968586387434556,
+      "grad_norm": 0.40647101402282715,
+      "learning_rate": 4.424707384416344e-06,
+      "loss": 0.7279,
+      "step": 124
+    },
+    {
+      "epoch": 2.6387434554973823,
+      "grad_norm": 0.3731394112110138,
+      "learning_rate": 3.4563125677897932e-06,
+      "loss": 0.7629,
+      "step": 126
+    },
+    {
+      "epoch": 2.680628272251309,
+      "grad_norm": 0.40869101881980896,
+      "learning_rate": 2.603632691643415e-06,
+      "loss": 0.7334,
+      "step": 128
+    },
+    {
+      "epoch": 2.7225130890052354,
+      "grad_norm": 0.33611902594566345,
+      "learning_rate": 1.8687876524993987e-06,
+      "loss": 0.6643,
+      "step": 130
+    },
+    {
+      "epoch": 2.7643979057591626,
+      "grad_norm": 0.38377344608306885,
+      "learning_rate": 1.2536043909088191e-06,
+      "loss": 0.7363,
+      "step": 132
+    },
+    {
+      "epoch": 2.806282722513089,
+      "grad_norm": 0.38260236382484436,
+      "learning_rate": 7.596123493895991e-07,
+      "loss": 0.7018,
+      "step": 134
+    },
+    {
+      "epoch": 2.8481675392670156,
+      "grad_norm": 0.3747893273830414,
+      "learning_rate": 3.8803966999139684e-07,
+      "loss": 0.8282,
+      "step": 136
+    },
+    {
+      "epoch": 2.8900523560209423,
+      "grad_norm": 0.40073326230049133,
+      "learning_rate": 1.3981014094099353e-07,
+      "loss": 0.6387,
+      "step": 138
+    },
+    {
+      "epoch": 2.931937172774869,
+      "grad_norm": 0.4021676480770111,
+      "learning_rate": 1.5540899959187727e-08,
+      "loss": 0.7076,
+      "step": 140
+    },
+    {
+      "epoch": 2.9528795811518327,
+      "step": 141,
+      "total_flos": 5.744674604829082e+16,
+      "train_loss": 0.7884446775659602,
+      "train_runtime": 510.788,
+      "train_samples_per_second": 8.957,
+      "train_steps_per_second": 0.276
+    }
+  ],
+  "logging_steps": 2,
+  "max_steps": 141,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": false,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.744674604829082e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43983893470f521a5d365e448af8448f4819056d6c2d89ad6ca9fb6d006dea24
+size 5304