Training in progress, step 180, checkpoint

checkpoint-180/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: Weni/WeniGPT-Agents-Mistral-1.0.0-SFT-merged
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.10.0

checkpoint-180/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Weni/WeniGPT-Agents-Mistral-1.0.0-SFT-merged",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "down_proj",
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "gate_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-180/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:328b1e951c7fd79cdee800f3245ba044a5cd6d04106a77b7b82e3cff721bb4fe
+size 167832240

checkpoint-180/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25c0bbf99788d3f9193284708fce6b95445c4745a1129e791fcc14c2b01e545b
+size 335922386

checkpoint-180/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae78313eb528c8d3695eebaf4de3539bd0a0bc6ee18c66af1ee183442f1758a0
+size 15024

checkpoint-180/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b38031f60d9e88601d369ef46bcdcf2b5b03f2cb4ba93853bcb2328df7ebb7c
+size 15024

checkpoint-180/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f58092375c93d237cd0e3149aecfbf83e2acdae46279e07a32920d01cb507e64
+size 15024

checkpoint-180/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83cd4bbff9962da7ec6787fcea8d65df7096917f9a5902e249ba7aee8887fe5f
+size 15024

checkpoint-180/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74f15868c98428b757deb41ceef3036d052c50d320ff1d3c76314d699ef0babc
+size 1064

checkpoint-180/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-180/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-180/tokenizer_config.json

@@ -0,0 +1,49 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}{% if message['role'] == 'user' %}{{ bos_token + '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'system' %}{{ '<<SYS>>\\n' + message['content'] + '\\n<</SYS>>\\n\\n' }}{% elif message['role'] == 'assistant' %}{{ ' '  + message['content'] + ' ' + eos_token }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "max_lenght": 8192,
+  "max_length": 8192,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<unk>",
+  "padding": true,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-180/trainer_state.json

@@ -0,0 +1,387 @@
+{
+  "best_metric": 0.021960575133562088,
+  "best_model_checkpoint": "./mistral/21-04-24-Weni-WeniGPT-Agents-Mistral-1.0.0-SFT-1.0.15-DPO_Experiment on DPO with other hyperparameters and best SFT model of WeniGPT-2_max_steps-180_batch_8_2024-04-21_ppid_9/checkpoint-180",
+  "epoch": 5.806451612903226,
+  "eval_steps": 30,
+  "global_step": 180,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.3225806451612903,
+      "grad_norm": 76.17081451416016,
+      "learning_rate": 4.166666666666667e-06,
+      "logits/chosen": -1.8212074041366577,
+      "logits/rejected": -1.836817979812622,
+      "logps/chosen": -181.83949279785156,
+      "logps/rejected": -258.220458984375,
+      "loss": 0.6886,
+      "rewards/accuracies": 0.25,
+      "rewards/chosen": 0.009336434304714203,
+      "rewards/margins": 0.0099251177161932,
+      "rewards/rejected": -0.0005886839935556054,
+      "step": 10
+    },
+    {
+      "epoch": 0.6451612903225806,
+      "grad_norm": 40.539878845214844,
+      "learning_rate": 4.741379310344828e-06,
+      "logits/chosen": -1.8025119304656982,
+      "logits/rejected": -1.8262531757354736,
+      "logps/chosen": -148.07679748535156,
+      "logps/rejected": -243.6903839111328,
+      "loss": 0.4144,
+      "rewards/accuracies": 0.949999988079071,
+      "rewards/chosen": 0.695365846157074,
+      "rewards/margins": 0.7868010997772217,
+      "rewards/rejected": -0.09143529087305069,
+      "step": 20
+    },
+    {
+      "epoch": 0.967741935483871,
+      "grad_norm": 13.742331504821777,
+      "learning_rate": 4.454022988505747e-06,
+      "logits/chosen": -1.7753016948699951,
+      "logits/rejected": -1.787701964378357,
+      "logps/chosen": -138.11192321777344,
+      "logps/rejected": -181.8013153076172,
+      "loss": 0.2469,
+      "rewards/accuracies": 0.949999988079071,
+      "rewards/chosen": 1.8171169757843018,
+      "rewards/margins": 2.026198387145996,
+      "rewards/rejected": -0.20908141136169434,
+      "step": 30
+    },
+    {
+      "epoch": 0.967741935483871,
+      "eval_logits/chosen": -1.8694961071014404,
+      "eval_logits/rejected": -1.8717602491378784,
+      "eval_logps/chosen": -128.06077575683594,
+      "eval_logps/rejected": -171.74044799804688,
+      "eval_loss": 0.2019422948360443,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 1.950033187866211,
+      "eval_rewards/margins": 2.1478688716888428,
+      "eval_rewards/rejected": -0.19783565402030945,
+      "eval_runtime": 17.0291,
+      "eval_samples_per_second": 1.644,
+      "eval_steps_per_second": 0.411,
+      "step": 30
+    },
+    {
+      "epoch": 1.2903225806451613,
+      "grad_norm": 25.29999542236328,
+      "learning_rate": 4.166666666666667e-06,
+      "logits/chosen": -1.832271933555603,
+      "logits/rejected": -1.852087378501892,
+      "logps/chosen": -138.46139526367188,
+      "logps/rejected": -207.39346313476562,
+      "loss": 0.1054,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.361999034881592,
+      "rewards/margins": 3.551447629928589,
+      "rewards/rejected": -1.1894484758377075,
+      "step": 40
+    },
+    {
+      "epoch": 1.6129032258064515,
+      "grad_norm": 22.02225112915039,
+      "learning_rate": 3.8793103448275865e-06,
+      "logits/chosen": -1.784176230430603,
+      "logits/rejected": -1.8057140111923218,
+      "logps/chosen": -140.75958251953125,
+      "logps/rejected": -195.9730224609375,
+      "loss": 0.0913,
+      "rewards/accuracies": 0.949999988079071,
+      "rewards/chosen": 2.6018612384796143,
+      "rewards/margins": 3.601649522781372,
+      "rewards/rejected": -0.9997884631156921,
+      "step": 50
+    },
+    {
+      "epoch": 1.935483870967742,
+      "grad_norm": 3.1534383296966553,
+      "learning_rate": 3.5919540229885056e-06,
+      "logits/chosen": -1.8954330682754517,
+      "logits/rejected": -1.9075534343719482,
+      "logps/chosen": -144.24533081054688,
+      "logps/rejected": -241.29345703125,
+      "loss": 0.1276,
+      "rewards/accuracies": 0.8500000238418579,
+      "rewards/chosen": 2.946927785873413,
+      "rewards/margins": 4.766872406005859,
+      "rewards/rejected": -1.8199446201324463,
+      "step": 60
+    },
+    {
+      "epoch": 1.935483870967742,
+      "eval_logits/chosen": -1.8808324337005615,
+      "eval_logits/rejected": -1.8824094533920288,
+      "eval_logps/chosen": -123.88575744628906,
+      "eval_logps/rejected": -172.878173828125,
+      "eval_loss": 0.09369345754384995,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 3.2025339603424072,
+      "eval_rewards/margins": 3.741692066192627,
+      "eval_rewards/rejected": -0.5391579866409302,
+      "eval_runtime": 16.97,
+      "eval_samples_per_second": 1.65,
+      "eval_steps_per_second": 0.412,
+      "step": 60
+    },
+    {
+      "epoch": 2.258064516129032,
+      "grad_norm": 2.812629461288452,
+      "learning_rate": 3.3045977011494256e-06,
+      "logits/chosen": -1.7766342163085938,
+      "logits/rejected": -1.7891451120376587,
+      "logps/chosen": -117.1172103881836,
+      "logps/rejected": -197.6320037841797,
+      "loss": 0.0476,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 4.011519432067871,
+      "rewards/margins": 5.832650184631348,
+      "rewards/rejected": -1.8211311101913452,
+      "step": 70
+    },
+    {
+      "epoch": 2.5806451612903225,
+      "grad_norm": 7.378849506378174,
+      "learning_rate": 3.017241379310345e-06,
+      "logits/chosen": -1.873382568359375,
+      "logits/rejected": -1.8968416452407837,
+      "logps/chosen": -115.59322357177734,
+      "logps/rejected": -202.29922485351562,
+      "loss": 0.0422,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 3.784104824066162,
+      "rewards/margins": 5.744203090667725,
+      "rewards/rejected": -1.9600980281829834,
+      "step": 80
+    },
+    {
+      "epoch": 2.903225806451613,
+      "grad_norm": 1.8571914434432983,
+      "learning_rate": 2.729885057471265e-06,
+      "logits/chosen": -1.8314573764801025,
+      "logits/rejected": -1.8384296894073486,
+      "logps/chosen": -135.9064483642578,
+      "logps/rejected": -216.98532104492188,
+      "loss": 0.0187,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 3.950857639312744,
+      "rewards/margins": 6.437536716461182,
+      "rewards/rejected": -2.4866783618927,
+      "step": 90
+    },
+    {
+      "epoch": 2.903225806451613,
+      "eval_logits/chosen": -1.8896868228912354,
+      "eval_logits/rejected": -1.8911399841308594,
+      "eval_logps/chosen": -121.66944885253906,
+      "eval_logps/rejected": -175.50460815429688,
+      "eval_loss": 0.05724393203854561,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 3.8674278259277344,
+      "eval_rewards/margins": 5.194509029388428,
+      "eval_rewards/rejected": -1.3270814418792725,
+      "eval_runtime": 16.9692,
+      "eval_samples_per_second": 1.65,
+      "eval_steps_per_second": 0.413,
+      "step": 90
+    },
+    {
+      "epoch": 3.225806451612903,
+      "grad_norm": 5.707949638366699,
+      "learning_rate": 2.4425287356321844e-06,
+      "logits/chosen": -1.8123356103897095,
+      "logits/rejected": -1.836766242980957,
+      "logps/chosen": -168.2237548828125,
+      "logps/rejected": -232.3643035888672,
+      "loss": 0.0175,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 4.8013739585876465,
+      "rewards/margins": 7.281676292419434,
+      "rewards/rejected": -2.4803037643432617,
+      "step": 100
+    },
+    {
+      "epoch": 3.5483870967741935,
+      "grad_norm": 3.4297404289245605,
+      "learning_rate": 2.1551724137931035e-06,
+      "logits/chosen": -1.8724552392959595,
+      "logits/rejected": -1.890865683555603,
+      "logps/chosen": -181.77548217773438,
+      "logps/rejected": -284.026611328125,
+      "loss": 0.0052,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 4.642284393310547,
+      "rewards/margins": 8.698179244995117,
+      "rewards/rejected": -4.0558953285217285,
+      "step": 110
+    },
+    {
+      "epoch": 3.870967741935484,
+      "grad_norm": 0.28286024928092957,
+      "learning_rate": 1.8678160919540231e-06,
+      "logits/chosen": -1.8464359045028687,
+      "logits/rejected": -1.8764251470565796,
+      "logps/chosen": -131.70431518554688,
+      "logps/rejected": -235.42318725585938,
+      "loss": 0.0118,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 4.332093238830566,
+      "rewards/margins": 7.9131927490234375,
+      "rewards/rejected": -3.581099271774292,
+      "step": 120
+    },
+    {
+      "epoch": 3.870967741935484,
+      "eval_logits/chosen": -1.8917609453201294,
+      "eval_logits/rejected": -1.8926684856414795,
+      "eval_logps/chosen": -120.6841049194336,
+      "eval_logps/rejected": -178.48434448242188,
+      "eval_loss": 0.029182463884353638,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 4.163033962249756,
+      "eval_rewards/margins": 6.384037494659424,
+      "eval_rewards/rejected": -2.221003770828247,
+      "eval_runtime": 16.9634,
+      "eval_samples_per_second": 1.651,
+      "eval_steps_per_second": 0.413,
+      "step": 120
+    },
+    {
+      "epoch": 4.193548387096774,
+      "grad_norm": 0.30069565773010254,
+      "learning_rate": 1.5804597701149427e-06,
+      "logits/chosen": -1.8492038249969482,
+      "logits/rejected": -1.8685414791107178,
+      "logps/chosen": -164.83790588378906,
+      "logps/rejected": -238.30447387695312,
+      "loss": 0.0108,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 5.031315326690674,
+      "rewards/margins": 8.202493667602539,
+      "rewards/rejected": -3.171177387237549,
+      "step": 130
+    },
+    {
+      "epoch": 4.516129032258064,
+      "grad_norm": 0.09980618953704834,
+      "learning_rate": 1.2931034482758623e-06,
+      "logits/chosen": -1.9228675365447998,
+      "logits/rejected": -1.9402902126312256,
+      "logps/chosen": -167.69627380371094,
+      "logps/rejected": -252.85574340820312,
+      "loss": 0.0283,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 3.7600669860839844,
+      "rewards/margins": 8.49266242980957,
+      "rewards/rejected": -4.7325944900512695,
+      "step": 140
+    },
+    {
+      "epoch": 4.838709677419355,
+      "grad_norm": 0.5666964054107666,
+      "learning_rate": 1.0057471264367817e-06,
+      "logits/chosen": -1.8569101095199585,
+      "logits/rejected": -1.872945785522461,
+      "logps/chosen": -123.6576156616211,
+      "logps/rejected": -158.0116424560547,
+      "loss": 0.0058,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 4.758939743041992,
+      "rewards/margins": 8.25415325164795,
+      "rewards/rejected": -3.4952139854431152,
+      "step": 150
+    },
+    {
+      "epoch": 4.838709677419355,
+      "eval_logits/chosen": -1.8856004476547241,
+      "eval_logits/rejected": -1.8862718343734741,
+      "eval_logps/chosen": -120.60621643066406,
+      "eval_logps/rejected": -180.44931030273438,
+      "eval_loss": 0.024126578122377396,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 4.186400890350342,
+      "eval_rewards/margins": 6.99689245223999,
+      "eval_rewards/rejected": -2.8104913234710693,
+      "eval_runtime": 16.9717,
+      "eval_samples_per_second": 1.65,
+      "eval_steps_per_second": 0.412,
+      "step": 150
+    },
+    {
+      "epoch": 5.161290322580645,
+      "grad_norm": 27.14858055114746,
+      "learning_rate": 7.183908045977011e-07,
+      "logits/chosen": -1.8459736108779907,
+      "logits/rejected": -1.8586574792861938,
+      "logps/chosen": -133.19772338867188,
+      "logps/rejected": -223.07730102539062,
+      "loss": 0.0116,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 4.50647497177124,
+      "rewards/margins": 8.688131332397461,
+      "rewards/rejected": -4.181654930114746,
+      "step": 160
+    },
+    {
+      "epoch": 5.483870967741936,
+      "grad_norm": 1.2454370260238647,
+      "learning_rate": 4.3103448275862073e-07,
+      "logits/chosen": -1.8669620752334595,
+      "logits/rejected": -1.8916997909545898,
+      "logps/chosen": -118.95484924316406,
+      "logps/rejected": -204.902099609375,
+      "loss": 0.0162,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 3.7229111194610596,
+      "rewards/margins": 8.270120620727539,
+      "rewards/rejected": -4.5472092628479,
+      "step": 170
+    },
+    {
+      "epoch": 5.806451612903226,
+      "grad_norm": 1.3749877214431763,
+      "learning_rate": 1.4367816091954023e-07,
+      "logits/chosen": -1.8054790496826172,
+      "logits/rejected": -1.8216861486434937,
+      "logps/chosen": -116.97957611083984,
+      "logps/rejected": -225.5679473876953,
+      "loss": 0.0058,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 4.90813684463501,
+      "rewards/margins": 9.645859718322754,
+      "rewards/rejected": -4.737723350524902,
+      "step": 180
+    },
+    {
+      "epoch": 5.806451612903226,
+      "eval_logits/chosen": -1.8866546154022217,
+      "eval_logits/rejected": -1.8871468305587769,
+      "eval_logps/chosen": -120.53617095947266,
+      "eval_logps/rejected": -180.97348022460938,
+      "eval_loss": 0.021960575133562088,
+      "eval_rewards/accuracies": 1.0,
+      "eval_rewards/chosen": 4.207414627075195,
+      "eval_rewards/margins": 7.175161838531494,
+      "eval_rewards/rejected": -2.967747211456299,
+      "eval_runtime": 16.9698,
+      "eval_samples_per_second": 1.65,
+      "eval_steps_per_second": 0.412,
+      "step": 180
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 180,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 6,
+  "save_steps": 90,
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-180/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d820e4ac8244824e226f350adbf946348a128d5fdf2ced2087647822264933d3
+size 5432