Model save

README.md

@@ -2,13 +2,12 @@
 license: apache-2.0
 library_name: peft
 tags:
-- alignment-handbook
 - trl
 - sft
 - generated_from_trainer
 base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
 datasets:
-- martimfasantos/openai-tldr-filtered
+- generator
 model-index:
 - name: tinyllama-1.1b-sum-sft-qlora
   results: []
@@ -19,9 +18,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # tinyllama-1.1b-sum-sft-qlora
 
-This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T) on the martimfasantos/openai-tldr-filtered dataset.
+This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.1466
+- Loss: 2.1440
 
 ## Model description
 
@@ -41,12 +40,12 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0004
-- train_batch_size: 32
+- train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
 - distributed_type: multi-GPU
 - gradient_accumulation_steps: 2
-- total_train_batch_size: 64
+- total_train_batch_size: 16
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_ratio: 0.1
@@ -54,10 +53,10 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| 2.1383        | 1.0   | 1351 | 2.1541          |
-| 2.1135        | 2.0   | 2702 | 2.1466          |
+| Training Loss | Epoch | Step  | Validation Loss |
+|:-------------:|:-----:|:-----:|:---------------:|
+| 2.1266        | 1.0   | 5403  | 2.1504          |
+| 2.1084        | 2.0   | 10806 | 2.1440          |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7e75c709e55a2a8b921f0828311723d61ed1dac6d9fc5399bf37abfa67010f06
+oid sha256:1c3e1bd0ce214d89dc7a8377b45c80d2c2f444218aa1f9434e54e32dbd47cc50
 size 25272360

all_results.json

@@ -1,13 +1,8 @@
 {
     "epoch": 2.0,
-    "eval_loss": 2.14662766456604,
-    "eval_runtime": 188.7756,
-    "eval_samples": 6553,
-    "eval_samples_per_second": 25.761,
-    "eval_steps_per_second": 3.221,
-    "train_loss": 2.1601854102158,
-    "train_runtime": 21984.2259,
+    "train_loss": 2.1470370752908954,
+    "train_runtime": 23954.8129,
     "train_samples": 116722,
-    "train_samples_per_second": 7.864,
-    "train_steps_per_second": 0.123
+    "train_samples_per_second": 7.217,
+    "train_steps_per_second": 0.451
 }

runs/May03_14-37-15_poseidon/events.out.tfevents.1714747047.poseidon.4188924.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b93f4275e11bb1f20cf1058dc6360f233b9b0a3b8b3cc6e973eb99a30544b435
-size 461421
+oid sha256:7ca2de38dbdedb01f6601557121ce68fb502aba80f771615b6d70a1882848f6c
+size 462257

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 2.0,
-    "train_loss": 2.1601854102158,
-    "train_runtime": 21984.2259,
+    "train_loss": 2.1470370752908954,
+    "train_runtime": 23954.8129,
     "train_samples": 116722,
-    "train_samples_per_second": 7.864,
-    "train_steps_per_second": 0.123
+    "train_samples_per_second": 7.217,
+    "train_steps_per_second": 0.451
 }