sanjeevl/llama38binstruct-summary-100s

README.md

@@ -1,13 +1,13 @@
 ---
-license: other
+base_model: NousResearch/Meta-Llama-3-8B-Instruct
+datasets:
+- generator
 library_name: peft
+license: other
 tags:
 - trl
 - sft
 - generated_from_trainer
-base_model: NousResearch/Meta-Llama-3-8B-Instruct
-datasets:
-- generator
 model-index:
 - name: llama38binstruct_summarize
   results: []
@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [NousResearch/Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.3460
+- Loss: 1.7929
 
 ## Model description
 
@@ -45,23 +45,23 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
-- lr_scheduler_warmup_steps: 0.03
+- lr_scheduler_warmup_steps: 3
 - training_steps: 100
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 1.5536        | 1.3889 | 25   | 1.5880          |
-| 0.6175        | 2.7778 | 50   | 1.8927          |
-| 0.2685        | 4.1667 | 75   | 1.9919          |
-| 0.0612        | 5.5556 | 100  | 2.3460          |
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 1.5444        | 1.25  | 25   | 1.4502          |
+| 0.4769        | 2.5   | 50   | 1.7111          |
+| 0.2309        | 3.75  | 75   | 1.7707          |
+| 0.13          | 5.0   | 100  | 1.7929          |
 
 
 ### Framework versions
 
 - PEFT 0.11.1
-- Transformers 4.41.2
+- Transformers 4.42.4
 - Pytorch 2.3.0+cu121
 - Datasets 2.20.0
 - Tokenizers 0.19.1

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "up_proj",
+    "k_proj",
     "down_proj",
+    "q_proj",
     "gate_proj",
-    "o_proj",
-    "k_proj",
     "v_proj",
-    "q_proj",
-    "up_proj"
+    "o_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:003eaf659e67572b4002048d7feb424158449e22446c197ba8fd6a0c2ac55fe0
+oid sha256:b6a8648d4b7e7bde406268bf26bfdd731a0862b15b596f49f6ec819664225192
 size 167832240

runs/Jul16_08-37-47_86f5e2bdd820/events.out.tfevents.1721119418.86f5e2bdd820.283.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42a944756f5097ad7d46ba92fb1f405fa895f75e407b608d831393edc853348f
+size 9261

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:69425bbb0ceb71772f12bd470389d94e1a482f642810773f1a86b80c2412f46c
-size 5368
+oid sha256:c3d8d6456dbc62bf599621e8ac7136d792623beddb1a96f2197897c607abcba1
+size 5432