End of training

README.md

@@ -15,9 +15,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.7718
-- Rouge Scores: {'rouge1': 91.95734764873546, 'rouge2': 82.32931400113223, 'rougeL': 85.99741932162073, 'rougeLsum': 91.94841950416424}
-- Bleu Scores: [0.9416174113277822, 0.927443422067012, 0.9093890905268396, 0.8902085653786349]
+- Loss: 0.7508
+- Rouge Scores: {'rouge1': 92.07934134490793, 'rouge2': 82.68216875143521, 'rougeL': 86.3418556340243, 'rougeLsum': 92.07165318568765}
+- Bleu Scores: [0.9430535279899742, 0.9289517504059885, 0.9111307023404618, 0.8922236591496603]
 - Gen Len: 2048.0
 
 ## Model description
@@ -50,10 +50,10 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Rouge Scores                                                                                                            | Bleu Scores                                                                      | Gen Len |
-|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------:|:-------:|
-| 0.8676        | 1.0   | 500  | 0.7965          | {'rouge1': 91.95351666069791, 'rouge2': 82.14665839576412, 'rougeL': 85.51462558903678, 'rougeLsum': 91.94448847318316} | [0.9428007493469047, 0.9283594260300487, 0.9099607650770003, 0.8903936529899328] | 2048.0  |
-| 0.6641        | 2.0   | 1000 | 0.7718          | {'rouge1': 91.95734764873546, 'rouge2': 82.32931400113223, 'rougeL': 85.99741932162073, 'rougeLsum': 91.94841950416424} | [0.9416174113277822, 0.927443422067012, 0.9093890905268396, 0.8902085653786349]  | 2048.0  |
+| Training Loss | Epoch | Step | Validation Loss | Rouge Scores                                                                                                           | Bleu Scores                                                                      | Gen Len |
+|:-------------:|:-----:|:----:|:---------------:|:----------------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------:|:-------:|
+| 0.8283        | 1.0   | 500  | 0.7644          | {'rouge1': 92.1717672607307, 'rouge2': 82.69254584175559, 'rougeL': 86.17480706939217, 'rougeLsum': 92.16499826848323} | [0.9428124451183093, 0.9288838577090098, 0.910999858543974, 0.8919623155075178]  | 2048.0  |
+| 0.5824        | 2.0   | 1000 | 0.7508          | {'rouge1': 92.07934134490793, 'rouge2': 82.68216875143521, 'rougeL': 86.3418556340243, 'rougeLsum': 92.07165318568765} | [0.9430535279899742, 0.9289517504059885, 0.9111307023404618, 0.8922236591496603] | 2048.0  |
 
 
 ### Framework versions

adapter_config.json

@@ -1,7 +1,7 @@
 {
   "alpha_pattern": {},
   "auto_mapping": null,
-  "base_model_name_or_path": null,
+  "base_model_name_or_path": "NousResearch/Llama-2-7b-hf",
   "bias": "none",
   "fan_in_fan_out": false,
   "inference_mode": true,
@@ -19,13 +19,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "w3",
-    "w2",
+    "up_proj",
     "q_proj",
-    "v_proj",
-    "w1",
+    "down_proj",
+    "k_proj",
+    "gate_proj",
     "o_proj",
-    "k_proj"
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_rslora": false

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:66ac4991e17da2f72047bf892ee8e69047adb02130b0f6efbeeb68b67d0b8899
-size 268474624
+oid sha256:5c669d10f2fd244dcd1703cc32869159b94974447418065516ba40e803d2fd2f
+size 639691872

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6e1a2508d3f5dfbb6e23296c3fea3922d1e40c2d1ea100010ef3e84ecf3c9326
+oid sha256:b09d3566ebec10da38d28d0274d38c8963f912f9859271df03d6b9a80e6faec3
 size 4792