End of training

README.md

@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 7.7076
+- Loss: 1.5101
 
 ## Model description
 
@@ -43,15 +43,16 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 12
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 3
+- num_epochs: 4
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 8.9719        | 0.9971 | 115  | 8.4008          |
-| 8.1061        | 1.9942 | 230  | 7.8448          |
-| 7.6961        | 2.9913 | 345  | 7.7076          |
+| Training Loss | Epoch  | Step  | Validation Loss |
+|:-------------:|:------:|:-----:|:---------------:|
+| 1.5134        | 0.9998 | 4651  | 1.5228          |
+| 1.402         | 1.9999 | 9303  | 1.5084          |
+| 1.2921        | 2.9999 | 13955 | 1.5076          |
+| 1.2776        | 3.9993 | 18604 | 1.5101          |
 
 
 ### Framework versions

adapter_config.json

@@ -23,9 +23,9 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "layers.0.self_attn.q_proj",
-    "layers.0.self_attn.v_proj",
-    "layers.0.self_attn.k_proj"
+    "dense",
+    "q_proj",
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:335431f0fb932a59e74de44a3ad8dd3e6a865c3e1ea6861cfed73b0ccf1a1a70
-size 1050748064
+oid sha256:3b1c8387dfa0d70a92f050462502afa32e89b614865ec5d44d7bec45d0f6a64f
+size 1111722280

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d03002142727fdec843e77baeab8b5ab068b477af88093783e8c0e0718e8d815
+oid sha256:934e7d7d3924eb1fa1951cd5ca644c6a8f9c05113af133a84b2f90057153ec97
 size 4984