End of training

README.md

@@ -12,12 +12,12 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/timmyafolami/huggingface/runs/w9qoroly)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/timmyafolami/huggingface/runs/dza6a6tk)
 # peft_phi_2
 
-This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on the None dataset.
+This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 8.4189
+- Loss: nan
 
 ## Model description
 
@@ -49,9 +49,9 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 8.0528        | 1.0   | 263  | 7.7273          |
-| 8.4081        | 2.0   | 526  | 8.4002          |
-| 8.4519        | 3.0   | 789  | 8.4189          |
+| 6.4514        | 1.0   | 287  | 6.5988          |
+| 0.0           | 2.0   | 574  | nan             |
+| 0.0           | 3.0   | 861  | nan             |
 
 
 ### Framework versions

adapter_config.json

@@ -20,9 +20,9 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "dense",
     "v_proj",
+    "dense",
+    "q_proj",
     "k_proj"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:116d780c8804bff796c19aa92508c944234221658d7a27a2cd2b2effe68f775c
+oid sha256:c16b7a7c84a6fc270c5f0d40f161714d6ef9544de4d276bb275a757581b613aa
 size 41977360

runs/Jul23_13-33-06_7e6c8afec7bd/events.out.tfevents.1721741592.7e6c8afec7bd.34.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e18b801f33f032f2f9623697f091b1ae56de1303b4ed354d8bc8ba39092db4e6
+size 15629

tokenizer.json

@@ -2,14 +2,12 @@
   "version": "1.0",
   "truncation": {
     "direction": "Right",
-    "max_length": 512,
+    "max_length": 2048,
     "strategy": "LongestFirst",
     "stride": 0
   },
   "padding": {
-    "strategy": {
-      "Fixed": 512
-    },
+    "strategy": "BatchLongest",
     "direction": "Right",
     "pad_to_multiple_of": null,
     "pad_id": 50256,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:33e4fc51715dbe01009bc97ab28fbddd62e79c1d37adc8ca8d21de7eab7b1b4f
+oid sha256:d3bcfe8268a59ca4611ee3edcb06a65f236d368c25248ed1e70d96028e4c3608
 size 5176