End of training

README.md

@@ -102,7 +102,7 @@ xformers_attention: null
 
 This model is a fine-tuned version of [aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct](https://huggingface.co/aisingapore/llama3-8b-cpt-sea-lionv2.1-instruct) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0435
+- Loss: 0.0519
 
 ## Model description
 
@@ -138,9 +138,9 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
 | 1.4927        | 0.0833 | 1    | 1.4978          |
-| 1.5735        | 0.25   | 3    | 1.4451          |
-| 1.0815        | 0.5    | 6    | 0.8162          |
-| 0.229         | 0.75   | 9    | 0.0435          |
+| 1.5735        | 0.25   | 3    | 1.4474          |
+| 1.0892        | 0.5    | 6    | 0.8217          |
+| 0.2385        | 0.75   | 9    | 0.0519          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,12 +20,12 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
     "o_proj",
-    "gate_proj",
+    "v_proj",
+    "q_proj",
     "down_proj",
     "k_proj",
-    "v_proj",
+    "gate_proj",
     "up_proj"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e29767d254322f76e4a7a492448794d0aed620a49b295ca886053e6cb71419a3
+oid sha256:72c2d9304942f834807d40db757cc852ec3bfa10c9ff76de8fc69bf16f624f17
 size 84047370

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7e7b3085613f5f73bc880d9364f4eef64991c08178bbf4fb9d8b05a2bd9a1fd9
+oid sha256:91f1a895b55a7ca6393f57cb37a757cb7adda2e6d6edadcdee65b54c9e99680e
 size 83945296

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:33ee77f9946e0ddfbac6f0c8ae58cf2871b3fdcb3ea1247868cf8374e044a7bd
+oid sha256:ab37ef2135f6555964c7f23e135ed3a58c3b21e0a8dfdabb28e1c2f6cc491bb9
 size 6776