Model save

README.md

@@ -19,7 +19,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [ytcheng/llama-3-8b-hf-sm-lora-merged](https://huggingface.co/ytcheng/llama-3-8b-hf-sm-lora-merged) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.4204
+- Loss: 3.0057
 
 ## Model description
 
@@ -38,7 +38,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 3e-05
+- learning_rate: 0.0001
 - train_batch_size: 32
 - eval_batch_size: 32
 - seed: 42
@@ -47,21 +47,21 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_ratio: 0.1
+- lr_scheduler_warmup_ratio: 0.3
 - num_epochs: 8
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 3.9305        | 0.9855 | 34   | 3.4654          |
-| 2.8968        | 2.0    | 69   | 2.6918          |
-| 2.5375        | 2.9855 | 103  | 2.5330          |
-| 2.4472        | 4.0    | 138  | 2.4676          |
-| 2.3782        | 4.9855 | 172  | 2.4321          |
-| 2.3757        | 6.0    | 207  | 2.4202          |
-| 2.3563        | 6.9855 | 241  | 2.4204          |
-| 2.3775        | 7.8841 | 272  | 2.4204          |
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 3.6712        | 1.0   | 33   | 3.2633          |
+| 2.6495        | 2.0   | 66   | 2.5112          |
+| 2.3212        | 3.0   | 99   | 2.3937          |
+| 2.0921        | 4.0   | 132  | 2.4587          |
+| 1.9862        | 5.0   | 165  | 2.8611          |
+| 1.9494        | 6.0   | 198  | 2.8478          |
+| 1.9216        | 7.0   | 231  | 3.0062          |
+| 1.9042        | 8.0   | 264  | 3.0057          |
 
 
 ### Framework versions