End of training

README.md

@@ -65,7 +65,7 @@ lora_model_dir: null
 lora_r: 8
 lora_target_linear: true
 lr_scheduler: cosine
-max_steps: 1
+max_steps: 50
 micro_batch_size: 2
 mlflow_experiment_name: /tmp/468123ffe07d0b34_train_data.json
 model_type: AutoModelForCausalLM
@@ -92,7 +92,7 @@ wandb_name: ed95d5b7-232d-47e8-8e58-64e8ef4d9bf9
 wandb_project: Gradients-On-Demand
 wandb_run: your_name
 wandb_runid: ed95d5b7-232d-47e8-8e58-64e8ef4d9bf9
-warmup_steps: 1
+warmup_steps: 10
 weight_decay: 0.0
 xformers_attention: null
 
@@ -103,6 +103,8 @@ xformers_attention: null
 # f07bb753-7195-aad3-6da6-d76624379f51
 
 This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 2.8098
 
 ## Model description
 
@@ -129,14 +131,19 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 8
 - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 2
-- training_steps: 1
+- lr_scheduler_warmup_steps: 10
+- training_steps: 50
 
 ### Training results
 
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
 | No log        | 0.0000 | 1    | 2.9149          |
+| 2.9402        | 0.0003 | 10   | 2.9024          |
+| 2.9541        | 0.0006 | 20   | 2.8376          |
+| 2.9321        | 0.0009 | 30   | 2.8198          |
+| 2.6967        | 0.0012 | 40   | 2.8110          |
+| 2.8701        | 0.0016 | 50   | 2.8098          |
 
 
 ### Framework versions

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:80fc3124efdd1b9b2381f84f06b232d24d4766336e1ab6de8139672688e5cfe9
+oid sha256:5c2d6bfa9010b5996d2e9b5de1719c49d4c11b362934a9a5e2b6be01766c007b
 size 28414730