wisdominanutshell
/

corrector_llama3

Generated from Trainer

4-bit precision

Model card Files Files and versions Community

iadithyan commited on 30 days ago

Commit

c8a07f0

•

1 Parent(s): 68afca9

End of training

Files changed (2) hide show

README.md +7 -9
adapter_model.bin +1 -1

README.md CHANGED Viewed

@@ -53,9 +53,9 @@ wandb_watch:
 wandb_name:
 wandb_log_model:
-gradient_accumulation_steps: 4
-micro_batch_size: 2
-num_epochs: 4
 optimizer: paged_adamw_32bit
 lr_scheduler: cosine
 learning_rate: 0.0002
@@ -75,7 +75,7 @@ xformers_attention:
 flash_attention: true
 warmup_steps: 10
-evals_per_epoch: 4
 eval_table_size:
 saves_per_epoch: 1
 debug:
@@ -114,15 +114,13 @@ More information needed
 The following hyperparameters were used during training:
 - learning_rate: 0.0002
-- train_batch_size: 2
-- eval_batch_size: 2
 - seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 8
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 10
-- num_epochs: 4
 ### Training results

 wandb_name:
 wandb_log_model:
+gradient_accumulation_steps: 1
+micro_batch_size: 1
+num_epochs: 20
 optimizer: paged_adamw_32bit
 lr_scheduler: cosine
 learning_rate: 0.0002
 flash_attention: true
 warmup_steps: 10
+evals_per_epoch: 1
 eval_table_size:
 saves_per_epoch: 1
 debug:
 The following hyperparameters were used during training:
 - learning_rate: 0.0002
+- train_batch_size: 1
+- eval_batch_size: 1
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 10
+- num_epochs: 20
 ### Training results

adapter_model.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bb5145f660beecc57ad98f3a51f272da10d0eeb894ac76ccadea4b6c7bd321d4
 size 167934026

 version https://git-lfs.github.com/spec/v1
+oid sha256:98220f15e89bc42672ebb4f8ac879067b9d5a0b33c83b4785774616b1cb336bf
 size 167934026