metadata
license: apache-2.0
language:
- en
pipeline_tag: text-generation
Sharathhebbar24/ssh_1.8B is a 1.8B model
The model is a modified version of qnguyen3/quan-1.8b-chat
Training hyperparameters
The following hyperparameters were used during training:
learning_rate: 2e-05 train_batch_size: 2 eval_batch_size: 2 seed: 42 distributed_type: multi-GPU num_devices: 4 gradient_accumulation_steps: 4 total_train_batch_size: 32 total_eval_batch_size: 8 optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 lr_scheduler_type: cosine lr_scheduler_warmup_steps: 100 num_epochs: 4