google_gemma-2-2b_HelpSteer2_sftsd1

This model is a fine-tuned version of google/gemma-2-2b on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0372

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 8e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 1
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0	0	1.3575
1.0884	0.0835	100	1.0875
1.0994	0.1669	200	1.0655
1.1144	0.2504	300	1.0564
1.0645	0.3339	400	1.0496
1.0857	0.4174	500	1.0451
1.0323	0.5008	600	1.0420
1.0663	0.5843	700	1.0401
1.062	0.6678	800	1.0388
1.0915	0.7513	900	1.0379
1.0923	0.8347	1000	1.0373
1.072	0.9182	1100	1.0373

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1