CBT-Gemma2-2B-Frac

This model is a fine-tuned version of google/gemma-2-2b-it on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8412

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 4
• eval_batch_size: 4
• seed: 3407
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.953	0.0572	100	0.9531
0.9325	0.1145	200	0.9239
0.9062	0.1717	300	0.9123
0.8994	0.2290	400	0.9025
0.9128	0.2862	500	0.8948
0.8969	0.3435	600	0.8887
0.8957	0.4007	700	0.8844
0.8665	0.4580	800	0.8764
0.8827	0.5152	900	0.8746
0.8395	0.5725	1000	0.8692
0.8978	0.6297	1100	0.8631
0.8497	0.6870	1200	0.8565
0.8722	0.7442	1300	0.8535
0.827	0.8015	1400	0.8495
0.8251	0.8587	1500	0.8434
0.8179	0.9160	1600	0.8436
0.8188	0.9732	1700	0.8361
0.7207	1.0305	1800	0.8458
0.6991	1.0877	1900	0.8428
0.7176	1.1450	2000	0.8412

Framework versions

• PEFT 0.13.2
• Transformers 4.45.2
• Pytorch 2.6.0+cu124
• Datasets 3.0.1
• Tokenizers 0.20.3