berel_finetuned_on_HB_2_epochs_general

This model is a fine-tuned version of dicta-il/BEREL on the None dataset. It achieves the following results on the evaluation set:

• Loss: 8.7244

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
7.9561	0.0568	500	8.7150
8.7343	0.1136	1000	8.7229
8.6403	0.1703	1500	8.6905
8.676	0.2271	2000	8.6748
8.6253	0.2839	2500	8.6821
8.7109	0.3407	3000	8.6823
8.6646	0.3975	3500	8.6498
8.6045	0.4542	4000	8.6458
8.6744	0.5110	4500	8.6375
8.6595	0.5678	5000	8.6310
8.6076	0.6246	5500	8.5505
8.6384	0.6814	6000	8.5585
9.636	0.7381	6500	36.3585
12.7509	0.7949	7000	9.5749
9.3313	0.8517	7500	9.2572
9.2503	0.9085	8000	9.1970
9.0888	0.9653	8500	nan
9.109	1.0220	9000	9.0959
9.0279	1.0788	9500	9.0480
8.918	1.1356	10000	9.0161
8.9248	1.1924	10500	nan
8.9299	1.2491	11000	8.9685
8.9322	1.3059	11500	8.9354
8.8491	1.3627	12000	8.9048
8.8492	1.4195	12500	8.8784
8.8958	1.4763	13000	8.8760
8.8851	1.5330	13500	8.8440
8.7776	1.5898	14000	8.8357
8.7646	1.6466	14500	8.8083
8.7689	1.7034	15000	8.8256
8.7576	1.7602	15500	8.8005
8.7357	1.8169	16000	8.7886
8.755	1.8737	16500	8.7802
8.6999	1.9305	17000	8.7594
8.6954	1.9873	17500	8.7244

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0