art-bert-base-cased

This model is a fine-tuned version of bert-base-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.5202

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
6.6432	3.23	100	5.8478
5.651	6.45	200	5.5368
5.1511	9.68	300	5.2206
4.77	12.9	400	4.9162
4.449	16.13	500	4.8133
4.18	19.35	600	4.5716
3.9485	22.58	700	4.3972
3.6496	25.81	800	4.2725
3.4384	29.03	900	4.1514
3.2557	32.26	1000	4.1532
3.0924	35.48	1100	3.9699
2.8789	38.71	1200	3.8153
2.7001	41.94	1300	3.8936
2.5654	45.16	1400	3.8185
2.4543	48.39	1500	3.9040
2.2817	51.61	1600	3.7283
2.2239	54.84	1700	3.6337
2.1119	58.06	1800	3.7746
1.9952	61.29	1900	3.5909
1.9466	64.52	2000	3.5679
1.8244	67.74	2100	3.6370
1.7837	70.97	2200	3.6295
1.6972	74.19	2300	3.6373
1.6845	77.42	2400	3.4213
1.6453	80.65	2500	3.5497
1.5759	83.87	2600	3.5886
1.5506	87.1	2700	3.4016
1.5294	90.32	2800	3.3665
1.4915	93.55	2900	3.3038
1.5035	96.77	3000	3.3139
1.4601	100.0	3100	3.5202

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2