bert-base-german-uncased

This model is a fine-tuned version of dbmdz/bert-base-german-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5523
• F1: 0.4333

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-07
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	F1
0.6943	1.0	189	0.6839	0.08
0.6718	2.0	378	0.6603	0.0339
0.66	3.0	567	0.6508	0.0476
0.6568	4.0	756	0.6457	0.05
0.6638	5.0	945	0.6439	0.0513
0.6552	6.0	1134	0.6387	0.0556
0.6709	7.0	1323	0.6386	0.0556
0.6541	8.0	1512	0.6369	0.0556
0.6644	9.0	1701	0.6434	0.0
0.6601	10.0	1890	0.6432	0.0
0.6558	11.0	2079	0.6421	0.0
0.6548	12.0	2268	0.6398	0.0
0.6593	13.0	2457	0.6401	0.0
0.6502	14.0	2646	0.6366	0.0
0.6457	15.0	2835	0.6292	0.0541
0.6478	16.0	3024	0.6267	0.0541
0.6378	17.0	3213	0.6311	0.0
0.6325	18.0	3402	0.6279	0.0
0.6467	19.0	3591	0.6253	0.0
0.6338	20.0	3780	0.6227	0.0
0.6281	21.0	3969	0.6194	0.0
0.6205	22.0	4158	0.6155	0.0
0.6288	23.0	4347	0.6113	0.0
0.6178	24.0	4536	0.6081	0.0976
0.6221	25.0	4725	0.6049	0.1778
0.612	26.0	4914	0.6011	0.25
0.6052	27.0	5103	0.5971	0.25
0.5997	28.0	5292	0.5925	0.2553
0.603	29.0	5481	0.5895	0.3265
0.5973	30.0	5670	0.5856	0.3200
0.594	31.0	5859	0.5821	0.3200
0.5825	32.0	6048	0.5786	0.3529
0.5787	33.0	6237	0.5753	0.3929
0.5848	34.0	6426	0.5724	0.3929
0.5703	35.0	6615	0.5698	0.4000
0.584	36.0	6804	0.5673	0.3929
0.5668	37.0	6993	0.5648	0.3860
0.5624	38.0	7182	0.5629	0.3860
0.5793	39.0	7371	0.5611	0.4138
0.5555	40.0	7560	0.5593	0.3860
0.5664	41.0	7749	0.5579	0.4407
0.553	42.0	7938	0.5566	0.4483
0.5597	43.0	8127	0.5555	0.4483
0.5555	44.0	8316	0.5546	0.4407
0.5589	45.0	8505	0.5538	0.4333
0.5578	46.0	8694	0.5531	0.4333
0.5499	47.0	8883	0.5528	0.4333
0.5434	48.0	9072	0.5524	0.4333
0.5607	49.0	9261	0.5523	0.4333
0.5478	50.0	9450	0.5523	0.4333

Framework versions

• Transformers 4.32.1
• Pytorch 2.1.2
• Datasets 2.12.0
• Tokenizers 0.13.3