gbert-base-amdi-synset

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6415
• Accuracy: 0.8330
• F1: 0.6477
• Precision: 0.6550
• Recall: 0.6579

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 48
• eval_batch_size: 48
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
3.3175	0.4587	50	2.2214	0.5594	0.2286	0.2022	0.2872
1.5824	0.9174	100	1.1227	0.6867	0.3880	0.4003	0.4337
0.9358	1.3761	150	0.8457	0.7866	0.5421	0.5293	0.5807
0.7884	1.8349	200	0.7147	0.7762	0.5535	0.5538	0.5913
0.6245	2.2936	250	0.6656	0.8055	0.5663	0.5539	0.6033
0.5484	2.7523	300	0.6216	0.7986	0.5762	0.5789	0.6072
0.462	3.2110	350	0.5902	0.8227	0.6267	0.6206	0.6518
0.4089	3.6697	400	0.6369	0.8072	0.5902	0.5842	0.6126
0.368	4.1284	450	0.6189	0.8158	0.6296	0.6384	0.6613
0.3232	4.5872	500	0.6415	0.8330	0.6477	0.6550	0.6579
0.2836	5.0459	550	0.6373	0.8124	0.6341	0.6491	0.6609
0.2212	5.5046	600	0.6843	0.8090	0.6315	0.6471	0.6501
0.2228	5.9633	650	0.5933	0.8365	0.6625	0.6898	0.6686
0.1838	6.4220	700	0.6382	0.8313	0.6452	0.6472	0.6626
0.1527	6.8807	750	0.6471	0.8330	0.6601	0.6751	0.6772
0.1393	7.3394	800	0.6751	0.8227	0.6279	0.6339	0.6434
0.1082	7.7982	850	0.6689	0.8382	0.6608	0.6836	0.6772
0.0812	8.2569	900	0.7124	0.8296	0.6670	0.6785	0.6802
0.0836	8.7156	950	0.7201	0.8244	0.6446	0.6597	0.6574
0.0816	9.1743	1000	0.7253	0.8296	0.6478	0.6722	0.6567
0.0645	9.6330	1050	0.7236	0.8262	0.6425	0.6655	0.6521

Framework versions

• Transformers 4.45.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.20.3