database_fixer_model

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0967
• Precision: 0.9054
• Recall: 0.9252
• F1: 0.9152
• Accuracy: 0.9478

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.8128	0.0801	100	0.5230	0.6682	0.7578	0.7102	0.8171
0.4072	0.1603	200	0.3203	0.7639	0.8298	0.7955	0.8804
0.2896	0.2404	300	0.2379	0.8258	0.8717	0.8481	0.9091
0.2443	0.3205	400	0.1952	0.8473	0.8862	0.8663	0.9197
0.2183	0.4006	500	0.1762	0.8629	0.8939	0.8781	0.9251
0.1944	0.4808	600	0.1567	0.8691	0.8999	0.8842	0.9299
0.1648	0.5609	700	0.1480	0.8729	0.9050	0.8887	0.9329
0.1658	0.6410	800	0.1451	0.8780	0.9046	0.8911	0.9332
0.1586	0.7212	900	0.1317	0.8822	0.9089	0.8953	0.9373
0.1353	0.8013	1000	0.1285	0.8920	0.9138	0.9028	0.9378
0.1435	0.8814	1100	0.1247	0.8870	0.9117	0.8992	0.9394
0.1434	0.9615	1200	0.1185	0.8896	0.9152	0.9022	0.9403
0.1132	1.0417	1300	0.1179	0.8878	0.9142	0.9008	0.9405
0.1052	1.1218	1400	0.1139	0.8895	0.9165	0.9028	0.9425
0.1163	1.2019	1500	0.1133	0.8894	0.9166	0.9028	0.9425
0.1095	1.2821	1600	0.1125	0.8959	0.9195	0.9076	0.9429
0.1115	1.3622	1700	0.1099	0.8961	0.9194	0.9076	0.9435
0.1045	1.4423	1800	0.1142	0.8793	0.9136	0.8961	0.9425
0.1135	1.5224	1900	0.1042	0.8920	0.9181	0.9049	0.9443
0.1077	1.6026	2000	0.1051	0.8977	0.9203	0.9089	0.9446
0.0945	1.6827	2100	0.1056	0.8975	0.9206	0.9089	0.9452
0.103	1.7628	2200	0.1069	0.8970	0.9194	0.9081	0.9437
0.103	1.8429	2300	0.1019	0.9011	0.9218	0.9113	0.9452
0.1043	1.9231	2400	0.0990	0.8998	0.9222	0.9109	0.9460
0.0984	2.0032	2500	0.0980	0.8985	0.9211	0.9097	0.9460
0.0881	2.0833	2600	0.1004	0.9147	0.9271	0.9208	0.9468
0.0828	2.1635	2700	0.1015	0.9172	0.9290	0.9231	0.9471
0.0851	2.2436	2800	0.0997	0.8850	0.9169	0.9007	0.9470
0.09	2.3237	2900	0.1030	0.9021	0.9236	0.9127	0.9462
0.0853	2.4038	3000	0.0969	0.9056	0.9244	0.9149	0.9474
0.0836	2.4840	3100	0.0967	0.8948	0.9221	0.9082	0.9480
0.0805	2.5641	3200	0.0975	0.8959	0.9211	0.9083	0.9478
0.086	2.6442	3300	0.0971	0.9076	0.9264	0.9169	0.9485
0.0884	2.7244	3400	0.0978	0.9131	0.9289	0.9209	0.9477
0.0821	2.8045	3500	0.0984	0.9059	0.9264	0.9160	0.9483
0.0841	2.8846	3600	0.0980	0.9086	0.9275	0.9180	0.9489
0.0808	2.9647	3700	0.0967	0.9054	0.9252	0.9152	0.9478

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1