DistilBERT-TC2000-10epochs

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0752
• Recall: {'recall': 0.98}
• Precision: {'precision': 0.9803145941921073}
• F1: {'f1': 0.9800242537313432}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Recall	Precision	F1
1.0272	0.18	20	0.8815	{'recall': 0.65}	{'precision': 0.7778791777580597}	{'f1': 0.6251215862860073}
0.8663	0.35	40	0.6770	{'recall': 0.905}	{'precision': 0.9120308312976535}	{'f1': 0.9054010850819201}
0.6016	0.53	60	0.4088	{'recall': 0.92}	{'precision': 0.9238949736347314}	{'f1': 0.9207242314918276}
0.3139	0.71	80	0.2508	{'recall': 0.93}	{'precision': 0.9322386382325532}	{'f1': 0.929768888773222}
0.2645	0.88	100	0.2048	{'recall': 0.955}	{'precision': 0.958280303030303}	{'f1': 0.954923196771023}
0.1811	1.06	120	0.1446	{'recall': 0.965}	{'precision': 0.9675925925925927}	{'f1': 0.9648852158183796}
0.1429	1.24	140	0.1245	{'recall': 0.975}	{'precision': 0.9762354497354496}	{'f1': 0.9749193929610656}
0.0941	1.42	160	0.1338	{'recall': 0.965}	{'precision': 0.9683561643835616}	{'f1': 0.9652805623632961}
0.1242	1.59	180	0.0872	{'recall': 0.975}	{'precision': 0.9759505494505496}	{'f1': 0.9750344590666455}
0.0893	1.77	200	0.0572	{'recall': 0.985}	{'precision': 0.9853867102396515}	{'f1': 0.9849564819176908}
0.0477	1.95	220	0.0794	{'recall': 0.975}	{'precision': 0.9762354497354496}	{'f1': 0.9749193929610656}
0.0128	2.12	240	0.0697	{'recall': 0.98}	{'precision': 0.9807447665056361}	{'f1': 0.9799368665956859}
0.0449	2.3	260	0.0635	{'recall': 0.97}	{'precision': 0.9725}	{'f1': 0.9702302752172594}
0.0996	2.48	280	0.0782	{'recall': 0.97}	{'precision': 0.9725}	{'f1': 0.9700752508361203}
0.0328	2.65	300	0.0127	{'recall': 0.995}	{'precision': 0.995060975609756}	{'f1': 0.9949962534538471}
0.0747	2.83	320	0.0380	{'recall': 0.975}	{'precision': 0.9767605633802816}	{'f1': 0.9751792302987906}
0.0413	3.01	340	0.0127	{'recall': 1.0}	{'precision': 1.0}	{'f1': 1.0}
0.0404	3.19	360	0.0120	{'recall': 0.995}	{'precision': 0.995060975609756}	{'f1': 0.9949915278995033}
0.0226	3.36	380	0.0085	{'recall': 1.0}	{'precision': 1.0}	{'f1': 1.0}
0.0543	3.54	400	0.0139	{'recall': 0.995}	{'precision': 0.9950925925925926}	{'f1': 0.9950042805165157}
0.0528	3.72	420	0.0408	{'recall': 0.985}	{'precision': 0.9856521739130435}	{'f1': 0.9850251572327045}
0.0051	3.89	440	0.0808	{'recall': 0.97}	{'precision': 0.9725}	{'f1': 0.9702302752172594}
0.014	4.07	460	0.0419	{'recall': 0.985}	{'precision': 0.985241846323936}	{'f1': 0.985017255463425}
0.051	4.25	480	0.0127	{'recall': 0.995}	{'precision': 0.9950925925925926}	{'f1': 0.9950042805165157}
0.0501	4.42	500	0.0200	{'recall': 0.985}	{'precision': 0.9850867537313434}	{'f1': 0.985009807126512}
0.0062	4.6	520	0.0247	{'recall': 0.985}	{'precision': 0.985241846323936}	{'f1': 0.985017255463425}
0.0118	4.78	540	0.0614	{'recall': 0.975}	{'precision': 0.975962157809984}	{'f1': 0.975047977706797}
0.0348	4.96	560	0.0516	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0226	5.13	580	0.0144	{'recall': 0.995}	{'precision': 0.995060975609756}	{'f1': 0.9949962534538471}
0.0159	5.31	600	0.0129	{'recall': 0.995}	{'precision': 0.995060975609756}	{'f1': 0.9949962534538471}
0.0026	5.49	620	0.0176	{'recall': 0.995}	{'precision': 0.995060975609756}	{'f1': 0.9949962534538471}
0.016	5.66	640	0.0404	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0433	5.84	660	0.0663	{'recall': 0.975}	{'precision': 0.9756772575250836}	{'f1': 0.975041928721174}
0.0354	6.02	680	0.0253	{'recall': 0.995}	{'precision': 0.995060975609756}	{'f1': 0.9949962534538471}
0.0041	6.19	700	0.0961	{'recall': 0.97}	{'precision': 0.9711688311688311}	{'f1': 0.9700614296351452}
0.0579	6.37	720	0.1336	{'recall': 0.965}	{'precision': 0.966783728687917}	{'f1': 0.9650813612906225}
0.0025	6.55	740	0.0424	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0328	6.73	760	0.0190	{'recall': 0.995}	{'precision': 0.995060975609756}	{'f1': 0.9949962534538471}
0.0217	6.9	780	0.0488	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0096	7.08	800	0.1115	{'recall': 0.97}	{'precision': 0.9711688311688311}	{'f1': 0.9700614296351452}
0.0106	7.26	820	0.0673	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0077	7.43	840	0.0354	{'recall': 0.985}	{'precision': 0.9850867537313434}	{'f1': 0.985009807126512}
0.0222	7.61	860	0.0410	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0026	7.79	880	0.0590	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0576	7.96	900	0.0596	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.018	8.14	920	0.0428	{'recall': 0.985}	{'precision': 0.9850867537313434}	{'f1': 0.985009807126512}
0.027	8.32	940	0.0425	{'recall': 0.985}	{'precision': 0.9850867537313434}	{'f1': 0.985009807126512}
0.036	8.5	960	0.0341	{'recall': 0.985}	{'precision': 0.9850867537313434}	{'f1': 0.985009807126512}
0.0094	8.67	980	0.0457	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0192	8.85	1000	0.0586	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.03	9.03	1020	0.0789	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0091	9.2	1040	0.0691	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0197	9.38	1060	0.0753	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0025	9.56	1080	0.0796	{'recall': 0.975}	{'precision': 0.9756772575250836}	{'f1': 0.975041928721174}
0.0414	9.73	1100	0.0791	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}
0.0075	9.91	1120	0.0756	{'recall': 0.98}	{'precision': 0.9803145941921073}	{'f1': 0.9800242537313432}

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.1
• Tokenizers 0.13.3