distilbert-base-uncased-finetuned-pos-kk-3080

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.6344
• Precision: 0.6506
• Recall: 0.6342
• F1: 0.6423
• Accuracy: 0.7199

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	59	2.3411	0.6246	0.6067	0.6155	0.7026
No log	2.0	118	2.1636	0.6382	0.625	0.6315	0.7093
No log	3.0	177	2.2138	0.6121	0.6044	0.6082	0.6920
No log	4.0	236	2.3509	0.6271	0.6307	0.6289	0.7045
No log	5.0	295	2.2692	0.6408	0.6239	0.6322	0.7113
No log	6.0	354	2.1764	0.6408	0.6261	0.6334	0.7103
No log	7.0	413	2.2618	0.6404	0.6147	0.6273	0.7093
No log	8.0	472	2.3162	0.6423	0.6239	0.6329	0.7122
0.0115	9.0	531	2.3318	0.6338	0.6193	0.6265	0.7093
0.0115	10.0	590	2.3152	0.6316	0.6273	0.6295	0.7036
0.0115	11.0	649	2.2334	0.6307	0.6170	0.6238	0.7026
0.0115	12.0	708	2.2705	0.6446	0.6261	0.6353	0.7161
0.0115	13.0	767	2.3049	0.6313	0.6204	0.6258	0.7055
0.0115	14.0	826	2.2713	0.6391	0.6193	0.6290	0.7151
0.0115	15.0	885	2.2914	0.6350	0.6204	0.6276	0.7122
0.0115	16.0	944	2.1958	0.6286	0.6193	0.6239	0.7055
0.0078	17.0	1003	2.3721	0.6200	0.6193	0.6196	0.7026
0.0078	18.0	1062	2.2756	0.6544	0.6319	0.6429	0.7238
0.0078	19.0	1121	2.3066	0.6585	0.6433	0.6508	0.7276
0.0078	20.0	1180	2.3065	0.6489	0.6273	0.6379	0.7141
0.0078	21.0	1239	2.3865	0.6314	0.6365	0.6339	0.7122
0.0078	22.0	1298	2.4231	0.6263	0.6284	0.6274	0.7084
0.0078	23.0	1357	2.3871	0.6247	0.6319	0.6283	0.7084
0.0078	24.0	1416	2.4611	0.6390	0.6273	0.6331	0.7170
0.0078	25.0	1475	2.3563	0.6345	0.6193	0.6268	0.7045
0.0043	26.0	1534	2.4208	0.6424	0.6261	0.6341	0.7190
0.0043	27.0	1593	2.3468	0.6343	0.6227	0.6285	0.7103
0.0043	28.0	1652	2.4458	0.6397	0.625	0.6323	0.7161
0.0043	29.0	1711	2.4779	0.6394	0.6284	0.6339	0.7113
0.0043	30.0	1770	2.3498	0.6466	0.6273	0.6368	0.7161
0.0043	31.0	1829	2.4026	0.6454	0.6388	0.6421	0.7141
0.0043	32.0	1888	2.4380	0.6394	0.6284	0.6339	0.7132
0.0043	33.0	1947	2.4099	0.6360	0.6193	0.6275	0.7084
0.0027	34.0	2006	2.4331	0.6370	0.6158	0.6262	0.7093
0.0027	35.0	2065	2.4166	0.6519	0.6399	0.6458	0.7199
0.0027	36.0	2124	2.5268	0.6303	0.6353	0.6328	0.7055
0.0027	37.0	2183	2.4152	0.6377	0.6319	0.6348	0.7074
0.0027	38.0	2242	2.5392	0.6293	0.6307	0.6300	0.7045
0.0027	39.0	2301	2.5672	0.6324	0.6353	0.6339	0.7093
0.0027	40.0	2360	2.5116	0.6323	0.6330	0.6327	0.7093
0.0027	41.0	2419	2.5884	0.6362	0.6376	0.6369	0.7113
0.0027	42.0	2478	2.5252	0.6512	0.6273	0.6390	0.7180
0.0021	43.0	2537	2.4763	0.6480	0.6353	0.6416	0.7151
0.0021	44.0	2596	2.4957	0.6455	0.6307	0.6381	0.7132
0.0021	45.0	2655	2.5187	0.6441	0.6330	0.6385	0.7064
0.0021	46.0	2714	2.4969	0.6524	0.6307	0.6414	0.7161
0.0021	47.0	2773	2.5839	0.6567	0.6296	0.6429	0.7180
0.0021	48.0	2832	2.4747	0.6647	0.6342	0.6491	0.7267
0.0021	49.0	2891	2.5119	0.6492	0.6411	0.6451	0.7218
0.0021	50.0	2950	2.5855	0.6382	0.6330	0.6356	0.7132
0.0016	51.0	3009	2.5679	0.6549	0.6376	0.6461	0.7190
0.0016	52.0	3068	2.4618	0.6631	0.6388	0.6507	0.7295
0.0016	53.0	3127	2.5270	0.6529	0.6170	0.6344	0.7141
0.0016	54.0	3186	2.5133	0.6485	0.6284	0.6383	0.7141
0.0016	55.0	3245	2.4895	0.6560	0.6342	0.6449	0.7180
0.0016	56.0	3304	2.5001	0.6650	0.6261	0.6450	0.7267
0.0016	57.0	3363	2.5202	0.6516	0.6284	0.6398	0.7180
0.0016	58.0	3422	2.4701	0.6715	0.6330	0.6517	0.7305
0.0016	59.0	3481	2.4988	0.6598	0.625	0.6419	0.7238
0.0015	60.0	3540	2.5555	0.6499	0.6216	0.6354	0.7161
0.0015	61.0	3599	2.5242	0.6487	0.6353	0.6419	0.7228
0.0015	62.0	3658	2.5146	0.6618	0.6284	0.6447	0.7190
0.0015	63.0	3717	2.5632	0.6496	0.625	0.6371	0.7170
0.0015	64.0	3776	2.5966	0.6486	0.6307	0.6395	0.7209
0.0015	65.0	3835	2.6079	0.6386	0.6261	0.6323	0.7170
0.0015	66.0	3894	2.5620	0.6355	0.6239	0.6296	0.7084
0.0015	67.0	3953	2.5748	0.6566	0.625	0.6404	0.7218
0.0009	68.0	4012	2.5582	0.6548	0.6353	0.6449	0.7209
0.0009	69.0	4071	2.5776	0.6549	0.6181	0.6360	0.7209
0.0009	70.0	4130	2.5435	0.6619	0.6330	0.6471	0.7267
0.0009	71.0	4189	2.5359	0.6489	0.6296	0.6391	0.7199
0.0009	72.0	4248	2.6138	0.6394	0.6284	0.6339	0.7151
0.0009	73.0	4307	2.6431	0.6385	0.6158	0.6270	0.7093
0.0009	74.0	4366	2.6701	0.6412	0.6353	0.6382	0.7151
0.0009	75.0	4425	2.6492	0.6511	0.6376	0.6443	0.7199
0.0009	76.0	4484	2.6477	0.6564	0.6376	0.6469	0.7218
0.0007	77.0	4543	2.6216	0.6422	0.6216	0.6317	0.7122
0.0007	78.0	4602	2.6166	0.6446	0.6261	0.6353	0.7132
0.0007	79.0	4661	2.7084	0.6382	0.6353	0.6368	0.7180
0.0007	80.0	4720	2.6783	0.6482	0.6296	0.6387	0.7199
0.0007	81.0	4779	2.7061	0.6472	0.6353	0.6412	0.7170
0.0007	82.0	4838	2.6468	0.6503	0.6376	0.6439	0.7190
0.0007	83.0	4897	2.6437	0.6404	0.6330	0.6367	0.7141
0.0007	84.0	4956	2.5965	0.6474	0.6296	0.6384	0.7170
0.0009	85.0	5015	2.6175	0.6524	0.6307	0.6414	0.7199
0.0009	86.0	5074	2.6304	0.6471	0.6307	0.6388	0.7161
0.0009	87.0	5133	2.6389	0.6524	0.6284	0.6402	0.7199
0.0009	88.0	5192	2.6132	0.6544	0.6296	0.6417	0.7218
0.0009	89.0	5251	2.5972	0.6475	0.6319	0.6396	0.7180
0.0009	90.0	5310	2.6066	0.6580	0.6376	0.6476	0.7257
0.0009	91.0	5369	2.6175	0.6611	0.6330	0.6467	0.7238
0.0009	92.0	5428	2.6420	0.6506	0.6365	0.6435	0.7238
0.0009	93.0	5487	2.6679	0.6480	0.6376	0.6428	0.7209
0.0007	94.0	5546	2.6318	0.6486	0.6330	0.6407	0.7199
0.0007	95.0	5605	2.6225	0.6553	0.6365	0.6457	0.7228
0.0007	96.0	5664	2.6299	0.6502	0.6330	0.6415	0.7199
0.0007	97.0	5723	2.6313	0.6514	0.6342	0.6426	0.7209
0.0007	98.0	5782	2.6338	0.6518	0.6353	0.6434	0.7209
0.0007	99.0	5841	2.6334	0.6518	0.6353	0.6434	0.7209
0.0007	100.0	5900	2.6344	0.6506	0.6342	0.6423	0.7199

Framework versions

• Transformers 4.34.1
• Pytorch 2.2.1+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1