finetuned-marktextepoch-n200

This model is a fine-tuned version of distilroberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0880

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
2.5742	1.0	1606	2.4071
2.4441	2.0	3212	2.2715
2.3699	3.0	4818	2.2896
2.3074	4.0	6424	2.2295
2.2667	5.0	8030	2.2147
2.2376	6.0	9636	2.1886
2.2161	7.0	11242	2.1816
2.1611	8.0	12848	2.1690
2.1418	9.0	14454	2.1541
2.1198	10.0	16060	2.1355
2.1033	11.0	17666	2.1132
2.0738	12.0	19272	2.1441
2.0581	13.0	20878	2.1068
2.0555	14.0	22484	2.1035
2.0375	15.0	24090	2.1000
2.0071	16.0	25696	2.1084
1.9942	17.0	27302	2.0711
1.9554	18.0	28908	2.0978
1.9469	19.0	30514	2.0705
1.9414	20.0	32120	2.0597
1.9331	21.0	33726	2.0782
1.9132	22.0	35332	2.0622
1.9003	23.0	36938	2.0426
1.9019	24.0	38544	2.0562
1.8733	25.0	40150	2.0419
1.8556	26.0	41756	2.0572
1.8399	27.0	43362	2.0453
1.8332	28.0	44968	2.0517
1.8296	29.0	46574	2.0580
1.788	30.0	48180	2.0454
1.7944	31.0	49786	2.0193
1.7716	32.0	51392	2.0595
1.7799	33.0	52998	2.0379
1.7633	34.0	54604	2.0392
1.7477	35.0	56210	2.0122
1.7407	36.0	57816	2.0293
1.7163	37.0	59422	2.0339
1.72	38.0	61028	1.9987
1.729	39.0	62634	2.0135
1.709	40.0	64240	2.0455
1.7019	41.0	65846	2.0206
1.6958	42.0	67452	2.0408
1.6789	43.0	69058	2.0470
1.6907	44.0	70664	2.0280
1.6531	45.0	72270	2.0514
1.6563	46.0	73876	2.0428
1.6364	47.0	75482	2.0305
1.6534	48.0	77088	2.0200
1.6312	49.0	78694	2.0444
1.6092	50.0	80300	2.0154
1.5998	51.0	81906	2.0249
1.5808	52.0	83512	2.0235
1.5945	53.0	85118	2.0286
1.6004	54.0	86724	2.0288
1.5802	55.0	88330	2.0346
1.5665	56.0	89936	2.0120
1.5723	57.0	91542	2.0257
1.5553	58.0	93148	2.0146
1.5445	59.0	94754	2.0333
1.5669	60.0	96360	2.0325
1.5318	61.0	97966	2.0250
1.5117	62.0	99572	2.0343
1.5248	63.0	101178	2.0183
1.5149	64.0	102784	2.0422
1.5087	65.0	104390	2.0236
1.5087	66.0	105996	2.0275
1.4938	67.0	107602	2.0384
1.5008	68.0	109208	2.0167
1.4871	69.0	110814	2.0456
1.4931	70.0	112420	2.0083
1.467	71.0	114026	2.0313
1.4519	72.0	115632	2.0254
1.448	73.0	117238	2.0289
1.4475	74.0	118844	2.0051
1.4522	75.0	120450	2.0378
1.4508	76.0	122056	2.0612
1.4428	77.0	123662	2.0479
1.4496	78.0	125268	2.0082
1.4305	79.0	126874	2.0376
1.4072	80.0	128480	2.0294
1.4148	81.0	130086	2.0565
1.4078	82.0	131692	2.0309
1.3931	83.0	133298	2.0371
1.4038	84.0	134904	2.0318
1.3893	85.0	136510	2.0413
1.3862	86.0	138116	2.0503
1.3782	87.0	139722	2.0182
1.3757	88.0	141328	2.0253
1.3879	89.0	142934	2.0357
1.3768	90.0	144540	2.0405
1.3494	91.0	146146	2.0495
1.3492	92.0	147752	2.0586
1.353	93.0	149358	2.0779
1.3397	94.0	150964	2.0564
1.3486	95.0	152570	2.0459
1.3262	96.0	154176	2.0692
1.349	97.0	155782	2.0765
1.3228	98.0	157388	2.0443
1.3291	99.0	158994	2.0617
1.3211	100.0	160600	2.0552
1.3344	101.0	162206	2.0626
1.307	102.0	163812	2.0492
1.2968	103.0	165418	2.0461
1.2919	104.0	167024	2.0725
1.3004	105.0	168630	2.0424
1.303	106.0	170236	2.0484
1.2847	107.0	171842	2.0083
1.2861	108.0	173448	2.0491
1.2763	109.0	175054	2.0505
1.2852	110.0	176660	2.0691
1.2611	111.0	178266	2.0711
1.2739	112.0	179872	2.0730
1.278	113.0	181478	2.0551
1.2581	114.0	183084	2.0554
1.2532	115.0	184690	2.0513
1.2322	116.0	186296	2.0292
1.2774	117.0	187902	2.0409
1.242	118.0	189508	2.0517
1.2476	119.0	191114	2.0612
1.2314	120.0	192720	2.0795
1.2379	121.0	194326	2.0679
1.2291	122.0	195932	2.0472
1.2515	123.0	197538	2.0829
1.2467	124.0	199144	2.0662
1.2437	125.0	200750	2.0962
1.2373	126.0	202356	2.0692
1.2099	127.0	203962	2.0688
1.1911	128.0	205568	2.0803
1.2311	129.0	207174	2.0765
1.2095	130.0	208780	2.0697
1.2093	131.0	210386	2.0507
1.2065	132.0	211992	2.0658
1.1964	133.0	213598	2.0542
1.2085	134.0	215204	2.0722
1.1871	135.0	216810	2.0806
1.1863	136.0	218416	2.0691
1.1763	137.0	220022	2.0869
1.1816	138.0	221628	2.0780
1.1854	139.0	223234	2.0462
1.1902	140.0	224840	2.0880
1.1762	141.0	226446	2.0682
1.1551	142.0	228052	2.0837
1.171	143.0	229658	2.1028
1.1571	144.0	231264	2.0726
1.1627	145.0	232870	2.0863
1.1537	146.0	234476	2.0857
1.1695	147.0	236082	2.0620
1.1477	148.0	237688	2.0817
1.1592	149.0	239294	2.0705
1.1478	150.0	240900	2.0841
1.1398	151.0	242506	2.0886
1.144	152.0	244112	2.0673
1.1646	153.0	245718	2.0620
1.12	154.0	247324	2.0821
1.1419	155.0	248930	2.0632
1.1436	156.0	250536	2.0817
1.1365	157.0	252142	2.0663
1.1318	158.0	253748	2.0796
1.1219	159.0	255354	2.0825
1.1306	160.0	256960	2.0837
1.1295	161.0	258566	2.0564
1.1261	162.0	260172	2.0722
1.1273	163.0	261778	2.1058
1.1143	164.0	263384	2.0963
1.1276	165.0	264990	2.0948
1.1238	166.0	266596	2.0695
1.1222	167.0	268202	2.0801
1.1145	168.0	269808	2.0768
1.1093	169.0	271414	2.0664
1.1141	170.0	273020	2.0903
1.0936	171.0	274626	2.1012
1.1048	172.0	276232	2.1033
1.0991	173.0	277838	2.0761
1.1164	174.0	279444	2.0689
1.0935	175.0	281050	2.0754
1.1032	176.0	282656	2.0810
1.1124	177.0	284262	2.0790
1.1107	178.0	285868	2.0762
1.085	179.0	287474	2.0697
1.093	180.0	289080	2.0856
1.1034	181.0	290686	2.0734
1.0983	182.0	292292	2.0837
1.0972	183.0	293898	2.1063
1.0909	184.0	295504	2.0873
1.0805	185.0	297110	2.0888
1.0893	186.0	298716	2.0498
1.096	187.0	300322	2.0906
1.0781	188.0	301928	2.0905
1.0981	189.0	303534	2.0767
1.093	190.0	305140	2.0695
1.0814	191.0	306746	2.0763
1.0862	192.0	308352	2.0890
1.0833	193.0	309958	2.1026
1.0806	194.0	311564	2.0978
1.0834	195.0	313170	2.1004
1.0807	196.0	314776	2.0953
1.0827	197.0	316382	2.1129
1.0826	198.0	317988	2.1069
1.0796	199.0	319594	2.0867
1.0881	200.0	321200	2.0880

Framework versions

• Transformers 4.21.1
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1