distilroberta-base-finetuned-marktextepoch_n200

This model is a fine-tuned version of distilroberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0531

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
2.2313	1.0	1500	2.1592
2.1731	2.0	3000	2.1277
2.153	3.0	4500	2.1144
2.1469	4.0	6000	2.1141
2.1281	5.0	7500	2.1374
2.1043	6.0	9000	2.1069
2.0834	7.0	10500	2.0993
2.0602	8.0	12000	2.0817
2.024	9.0	13500	2.0918
2.0261	10.0	15000	2.0793
1.9889	11.0	16500	2.0567
1.9915	12.0	18000	2.0700
1.9532	13.0	19500	2.0436
1.9362	14.0	21000	2.0596
1.9024	15.0	22500	2.0189
1.9262	16.0	24000	2.0435
1.8883	17.0	25500	2.0430
1.8867	18.0	27000	2.0416
1.8807	19.0	28500	2.0051
1.8517	20.0	30000	2.0338
1.8357	21.0	31500	2.0166
1.8241	22.0	33000	2.0355
1.7985	23.0	34500	2.0073
1.8061	24.0	36000	2.0473
1.7996	25.0	37500	2.0446
1.7786	26.0	39000	2.0086
1.771	27.0	40500	2.0294
1.7549	28.0	42000	2.0127
1.7726	29.0	43500	2.0191
1.7275	30.0	45000	2.0182
1.708	31.0	46500	2.0130
1.7345	32.0	48000	2.0155
1.7044	33.0	49500	1.9898
1.7126	34.0	51000	2.0166
1.698	35.0	52500	1.9879
1.6637	36.0	54000	2.0311
1.6854	37.0	55500	2.0355
1.6585	38.0	57000	2.0094
1.6418	39.0	58500	2.0042
1.667	40.0	60000	2.0116
1.6507	41.0	61500	2.0095
1.622	42.0	63000	2.0158
1.6381	43.0	64500	2.0339
1.6099	44.0	66000	2.0082
1.6076	45.0	67500	2.0207
1.5805	46.0	69000	2.0172
1.5862	47.0	70500	2.0132
1.5806	48.0	72000	2.0198
1.574	49.0	73500	2.0181
1.5718	50.0	75000	2.0086
1.5591	51.0	76500	1.9832
1.5468	52.0	78000	2.0167
1.5637	53.0	79500	2.0118
1.5117	54.0	81000	2.0290
1.5363	55.0	82500	2.0011
1.4976	56.0	84000	2.0160
1.5129	57.0	85500	2.0224
1.4964	58.0	87000	2.0219
1.4906	59.0	88500	2.0212
1.4941	60.0	90000	2.0255
1.4876	61.0	91500	2.0116
1.4837	62.0	93000	2.0176
1.4661	63.0	94500	2.0388
1.4634	64.0	96000	2.0165
1.4449	65.0	97500	2.0185
1.468	66.0	99000	2.0246
1.4567	67.0	100500	2.0244
1.4367	68.0	102000	2.0093
1.4471	69.0	103500	2.0101
1.4255	70.0	105000	2.0248
1.4203	71.0	106500	2.0224
1.42	72.0	108000	2.0279
1.4239	73.0	109500	2.0295
1.4126	74.0	111000	2.0196
1.4038	75.0	112500	2.0225
1.3874	76.0	114000	2.0456
1.3758	77.0	115500	2.0423
1.3924	78.0	117000	2.0184
1.3744	79.0	118500	2.0555
1.3622	80.0	120000	2.0387
1.3653	81.0	121500	2.0344
1.3724	82.0	123000	2.0184
1.3684	83.0	124500	2.0285
1.3576	84.0	126000	2.0544
1.348	85.0	127500	2.0412
1.3387	86.0	129000	2.0459
1.3416	87.0	130500	2.0329
1.3421	88.0	132000	2.0274
1.3266	89.0	133500	2.0233
1.3183	90.0	135000	2.0319
1.322	91.0	136500	2.0080
1.32	92.0	138000	2.0472
1.304	93.0	139500	2.0538
1.3061	94.0	141000	2.0340
1.3199	95.0	142500	2.0456
1.2985	96.0	144000	2.0167
1.3021	97.0	145500	2.0204
1.2787	98.0	147000	2.0645
1.2879	99.0	148500	2.0345
1.2695	100.0	150000	2.0340
1.2884	101.0	151500	2.0602
1.2747	102.0	153000	2.0667
1.2607	103.0	154500	2.0551
1.2551	104.0	156000	2.0544
1.2557	105.0	157500	2.0553
1.2495	106.0	159000	2.0370
1.26	107.0	160500	2.0568
1.2499	108.0	162000	2.0427
1.2438	109.0	163500	2.0184
1.2496	110.0	165000	2.0227
1.2332	111.0	166500	2.0621
1.2231	112.0	168000	2.0661
1.211	113.0	169500	2.0673
1.217	114.0	171000	2.0544
1.2206	115.0	172500	2.0542
1.2083	116.0	174000	2.0592
1.2205	117.0	175500	2.0451
1.2065	118.0	177000	2.0402
1.1988	119.0	178500	2.0615
1.218	120.0	180000	2.0374
1.1917	121.0	181500	2.0349
1.1854	122.0	183000	2.0790
1.1819	123.0	184500	2.0766
1.2029	124.0	186000	2.0364
1.1851	125.0	187500	2.0568
1.1734	126.0	189000	2.0445
1.1701	127.0	190500	2.0770
1.1824	128.0	192000	2.0566
1.1604	129.0	193500	2.0542
1.1733	130.0	195000	2.0525
1.1743	131.0	196500	2.0577
1.1692	132.0	198000	2.0723
1.1519	133.0	199500	2.0567
1.1401	134.0	201000	2.0795
1.1692	135.0	202500	2.0625
1.157	136.0	204000	2.0793
1.1495	137.0	205500	2.0782
1.1479	138.0	207000	2.0392
1.1247	139.0	208500	2.0796
1.143	140.0	210000	2.0369
1.1324	141.0	211500	2.0699
1.1341	142.0	213000	2.0694
1.1317	143.0	214500	2.0569
1.1254	144.0	216000	2.0545
1.1156	145.0	217500	2.0708
1.1353	146.0	219000	2.0767
1.1312	147.0	220500	2.0523
1.1224	148.0	222000	2.0565
1.106	149.0	223500	2.0696
1.1069	150.0	225000	2.0478
1.1011	151.0	226500	2.0475
1.0985	152.0	228000	2.0888
1.1107	153.0	229500	2.0756
1.1058	154.0	231000	2.0812
1.1027	155.0	232500	2.0597
1.0996	156.0	234000	2.0684
1.0987	157.0	235500	2.0629
1.0881	158.0	237000	2.0701
1.1143	159.0	238500	2.0740
1.0823	160.0	240000	2.0869
1.0925	161.0	241500	2.0567
1.1034	162.0	243000	2.0833
1.0759	163.0	244500	2.0585
1.0998	164.0	246000	2.0293
1.0891	165.0	247500	2.0608
1.1036	166.0	249000	2.0831
1.076	167.0	250500	2.0979
1.0895	168.0	252000	2.0882
1.0825	169.0	253500	2.0742
1.0793	170.0	255000	2.0841
1.079	171.0	256500	2.0829
1.0653	172.0	258000	2.0888
1.0834	173.0	259500	2.0784
1.0721	174.0	261000	2.0859
1.0712	175.0	262500	2.0810
1.0494	176.0	264000	2.0605
1.0654	177.0	265500	2.0623
1.077	178.0	267000	2.0756
1.056	179.0	268500	2.0782
1.0523	180.0	270000	2.0966
1.0656	181.0	271500	2.0750
1.0636	182.0	273000	2.0769
1.0851	183.0	274500	2.0872
1.0562	184.0	276000	2.0893
1.0534	185.0	277500	2.0661
1.0514	186.0	279000	2.0712
1.062	187.0	280500	2.0769
1.0683	188.0	282000	2.0765
1.0606	189.0	283500	2.0735
1.0555	190.0	285000	2.0710
1.0568	191.0	286500	2.0860
1.0502	192.0	288000	2.0587
1.0437	193.0	289500	2.0998
1.0534	194.0	291000	2.0418
1.062	195.0	292500	2.0724
1.0457	196.0	294000	2.0612
1.0501	197.0	295500	2.1012
1.0728	198.0	297000	2.0721
1.0413	199.0	298500	2.0535
1.0461	200.0	300000	2.0531

Framework versions

• Transformers 4.21.0
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1