chemical-bert-uncased-finetuned-cust-c1-cust

This model is a fine-tuned version of shafin/chemical-bert-uncased-finetuned-cust on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5420

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.96	1.0	63	1.6719
1.7095	2.0	126	1.5305
1.5634	3.0	189	1.2972
1.4785	4.0	252	1.3354
1.3991	5.0	315	1.2542
1.3482	6.0	378	1.1870
1.2984	7.0	441	1.1844
1.2589	8.0	504	1.1262
1.1762	9.0	567	1.1176
1.1724	10.0	630	1.0312
1.1222	11.0	693	1.0113
1.1021	12.0	756	1.0518
1.0646	13.0	819	1.0433
1.0273	14.0	882	0.9634
1.0187	15.0	945	0.9299
0.9854	16.0	1008	0.9458
0.9799	17.0	1071	0.9733
0.95	18.0	1134	0.9169
0.934	19.0	1197	0.9246
0.907	20.0	1260	0.8939
0.8974	21.0	1323	0.8575
0.8749	22.0	1386	0.8513
0.8526	23.0	1449	0.8089
0.8359	24.0	1512	0.8600
0.8292	25.0	1575	0.8517
0.8263	26.0	1638	0.8293
0.8033	27.0	1701	0.7747
0.7999	28.0	1764	0.8169
0.7778	29.0	1827	0.7981
0.7574	30.0	1890	0.7457
0.7581	31.0	1953	0.7504
0.7404	32.0	2016	0.7637
0.7332	33.0	2079	0.7902
0.7314	34.0	2142	0.7185
0.7209	35.0	2205	0.7534
0.6902	36.0	2268	0.7334
0.6973	37.0	2331	0.7069
0.687	38.0	2394	0.6820
0.6658	39.0	2457	0.7155
0.6697	40.0	2520	0.7149
0.6584	41.0	2583	0.7413
0.6638	42.0	2646	0.7245
0.6282	43.0	2709	0.7177
0.6418	44.0	2772	0.6653
0.6323	45.0	2835	0.7715
0.6256	46.0	2898	0.7269
0.6109	47.0	2961	0.6744
0.6133	48.0	3024	0.6816
0.595	49.0	3087	0.6969
0.6058	50.0	3150	0.6965
0.5961	51.0	3213	0.6988
0.587	52.0	3276	0.6727
0.5861	53.0	3339	0.6327
0.5758	54.0	3402	0.6538
0.5692	55.0	3465	0.6612
0.567	56.0	3528	0.5989
0.5514	57.0	3591	0.6776
0.5526	58.0	3654	0.6440
0.556	59.0	3717	0.6682
0.5476	60.0	3780	0.6254
0.536	61.0	3843	0.6239
0.526	62.0	3906	0.6606
0.532	63.0	3969	0.6565
0.5189	64.0	4032	0.6586
0.5075	65.0	4095	0.6286
0.5131	66.0	4158	0.6646
0.498	67.0	4221	0.6486
0.4979	68.0	4284	0.6313
0.4885	69.0	4347	0.6419
0.4875	70.0	4410	0.6313
0.4904	71.0	4473	0.6602
0.4712	72.0	4536	0.6200
0.4798	73.0	4599	0.5912
0.4802	74.0	4662	0.6001
0.4704	75.0	4725	0.6303
0.4709	76.0	4788	0.5871
0.465	77.0	4851	0.6344
0.4651	78.0	4914	0.6030
0.4501	79.0	4977	0.5998
0.4584	80.0	5040	0.5926
0.4651	81.0	5103	0.6134
0.438	82.0	5166	0.6254
0.448	83.0	5229	0.6260
0.4295	84.0	5292	0.5866
0.434	85.0	5355	0.5740
0.4261	86.0	5418	0.5691
0.4312	87.0	5481	0.6243
0.4289	88.0	5544	0.5781
0.4255	89.0	5607	0.6226
0.4254	90.0	5670	0.5538
0.4231	91.0	5733	0.5874
0.4107	92.0	5796	0.6054
0.4082	93.0	5859	0.5898
0.4144	94.0	5922	0.5826
0.4225	95.0	5985	0.5501
0.3964	96.0	6048	0.5886
0.3972	97.0	6111	0.5831
0.4165	98.0	6174	0.5164
0.4024	99.0	6237	0.5714
0.4013	100.0	6300	0.5734
0.3933	101.0	6363	0.5727
0.3821	102.0	6426	0.5985
0.3904	103.0	6489	0.5571
0.3965	104.0	6552	0.5837
0.3789	105.0	6615	0.5989
0.3733	106.0	6678	0.5405
0.3907	107.0	6741	0.6059
0.3794	108.0	6804	0.5602
0.3689	109.0	6867	0.5590
0.3603	110.0	6930	0.5886
0.3747	111.0	6993	0.5294
0.3667	112.0	7056	0.5759
0.3754	113.0	7119	0.5821
0.3676	114.0	7182	0.5653
0.3524	115.0	7245	0.5537
0.3624	116.0	7308	0.5523
0.3527	117.0	7371	0.5799
0.3588	118.0	7434	0.6346
0.3539	119.0	7497	0.5116
0.3553	120.0	7560	0.5716
0.3483	121.0	7623	0.5721
0.3625	122.0	7686	0.5393
0.3354	123.0	7749	0.5800
0.3392	124.0	7812	0.5389
0.344	125.0	7875	0.5455
0.3451	126.0	7938	0.5428
0.3374	127.0	8001	0.5580
0.3428	128.0	8064	0.5339
0.3386	129.0	8127	0.5447
0.3318	130.0	8190	0.5738
0.3388	131.0	8253	0.5667
0.3335	132.0	8316	0.5407
0.3383	133.0	8379	0.5679
0.3299	134.0	8442	0.5846
0.327	135.0	8505	0.5511
0.3354	136.0	8568	0.5649
0.32	137.0	8631	0.5358
0.3265	138.0	8694	0.5528
0.319	139.0	8757	0.5926
0.3304	140.0	8820	0.5531
0.3191	141.0	8883	0.5379
0.3298	142.0	8946	0.5468
0.3134	143.0	9009	0.5623
0.3186	144.0	9072	0.5162
0.3179	145.0	9135	0.5570
0.3175	146.0	9198	0.5379
0.3051	147.0	9261	0.5437
0.312	148.0	9324	0.5301
0.3093	149.0	9387	0.5393
0.3227	150.0	9450	0.5531
0.3125	151.0	9513	0.5794
0.3162	152.0	9576	0.5677
0.3006	153.0	9639	0.5668
0.3011	154.0	9702	0.5797
0.3208	155.0	9765	0.5450
0.3048	156.0	9828	0.5465
0.3092	157.0	9891	0.5358
0.3125	158.0	9954	0.5043
0.3083	159.0	10017	0.5321
0.3	160.0	10080	0.5526
0.2968	161.0	10143	0.5324
0.3068	162.0	10206	0.5471
0.3129	163.0	10269	0.5575
0.3061	164.0	10332	0.5796
0.2943	165.0	10395	0.5544
0.2967	166.0	10458	0.5422
0.2959	167.0	10521	0.5149
0.2987	168.0	10584	0.5685
0.3045	169.0	10647	0.5176
0.2975	170.0	10710	0.5044
0.2948	171.0	10773	0.5264
0.3	172.0	10836	0.5174
0.2967	173.0	10899	0.5658
0.2873	174.0	10962	0.4988
0.2939	175.0	11025	0.5512
0.2954	176.0	11088	0.5139
0.301	177.0	11151	0.6007
0.2948	178.0	11214	0.5167
0.2898	179.0	11277	0.5443
0.2869	180.0	11340	0.5544
0.2973	181.0	11403	0.5644
0.2985	182.0	11466	0.5153
0.2904	183.0	11529	0.5561
0.2872	184.0	11592	0.5610
0.2894	185.0	11655	0.5511
0.297	186.0	11718	0.5408
0.2904	187.0	11781	0.5574
0.2818	188.0	11844	0.5182
0.2873	189.0	11907	0.5425
0.2973	190.0	11970	0.5198
0.2913	191.0	12033	0.5119
0.2931	192.0	12096	0.5585
0.2859	193.0	12159	0.5368
0.2853	194.0	12222	0.5274
0.294	195.0	12285	0.5685
0.2885	196.0	12348	0.5581
0.295	197.0	12411	0.4987
0.2807	198.0	12474	0.5168
0.289	199.0	12537	0.5284
0.2893	200.0	12600	0.5420

Framework versions

• Transformers 4.24.0
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.2