xlmr-lstm-crf-resume-ner2

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3688
• Precision: 0.7289
• Recall: 0.7578
• F1: 0.7431
• Accuracy: 0.9403

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 8e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
2.6224	1.0	17	1.1537	0.9517	0.0367	0.0707	0.8445
1.0702	2.0	34	0.9869	0.0	0.0	0.0	0.8483
0.8288	3.0	51	0.6899	0.0287	0.0029	0.0053	0.8586
0.6586	4.0	68	0.5719	0.1512	0.0378	0.0605	0.8705
0.5433	5.0	85	0.4887	0.2649	0.0873	0.1313	0.8745
0.4696	6.0	102	0.4358	0.1852	0.0631	0.0941	0.8822
0.4114	7.0	119	0.3901	0.4455	0.3463	0.3897	0.8914
0.3631	8.0	136	0.3684	0.4111	0.3891	0.3998	0.9006
0.3239	9.0	153	0.3457	0.4668	0.4991	0.4824	0.9024
0.3047	10.0	170	0.3195	0.5824	0.4693	0.5198	0.9142
0.2775	11.0	187	0.3110	0.5384	0.5206	0.5294	0.9130
0.2518	12.0	204	0.3078	0.6492	0.4703	0.5455	0.9176
0.2362	13.0	221	0.3036	0.5136	0.5739	0.5420	0.9130
0.2174	14.0	238	0.2983	0.5499	0.6023	0.5749	0.9146
0.2037	15.0	255	0.2909	0.6167	0.5656	0.5900	0.9234
0.1842	16.0	272	0.3100	0.5866	0.6141	0.6000	0.9201
0.1706	17.0	289	0.2949	0.6067	0.6234	0.6149	0.9231
0.1648	18.0	306	0.2992	0.6047	0.6188	0.6117	0.9239
0.1485	19.0	323	0.2972	0.6012	0.6761	0.6364	0.9228
0.1381	20.0	340	0.2910	0.6372	0.6423	0.6397	0.9282
0.1259	21.0	357	0.2822	0.6575	0.6534	0.6555	0.9310
0.1178	22.0	374	0.3007	0.6297	0.6862	0.6567	0.9278
0.1123	23.0	391	0.2864	0.6537	0.6859	0.6694	0.9308
0.1017	24.0	408	0.2988	0.6924	0.6849	0.6886	0.9360
0.0961	25.0	425	0.3043	0.6219	0.7080	0.6622	0.9299
0.091	26.0	442	0.3092	0.6389	0.7298	0.6813	0.9293
0.0866	27.0	459	0.3121	0.6346	0.6806	0.6568	0.9278
0.0808	28.0	476	0.2988	0.7084	0.7040	0.7062	0.9376
0.0723	29.0	493	0.2962	0.6888	0.7112	0.6998	0.9372
0.0692	30.0	510	0.3080	0.6906	0.7248	0.7073	0.9365
0.0627	31.0	527	0.3178	0.6683	0.7077	0.6874	0.9342
0.0647	32.0	544	0.3044	0.7079	0.7211	0.7144	0.9380
0.0557	33.0	561	0.3157	0.7206	0.7200	0.7203	0.9382
0.0532	34.0	578	0.3220	0.6841	0.7501	0.7156	0.9371
0.0496	35.0	595	0.3206	0.6452	0.7565	0.6964	0.9314
0.0494	36.0	612	0.3203	0.6901	0.7533	0.7203	0.9376
0.0426	37.0	629	0.3348	0.7123	0.7408	0.7263	0.9374
0.0416	38.0	646	0.3317	0.7065	0.7389	0.7224	0.9376
0.0418	39.0	663	0.3323	0.7099	0.7378	0.7236	0.9379
0.0372	40.0	680	0.3322	0.7087	0.7543	0.7308	0.9383
0.0349	41.0	697	0.3295	0.7213	0.7261	0.7237	0.9381
0.0357	42.0	714	0.3474	0.7	0.7471	0.7228	0.9368
0.034	43.0	731	0.3342	0.7158	0.7554	0.7350	0.9384
0.0301	44.0	748	0.3417	0.7271	0.7423	0.7346	0.9397
0.0297	45.0	765	0.3416	0.7284	0.7501	0.7391	0.9397
0.0278	46.0	782	0.3583	0.7254	0.7567	0.7408	0.9403
0.0264	47.0	799	0.3515	0.7246	0.7583	0.7411	0.9405
0.0254	48.0	816	0.3544	0.7147	0.7628	0.7380	0.9405
0.0239	49.0	833	0.3555	0.7161	0.7706	0.7423	0.9392
0.0227	50.0	850	0.3611	0.7164	0.7687	0.7417	0.9400
0.023	51.0	867	0.3646	0.7080	0.7687	0.7371	0.9389
0.0217	52.0	884	0.3718	0.7344	0.7639	0.7489	0.9404
0.0214	53.0	901	0.3656	0.7137	0.7618	0.7370	0.9397
0.0197	54.0	918	0.3700	0.7060	0.7612	0.7326	0.9387
0.019	55.0	935	0.3764	0.7166	0.7762	0.7452	0.9401
0.0183	56.0	952	0.3688	0.7289	0.7578	0.7431	0.9403

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0