xlm-roberta-base-finetuned-semantics

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0033
• F1: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 10
• eval_batch_size: 10
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	F1
0.7438	1.0	10	0.3942	0.7160
0.3045	2.0	20	0.1455	0.8608
0.1072	3.0	30	0.0673	0.9114
0.0647	4.0	40	0.0389	0.9620
0.0477	5.0	50	0.0212	0.9620
0.0164	6.0	60	0.0258	0.9744
0.0215	7.0	70	0.0303	0.9744
0.022	8.0	80	0.0096	1.0
0.0057	9.0	90	0.0104	1.0
0.0069	10.0	100	0.0101	1.0
0.0028	11.0	110	0.0085	1.0
0.0109	12.0	120	0.0128	0.9744
0.0048	13.0	130	0.0136	0.9744
0.0045	14.0	140	0.0145	0.9744
0.003	15.0	150	0.0142	0.9744
0.0013	16.0	160	0.0140	0.9744
0.0025	17.0	170	0.0134	0.9744
0.001	18.0	180	0.0110	0.9744
0.001	19.0	190	0.0085	0.9744
0.0007	20.0	200	0.0038	1.0
0.0007	21.0	210	0.0008	1.0
0.0043	22.0	220	0.0009	1.0
0.0045	23.0	230	0.0011	1.0
0.0008	24.0	240	0.0012	1.0
0.0009	25.0	250	0.0013	1.0
0.0006	26.0	260	0.0012	1.0
0.0006	27.0	270	0.0011	1.0
0.0013	28.0	280	0.0012	1.0
0.0006	29.0	290	0.0013	1.0
0.0017	30.0	300	0.0004	1.0
0.0019	31.0	310	0.0005	1.0
0.0137	32.0	320	0.0042	1.0
0.0023	33.0	330	0.0280	0.9744
0.0099	34.0	340	0.0327	0.9744
0.0085	35.0	350	0.0196	0.9744
0.001	36.0	360	0.0043	1.0
0.0014	37.0	370	0.0006	1.0
0.0005	38.0	380	0.0005	1.0
0.0016	39.0	390	0.0004	1.0
0.0005	40.0	400	0.0006	1.0
0.009	41.0	410	0.0007	1.0
0.0008	42.0	420	0.0006	1.0
0.0005	43.0	430	0.0006	1.0
0.0004	44.0	440	0.0006	1.0
0.0004	45.0	450	0.0006	1.0
0.001	46.0	460	0.0004	1.0
0.0006	47.0	470	0.0004	1.0
0.0006	48.0	480	0.0012	1.0
0.0003	49.0	490	0.0022	1.0
0.0004	50.0	500	0.0025	1.0
0.0003	51.0	510	0.0025	1.0
0.0003	52.0	520	0.0025	1.0
0.0014	53.0	530	0.0026	1.0
0.0003	54.0	540	0.0033	1.0
0.0003	55.0	550	0.0034	1.0
0.0029	56.0	560	0.0033	1.0
0.0022	57.0	570	0.0032	1.0
0.0024	58.0	580	0.0032	1.0
0.0003	59.0	590	0.0030	1.0
0.0006	60.0	600	0.0068	0.9744
0.0003	61.0	610	0.0093	0.9744
0.0002	62.0	620	0.0098	0.9744
0.0003	63.0	630	0.0096	0.9744
0.0002	64.0	640	0.0090	0.9744
0.0003	65.0	650	0.0073	0.9744
0.0002	66.0	660	0.0058	1.0
0.0079	67.0	670	0.0024	1.0
0.0005	68.0	680	0.0006	1.0
0.0028	69.0	690	0.0004	1.0
0.0077	70.0	700	0.0005	1.0
0.0004	71.0	710	0.0004	1.0
0.0078	72.0	720	0.0002	1.0
0.0047	73.0	730	0.0029	1.0
0.0004	74.0	740	0.0067	1.0
0.0004	75.0	750	0.0077	1.0
0.0003	76.0	760	0.0078	1.0
0.0021	77.0	770	0.0075	1.0
0.0003	78.0	780	0.0071	1.0
0.0019	79.0	790	0.0066	1.0
0.0003	80.0	800	0.0062	1.0
0.0003	81.0	810	0.0057	1.0
0.0034	82.0	820	0.0052	1.0
0.0002	83.0	830	0.0048	1.0
0.0003	84.0	840	0.0046	1.0
0.0002	85.0	850	0.0044	1.0
0.0002	86.0	860	0.0043	1.0
0.0002	87.0	870	0.0041	1.0
0.0053	88.0	880	0.0037	1.0
0.0034	89.0	890	0.0036	1.0
0.0002	90.0	900	0.0036	1.0
0.0002	91.0	910	0.0035	1.0
0.0002	92.0	920	0.0035	1.0
0.0002	93.0	930	0.0035	1.0
0.0002	94.0	940	0.0034	1.0
0.0002	95.0	950	0.0034	1.0
0.0023	96.0	960	0.0034	1.0
0.0002	97.0	970	0.0034	1.0
0.0002	98.0	980	0.0034	1.0
0.0021	99.0	990	0.0033	1.0
0.0003	100.0	1000	0.0033	1.0

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1
• Datasets 2.14.6
• Tokenizers 0.14.1