xlnet-base-cased_winobias_finetuned

This model is a fine-tuned version of xlnet-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6133
• Accuracy: 0.8232
• Tp: 0.4129
• Tn: 0.4104
• Fp: 0.0896
• Fn: 0.0871

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Tp	Tn	Fp	Fn
0.7262	0.8	20	0.6972	0.5	0.5	0.0	0.5	0.0
0.7214	1.6	40	0.7030	0.5	0.5	0.0	0.5	0.0
0.7094	2.4	60	0.6976	0.5	0.5	0.0	0.5	0.0
0.7102	3.2	80	0.6932	0.4975	0.0518	0.4457	0.0543	0.4482
0.6953	4.0	100	0.6932	0.5	0.5	0.0	0.5	0.0
0.7017	4.8	120	0.6931	0.5044	0.4173	0.0871	0.4129	0.0827
0.7098	5.6	140	0.6950	0.5	0.0	0.5	0.0	0.5
0.7075	6.4	160	0.6932	0.5	0.4811	0.0189	0.4811	0.0189
0.7138	7.2	180	0.6935	0.5	0.0	0.5	0.0	0.5
0.7009	8.0	200	0.6932	0.5	0.5	0.0	0.5	0.0
0.7007	8.8	220	0.6932	0.5006	0.4949	0.0057	0.4943	0.0051
0.6982	9.6	240	0.6929	0.5107	0.0638	0.4470	0.0530	0.4362
0.6991	10.4	260	0.6943	0.5	0.5	0.0	0.5	0.0
0.7027	11.2	280	0.6937	0.5	0.0	0.5	0.0	0.5
0.6944	12.0	300	0.6952	0.5	0.0	0.5	0.0	0.5
0.6928	12.8	320	0.6931	0.5	0.0	0.5	0.0	0.5
0.707	13.6	340	0.6938	0.5	0.0	0.5	0.0	0.5
0.7059	14.4	360	0.6935	0.5	0.5	0.0	0.5	0.0
0.7034	15.2	380	0.6939	0.5	0.0	0.5	0.0	0.5
0.7048	16.0	400	0.6935	0.5	0.5	0.0	0.5	0.0
0.6967	16.8	420	0.6929	0.5	0.5	0.0	0.5	0.0
0.705	17.6	440	0.6924	0.5	0.0	0.5	0.0	0.5
0.7008	18.4	460	0.6909	0.5	0.5	0.0	0.5	0.0
0.565	19.2	480	0.5361	0.7418	0.4779	0.2639	0.2361	0.0221
0.4145	20.0	500	0.4368	0.7336	0.4956	0.2380	0.2620	0.0044
0.3663	20.8	520	0.4434	0.7355	0.4943	0.2412	0.2588	0.0057
0.3586	21.6	540	0.5477	0.7670	0.3491	0.4179	0.0821	0.1509
0.3668	22.4	560	0.3964	0.7765	0.4154	0.3611	0.1389	0.0846
0.3007	23.2	580	0.5550	0.7797	0.4394	0.3403	0.1597	0.0606
0.3052	24.0	600	0.4791	0.7992	0.4186	0.3807	0.1193	0.0814
0.2277	24.8	620	0.6168	0.8144	0.4179	0.3965	0.1035	0.0821
0.1419	25.6	640	1.0091	0.8131	0.4097	0.4034	0.0966	0.0903
0.1323	26.4	660	0.6764	0.8106	0.4097	0.4009	0.0991	0.0903
0.1191	27.2	680	0.5545	0.8125	0.4236	0.3889	0.1111	0.0764
0.0766	28.0	700	0.9580	0.8169	0.4167	0.4003	0.0997	0.0833
0.0452	28.8	720	1.1316	0.8157	0.4009	0.4148	0.0852	0.0991
0.0491	29.6	740	1.2071	0.8119	0.4154	0.3965	0.1035	0.0846
0.073	30.4	760	1.1414	0.8163	0.3914	0.4249	0.0751	0.1086
0.0496	31.2	780	1.0776	0.8182	0.4255	0.3927	0.1073	0.0745
0.0296	32.0	800	1.1979	0.8188	0.4078	0.4110	0.0890	0.0922
0.0379	32.8	820	1.2639	0.8201	0.4167	0.4034	0.0966	0.0833
0.0165	33.6	840	1.3292	0.8125	0.3958	0.4167	0.0833	0.1042
0.0132	34.4	860	1.4465	0.8169	0.3971	0.4198	0.0802	0.1029
0.0205	35.2	880	1.3776	0.8201	0.4091	0.4110	0.0890	0.0909
0.0051	36.0	900	1.3503	0.8207	0.4116	0.4091	0.0909	0.0884
0.0013	36.8	920	1.4425	0.8213	0.4072	0.4141	0.0859	0.0928
0.0002	37.6	940	1.4601	0.8226	0.4028	0.4198	0.0802	0.0972
0.0007	38.4	960	1.5047	0.8213	0.4034	0.4179	0.0821	0.0966
0.0002	39.2	980	1.5297	0.8207	0.4072	0.4135	0.0865	0.0928
0.0002	40.0	1000	1.5409	0.8226	0.4122	0.4104	0.0896	0.0878
0.0002	40.8	1020	1.5452	0.8239	0.4097	0.4141	0.0859	0.0903
0.0001	41.6	1040	1.6013	0.8194	0.4097	0.4097	0.0903	0.0903
0.0001	42.4	1060	1.6177	0.8201	0.4104	0.4097	0.0903	0.0896
0.0001	43.2	1080	1.6290	0.8201	0.4104	0.4097	0.0903	0.0896
0.0001	44.0	1100	1.6365	0.8201	0.4097	0.4104	0.0896	0.0903
0.0001	44.8	1120	1.6425	0.8194	0.4097	0.4097	0.0903	0.0903
0.0142	45.6	1140	1.6429	0.8220	0.4110	0.4110	0.0890	0.0890
0.0001	46.4	1160	1.6080	0.8239	0.4135	0.4104	0.0896	0.0865
0.0001	47.2	1180	1.6113	0.8239	0.4135	0.4104	0.0896	0.0865
0.0001	48.0	1200	1.6046	0.8239	0.4129	0.4110	0.0890	0.0871
0.0001	48.8	1220	1.6066	0.8239	0.4135	0.4104	0.0896	0.0865
0.0001	49.6	1240	1.6133	0.8232	0.4129	0.4104	0.0896	0.0871

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1
• Datasets 2.10.1
• Tokenizers 0.13.2