refusal-classifier

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3054
• F1: 1.0
• Roc Auc: 1.0
• Accuracy: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 384
• eval_batch_size: 384
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 2
• label_smoothing_factor: 0.05

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc Auc	Accuracy
No log	0.0076	100	2.4712	0.1582	0.7093	0.2062
No log	0.0151	200	2.0531	0.4984	0.9014	0.5231
No log	0.0227	300	1.3834	0.6824	0.9524	0.6938
No log	0.0302	400	1.0466	0.7584	0.9684	0.7623
1.7761	0.0378	500	0.8311	0.8162	0.9822	0.8177
1.7761	0.0454	600	0.7246	0.8474	0.9878	0.8469
1.7761	0.0529	700	0.6630	0.8680	0.9908	0.8677
1.7761	0.0605	800	0.6067	0.8865	0.9933	0.8862
1.7761	0.0681	900	0.5693	0.8948	0.9948	0.8946
0.6613	0.0756	1000	0.5578	0.8993	0.9957	0.8992
0.6613	0.0832	1100	0.5338	0.9103	0.9963	0.9108
0.6613	0.0907	1200	0.5117	0.9154	0.9971	0.9154
0.6613	0.0983	1300	0.5015	0.9253	0.9972	0.9254
0.6613	0.1059	1400	0.4856	0.9277	0.9977	0.9277
0.5142	0.1134	1500	0.4666	0.9448	0.9980	0.9446
0.5142	0.1210	1600	0.4659	0.9391	0.9981	0.9392
0.5142	0.1285	1700	0.4543	0.9399	0.9984	0.94
0.5142	0.1361	1800	0.4557	0.9392	0.9985	0.9392
0.5142	0.1437	1900	0.4472	0.9454	0.9985	0.9454
0.4593	0.1512	2000	0.4354	0.9523	0.9987	0.9523
0.4593	0.1588	2100	0.4317	0.9507	0.9989	0.9508
0.4593	0.1664	2200	0.4296	0.9479	0.9990	0.9477
0.4593	0.1739	2300	0.4335	0.9479	0.9989	0.9477
0.4593	0.1815	2400	0.4237	0.9525	0.9991	0.9523
0.4301	0.1890	2500	0.4158	0.9592	0.9991	0.9592
0.4301	0.1966	2600	0.4071	0.9537	0.9993	0.9538
0.4301	0.2042	2700	0.4111	0.9555	0.9993	0.9554
0.4301	0.2117	2800	0.3970	0.9623	0.9994	0.9623
0.4301	0.2193	2900	0.3992	0.9559	0.9994	0.9562
0.4091	0.2268	3000	0.3954	0.9631	0.9994	0.9631
0.4091	0.2344	3100	0.3868	0.9676	0.9996	0.9677
0.4091	0.2420	3200	0.3947	0.9615	0.9995	0.9615
0.4091	0.2495	3300	0.3935	0.9653	0.9994	0.9654
0.4091	0.2571	3400	0.3843	0.9661	0.9995	0.9662
0.3959	0.2647	3500	0.3811	0.9653	0.9995	0.9654
0.3959	0.2722	3600	0.3813	0.9654	0.9996	0.9654
0.3959	0.2798	3700	0.3870	0.9646	0.9995	0.9646
0.3959	0.2873	3800	0.3768	0.9692	0.9996	0.9692
0.3959	0.2949	3900	0.3825	0.9669	0.9996	0.9669
0.3846	0.3025	4000	0.3741	0.9677	0.9996	0.9677
0.3846	0.3100	4100	0.3739	0.9716	0.9996	0.9715
0.3846	0.3176	4200	0.3645	0.9769	0.9997	0.9769
0.3846	0.3251	4300	0.3651	0.9739	0.9997	0.9738
0.3846	0.3327	4400	0.3644	0.9761	0.9997	0.9762
0.3752	0.3403	4500	0.3682	0.9753	0.9997	0.9754
0.3752	0.3478	4600	0.3634	0.9731	0.9998	0.9731
0.3752	0.3554	4700	0.3577	0.9777	0.9998	0.9777
0.3752	0.3629	4800	0.3575	0.9777	0.9998	0.9777
0.3752	0.3705	4900	0.3605	0.9785	0.9998	0.9785
0.3685	0.3781	5000	0.3569	0.9762	0.9998	0.9762
0.3685	0.3856	5100	0.3546	0.9769	0.9999	0.9769
0.3685	0.3932	5200	0.3580	0.9792	0.9998	0.9792
0.3685	0.4008	5300	0.3596	0.9738	0.9998	0.9738
0.3685	0.4083	5400	0.3554	0.9777	0.9999	0.9777
0.3616	0.4159	5500	0.3491	0.9839	0.9999	0.9838
0.3616	0.4234	5600	0.3487	0.9807	0.9999	0.9808
0.3616	0.4310	5700	0.3481	0.9831	0.9999	0.9831
0.3616	0.4386	5800	0.3437	0.9839	0.9999	0.9838
0.3616	0.4461	5900	0.3467	0.9823	0.9999	0.9823
0.3556	0.4537	6000	0.3411	0.9823	0.9999	0.9823
0.3556	0.4612	6100	0.3422	0.9846	0.9999	0.9846
0.3556	0.4688	6200	0.3482	0.9823	0.9999	0.9823
0.3556	0.4764	6300	0.3406	0.9839	0.9999	0.9838
0.3556	0.4839	6400	0.3368	0.9854	0.9999	0.9854
0.3513	0.4915	6500	0.3416	0.9800	0.9999	0.98
0.3513	0.4991	6600	0.3382	0.9862	0.9999	0.9862
0.3513	0.5066	6700	0.3361	0.9869	1.0000	0.9869
0.3513	0.5142	6800	0.3329	0.9854	1.0000	0.9854
0.3513	0.5217	6900	0.3303	0.9877	1.0000	0.9877
0.3487	0.5293	7000	0.3393	0.9838	1.0000	0.9838
0.3487	0.5369	7100	0.3298	0.9892	1.0000	0.9892
0.3487	0.5444	7200	0.3319	0.9831	1.0000	0.9831
0.3487	0.5520	7300	0.3365	0.9815	1.0000	0.9815
0.3487	0.5595	7400	0.3331	0.9908	1.0000	0.9908
0.3456	0.5671	7500	0.3406	0.9838	0.9999	0.9838
0.3456	0.5747	7600	0.3320	0.9877	1.0000	0.9877
0.3456	0.5822	7700	0.3341	0.9877	1.0000	0.9877
0.3456	0.5898	7800	0.3296	0.9885	1.0000	0.9885
0.3456	0.5974	7900	0.3319	0.9884	1.0000	0.9885
0.3419	0.6049	8000	0.3282	0.9908	1.0000	0.9908
0.3419	0.6125	8100	0.3272	0.9908	1.0000	0.9908
0.3419	0.6200	8200	0.3244	0.9923	1.0000	0.9923
0.3419	0.6276	8300	0.3304	0.9860	1.0000	0.9862
0.3419	0.6352	8400	0.3301	0.9877	1.0000	0.9877
0.3392	0.6427	8500	0.3240	0.9923	1.0000	0.9923
0.3392	0.6503	8600	0.3229	0.9908	1.0000	0.9908
0.3392	0.6578	8700	0.3247	0.9923	1.0000	0.9923
0.3392	0.6654	8800	0.3254	0.9900	1.0000	0.99
0.3392	0.6730	8900	0.3248	0.9900	1.0000	0.99
0.3374	0.6805	9000	0.3235	0.9900	1.0000	0.99
0.3374	0.6881	9100	0.3250	0.9915	1.0000	0.9915
0.3374	0.6957	9200	0.3232	0.9938	1.0000	0.9938
0.3374	0.7032	9300	0.3236	0.9915	1.0000	0.9915
0.3374	0.7108	9400	0.3237	0.9908	1.0000	0.9908
0.3352	0.7183	9500	0.3241	0.9908	1.0000	0.9908
0.3352	0.7259	9600	0.3160	0.9954	1.0000	0.9954
0.3352	0.7335	9700	0.3185	0.9938	1.0000	0.9938
0.3352	0.7410	9800	0.3187	0.9939	1.0000	0.9938
0.3352	0.7486	9900	0.3247	0.9892	1.0000	0.9892
0.3333	0.7561	10000	0.3174	0.9939	1.0000	0.9938
0.3333	0.7637	10100	0.3207	0.9938	1.0000	0.9938
0.3333	0.7713	10200	0.3237	0.9923	1.0000	0.9923
0.3333	0.7788	10300	0.3197	0.9938	1.0000	0.9938
0.3333	0.7864	10400	0.3194	0.9931	1.0000	0.9931
0.3309	0.7940	10500	0.3209	0.9923	1.0000	0.9923
0.3309	0.8015	10600	0.3178	0.9946	1.0000	0.9946
0.3309	0.8091	10700	0.3159	0.9946	1.0000	0.9946
0.3309	0.8166	10800	0.3138	0.9954	1.0000	0.9954
0.3309	0.8242	10900	0.3142	0.9954	1.0000	0.9954
0.3298	0.8318	11000	0.3131	0.9954	1.0000	0.9954
0.3298	0.8393	11100	0.3171	0.9939	1.0000	0.9938
0.3298	0.8469	11200	0.3167	0.9946	1.0000	0.9946
0.3298	0.8544	11300	0.3182	0.9938	1.0000	0.9938
0.3298	0.8620	11400	0.3184	0.9923	1.0000	0.9923
0.3287	0.8696	11500	0.3173	0.9962	1.0000	0.9962
0.3287	0.8771	11600	0.3195	0.9946	1.0000	0.9946
0.3287	0.8847	11700	0.3142	0.9962	1.0000	0.9962
0.3287	0.8922	11800	0.3148	0.9946	1.0000	0.9946
0.3287	0.8998	11900	0.3139	0.9962	1.0000	0.9962
0.3265	0.9074	12000	0.3162	0.9954	1.0000	0.9954
0.3265	0.9149	12100	0.3129	0.9962	1.0000	0.9962
0.3265	0.9225	12200	0.3176	0.9931	1.0000	0.9931
0.3265	0.9301	12300	0.3221	0.9931	0.9999	0.9931
0.3265	0.9376	12400	0.3168	0.9962	1.0000	0.9962
0.3257	0.9452	12500	0.3129	0.9969	1.0000	0.9969
0.3257	0.9527	12600	0.3140	0.9954	1.0000	0.9954
0.3257	0.9603	12700	0.3160	0.9954	1.0000	0.9954
0.3257	0.9679	12800	0.3133	0.9954	1.0000	0.9954
0.3257	0.9754	12900	0.3116	0.9977	1.0000	0.9977
0.3242	0.9830	13000	0.3091	0.9985	1.0000	0.9985
0.3242	0.9905	13100	0.3124	0.9969	1.0000	0.9969
0.3242	0.9981	13200	0.3148	0.9946	1.0000	0.9946
0.3242	1.0057	13300	0.3191	0.9946	1.0000	0.9946
0.3242	1.0132	13400	0.3135	0.9946	1.0000	0.9946
0.3213	1.0208	13500	0.3191	0.9915	1.0000	0.9915
0.3213	1.0284	13600	0.3145	0.9939	1.0000	0.9938
0.3213	1.0359	13700	0.3135	0.9946	1.0000	0.9946
0.3213	1.0435	13800	0.3126	0.9931	1.0	0.9931
0.3213	1.0510	13900	0.3125	0.9962	1.0000	0.9962
0.32	1.0586	14000	0.3129	0.9961	1.0000	0.9962
0.32	1.0662	14100	0.3099	0.9969	1.0000	0.9969
0.32	1.0737	14200	0.3113	0.9977	1.0000	0.9977
0.32	1.0813	14300	0.3111	0.9969	1.0000	0.9969
0.32	1.0888	14400	0.3074	0.9985	1.0	0.9985
0.3189	1.0964	14500	0.3088	0.9969	1.0000	0.9969
0.3189	1.1040	14600	0.3087	0.9977	1.0	0.9977
0.3189	1.1115	14700	0.3110	0.9962	1.0000	0.9962
0.3189	1.1191	14800	0.3097	0.9961	1.0000	0.9962
0.3189	1.1267	14900	0.3086	0.9977	1.0	0.9977
0.3183	1.1342	15000	0.3087	0.9962	1.0000	0.9962
0.3183	1.1418	15100	0.3133	0.9946	1.0000	0.9946
0.3183	1.1493	15200	0.3098	0.9969	1.0	0.9969
0.3183	1.1569	15300	0.3111	0.9962	1.0000	0.9962
0.3183	1.1645	15400	0.3149	0.9946	1.0000	0.9946
0.3172	1.1720	15500	0.3143	0.9954	1.0000	0.9954
0.3172	1.1796	15600	0.3090	0.9969	1.0	0.9969
0.3172	1.1871	15700	0.3074	0.9977	1.0000	0.9977
0.3172	1.1947	15800	0.3116	0.9969	1.0000	0.9969
0.3172	1.2023	15900	0.3113	0.9969	1.0000	0.9969
0.3163	1.2098	16000	0.3101	0.9977	1.0	0.9977
0.3163	1.2174	16100	0.3081	0.9977	1.0000	0.9977
0.3163	1.2250	16200	0.3163	0.9938	0.9999	0.9938
0.3163	1.2325	16300	0.3141	0.9946	1.0000	0.9946
0.3163	1.2401	16400	0.3134	0.9946	1.0000	0.9946
0.3157	1.2476	16500	0.3131	0.9954	1.0000	0.9954
0.3157	1.2552	16600	0.3168	0.9946	1.0000	0.9946
0.3157	1.2628	16700	0.3112	0.9962	1.0000	0.9962
0.3157	1.2703	16800	0.3158	0.9962	1.0000	0.9962
0.3157	1.2779	16900	0.3118	0.9969	1.0000	0.9969
0.3156	1.2854	17000	0.3115	0.9969	1.0000	0.9969
0.3156	1.2930	17100	0.3091	0.9969	1.0000	0.9969
0.3156	1.3006	17200	0.3087	0.9969	1.0000	0.9969
0.3156	1.3081	17300	0.3087	0.9985	1.0000	0.9985
0.3156	1.3157	17400	0.3095	0.9969	1.0000	0.9969
0.315	1.3233	17500	0.3075	0.9985	1.0000	0.9985
0.315	1.3308	17600	0.3087	0.9969	1.0000	0.9969
0.315	1.3384	17700	0.3095	0.9985	1.0000	0.9985
0.315	1.3459	17800	0.3082	0.9977	1.0	0.9977
0.315	1.3535	17900	0.3074	0.9977	1.0	0.9977
0.3145	1.3611	18000	0.3086	0.9977	1.0000	0.9977
0.3145	1.3686	18100	0.3069	0.9969	1.0000	0.9969
0.3145	1.3762	18200	0.3092	0.9977	1.0000	0.9977
0.3145	1.3837	18300	0.3081	0.9969	1.0000	0.9969
0.3145	1.3913	18400	0.3129	0.9954	1.0000	0.9954
0.314	1.3989	18500	0.3086	0.9977	1.0	0.9977
0.314	1.4064	18600	0.3093	0.9962	1.0000	0.9962
0.314	1.4140	18700	0.3085	0.9969	1.0000	0.9969
0.314	1.4216	18800	0.3081	0.9977	1.0	0.9977
0.314	1.4291	18900	0.3100	0.9962	1.0	0.9962
0.3138	1.4367	19000	0.3067	0.9985	1.0	0.9985
0.3138	1.4442	19100	0.3080	0.9969	1.0	0.9969
0.3138	1.4518	19200	0.3080	0.9977	1.0000	0.9977
0.3138	1.4594	19300	0.3116	0.9954	1.0000	0.9954
0.3138	1.4669	19400	0.3117	0.9946	1.0000	0.9946
0.313	1.4745	19500	0.3106	0.9962	1.0000	0.9962
0.313	1.4820	19600	0.3091	0.9969	1.0000	0.9969
0.313	1.4896	19700	0.3086	0.9977	1.0	0.9977
0.313	1.4972	19800	0.3071	0.9977	1.0	0.9977
0.313	1.5047	19900	0.3081	0.9977	1.0000	0.9977
0.3131	1.5123	20000	0.3090	0.9961	1.0000	0.9962
0.3131	1.5198	20100	0.3071	0.9977	1.0000	0.9977
0.3131	1.5274	20200	0.3089	0.9969	1.0000	0.9969
0.3131	1.5350	20300	0.3092	0.9969	1.0000	0.9969
0.3131	1.5425	20400	0.3064	0.9985	1.0	0.9985
0.3123	1.5501	20500	0.3113	0.9961	1.0000	0.9962
0.3123	1.5577	20600	0.3096	0.9954	1.0000	0.9954
0.3123	1.5652	20700	0.3078	0.9977	1.0000	0.9977
0.3123	1.5728	20800	0.3093	0.9969	1.0000	0.9969
0.3123	1.5803	20900	0.3077	0.9985	1.0000	0.9985
0.3119	1.5879	21000	0.3073	0.9977	1.0000	0.9977
0.3119	1.5955	21100	0.3053	0.9992	1.0	0.9992
0.3119	1.6030	21200	0.3066	0.9985	1.0	0.9985
0.3119	1.6106	21300	0.3085	0.9985	1.0	0.9985
0.3119	1.6181	21400	0.3082	0.9969	1.0000	0.9969
0.3114	1.6257	21500	0.3079	0.9977	1.0000	0.9977
0.3114	1.6333	21600	0.3063	0.9977	1.0	0.9977
0.3114	1.6408	21700	0.3065	0.9985	1.0	0.9985
0.3114	1.6484	21800	0.3083	0.9969	1.0	0.9969
0.3114	1.6560	21900	0.3066	0.9977	1.0	0.9977
0.3118	1.6635	22000	0.3071	0.9977	1.0	0.9977
0.3118	1.6711	22100	0.3075	0.9977	1.0	0.9977
0.3118	1.6786	22200	0.3076	0.9969	1.0	0.9969
0.3118	1.6862	22300	0.3070	0.9977	1.0	0.9977
0.3118	1.6938	22400	0.3069	0.9985	1.0	0.9985
0.3113	1.7013	22500	0.3078	0.9969	1.0	0.9969
0.3113	1.7089	22600	0.3063	0.9985	1.0	0.9985
0.3113	1.7164	22700	0.3057	0.9992	1.0	0.9992
0.3113	1.7240	22800	0.3070	0.9977	1.0	0.9977
0.3113	1.7316	22900	0.3067	0.9977	1.0	0.9977
0.3107	1.7391	23000	0.3057	0.9985	1.0	0.9985
0.3107	1.7467	23100	0.3061	0.9992	1.0	0.9992
0.3107	1.7543	23200	0.3059	0.9992	1.0	0.9992
0.3107	1.7618	23300	0.3055	0.9992	1.0	0.9992
0.3107	1.7694	23400	0.3054	1.0	1.0	1.0
0.3114	1.7769	23500	0.3055	0.9985	1.0	0.9985
0.3114	1.7845	23600	0.3067	0.9985	1.0	0.9985
0.3114	1.7921	23700	0.3063	0.9985	1.0	0.9985
0.3114	1.7996	23800	0.3053	0.9992	1.0	0.9992
0.3114	1.8072	23900	0.3055	0.9985	1.0	0.9985
0.3105	1.8147	24000	0.3057	0.9992	1.0	0.9992
0.3105	1.8223	24100	0.3053	0.9992	1.0	0.9992
0.3105	1.8299	24200	0.3065	0.9992	1.0	0.9992
0.3105	1.8374	24300	0.3064	0.9977	1.0	0.9977
0.3105	1.8450	24400	0.3066	0.9985	1.0	0.9985
0.3106	1.8526	24500	0.3070	0.9977	1.0000	0.9977
0.3106	1.8601	24600	0.3054	0.9992	1.0	0.9992
0.3106	1.8677	24700	0.3062	0.9985	1.0	0.9985
0.3106	1.8752	24800	0.3060	0.9992	1.0	0.9992
0.3106	1.8828	24900	0.3063	0.9992	1.0000	0.9992
0.3101	1.8904	25000	0.3058	0.9992	1.0	0.9992
0.3101	1.8979	25100	0.3062	0.9985	1.0	0.9985
0.3101	1.9055	25200	0.3061	0.9992	1.0000	0.9992
0.3101	1.9130	25300	0.3064	0.9985	1.0000	0.9985
0.3101	1.9206	25400	0.3057	0.9992	1.0	0.9992
0.3102	1.9282	25500	0.3060	0.9992	1.0	0.9992
0.3102	1.9357	25600	0.3059	0.9985	1.0	0.9985
0.3102	1.9433	25700	0.3053	0.9992	1.0	0.9992
0.3102	1.9509	25800	0.3049	1.0	1.0	1.0
0.3102	1.9584	25900	0.3053	0.9992	1.0	0.9992
0.3098	1.9660	26000	0.3052	0.9992	1.0	0.9992
0.3098	1.9735	26100	0.3051	1.0	1.0	1.0
0.3098	1.9811	26200	0.3052	0.9992	1.0	0.9992
0.3098	1.9887	26300	0.3051	0.9992	1.0	0.9992
0.3098	1.9962	26400	0.3051	0.9992	1.0	0.9992

Framework versions

• Transformers 4.46.3
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.3