scenario-kd-from-post-finetune-gold-silver-div-2-data-hate_speech_filipino-model

This model is a fine-tuned version of haryoaw/scenario-teacher-data-hate_speech_filipino-model-xlm-roberta-base on the hate_speech_filipino dataset. It achieves the following results on the evaluation set:

• Loss: 0.8676
• Accuracy: 0.7836
• F1: 0.7765

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6969

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.32	100	1.4538	0.7609	0.7566
No log	0.64	200	1.2488	0.7625	0.7594
No log	0.96	300	1.1488	0.7691	0.7658
No log	1.28	400	1.0930	0.7836	0.7666
1.2528	1.6	500	1.1216	0.7802	0.7576
1.2528	1.92	600	1.0502	0.7696	0.7658
1.2528	2.24	700	1.1185	0.7698	0.7729
1.2528	2.56	800	1.1002	0.7739	0.7621
1.2528	2.88	900	1.0090	0.7807	0.7662
0.8257	3.19	1000	1.0320	0.7836	0.7673
0.8257	3.51	1100	1.0162	0.7869	0.7673
0.8257	3.83	1200	0.9923	0.7880	0.7677
0.8257	4.15	1300	1.0061	0.7845	0.7653
0.8257	4.47	1400	0.9962	0.7878	0.7710
0.6686	4.79	1500	1.0640	0.7791	0.7774
0.6686	5.11	1600	0.9700	0.7864	0.7634
0.6686	5.43	1700	0.9535	0.7885	0.7680
0.6686	5.75	1800	0.9281	0.7961	0.7730
0.6686	6.07	1900	0.9843	0.7776	0.7754
0.5743	6.39	2000	0.9219	0.7937	0.7721
0.5743	6.71	2100	0.9237	0.7880	0.7760
0.5743	7.03	2200	0.9497	0.7899	0.7703
0.5743	7.35	2300	0.9365	0.7859	0.7747
0.5743	7.67	2400	1.0137	0.7663	0.7722
0.5272	7.99	2500	0.9682	0.7895	0.7678
0.5272	8.31	2600	0.9419	0.7843	0.7752
0.5272	8.63	2700	0.9684	0.7864	0.7792
0.5272	8.95	2800	0.9374	0.7935	0.7745
0.5272	9.27	2900	0.9877	0.7935	0.7790
0.4948	9.58	3000	0.9243	0.7859	0.7747
0.4948	9.9	3100	0.8968	0.7885	0.7710
0.4948	10.22	3200	0.8843	0.7845	0.7715
0.4948	10.54	3300	0.9175	0.7911	0.7738
0.4948	10.86	3400	0.9012	0.7932	0.7754
0.4658	11.18	3500	0.8939	0.7836	0.7739
0.4658	11.5	3600	0.9108	0.7819	0.7774
0.4658	11.82	3700	0.9651	0.7788	0.7771
0.4658	12.14	3800	0.8833	0.7949	0.7732
0.4658	12.46	3900	0.8860	0.7968	0.7777
0.4449	12.78	4000	0.8929	0.7873	0.7745
0.4449	13.1	4100	0.9065	0.7885	0.7824
0.4449	13.42	4200	0.9258	0.7802	0.7756
0.4449	13.74	4300	0.9231	0.7833	0.7818
0.4449	14.06	4400	0.8578	0.8001	0.7825
0.4249	14.38	4500	0.8835	0.7873	0.7838
0.4249	14.7	4600	0.9011	0.7909	0.7847
0.4249	15.02	4700	0.9035	0.7852	0.7816
0.4249	15.34	4800	0.8461	0.7956	0.7776
0.4249	15.65	4900	0.8402	0.7928	0.7835
0.4159	15.97	5000	0.8629	0.7888	0.7782
0.4159	16.29	5100	0.8737	0.7925	0.7737
0.4159	16.61	5200	0.8610	0.7916	0.7736
0.4159	16.93	5300	0.8533	0.7977	0.7819
0.4159	17.25	5400	0.8838	0.7906	0.7778
0.4032	17.57	5500	0.8602	0.7925	0.7793
0.4032	17.89	5600	0.8505	0.7942	0.7805
0.4032	18.21	5700	0.8540	0.7824	0.7782
0.4032	18.53	5800	0.9424	0.7916	0.7589
0.4032	18.85	5900	0.8829	0.7859	0.7796
0.3915	19.17	6000	0.8206	0.7975	0.7844
0.3915	19.49	6100	0.8676	0.7836	0.7765

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1
• Datasets 2.14.5
• Tokenizers 0.13.3