scenario-non-kd-from-scratch-data-indolem_sentiment-model-xlm-roberta-base

This model is a fine-tuned version of xlm-roberta-base on the indolem_sentiment dataset. It achieves the following results on the evaluation set:

• Loss: 1.0766
• Accuracy: 0.8346
• F1: 0.7130

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6969

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.88	100	0.6919	0.6792	0.6025
No log	1.75	200	0.6092	0.7619	0.3357
No log	2.63	300	0.5160	0.7669	0.6353
No log	3.51	400	0.9501	0.7093	0.6352
0.3526	4.39	500	0.7742	0.7794	0.6857
0.3526	5.26	600	0.8757	0.7895	0.6794
0.3526	6.14	700	0.6974	0.8195	0.6897
0.3526	7.02	800	1.0621	0.8221	0.6203
0.3526	7.89	900	1.5238	0.7694	0.6933
0.0789	8.77	1000	1.3006	0.7820	0.6969
0.0789	9.65	1100	1.1286	0.8246	0.7083
0.0789	10.53	1200	0.8404	0.8346	0.736
0.0789	11.4	1300	1.1477	0.8321	0.7100
0.0789	12.28	1400	1.0099	0.8421	0.7042
0.0257	13.16	1500	1.1806	0.8371	0.7032
0.0257	14.04	1600	1.1848	0.8170	0.6332
0.0257	14.91	1700	1.1102	0.8321	0.6996
0.0257	15.79	1800	1.0778	0.8271	0.6667
0.0257	16.67	1900	1.2473	0.8296	0.6634
0.022	17.54	2000	1.2310	0.8221	0.7054
0.022	18.42	2100	1.0758	0.8346	0.6887
0.022	19.3	2200	1.4127	0.8070	0.7220
0.022	20.18	2300	1.0067	0.8246	0.6789
0.022	21.05	2400	1.2720	0.8045	0.6829
0.0209	21.93	2500	1.2202	0.8296	0.6731
0.0209	22.81	2600	1.3424	0.8221	0.6468
0.0209	23.68	2700	1.0766	0.8346	0.7130

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1
• Datasets 2.14.5
• Tokenizers 0.13.3