scenario-non-kd-from-post-finetune-div-2-data-indolem_sentiment-model-haryoaw-sc

This model is a fine-tuned version of haryoaw/scenario-normal-finetune-clf-data-indolem_sentiment-model-xlm-roberta-base on the indolem_sentiment dataset. It achieves the following results on the evaluation set:

• Loss: 1.0411
• Accuracy: 0.8647
• F1: 0.7823

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6969

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	0.88	100	0.6692	0.7594	0.6962
No log	1.75	200	0.3987	0.8195	0.7429
No log	2.63	300	0.7587	0.8170	0.7591
No log	3.51	400	0.5380	0.8521	0.7592
0.2364	4.39	500	0.9505	0.8396	0.7576
0.2364	5.26	600	0.7528	0.8571	0.7692
0.2364	6.14	700	0.7176	0.8571	0.7511
0.2364	7.02	800	0.6990	0.8446	0.7395
0.2364	7.89	900	0.6388	0.8672	0.7745
0.0693	8.77	1000	0.5099	0.8596	0.7647
0.0693	9.65	1100	0.8001	0.8471	0.7608
0.0693	10.53	1200	0.6682	0.8847	0.8115
0.0693	11.4	1300	0.8207	0.8797	0.8140
0.0693	12.28	1400	0.9404	0.8596	0.7522
0.0412	13.16	1500	0.7976	0.8596	0.7586
0.0412	14.04	1600	0.8785	0.8672	0.7854
0.0412	14.91	1700	0.8712	0.8571	0.7532
0.0412	15.79	1800	1.0122	0.8596	0.7477
0.0412	16.67	1900	0.7452	0.8747	0.7899
0.0303	17.54	2000	1.0122	0.8622	0.7791
0.0303	18.42	2100	0.8764	0.8647	0.7611
0.0303	19.3	2200	0.8529	0.8697	0.7570
0.0303	20.18	2300	0.8582	0.8847	0.8034
0.0303	21.05	2400	1.0437	0.8571	0.7782
0.0204	21.93	2500	0.9941	0.8647	0.7632
0.0204	22.81	2600	1.0960	0.8546	0.7411
0.0204	23.68	2700	0.7326	0.8697	0.7615
0.0204	24.56	2800	1.0411	0.8647	0.7823

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1
• Datasets 2.14.5
• Tokenizers 0.13.3