distilroberta-propaganda-2class

This model is a fine-tuned version of distilroberta-base on the QCRI propaganda dataset.

It achieves the following results on the evaluation set:

• Loss: 0.5087
• Acc: 0.7424

Training and evaluation data

Training data is the 19-class QCRI propaganda data, with all propaganda classes collapsed to a single catch-all 'prop' class.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 12345
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 16
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Acc
0.5737	1.0	493	0.5998	0.6515
0.4954	2.0	986	0.5530	0.7080
0.4774	3.0	1479	0.5331	0.7258
0.4846	4.0	1972	0.5247	0.7339
0.4749	5.0	2465	0.5392	0.7199
0.502	6.0	2958	0.5124	0.7466
0.457	7.0	3451	0.5167	0.7432
0.4899	8.0	3944	0.5160	0.7428
0.4833	9.0	4437	0.5280	0.7339
0.5114	10.0	4930	0.5112	0.7436
0.4419	11.0	5423	0.5060	0.7525
0.4743	12.0	5916	0.5031	0.7547
0.4597	13.0	6409	0.5043	0.7517
0.4861	14.0	6902	0.5055	0.7487
0.499	15.0	7395	0.5091	0.7419
0.501	16.0	7888	0.5037	0.7521
0.4659	17.0	8381	0.5087	0.7424

Framework versions

• Transformers 4.11.2
• Pytorch 1.7.1
• Datasets 1.11.0
• Tokenizers 0.10.3