Clickbait1
This model is a fine-tuned version of microsoft/Multilingual-MiniLM-L12-H384 on the Webis-Clickbait-17 dataset. It achieves the following results on the evaluation set:
- Loss: 0.0257
Model description
MiniLM is a distilled model from the paper "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers".
We fine tune this model to evaluate (regression) the clickbait level of title news.
Intended uses & limitations
Model looks like the model described in the paper Predicting Clickbait Strength in Online Social Media by Indurthi Vijayasaradhi, Syed Bakhtiyar, Gupta Manish, Varma Vasudeva.
The model was trained with english titles.
Training and evaluation data
We trained the model with the official training data for the chalenge (clickbait17-train-170630.zip (894 MiB, 19538 posts), plus another set that was just available after the end of the challenge (clickbait17-train-170331.zip (157 MiB, 2459 posts).
Training procedure
Code can be find in Github.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
No log | 0.05 | 50 | 0.0571 |
No log | 0.09 | 100 | 0.0448 |
No log | 0.14 | 150 | 0.0391 |
No log | 0.18 | 200 | 0.0326 |
No log | 0.23 | 250 | 0.0343 |
No log | 0.27 | 300 | 0.0343 |
No log | 0.32 | 350 | 0.0343 |
No log | 0.36 | 400 | 0.0346 |
No log | 0.41 | 450 | 0.0343 |
0.0388 | 0.46 | 500 | 0.0297 |
0.0388 | 0.5 | 550 | 0.0293 |
0.0388 | 0.55 | 600 | 0.0301 |
0.0388 | 0.59 | 650 | 0.0290 |
0.0388 | 0.64 | 700 | 0.0326 |
0.0388 | 0.68 | 750 | 0.0285 |
0.0388 | 0.73 | 800 | 0.0285 |
0.0388 | 0.77 | 850 | 0.0275 |
0.0388 | 0.82 | 900 | 0.0314 |
0.0388 | 0.87 | 950 | 0.0309 |
0.0297 | 0.91 | 1000 | 0.0277 |
0.0297 | 0.96 | 1050 | 0.0281 |
0.0297 | 1.0 | 1100 | 0.0273 |
0.0297 | 1.05 | 1150 | 0.0270 |
0.0297 | 1.09 | 1200 | 0.0291 |
0.0297 | 1.14 | 1250 | 0.0293 |
0.0297 | 1.18 | 1300 | 0.0269 |
0.0297 | 1.23 | 1350 | 0.0276 |
0.0297 | 1.28 | 1400 | 0.0279 |
0.0297 | 1.32 | 1450 | 0.0267 |
0.0265 | 1.37 | 1500 | 0.0270 |
0.0265 | 1.41 | 1550 | 0.0300 |
0.0265 | 1.46 | 1600 | 0.0274 |
0.0265 | 1.5 | 1650 | 0.0274 |
0.0265 | 1.55 | 1700 | 0.0266 |
0.0265 | 1.59 | 1750 | 0.0267 |
0.0265 | 1.64 | 1800 | 0.0267 |
0.0265 | 1.68 | 1850 | 0.0280 |
0.0265 | 1.73 | 1900 | 0.0274 |
0.0265 | 1.78 | 1950 | 0.0272 |
0.025 | 1.82 | 2000 | 0.0261 |
0.025 | 1.87 | 2050 | 0.0268 |
0.025 | 1.91 | 2100 | 0.0268 |
0.025 | 1.96 | 2150 | 0.0259 |
0.025 | 2.0 | 2200 | 0.0257 |
0.025 | 2.05 | 2250 | 0.0260 |
0.025 | 2.09 | 2300 | 0.0263 |
0.025 | 2.14 | 2350 | 0.0262 |
0.025 | 2.19 | 2400 | 0.0269 |
0.025 | 2.23 | 2450 | 0.0262 |
0.0223 | 2.28 | 2500 | 0.0262 |
0.0223 | 2.32 | 2550 | 0.0267 |
0.0223 | 2.37 | 2600 | 0.0260 |
0.0223 | 2.41 | 2650 | 0.0260 |
0.0223 | 2.46 | 2700 | 0.0259 |
Framework versions
- Transformers 4.18.0
- Pytorch 1.11.0a0+17540c5
- Datasets 2.1.0
- Tokenizers 0.12.1
- Downloads last month
- 114