File size: 1,491 Bytes
9df7b95 2dce7b8 9df7b95 2dce7b8 9df7b95 2dce7b8 9df7b95 2dce7b8 9df7b95 233b549 2dce7b8 9df7b95 2dce7b8 9df7b95 2dce7b8 9df7b95 2dce7b8 9df7b95 2da0010 2dce7b8 9df7b95 2dce7b8 9df7b95 2dce7b8 2da0010 2dce7b8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
---
library_name: transformers
datasets:
- vector-institute/newsmediabias-plus
language:
- en
metrics:
- accuracy
- precision
- recall
- f1
base_model:
- google-bert/bert-base-uncased
pipeline_tag: text-classification
---
# BERT NMB+ (Disinformation Sequence Classification):
Classifies sentences as "Likely" or "Unlikely" biased/disinformation (max token len 128).
Fine-tuned BERT ([bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased)) on the `headline` and `text_label` fields in the [News Media Bias Plus Dataset](https://huggingface.co/datasets/vector-institute/newsmediabias-plus).
**This model was trained without weighted sampling, and the dataset contains 81.9% 'Likely' and 18.1% 'Unlikely' examples.** The same model trained with weighted sampling preformed better when evaluated by gpt-4o-mini as a judge and is available [here](https://huggingface.co/maximuspowers/nmbp-bert-headlines-balanced).
### Metics
*Evaluated on a 0.1 random sample of the NMB+ dataset, unseen during training*
- Accuracy: 0.7990
- Precision: 0.8096
- Recall: 0.9556
- F1 Score: 0.8766
## How to Use:
```python
from transformers import pipeline
classifier = pipeline("text-classification", model="maximuspowers/nmbp-bert-headlines")
result = classifier("He was a terrible politician.", top_k=2)
```
### Example Response:
```json
[
{
'label': 'Likely',
'score': 0.9967995882034302
},
{
'label': 'Unlikely',
'score': 0.003200419945642352
}
]
``` |