Byline Detection
Model description
byline_detection is a fine-tuned DistilBERT token classification model, which tags bylines and datelines in news articles.
It is trained to deal with OCR noise.
Intended uses
You can use this model with Transformers pipeline for NER.
from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline
tokenizer = AutoTokenizer.from_pretrained("dell-research-harvard/byline-detection")
model = AutoModelForTokenClassification.from_pretrained("dell-research-harvard/byline-detection")
nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "NEW ORLEANS, (UP) — The Roman Catholic Church, through its leaders in the United States today appealed "
ner_results = nlp(example)
print(ner_results)
Limitations and bias
This model was trained on historical news and may reflect biases from a specific period of time. It may also not generalise well to other setting. Additionally, the model occasionally tags subword tokens as entities and post-processing of results may be necessary to handle those cases.
Training data
This model was fine-tuned on historical English-language news that had been OCRd from American newspapers.
# of training examples per entity type
Dataset | Count |
---|---|
Train | 1,392 |
Dev | 464 |
Test | 464 |
Training procedure
The data was used to fine-tune a DistilBERT model at a learning rate of 2e−5 with a batch size of 16 for 25 epochs.
Eval results
Statistic | Result |
---|---|
F1 | 0.96 |
Notes
This model card was influence by that of dslim/bert-base-NER
Citation
If you use this model, you can cite the following paper:
@misc{silcock2024newswirelargescalestructureddatabase,
title={Newswire: A Large-Scale Structured Database of a Century of Historical News},
author={Emily Silcock and Abhishek Arora and Luca D'Amico-Wong and Melissa Dell},
year={2024},
eprint={2406.09490},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2406.09490},
}
Applications
We applied this model to a century of historical news articles, and georeference the bylines. You can see them all in the NEWSWIRE dataset.
- Downloads last month
- 9