Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Punctuator for Uncased English
|
| 2 |
+
|
| 3 |
+
The model is fine-tuned based on `DistilBertForTokenClassification` for adding punctuations to plain text (uncased English)
|
| 4 |
+
|
| 5 |
+
## Usage
|
| 6 |
+
|
| 7 |
+
```python
|
| 8 |
+
from transformers import DistilBertForTokenClassification, DistilBertTokenizerFast
|
| 9 |
+
|
| 10 |
+
model = DistilBertForTokenClassification.from_pretrained("Qishuai/distilbert_punctuator_en")
|
| 11 |
+
tokenizer = DistilBertTokenizerFast.from_pretrained("Qishuai/distilbert_punctuator_en")
|
| 12 |
+
```
|
| 13 |
+
|
| 14 |
+
## Model Overview
|
| 15 |
+
|
| 16 |
+
### Training data
|
| 17 |
+
Combination of following three dataset:
|
| 18 |
+
|
| 19 |
+
- BBC news: From BBC news website corresponding to stories in five topical areas from 2004-2005. [Reference](https://www.kaggle.com/hgultekin/bbcnewsarchive)
|
| 20 |
+
- News articles: 20000 samples of short news articles scraped from Hindu, Indian times and Guardian between Feb 2017 and Aug 2017 [Reference](https://www.kaggle.com/sunnysai12345/news-summary?select=news_summary_more.csv)
|
| 21 |
+
- Ted talks: transcripts of over 4,000 TED talks between 2004 and 2019 [Reference](https://www.kaggle.com/miguelcorraljr/ted-ultimate-dataset)
|
| 22 |
+
|
| 23 |
+
### Model Performance
|
| 24 |
+
- Validation with 500 samples of dataset scraped from https://www.thenews.com.pk website. [Reference](https://www.kaggle.com/asad1m9a9h6mood/news-articles)
|
| 25 |
+
- Metrics Report:
|
| 26 |
+
|
| 27 |
+
| | precision | recall | f1-score | support |
|
| 28 |
+
|:--------------:|:---------:|:------:|:--------:|:-------:|
|
| 29 |
+
| COMMA | 0.66 | 0.55 | 0.60 | 7064 |
|
| 30 |
+
| EXLAMATIONMARK | 1.00 | 0.00 | 0.00 | 5 |
|
| 31 |
+
| PERIOD | 0.73 | 0.63 | 0.68 | 6573 |
|
| 32 |
+
| QUESTIONMARK | 0.54 | 0.41 | 0.47 | 17 |
|
| 33 |
+
| micro avg | 0.69 | 0.59 | 0.64 | 13659 |
|
| 34 |
+
| macro avg | 0.73 | 0.40 | 0.44 | 13659 |
|
| 35 |
+
| weighted avg | 0.69 | 0.59 | 0.64 | 13659 |
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
- Validation with 86 news ted talks of 2020 which are not included in training dataset [Reference](https://www.kaggle.com/thegupta/ted-talk)
|
| 39 |
+
- Metrics Report:
|
| 40 |
+
| | precision | recall | f1-score | support |
|
| 41 |
+
|:--------------:|:---------:|:------:|:--------:|:-------:|
|
| 42 |
+
| COMMA | 0.71 | 0.56 | 0.63 | 10712 |
|
| 43 |
+
| EXLAMATIONMARK | 0.45 | 0.07 | 0.12 | 75 |
|
| 44 |
+
| PERIOD | 0.75 | 0.65 | 0.70 | 7921 |
|
| 45 |
+
| QUESTIONMARK | 0.73 | 0.67 | 0.70 | 827 |
|
| 46 |
+
| micro avg | 0.73 | 0.60 | 0.66 | 19535 |
|
| 47 |
+
| macro avg | 0.66 | 0.49 | 0.53 | 19535 |
|
| 48 |
+
| weighted avg | 0.73 | 0.60 | 0.66 | 19535 |
|
| 49 |
+
|
| 50 |
+
|