# Punctuator for Uncased English

The model is fine-tuned based on `DistilBertForTokenClassification` for adding punctuations to plain text (uncased English)

## Usage

```python
from transformers import DistilBertForTokenClassification, DistilBertTokenizerFast

model = DistilBertForTokenClassification.from_pretrained("Qishuai/distilbert_punctuator_en")
tokenizer = DistilBertTokenizerFast.from_pretrained("Qishuai/distilbert_punctuator_en")
```

## Model Overview

### Training data
Combination of following three dataset:

- BBC news: From BBC news website corresponding to stories in five topical areas from 2004-2005. [Reference](https://www.kaggle.com/hgultekin/bbcnewsarchive)
- News articles: 20000 samples of short news articles scraped from Hindu, Indian times and Guardian between Feb 2017 and Aug 2017 [Reference](https://www.kaggle.com/sunnysai12345/news-summary?select=news_summary_more.csv)
- Ted talks: transcripts of over 4,000 TED talks between 2004 and 2019 [Reference](https://www.kaggle.com/miguelcorraljr/ted-ultimate-dataset)

### Model Performance
- Validation with 500 samples of dataset scraped from https://www.thenews.com.pk website. [Reference](https://www.kaggle.com/asad1m9a9h6mood/news-articles)
- Metrics Report:

    |                | precision | recall | f1-score | support |
    |:--------------:|:---------:|:------:|:--------:|:-------:|
    |      COMMA     |    0.66   |  0.55  |   0.60   |   7064  |
    | EXLAMATIONMARK |    1.00   |  0.00  |   0.00   |    5    |
    |     PERIOD     |    0.73   |  0.63  |   0.68   |   6573  |
    |  QUESTIONMARK  |    0.54   |  0.41  |   0.47   |    17   |
    |    micro avg   |    0.69   |  0.59  |   0.64   |  13659  |
    |    macro avg   |    0.73   |  0.40  |   0.44   |  13659  |
    |  weighted avg  |    0.69   |  0.59  |   0.64   |  13659  |


- Validation with 86 news ted talks of 2020 which are not included in training dataset [Reference](https://www.kaggle.com/thegupta/ted-talk)
- Metrics Report:
    |                | precision | recall | f1-score | support |
    |:--------------:|:---------:|:------:|:--------:|:-------:|
    |      COMMA     |    0.71   |  0.56  |   0.63   |  10712  |
    | EXLAMATIONMARK |    0.45   |  0.07  |   0.12   |    75   |
    |     PERIOD     |    0.75   |  0.65  |   0.70   |   7921  |
    |  QUESTIONMARK  |    0.73   |  0.67  |   0.70   |   827   |
    |    micro avg   |    0.73   |  0.60  |   0.66   |  19535  |
    |    macro avg   |    0.66   |  0.49  |   0.53   |  19535  |
    |  weighted avg  |    0.73   |  0.60  |   0.66   |  19535  |