README.md

---
license: mit
tags:
- generated_from_trainer
datasets:
- banking77
metrics:
- accuracy
widget:
- text: 'Can I track the card you sent to me? '
  example_title: Card Arrival Example
- text: Can you explain your exchange rate policy to me?
  example_title: Exchange Rate Example
- text: I can't pay by my credit card
  example_title: Card Not Working Example
base_model: distilbert-base-uncased
model-index:
- name: distilbert-base-uncased-banking77-classification
  results:
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      name: banking77
      type: banking77
      args: default
    metrics:
    - type: accuracy
      value: 0.924025974025974
      name: Accuracy
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      name: banking77
      type: banking77
      config: default
      split: test
    metrics:
    - type: accuracy
      value: 0.924025974025974
      name: Accuracy
      verified: true
    - type: precision
      value: 0.9278003086307286
      name: Precision Macro
      verified: true
    - type: precision
      value: 0.924025974025974
      name: Precision Micro
      verified: true
    - type: precision
      value: 0.9278003086307287
      name: Precision Weighted
      verified: true
    - type: recall
      value: 0.9240259740259743
      name: Recall Macro
      verified: true
    - type: recall
      value: 0.924025974025974
      name: Recall Micro
      verified: true
    - type: recall
      value: 0.924025974025974
      name: Recall Weighted
      verified: true
    - type: f1
      value: 0.9243068139192414
      name: F1 Macro
      verified: true
    - type: f1
      value: 0.924025974025974
      name: F1 Micro
      verified: true
    - type: f1
      value: 0.9243068139192416
      name: F1 Weighted
      verified: true
    - type: loss
      value: 0.31516405940055847
      name: loss
      verified: true
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# distilbert-base-uncased-banking77-classification

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the banking77 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3152
- Accuracy: 0.9240
- F1 Score: 0.9243

## Model description

This is my first fine-tuning experiment using Hugging Face.
Using distilBERT as a pretrained model, I trained a classifier for online banking queries.
It could be useful for addressing tickets.

## Intended uses & limitations

The model can be used on text classification. In particular is fine tuned on banking domain.

## Training and evaluation data

The dataset used is [banking77](https://huggingface.co/datasets/banking77)

The 77 labels are:

|label|intent|
|:---:|:----:|
|0|activate_my_card|
|1|age_limit|
|2|apple_pay_or_google_pay|
|3|atm_support|
|4|automatic_top_up|
|5|balance_not_updated_after_bank_transfer|
|6|balance_not_updated_after_cheque_or_cash_deposit|
|7|beneficiary_not_allowed|
|8|cancel_transfer|
|9|card_about_to_expire|
|10|card_acceptance|
|11|card_arrival|
|12|card_delivery_estimate|
|13|card_linking|
|14|card_not_working|
|15|card_payment_fee_charged|
|16|card_payment_not_recognised|
|17|card_payment_wrong_exchange_rate|
|18|card_swallowed|
|19|cash_withdrawal_charge|
|20|cash_withdrawal_not_recognised|
|21|change_pin|
|22|compromised_card|
|23|contactless_not_working|
|24|country_support|
|25|declined_card_payment|
|26|declined_cash_withdrawal|
|27|declined_transfer|
|28|direct_debit_payment_not_recognised|
|29|disposable_card_limits|
|30|edit_personal_details|
|31|exchange_charge|
|32|exchange_rate|
|33|exchange_via_app|
|34|extra_charge_on_statement|
|35|failed_transfer|
|36|fiat_currency_support|
|37|get_disposable_virtual_card|
|38|get_physical_card|
|39|getting_spare_card|
|40|getting_virtual_card|
|41|lost_or_stolen_card|
|42|lost_or_stolen_phone|
|43|order_physical_card|
|44|passcode_forgotten|
|45|pending_card_payment|
|46|pending_cash_withdrawal|
|47|pending_top_up|
|48|pending_transfer|
|49|pin_blocked|
|50|receiving_money|
|51|Refund_not_showing_up|
|52|request_refund|
|53|reverted_card_payment?|
|54|supported_cards_and_currencies|
|55|terminate_account|
|56|top_up_by_bank_transfer_charge|
|57|top_up_by_card_charge|
|58|top_up_by_cash_or_cheque|
|59|top_up_failed|
|60|top_up_limits|
|61|top_up_reverted|
|62|topping_up_by_card|
|63|transaction_charged_twice|
|64|transfer_fee_charged|
|65|transfer_into_account|
|66|transfer_not_received_by_recipient|
|67|transfer_timing|
|68|unable_to_verify_identity|
|69|verify_my_identity|
|70|verify_source_of_funds|
|71|verify_top_up|
|72|virtual_card_not_working|
|73|visa_or_mastercard|
|74|why_verify_identity|
|75|wrong_amount_of_cash_received|
|76|wrong_exchange_rate_for_cash_withdrawal|


## Training procedure

```
from transformers import pipeline

pipe = pipeline("text-classification", model="nickprock/distilbert-base-uncased-banking77-classification")
pipe("I can't pay by my credit card")
```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Score |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|
| 3.8732        | 1.0   | 157  | 3.1476          | 0.5370   | 0.4881   |
| 2.5598        | 2.0   | 314  | 1.9780          | 0.6916   | 0.6585   |
| 1.5863        | 3.0   | 471  | 1.2239          | 0.8042   | 0.7864   |
| 0.9829        | 4.0   | 628  | 0.8067          | 0.8565   | 0.8487   |
| 0.6274        | 5.0   | 785  | 0.5837          | 0.8799   | 0.8752   |
| 0.4304        | 6.0   | 942  | 0.4630          | 0.9042   | 0.9040   |
| 0.3106        | 7.0   | 1099 | 0.3982          | 0.9088   | 0.9087   |
| 0.2238        | 8.0   | 1256 | 0.3587          | 0.9110   | 0.9113   |
| 0.1708        | 9.0   | 1413 | 0.3351          | 0.9208   | 0.9208   |
| 0.1256        | 10.0  | 1570 | 0.3242          | 0.9179   | 0.9182   |
| 0.0981        | 11.0  | 1727 | 0.3136          | 0.9211   | 0.9214   |
| 0.0745        | 12.0  | 1884 | 0.3151          | 0.9211   | 0.9213   |
| 0.0601        | 13.0  | 2041 | 0.3089          | 0.9218   | 0.9220   |
| 0.0482        | 14.0  | 2198 | 0.3158          | 0.9214   | 0.9216   |
| 0.0402        | 15.0  | 2355 | 0.3126          | 0.9224   | 0.9226   |
| 0.0344        | 16.0  | 2512 | 0.3143          | 0.9231   | 0.9233   |
| 0.0298        | 17.0  | 2669 | 0.3156          | 0.9231   | 0.9233   |
| 0.0272        | 18.0  | 2826 | 0.3134          | 0.9244   | 0.9247   |
| 0.0237        | 19.0  | 2983 | 0.3156          | 0.9244   | 0.9246   |
| 0.0229        | 20.0  | 3140 | 0.3152          | 0.9240   | 0.9243   |


### Framework versions

- Transformers 4.20.1
- Pytorch 1.12.0+cu113
- Datasets 2.3.2
- Tokenizers 0.12.1