lewtun's picture
lewtun HF staff
Add evaluation results on banking77 dataset
641a113
|
raw
history blame
2.23 kB
---
language: en
tags:
- banking
- intent
- multiclass
datasets:
- banking77
widget:
- text: How long until my transfer goes through?
model-index:
- name: mrm8488/distilroberta-finetuned-banking77
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: banking77
type: banking77
config: default
split: test
metrics:
- name: Accuracy
type: accuracy
value: 0.8961038961038961
verified: true
- name: Precision Macro
type: precision
value: 0.9063619413451185
verified: true
- name: Precision Micro
type: precision
value: 0.8961038961038961
verified: true
- name: Precision Weighted
type: precision
value: 0.906361941345118
verified: true
- name: Recall Macro
type: recall
value: 0.8961038961038963
verified: true
- name: Recall Micro
type: recall
value: 0.8961038961038961
verified: true
- name: Recall Weighted
type: recall
value: 0.8961038961038961
verified: true
- name: F1 Macro
type: f1
value: 0.8914970725184695
verified: true
- name: F1 Micro
type: f1
value: 0.8961038961038962
verified: true
- name: F1 Weighted
type: f1
value: 0.8914970725184699
verified: true
- name: loss
type: loss
value: 0.5607758164405823
verified: true
---
# distilroberta-base fine-tuned on banking77 dataset for intent classification
Test set accuray: 0.896
## How to use
```py
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline
ckpt = 'mrm8488/distilroberta-finetuned-banking77'
tokenizer = AutoTokenizer.from_pretrained(ckpt)
model = AutoModelForSequenceClassification.from_pretrained(ckpt)
classifier = pipeline('text-classification', tokenizer=tokenizer, model=model)
classifier('What is the base of the exchange rates?')
# Output: [{'label': 'exchange_rate', 'score': 0.8509947657585144}]
```
> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/)
> Made with <span style="color: #e25555;">&hearts;</span> in Spain