README.md

metadata

tags:
  - optimum
datasets:
  - banking77
metrics:
  - accuracy
model-index:
  - name: quantized-distilbert-banking77
    results:
      - task:
          name: Text Classification
          type: text-classification
        dataset:
          name: banking77
          type: banking77
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.9244

Quantized-distilbert-banking77

This model is a dynamically quantized version of optimum/distilbert-base-uncased-finetuned-banking77 on the banking77 dataset.

The model was created using the dynamic-quantization notebook from a workshop presented at MLOps World 2022.

It achieves the following results on the evaluation set:

Accuracy

• Vanilla model: 92.5%
• Quantized model: 92.44%

The quantized model achieves 99.93% accuracy of the FP32 model

Latency

Payload sequence length: 128
Instance type: AWS c6i.xlarge

latency	vanilla transformers	quantized optimum model	improvement
p95	63.24ms	37.06ms	1.71x
avg	62.87ms	37.93ms	1.66x

How to use

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import pipeline, AutoTokenizer

model = ORTModelForSequenceClassification.from_pretrained("lewtun/quantized-distilbert-banking77")
tokenizer = AutoTokenizer.from_pretrained("lewtun/quantized-distilbert-banking77")

classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)
classifier("What is the exchange rate like on this app?")