File size: 1,990 Bytes
5fea9ee 3665cbf 327bec4 3665cbf 25503ec 3665cbf 25503ec |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
---
license: apache-2.0
---
This repo has an optimized version of [Detoxify](https://github.com/unitaryai/detoxify/), which needs less disk space and less memory at the cost of just a little bit of accuracy.
This is an experiment for me to learn how to use [🤗 Optimum](https://huggingface.co/docs/optimum/index).
# Usage
Loading the model requires the [🤗 Optimum](https://huggingface.co/docs/optimum/index) library installed.
```python
from optimum.onnxruntime import ORTModelForSequenceClassification
from optimum.pipelines import pipeline as opt_pipeline
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("dcferreira/detoxify-optimized")
model = ORTModelForSequenceClassification.from_pretrained("dcferreira/detoxify-optimized")
pipe = opt_pipeline(
model=model,
task="text-classification",
function_to_apply="sigmoid",
accelerator="ort",
tokenizer=tokenizer,
top_k=None, # return scores for all the labels, model was trained as multilabel
)
print(pipe(['example text','exemple de texte','texto de ejemplo','testo di esempio','texto de exemplo','örnek metin','пример текста']))
```
# Performance
The table below compares some statistics on running the original model, vs the original model with the [onnxruntime](https://onnxruntime.ai/), vs optimizing the model with onnxruntime.
| model | Accuracy (%) | Samples p/ second (CPU) | Samples p/ second (GPU) | GPU VRAM | Disk Space |
|----------------|----------|-------------------------|-------------------------|----------|------------|
| original | 92.1083 | 16 | 250 | 3GB | 1.1GB |
| ort | 92.1067 | 19 | 340 | 4GB | 1.1GB |
| optimized (O4) | 92.1031 | 14 | 650 | 2GB | 540MB |
For details on how these numbers were reached, check out `evaluate.py` in this repo.
|