echarlaix HF staff commited on
Commit
5171d2a
1 Parent(s): cf544a6

update code snippet

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -10,6 +10,7 @@ tags:
10
  - text-classification
11
  - neural-compressor
12
  - int8
 
13
  ---
14
 
15
  # Dynamically quantized DistilBERT base uncased finetuned SST-2
@@ -27,14 +28,17 @@ tags:
27
 
28
  ## How to Get Started With the Model
29
 
 
 
 
30
  To load the quantized model and run inference using the Transformers [pipelines](https://huggingface.co/docs/transformers/main/en/main_classes/pipelines), you can do as follows:
31
 
32
  ```python
33
  from transformers import AutoTokenizer, pipeline
34
- from optimum.intel.neural_compressor import IncQuantizedModelForSequenceClassification
35
 
36
  model_id = "echarlaix/distilbert-base-uncased-finetuned-sst-2-english-int8-dynamic"
37
- model = IncQuantizedModelForSequenceClassification.from_pretrained(model_id)
38
  tokenizer = AutoTokenizer.from_pretrained(model_id)
39
  cls_pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
40
  text = "He's a dreadful magician."
10
  - text-classification
11
  - neural-compressor
12
  - int8
13
+ - 8-bit
14
  ---
15
 
16
  # Dynamically quantized DistilBERT base uncased finetuned SST-2
28
 
29
  ## How to Get Started With the Model
30
 
31
+ This requires to install Optimum :
32
+ `pip install optimum[neural-compressor]`
33
+
34
  To load the quantized model and run inference using the Transformers [pipelines](https://huggingface.co/docs/transformers/main/en/main_classes/pipelines), you can do as follows:
35
 
36
  ```python
37
  from transformers import AutoTokenizer, pipeline
38
+ from optimum.intel import INCModelForSequenceClassification
39
 
40
  model_id = "echarlaix/distilbert-base-uncased-finetuned-sst-2-english-int8-dynamic"
41
+ model = INCModelForSequenceClassification.from_pretrained(model_id)
42
  tokenizer = AutoTokenizer.from_pretrained(model_id)
43
  cls_pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
44
  text = "He's a dreadful magician."