SamLowe commited on
Commit
1d7182a
1 Parent(s): 8bfb1d4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +33 -1
README.md CHANGED
@@ -41,7 +41,39 @@ This model is the ONNX version of [https://huggingface.co/SamLowe/roberta-base-g
41
 
42
  #### Using Optimum Library ONNX Classes
43
 
44
- To follow.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
  #### Using ONNXRuntime
47
 
 
41
 
42
  #### Using Optimum Library ONNX Classes
43
 
44
+ Optimum library has equivalents (starting `ORT`) for the main Transformers classes, so these models can be used with the familiar constructs. The only extra property needed is `file_name` on the model creation, which in the below example specifies the quantized (INT8) model.
45
+
46
+ ```python
47
+ sentences = ["ONNX is seriously fast for small batches. Impressive"]
48
+
49
+ from transformers import AutoTokenizer, pipeline
50
+ from optimum.onnxruntime import ORTModelForSequenceClassification
51
+
52
+ model_id = "SamLowe/roberta-base-go_emotions-onnx"
53
+ file_name = "onnx/model_quantized.onnx"
54
+
55
+ model = ORTModelForSequenceClassification.from_pretrained(model_id, file_name=file_name)
56
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
57
+
58
+ onnx_classifier = pipeline(
59
+ task="text-classification",
60
+ model=model,
61
+ tokenizer=tokenizer,
62
+ top_k=None,
63
+ function_to_apply="sigmoid", # optional as is the default for the task
64
+ )
65
+
66
+ model_outputs = onnx_classifier(sentences)
67
+ # gives a list of outputs, each a list of dicts (one per label)
68
+
69
+ print(model_outputs)
70
+ # E.g.
71
+ # [[{'label': 'admiration', 'score': 0.9203393459320068},
72
+ # {'label': 'approval', 'score': 0.0560273639857769},
73
+ # {'label': 'neutral', 'score': 0.04265536740422249},
74
+ # {'label': 'gratitude', 'score': 0.015126707963645458},
75
+ # ...
76
+ ```
77
 
78
  #### Using ONNXRuntime
79