Teja-Gollapudi commited on
Commit
8841d1a
1 Parent(s): 0dd61a3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -1,3 +1,23 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ Int-8 dynamic quantized version of (VMware/tinyroberta-mrqa)[https://huggingface.co/VMware/tinyroberta-mrqa].
5
+
6
+ ```
7
+ from optimum.onnxruntime import ORTModelForQuestionAnswering
8
+ from transformers import pipeline, AutoTokenizer
9
+
10
+ model_name = 'VMware/tinyroberta-quantized-mrqa'
11
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
12
+ quantized_model = ORTModelForQuestionAnswering.from_pretrained(model_name, file_name="model_quantize.onnx")
13
+
14
+ qa_model = pipeline('question-answering', model=quantized_model, tokenizer=tokenizer)
15
+
16
+ qa_input = {
17
+ 'question': '',
18
+ 'context': ''
19
+ }
20
+
21
+ qa_answers = qa_model(qa_input)
22
+
23
+ ```