lvkaokao commited on
Commit
9a2ab64
1 Parent(s): a8c956d

update README.md.

Browse files
Files changed (1) hide show
  1. README.md +37 -1
README.md CHANGED
@@ -1,3 +1,39 @@
1
  ---
2
- license: mit
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: mlt
3
+ tags:
4
+ - int8
5
+ - Intel® Neural Compressor
6
+ - PostTrainingStatic
7
+ datasets:
8
+ - mnli
9
+ metrics:
10
+ - accuracy
11
  ---
12
+
13
+ # INT8 RoBERT large finetuned on MNLI
14
+
15
+ ### Post-training static quantization
16
+
17
+ This is an INT8 PyTorch model quantized with [Intel® Neural Compressor](https://github.com/intel/neural-compressor).
18
+
19
+ The original fp32 model comes from the fine-tuned model [roberta-large-mnli](https://huggingface.co/roberta-large-mnli).
20
+
21
+ The calibration dataloader is the train dataloader. The default calibration sampling size 100 isn't divisible exactly by batch size 8, so the real sampling size is 104.
22
+
23
+ The linear modules **roberta.encoder.layer.16.output.dense**, **roberta.encoder.layer.17.output.dense**, **roberta.encoder.layer.18.output.dense**, fall back to fp32 for less than 1% relative accuracy loss.
24
+
25
+ ### Evaluation result
26
+
27
+ | |INT8|FP32|
28
+ |---|:---:|:---:|
29
+ | **Accuracy (eval-acc)** |89.8624|90.5960|
30
+ | **Model size (MB)** |381M|1.4G|
31
+
32
+ ### Load with Intel® Neural Compressor:
33
+
34
+ ```python
35
+ from neural_compressor.utils.load_huggingface import OptimizedModel
36
+ int8_model = OptimizedModel.from_pretrained(
37
+ 'Intel/roberta-base-squad2-int8-static',
38
+ )
39
+ ```