lvkaokao commited on
Commit
c7c2381
1 Parent(s): e1c3c09

update README.md.

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -1,3 +1,39 @@
1
  ---
2
  license: cc-by-4.0
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-4.0
3
+ tags:
4
+ - int8
5
+ - Intel® Neural Compressor
6
+ - PostTrainingStatic
7
+ datasets:
8
+ - squad2
9
+ metrics:
10
+ - f1
11
  ---
12
+
13
+ # INT8 RoBERT base finetuned on Squad2
14
+
15
+ ### Post-training static quantization
16
+
17
+ This is an INT8 PyTorch model quantized with [Intel® Neural Compressor](https://github.com/intel/neural-compressor).
18
+
19
+ The original fp32 model comes from the fine-tuned model [deepset/roberta-base-squad2](https://huggingface.co/deepset/roberta-base-squad2).
20
+
21
+ The calibration dataloader is the train dataloader. The default calibration sampling size 100 isn't divisible exactly by batch size 8, so the real sampling size is 104.
22
+
23
+ The linear modules **roberta.encoder.layer.7.output.dense**, **roberta.encoder.layer.8.output.dense**, **roberta.encoder.layer.9.output.dense**, fall back to fp32 for less than 1% relative accuracy loss.
24
+
25
+ ### Test result
26
+
27
+ | |INT8|FP32|
28
+ |---|:---:|:---:|
29
+ | **Accuracy (eval-f1)** |82.3122|82.9231|
30
+ | **Model size (MB)** |141|474|
31
+
32
+ ### Load with Intel® Neural Compressor:
33
+
34
+ ```python
35
+ from neural_compressor.utils.load_huggingface import OptimizedModel
36
+ int8_model = OptimizedModel.from_pretrained(
37
+ 'Intel/roberta-base-squad2-int8-static',
38
+ )
39
+ ```