xinhe commited on
Commit
fe99803
1 Parent(s): 0d8a72c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -1,3 +1,39 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ tags:
4
+ - int8
5
+ - Intel® Neural Compressor
6
+ - PostTrainingStatic
7
+ datasets:
8
+ - squad
9
+ metrics:
10
+ - f1
11
  ---
12
+
13
+ # INT8 DistilBERT base uncased finetuned on Squad
14
+
15
+ ### Post-training static quantization
16
+
17
+ This is an INT8 PyTorch model quantized with [Intel® Neural Compressor](https://github.com/intel/neural-compressor).
18
+
19
+ The original fp32 model comes from the fine-tuned model [distilbert-base-uncased-distilled-squad](https://huggingface.co/distilbert-base-uncased-distilled-squad).
20
+
21
+ The calibration dataloader is the train dataloader. The default calibration sampling size 300 isn't divisible exactly by batch size 8, so the real sampling size is 304.
22
+
23
+ The linear module **distilbert.transformer.layer.1.ffn.lin2** falls back to fp32 to meet the 1% relative accuracy loss.
24
+
25
+ ### Test result
26
+
27
+ | |INT8|FP32|
28
+ |---|:---:|:---:|
29
+ | **Accuracy (eval-f1)** |86.1069|86.8374|
30
+ | **Model size (MB)** |74.7|265|
31
+
32
+ ### Load with Intel® Neural Compressor:
33
+
34
+ ```python
35
+ from neural_compressor.utils.load_huggingface import OptimizedModel
36
+ int8_model = OptimizedModel.from_pretrained(
37
+ 'Intel/distilbert-base-uncased-distilled-squad-int8-static',
38
+ )
39
+ ```