xinhe commited on
Commit
a8355b3
1 Parent(s): a4be64f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +33 -0
README.md CHANGED
@@ -1,3 +1,36 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ tags:
4
+ - int8
5
+ - Intel® Neural Compressor
6
+ - neural-compressor
7
+ - PostTrainingDynamic
8
+ datasets:
9
+ - cnn_daily
10
+ metrics:
11
+ - rougeLsum
12
  ---
13
+
14
+ # INT8 T5 base finetuned on CNN DailyMail
15
+
16
+ ### Post-training dynamic quantization
17
+
18
+ This is an INT8 PyTorch model quantized with [huggingface/optimum-intel](https://github.com/huggingface/optimum-intel) through the usage of [Intel® Neural Compressor](https://github.com/intel/neural-compressor).
19
+
20
+ The original fp32 model comes from the fine-tuned model [flax-community/t5-base-cnn-dm](https://huggingface.co/flax-community/t5-base-cnn-dm).
21
+
22
+ ### Evaluation result
23
+
24
+ | |INT8|FP32|
25
+ |---|:---:|:---:|
26
+ | **Accuracy (eval-rougeLsum)** | 36.5661 |36.5959|
27
+ | **Model size** |326M|892M|
28
+
29
+ ### Load with optimum:
30
+
31
+ ```python
32
+ from optimum.intel.neural_compressor.quantization import IncQuantizedModelForSeq2SeqLM
33
+ int8_model = IncQuantizedModelForSeq2SeqLM.from_pretrained(
34
+ 'Intel/t5-base-cnn-dm-int8-dynamic',
35
+ )
36
+ ```