Intel
/

t5-base-cnn-dm-int8-dynamic-inc

Text2Text Generation Transformers PyTorch

t5 int8 Intel® Neural Compressor neural-compressor PostTrainingDynamic Inference Endpoints text-generation-inference

Model card Files Files and versions Community

Edit model card

INT8 T5 base finetuned on CNN DailyMail

Post-training dynamic quantization

This is an INT8 PyTorch model quantized with huggingface/optimum-intel through the usage of Intel® Neural Compressor.

The original fp32 model comes from the fine-tuned model flax-community/t5-base-cnn-dm.

Evaluation result

	INT8	FP32
Accuracy (eval-rougeLsum)	36.5661	36.5959
Model size	326M	892M

Load with optimum:

from optimum.intel import INCModelForSeq2SeqLM

model_id = "Intel/t5-base-cnn-dm-int8-dynamic"
int8_model = INCModelForSeq2SeqLM.from_pretrained(model_id)

Downloads last month: 15

Dataset used to train Intel/t5-base-cnn-dm-int8-dynamic-inc

Collection including Intel/t5-base-cnn-dm-int8-dynamic-inc

T5

Originally from Google: Text-To-Text Transfer Transformer (T5) • 4 items • Updated Mar 19