README.md

# Model Details: int8 1x4 Sparse Distilbert
The article discusses the how to make inference of transformer-based models more efficient on Intel hardware. The authors propose sparse pattern 1x4 to fit Intel instructions and improve the performance. We implement 1x4 block pruning and get an 80% sparse model on the SQuAD1.1 dataset. Combined with quantization, it achieves up to **x24.2 speedup with less than 1% accuracy loss**. The article also shows performance gains of other models with this approach.
The model card has been written by Intel.

### Model license
Licensed under MIT license.

| Model Detail | Description |
| ----         | ---         |
| language: | en |
| Model Authors Company | Intel |
| Date | June 7, 2023 |
| Version | 1 |
| Type | NLP - Tiny language model|
| Architecture | " we propose a new pipeline for creating and running Fast Transformer models on CPUs, utilizing hardware-aware pruning, knowledge distillation, quantization, and our own Transformer inference runtime engine with optimized kernels for sparse and quantized operators. We demonstrate the efficiency of our pipeline by creating a Fast DistilBERT model showing minimal accuracy loss on the question-answering SQuADv1.1 benchmark, and throughput results under typical production constraints and environments. Our results outperform existing state-of-the-art Neural Magic's DeepSparse runtime performance by up to 50% and up to 4.1x performance speedup over ONNX Runtime." |
| Paper or Other Resources | https://arxiv.org/abs/2211.07715.pdf |
| License | TBD |

### How to use
Please follow Readme in https://github.com/intel/intel-extension-for-transformers/tree/main/examples/huggingface/pytorch/text-classification/deployment/sparse/distilbert_base_uncased