Package: neural-compressor-full
Intel® Neural CompressorAn open-source Python library supporting popular model compression techniques on all mainstream deep learning frameworks (TensorFlow, PyTorch, ONNX Runtime, and MXNet)Architecture|Workflow|Results|Examples|DocumentationsIntel® Neural Compressor aims to provide popular model compression techniques such as quantization, pruning (sparsity), distillation, and neural architecture search on mainstream frameworks such asTensorFlow,PyTorch,ONNX Runtime, andMXNet,
as well as Intel extensions such asIntel Extension for TensorFlowandIntel Extension for PyTorch.
In particular, the tool provides the key features, typical examples, and open collaborations as below:Support a wide range of Intel hardware such asIntel Xeon Scalable processor,Intel Xeon CPU Max Series,Intel Data Center GPU Flex Series, andIntel Data Center GPU Max Serieswith extensive testing; support AMD CPU, ARM CPU, and NVidia GPU through ONNX Runtime with limited testingValidate more than 10,000 models such asBloom-176B,OPT-6.7B,Stable Diffusion,GPT-J,BERT-Large, andResNet50from popular model hubs such asHugging Face,Torch Vision, andONNX Model Zoo, by leveraging zero-code optimization solutionNeural Coderand automaticaccuracy-drivenquantization strategiesCollaborate with cloud marketplace such asGoogle Cloud Platform,Amazon Web Services, andAzure, software platforms such asAlibaba CloudandTencent TACO, and open AI ecosystem such asHugging Face,PyTorch,ONNX, andLightning AIInstallationInstall from pypipipinstallneural-compressorMore installation methods can be found atInstallation Guide. Please check out ourFAQfor more details.Getting StartedQuantization with Python API# Install Intel Neural Compressor and TensorFlowpipinstallneural-compressorpipinstalltensorflow# Prepare fp32 modelwgethttps://storage.googleapis.com/intel-optimized-tensorflow/models/v1_6/mobilenet_v1_1.0_224_frozen.pbfromneural_compressor.configimportPostTrainingQuantConfigfromneural_compressor.dataimportDataLoaderfromneural_compressor.dataimportDatasetsdataset=Datasets('tensorflow')['dummy'](shape=(1,224,224,3))dataloader=DataLoader(framework='tensorflow',dataset=dataset)fromneural_compressor.quantizationimportfitq_model=fit(model="./mobilenet_v1_1.0_224_frozen.pb",conf=PostTrainingQuantConfig(),calib_dataloader=dataloader,eval_dataloader=dataloader)More quick samples can be found inGet Started Page.DocumentationOverviewArchitectureWorkflowAPIsGUINotebookExamplesIntel oneAPI AI Analytics ToolkitPython-based APIsQuantizationAdvanced Mixed PrecisionPruning (Sparsity)DistillationOrchestrationBenchmarkingDistributed CompressionModel ExportNeural Coder (Zero-code Optimization)LauncherJupyterLab ExtensionVisual Studio Code ExtensionSupported MatrixAdvanced TopicsAdaptorStrategyDistillation for QuantizationSmoothQuantSelected Publications/EventsBlog on Medium:Effective Post-training Quantization for Large Language Models with Enhanced SmoothQuant Approach(Apr 2023)Blog by Intel:Intel® Xeon® Processors Are Still the Only CPU With MLPerf Results, Raising the Bar By 5x(Apr 2023)Post on Social Media:Adopt with Tencent TACO: Heterogeneous optimization is also key to improving AI computing power(Mar 2023)Post on Social Media:Training and Inference for Stable Diffusion | Intel Business(Jan 2023)NeurIPS'2022:Fast Distilbert on CPUs(Oct 2022)NeurIPS'2022:QuaLA-MiniLM: a Quantized Length Adaptive MiniLM(Oct 2022)View ourFull Publication List.Additional ContentRelease InformationContribution GuidelinesLegal InformationSecurity PolicyResearch CollaborationsWelcome to raise any interesting research ideas on model compression techniques and feel free to reach us (inc.maintainers@intel.com). Look forward to our collaborations on Intel Neural Compressor!