Package: micronet
micronet"目前在深度学习领域分类两个派别，一派为学院派，研究强大、复杂的模型网络和实验方法，为了追求更高的性能；另一派为工程派，旨在将算法更稳定、高效的落地在硬件平台上，效率是其追求的目标。复杂的模型固然具有更好的性能，但是高额的存储空间、计算资源消耗是使其难以有效的应用在各硬件平台上的重要原因。所以，深度神经网络日益增长的规模为深度学习在移动端的部署带来了巨大的挑战，深度学习模型压缩与部署成为了学术界和工业界都重点关注的研究领域之一"项目简介micronet, a model compression and deploy lib.压缩量化：High-Bit(>2b): QAT, PTQ, QAFT; Low-Bit(≤2b)/Ternary and Binary: QAT剪枝：正常、规整和分组卷积结构剪枝针对特征(A)二值量化的BN融合(训练量化后，BN参数 —> conv的偏置b)High-Bit量化的BN融合(训练量化中，先融合再量化，融合：BN参数 —> conv的权重w和偏置b)部署TensorRT(fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape等)代码结构micronet
├── __init__.py
├── base_module
│   ├── __init__.py
│   └── op.py
├── compression
│   ├── README.md
│   ├── __init__.py
│   ├── pruning
│   │   ├── README.md
│   │   ├── __init__.py
│   │   ├── gc_prune.py
│   │   ├── main.py
│   │   ├── models_save
│   │   │   └── models_save.txt
│   │   └── normal_regular_prune.py
│   └── quantization
│       ├── README.md
│       ├── __init__.py
│       ├── wbwtab
│       │   ├── __init__.py
│       │   ├── bn_fuse
│       │   │   ├── bn_fuse.py
│       │   │   ├── bn_fused_model_test.py
│       │   │   └── models_save
│       │   │       └── models_save.txt
│       │   ├── main.py
│       │   ├── models_save
│       │   │   └── models_save.txt
│       │   └── quantize.py
│       └── wqaq
│           ├── __init__.py
│           ├── dorefa
│           │   ├── __init__.py
│           │   ├── main.py
│           │   ├── models_save
│           │   │   └── models_save.txt
│           │   ├── quant_model_test
│           │   │   ├── models_save
│           │   │   │   └── models_save.txt
│           │   │   ├── quant_model_para.py
│           │   │   └── quant_model_test.py
│           │   └── quantize.py
│           └── iao
│               ├── __init__.py
│               ├── bn_fuse
│               │   ├── bn_fuse.py
│               │   ├── bn_fused_model_test.py
│               │   └── models_save
│               │       └── models_save.txt
│               ├── main.py
│               ├── models_save
│               │   └── models_save.txt
│               └── quantize.py
├── data
│   └── data.txt
├── deploy
│   ├── README.md
│   ├── __init__.py
│   └── tensorrt
│       ├── README.md
│       ├── __init__.py
│       ├── calibrator.py
│       ├── eval_trt.py
│       ├── models
│       │   ├── __init__.py
│       │   └── models_trt.py
│       ├── models_save
│       │   └── calibration_seg.cache
│       ├── test_trt.py
│       └── util_trt.py
├── models
│   ├── __init__.py
│   ├── nin.py
│   ├── nin_gc.py
│   └── resnet.py
└── readme_imgs
    ├── code_structure.jpg
    └── micronet.xmind项目进展2019.12.4, 初次提交12.8, DoReFa特征(A)量化前先进行缩放(* 0.1)，然后再截断，以减小截断误差12.11, 增加项目代码结构图12.12, 完善使用示例12.14, 增加:1、BN融合量化情况(W三值/二值)可选，即训练量化时选择W三/二值，这里则对应选择; 2、BN融合时对卷积核(conv)不带偏置(bias)的处理12.17, 增加模型压缩前后数据对比(示例)12.20, 增加设备可选(cpu、gpu(单卡、多卡))12.27, 补充相关论文12.29, 取消High-Bit量化8-bit以内的限制，即现在可以量化至10-bit、16-bit等2020.2.17, 1、精简W三值/二值量化代码; 2、加速W三值量化训练2.18, 优化针对特征(A)二值的BN融合:去除对BN层gamma参数的限制，即现在此情况下融合时BN可正常训练2.24, 再次优化三/二值量化代码组织结构，增强可移植性，旧版确实不太好移植。目前移植方法：将想要量化的Conv用compression/quantization/wbwtab/models/util_wbwtab.py中的QuantConv2d替换即可，可参照该路径下nin_gc.py中的使用方法3.1, 新增：1、google的High-Bit量化方法; 2、训练中High-Bit量化的BN融合3.2、3.3, 规整量化代码整体结构，目前所有量化方法都可采取类似的移植方式：将想要量化的Conv(或FC，目前dorefa支持，其他方法类似可写)用models/util_wxax.py中的QuantConv2d(或QuantLinear)替换即可，可分别参照该路径下nin_gc.py中的使用方法进行移植（分类、检测、分割等均适用，但需要据实际情况具体调试）3.4, 规整优化wbwtab/bn_fuse中“针对特征(A)二值的BN融合”的相关实现代码，可进行BN融合及融合前后模型对比测试(精度/速度/(大小))3.11, 调整compression/wqaq/iao中的BN层momentum参数(0.1 —> 0.01),削弱batch统计参数占比,一定程度抑制量化带来的抖动。经实验,量化训练更稳定,acc提升1%左右3.13, 更新代码结构图4.6, 修正二值量化训练中W_clip的相关问题(之前由于这个，导致二值量化训练精度上不去，现在已可正常使用)(同时修正无法找到一些模块如models/util_wxax.py的问题)12.14, 1、improve code structure; 2、add deploy-tensorrt(main module, but not running yet)12.18, 1、improve code structure/module reference/module_name; 2、add transfer-use demo12.21, improve pruning-quantization pipeline and code2021.1.4, add other quant_op1.5, add quant_weight's per-channel and per-layer selection1.7, fix iao's loss-nan bug. The bug is due to per-channel min/max error1.8, 1、improve quant_para save. Now, only save scale and zero_point; 2、add optional weight_observer(MinMaxObserver or MovingAverageMinMaxObserver)1.11, fix bug in binary_a(1/0) and binary_w preprocessing1.12, add "pip install"1.22, add auto_insert_quant_op(this still needs to be improved)1.27, improve auto_insert_quant_op(now you can easily use quantization, asquant_test_auto)1.28, 1、fix prune-quantization pipeline and code; 2、improve code structure2.1, improve wbwtab_bn_fuse2.4, 1、add wqaq_bn_fuse; 2、add quant_model_inference_simulation; 3、improve code format4.30, 1、update code_structure img; 2、fix iao's quant_weight_range, quant_contrans and quant_bn_fuse_conv pretrained_model bn_para load bug5.4, addqaft, it's beneficial to improve the quantization accuracy5.6, addptq, its quantization accuracy is also good5.11, add bn_fuse_calib flag5.14, 1、changestetoclip_ste, it's beneficial to improve the quant_train；2、remove quant_relu and add quant_leaky_relu5.15, fix bug in quant_model_para post-processing6.7, add quant_add(need use base_module's op) and quant_resnet demo6.9, iao_quant supports multi gpus6.16, fix quant_round() and quant_binary()环境要求python >= 3.5torch >= 1.1.0torchvison >= 0.3.0numpyonnx == 1.6.0tensorrt == 7.0.0.11安装PyPIpipinstallmicronet-ihttps://pypi.org/simpleGitHubgitclonehttps://github.com/666DZY666/micronet.gitcdmicronet
pythonsetup.pyinstall验证python-c"import micronet; print(micronet.__version__)"测试Install from github压缩量化--refine,可加载预训练浮点模型参数,在其基础上做量化wbwtab--W --A, 权重W和特征A量化取值cdmicronet/compression/quantization/wbwtabWbAbpythonmain.py--W2--A2WbA32pythonmain.py--W2--A32WtAbpythonmain.py--W3--A2WtA32pythonmain.py--W3--A32wqaq--w_bits --a_bits, 权重W和特征A量化位数dorefacdmicronet/compression/quantization/wqaq/dorefaW16A16pythonmain.py--w_bits16--a_bits16W8A8pythonmain.py--w_bits8--a_bits8W4A4pythonmain.py--w_bits4--a_bits4其他bits情况类比iaocdmicronet/compression/quantization/wqaq/iao量化位数选择同dorefa单卡QAT/PTQ  —>  QAFT! 注意，需要在QAT/PTQ之后再做QAFT !--q_type, 量化类型(0-对称, 1-非对称)--q_level, 权重量化级别(0-通道级, 1-层级)--weight_observer, weight_observer选择(0-MinMaxObserver, 1-MovingAverageMinMaxObserver)--bn_fuse, 量化中bn融合标志--bn_fuse_calib, 量化中bn融合校准标志--pretrained_model, 预训练浮点模型--qaft, qaft标志--ptq, ptq_observer--ptq_control, ptq_control--ptq_batch, ptq的batch数量--percentile, ptq校准的比例QAT默认: 对称、(权重)通道级量化, bn不融合, weight_observer-MinMaxObserver, 不加载预训练浮点模型, 进行qatpythonmain.py--q_type0--q_level0--weight_observer0对称、(权重)通道级量化, bn不融合, weight_observer-MovingAverageMinMaxObserverpythonmain.py--q_type0--q_level0--weight_observer1对称、(权重)层级量化, bn不融合pythonmain.py--q_type0--q_level1非对称、(权重)通道级量化, bn不融合pythonmain.py--q_type1--q_level0非对称、(权重)层级量化, bn不融合pythonmain.py--q_type1--q_level1对称、(权重)通道级量化, bn融合pythonmain.py--q_type0--q_level0--bn_fuse对称、(权重)层级量化, bn融合pythonmain.py--q_type0--q_level1--bn_fuse非对称、(权重)通道级量化, bn融合pythonmain.py--q_type1--q_level0--bn_fuse非对称、(权重)层级量化, bn融合pythonmain.py--q_type1--q_level1--bn_fuse对称、(权重)通道级量化, bn融合校准pythonmain.py--q_type0--q_level0--bn_fuse--bn_fuse_calibPTQ需要加载预训练浮点模型,本项目中其可由剪枝中采用正常训练获取对称、(权重)通道级量化, bn融合pythonmain.py--refine../../../pruning/models_save/nin_gc.pth--q_level0--bn_fuse--pretrained_model--ptq_control--ptq--batch_size32--ptq_batch200--percentile0.999999其他情况类比QAFT! 注意，需要在QAT/PTQ之后再做QAFT !QAT  —>  QAFT对称、(权重)通道级量化, bn融合pythonmain.py--resumemodels_save/nin_gc_bn_fused.pth--q_type0--q_level0--bn_fuse--qaft--lr0.00001其他情况类比PTQ  —>  QAFT对称、(权重)通道级量化, bn融合pythonmain.py--resumemodels_save/nin_gc_bn_fused.pth--q_level0--bn_fuse--qaft--lr0.00001--ptq其他情况类比剪枝稀疏训练  —>  剪枝  —>  微调cdmicronet/compression/pruning稀疏训练-sr 稀疏标志--s 稀疏率(需根据dataset、model情况具体调整)--model_type 模型类型(0-nin, 1-nin_gc)nin(正常卷积结构)pythonmain.py-sr--s0.0001--model_type0nin_gc(含分组卷积结构)pythonmain.py-sr--s0.001--model_type1剪枝--percent 剪枝率--normal_regular 正常、规整剪枝标志及规整剪枝基数(如设置为N,则剪枝后模型每层filter个数即为N的倍数)--model 稀疏训练后的model路径--save 剪枝后保存的model路径（路径默认已给出, 可据实际情况更改）正常剪枝(nin)pythonnormal_regular_prune.py--percent0.5--modelmodels_save/nin_sparse.pth--savemodels_save/nin_prune.pth规整剪枝(nin)pythonnormal_regular_prune.py--percent0.5--normal_regular8--modelmodels_save/nin_sparse.pth--savemodels_save/nin_prune.pth或pythonnormal_regular_prune.py--percent0.5--normal_regular16--modelmodels_save/nin_sparse.pth--savemodels_save/nin_prune.pth分组卷积结构剪枝(nin_gc)pythongc_prune.py--percent0.4--modelmodels_save/nin_gc_sparse.pth微调--prune_refine 剪枝后的model路径（在其基础上做微调）ninpythonmain.py--model_type0--prune_refinemodels_save/nin_prune.pthnin_gc需要传入剪枝后得到的新模型的cfg如pythonmain.py--model_type1--gc_prune_refine154162144304320320608584剪枝 —> 量化（注意剪枝率和量化率平衡）加载剪枝后的浮点模型再做量化剪枝 —> 量化（高位）（剪枝率偏大、量化率偏小）w8a8(dorefa)cdmicronet/compression/quantization/wqaq/dorefanin(正常卷积结构)pythonmain.py--w_bits8--a_bits8--model_type0--prune_quant../../../pruning/models_save/nin_finetune.pthnin_gc(含分组卷积结构)pythonmain.py--w_bits8--a_bits8--model_type1--prune_quant../../../pruning/models_save/nin_gc_retrain.pthw8a8(iao)cdmicronet/compression/quantization/wqaq/iaoQAT/PTQ  —>  QAFT! 注意，需要在QAT/PTQ之后再做QAFT !QATbn不融合nin(正常卷积结构)pythonmain.py--w_bits8--a_bits8--model_type0--prune_quant../../../pruning/models_save/nin_finetune.pth--lr0.001nin_gc(含分组卷积结构)pythonmain.py--w_bits8--a_bits8--model_type1--prune_quant../../../pruning/models_save/nin_gc_retrain.pth--lr0.001bn融合nin(正常卷积结构)pythonmain.py--w_bits8--a_bits8--model_type0--prune_quant../../../pruning/models_save/nin_finetune.pth--bn_fuse--pretrained_model--lr0.001nin_gc(含分组卷积结构)pythonmain.py--w_bits8--a_bits8--model_type1--prune_quant../../../pruning/models_save/nin_gc_retrain.pth--bn_fuse--pretrained_model--lr0.001PTQnin(正常卷积结构)pythonmain.py--w_bits8--a_bits8--model_type0--prune_quant../../../pruning/models_save/nin_finetune.pth--bn_fuse--pretrained_model--ptq_control--ptq--batch_size32--ptq_batch200--percentile0.999999其他情况类比QAFT! 注意，需要在QAT/PTQ之后再做QAFT !QAT  —>  QAFTbn不融合nin(正常卷积结构)pythonmain.py--w_bits8--a_bits8--model_type0--prune_qaftmodels_save/nin.pth--qaft--lr0.00001nin_gc(含分组卷积结构)pythonmain.py--w_bits8--a_bits8--model_type1--prune_qaftmodels_save/nin_gc.pth--qaft--lr0.00001bn融合nin(正常卷积结构)pythonmain.py--w_bits8--a_bits8--model_type0--prune_qaftmodels_save/nin_bn_fused.pth--bn_fuse--qaft--lr0.00001nin_gc(含分组卷积结构)pythonmain.py--w_bits8--a_bits8--model_type1--prune_qaftmodels_save/nin_gc_bn_fused.pth--bn_fuse--qaft--lr0.00001PTQ  —>  QAFTbn不融合nin(正常卷积结构)pythonmain.py--w_bits8--a_bits8--model_type0--prune_qaftmodels_save/nin.pth--qaft--lr0.00001--ptqnin_gc(含分组卷积结构)pythonmain.py--w_bits8--a_bits8--model_type1--prune_qaftmodels_save/nin_gc.pth--qaft--lr0.00001--ptqbn融合nin(正常卷积结构)pythonmain.py--w_bits8--a_bits8--model_type0--prune_qaftmodels_save/nin_bn_fused.pth--bn_fuse--qaft--lr0.00001--ptqnin_gc(含分组卷积结构)pythonmain.py--w_bits8--a_bits8--model_type1--prune_qaftmodels_save/nin_gc_bn_fused.pth--bn_fuse--qaft--lr0.00001--ptq其他可选量化配置类比剪枝 —> 量化（低位）（剪枝率偏小、量化率偏大）cdmicronet/compression/quantization/wbwtabwbabnin(正常卷积结构)pythonmain.py--W2--A2--model_type0--prune_quant../../pruning/models_save/nin_finetune.pthnin_gc(含分组卷积结构)pythonmain.py--W2--A2--model_type1--prune_quant../../pruning/models_save/nin_gc_retrain.pth其他取值情况类比BN融合与量化推理仿真测试wbwtabcdmicronet/compression/quantization/wbwtab/bn_fusebn_fuse(得到quant_model_train和quant_bn_fused_model_inference的结构和参数)--model_type, 1 - nin_gc(含分组卷积结构); 0 - nin(正常卷积结构)--prune_quant, 剪枝_量化模型标志--W, weight量化取值均需要与量化训练保持一致,可直接用默认nin_gc, quant_model, wbpythonbn_fuse.py--model_type1--W2nin_gc, prune_quant_model, wbpythonbn_fuse.py--model_type1--prune_quant--W2nin_gc, quant_model, wtpythonbn_fuse.py--model_type1--W3nin, quant_model, wbpythonbn_fuse.py--model_type0--W2bn_fused_model_test(对quant_model_train和quant_bn_fused_model_inference进行测试)pythonbn_fused_model_test.pydorefacdmicronet/compression/quantization/wqaq/dorefa/quant_model_testquant_model_para(得到quant_model_train和quant_model_inference的结构和参数)--model_type, 1 - nin_gc(含分组卷积结构); 0 - nin(正常卷积结构)--prune_quant, 剪枝_量化模型标志--w_bits, weight量化位数; --a_bits, activation量化位数均需要与量化训练保持一致,可直接用默认nin_gc, quant_model, w8a8pythonquant_model_para.py--model_type1--w_bits8--a_bits8nin_gc, prune_quant_model, w8a8pythonquant_model_para.py--model_type1--prune_quant--w_bits8--a_bits8nin, quant_model, w8a8pythonquant_model_para.py--model_type0--w_bits8--a_bits8quant_model_test(对quant_model_train和quant_model_inference进行测试)pythonquant_model_test.pyiao注意,量化训练时 --bn_fuse 需要设置为 Truecdmicronet/compression/quantization/wqaq/iao/bn_fusebn_fuse(得到quant_bn_fused_model_train和quant_bn_fused_model_inference的结构和参数)--model_type, 1 - nin_gc(含分组卷积结构); 0 - nin(正常卷积结构)--prune_quant, 剪枝_量化模型标志--w_bits, weight量化位数; --a_bits, activation量化位数--q_type, 0 - 对称; 1 - 非对称--q_level, 0 - 通道级; 1 - 层级均需要与量化训练保持一致,可直接用默认nin_gc, quant_model, w8a8pythonbn_fuse.py--model_type1--w_bits8--a_bits8nin_gc, prune_quant_model, w8a8pythonbn_fuse.py--model_type1--prune_quant--w_bits8--a_bits8nin, quant_model, w8a8pythonbn_fuse.py--model_type0--w_bits8--a_bits8nin_gc, quant_model, w8a8, 非对称, 层级pythonbn_fuse.py--model_type0--w_bits8--a_bits8--q_type1--q_level1bn_fused_model_test(对quant_bn_fused_model_train和quant_bn_fused_model_inference进行测试)pythonbn_fused_model_test.py设备选取现支持cpu、gpu(单卡、多卡)--cpu 使用cpu，--gpu_id 使用并选择gpucpupythonmain.py--cpugpu单卡pythonmain.py--gpu_id0或pythonmain.py--gpu_id1gpu多卡pythonmain.py--gpu_id0,1或pythonmain.py--gpu_id0,1,2默认，使用服务器全卡部署TensorRT目前仅提供相关核心模块代码，后续再加入完整可运行demo相关解读tensorrt-基础tensorrt-op/dynamic_shape迁移量化训练LeNet examplequant_test_manual.pyA model can be quantized(High-Bit(>2b)、Low-Bit(≤2b)/Ternary and Binary) by simply replacingopwithquant_op.importtorch.nnasnnimporttorch.nn.functionalasF# some base_op, such as ``Add``、``Concat``frommicronet.base_module.opimport*# ``quantize`` is quant_module, ``QuantConv2d``, ``QuantLinear``, ``QuantMaxPool2d``, ``QuantReLU`` are quant_opfrommicronet.compression.quantization.wbwtab.quantizeimport(QuantConv2dasquant_conv_wbwtab,)frommicronet.compression.quantization.wbwtab.quantizeimport(ActivationQuantizerasquant_relu_wbwtab,)frommicronet.compression.quantization.wqaq.dorefa.quantizeimport(QuantConv2dasquant_conv_dorefa,)frommicronet.compression.quantization.wqaq.dorefa.quantizeimport(QuantLinearasquant_linear_dorefa,)frommicronet.compression.quantization.wqaq.iao.quantizeimport(QuantConv2dasquant_conv_iao,)frommicronet.compression.quantization.wqaq.iao.quantizeimport(QuantLinearasquant_linear_iao,)frommicronet.compression.quantization.wqaq.iao.quantizeimport(QuantMaxPool2dasquant_max_pool_iao,)frommicronet.compression.quantization.wqaq.iao.quantizeimport(QuantReLUasquant_relu_iao,)classLeNet(nn.Module):def__init__(self):super(LeNet,self).__init__()self.conv1=nn.Conv2d(1,10,kernel_size=5)self.conv2=nn.Conv2d(10,20,kernel_size=5)self.fc1=nn.Linear(320,50)self.fc2=nn.Linear(50,10)self.max_pool=nn.MaxPool2d(kernel_size=2)self.relu=nn.ReLU(inplace=True)defforward(self,x):x=self.relu(self.max_pool(self.conv1(x)))x=self.relu(self.max_pool(self.conv2(x)))x=x.view(-1,320)x=self.relu(self.fc1(x))x=F.dropout(x,training=self.training)x=self.fc2(x)returnF.log_softmax(x,dim=1)classQuantLeNetWbWtAb(nn.Module):def__init__(self):super(QuantLeNetWbWtAb,self).__init__()self.conv1=quant_conv_wbwtab(1,10,kernel_size=5)self.conv2=quant_conv_wbwtab(10,20,kernel_size=5)self.fc1=nn.Linear(320,50)self.fc2=nn.Linear(50,10)self.max_pool=nn.MaxPool2d(kernel_size=2)self.relu=quant_relu_wbwtab()defforward(self,x):x=self.relu(self.max_pool(self.conv1(x)))x=self.relu(self.max_pool(self.conv2(x)))x=x.view(-1,320)x=self.relu(self.fc1(x))x=F.dropout(x,training=self.training)x=self.fc2(x)returnF.log_softmax(x,dim=1)classQuantLeNetDoReFa(nn.Module):def__init__(self):super(QuantLeNetDoReFa,self).__init__()self.conv1=quant_conv_dorefa(1,10,kernel_size=5)self.conv2=quant_conv_dorefa(10,20,kernel_size=5)self.fc1=quant_linear_dorefa(320,50)self.fc2=quant_linear_dorefa(50,10)self.max_pool=nn.MaxPool2d(kernel_size=2)self.relu=nn.ReLU(inplace=True)defforward(self,x):x=self.relu(self.max_pool(self.conv1(x)))x=self.relu(self.max_pool(self.conv2(x)))x=x.view(-1,320)x=self.relu(self.fc1(x))x=F.dropout(x,training=self.training)x=self.fc2(x)returnF.log_softmax(x,dim=1)classQuantLeNetIAO(nn.Module):def__init__(self):super(QuantLeNetIAO,self).__init__()self.conv1=quant_conv_iao(1,10,kernel_size=5)self.conv2=quant_conv_iao(10,20,kernel_size=5)self.fc1=quant_linear_iao(320,50)self.fc2=quant_linear_iao(50,10)self.max_pool=quant_max_pool_iao(kernel_size=2)self.relu=nn.ReLU(inplace=True)defforward(self,x):x=self.relu(self.max_pool(self.conv1(x)))x=self.relu(self.max_pool(self.conv2(x)))x=x.view(-1,320)x=self.relu(self.fc1(x))x=F.dropout(x,training=self.training)x=self.fc2(x)returnF.log_softmax(x,dim=1)lenet=LeNet()quant_lenet_wbwtab=QuantLeNetWbWtAb()quant_lenet_dorefa=QuantLeNetDoReFa()quant_lenet_iao=QuantLeNetIAO()print("***ori_model***\n",lenet)print("\n***quant_model_wbwtab***\n",quant_lenet_wbwtab)print("\n***quant_model_dorefa***\n",quant_lenet_dorefa)print("\n***quant_model_iao***\n",quant_lenet_iao)print("\nquant_model is ready")print("micronet is ready")quant_test_auto.pyA model can be quantized(High-Bit(>2b)、Low-Bit(≤2b)/Ternary and Binary) by simply usingmicronet.compression.quantization.quantize.prepare(model).importtorch.nnasnnimporttorch.nn.functionalasF# some base_op, such as ``Add``、``Concat``frommicronet.base_module.opimport*importmicronet.compression.quantization.wqaq.dorefa.quantizeasquant_dorefaimportmicronet.compression.quantization.wqaq.iao.quantizeasquant_iaoclassLeNet(nn.Module):def__init__(self):super(LeNet,self).__init__()self.conv1=nn.Conv2d(1,10,kernel_size=5)self.conv2=nn.Conv2d(10,20,kernel_size=5)self.fc1=nn.Linear(320,50)self.fc2=nn.Linear(50,10)self.max_pool=nn.MaxPool2d(kernel_size=2)self.relu=nn.ReLU(inplace=True)defforward(self,x):x=self.relu(self.max_pool(self.conv1(x)))x=self.relu(self.max_pool(self.conv2(x)))x=x.view(-1,320)x=self.relu(self.fc1(x))x=F.dropout(x,training=self.training)x=self.fc2(x)returnF.log_softmax(x,dim=1)"""--w_bits --a_bits, 权重W和特征A量化位数--q_type, 量化类型(0-对称, 1-非对称)--q_level, 权重量化级别(0-通道级, 1-层级)--weight_observer, weight_observer选择(0-MinMaxObserver, 1-MovingAverageMinMaxObserver)--bn_fuse, 量化中bn融合标志--bn_fuse_calib, 量化中bn融合校准标志--pretrained_model, 预训练浮点模型--qaft, qaft标志--ptq, ptq标志--percentile, ptq校准的比例"""lenet=LeNet()quant_lenet_dorefa=quant_dorefa.prepare(lenet,inplace=False,a_bits=8,w_bits=8)quant_lenet_iao=quant_iao.prepare(lenet,inplace=False,a_bits=8,w_bits=8,q_type=0,q_level=0,weight_observer=0,bn_fuse=False,bn_fuse_calib=False,pretrained_model=False,qaft=False,ptq=False,percentile=0.9999,)# if ptq == False, do qat/qaft, need train# if ptq == True, do ptq, don't need train# you can refer to micronet/compression/quantization/wqaq/iao/main.pyprint("***ori_model***\n",lenet)print("\n***quant_model_dorefa***\n",quant_lenet_dorefa)print("\n***quant_model_iao***\n",quant_lenet_iao)print("\nquant_model is ready")print("micronet is ready")testquant_test_manualpython-c"import micronet; micronet.quant_test_manual()"quant_test_autopython-c"import micronet; micronet.quant_test_auto()"when outputting "quant_model is ready", micronet is ready.量化推理参考BN融合与量化推理仿真测试模型压缩数据对比（仅供参考）以下为cifar10示例，可在更冗余模型、更大数据集上尝试其他组合压缩方式类型W(Bits)A(Bits)AccGFLOPsPara(M)Size(MB)压缩率损失原模型(nin)FP32FP3291.01%0.150.672.68******采用分组卷积结构(nin_gc)FP32FP3291.04%0.150.582.3213.43%-0.03%剪枝FP32FP3290.26%0.090.321.2852.24%0.75%量化1FP3290.93%***0.580.20492.39%0.08%量化1.5FP3291%***0.580.27289.85%0.01%量化1186.23%***0.580.20492.39%4.78%量化1.5186.48%***0.580.27289.85%4.53%量化(DoReFa)8891.03%***0.580.59677.76%-0.02%量化(IAO,全量化,symmetric/per-channel/bn_fuse)8890.99%***0.580.59677.76%0.02%分组+剪枝+量化1.5186.13%***0.320.1992.91%4.88%--train_batch_size 256, 单卡相关资料压缩量化QAT二值BinarizedNeuralNetworks: TrainingNeuralNetworkswithWeightsand ActivationsConstrainedto +1 or−1XNOR-Net:ImageNetClassiﬁcationUsingBinary ConvolutionalNeuralNetworksAN EMPIRICAL STUDY OF BINARY NEURAL NETWORKS’ OPTIMISATIONA Review of Binarized Neural Networks三值Ternary weight networksHigh-BitDoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth GradientsQuantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only InferenceQuantizing deep convolutional networks for efficient inference: A whitepaperPTQHigh-Bittensorrt-ptq-8-bit剪枝Learning Efficient Convolutional Networks through Network SlimmingRETHINKING THE VALUE OF NETWORK PRUNING适配专用芯片的模型压缩Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing部署TensorRTgithubptqtensorrt-基础tensorrt-op/dynamic_shapesummary后续tensorrt完整demo其他压缩算法(量化/剪枝/蒸馏/NAS等)其他部署框架(mnn/tnn/tengine等)压缩 —> 部署