Package: antialiased-cnns
Antialiased CNNs[Project Page][Paper][Talk]Making Convolutional Networks Shift-Invariant AgainRichard Zhang. InICML, 2019.Quick & easy startRunpip install antialiased-cnnsimportantialiased_cnnsmodel=antialiased_cnns.resnet50(pretrained=True)If you have a model already and want to antialias and continue training, copy your old weights over:importtorchvision.modelsasmodelsold_model=models.resnet50(pretrained=True)# old (aliased) modelantialiased_cnns.copy_params_buffers(old_model,model)# copy the weights overIf you want to modify your own model, use the BlurPool layer. More information about our provided models and how to use BlurPool is below.C=10# example feature channel sizeblurpool=antialiased_cnns.BlurPool(C,stride=2)# BlurPool layer; use to downsample a feature mapex_tens=torch.Tensor(1,C,128,128)print(blurpool(ex_tens).shape)# 1xCx64x64 tensorUpdates(Oct 2020) FinetuneI initialize the antialiased model with weights from baseline model, and finetune. Before, I was training from scratch. The results are better.(Oct 2020) Additional modelsWe now have 23 total model variants. I added variants of vgg, densenet, resnext, wide resnet varieties! The same conclusions hold.(Sept 2020) Pip installYou can also nowpip install antialiased-cnnsand load models with thepretrained=Trueflag.(Sept 2020) Kernel 4I have added kernel size 4 experiments. When downsampling an even sized feature map (e.g., a 128x128-->64x64), this is actually the correct size to use to keep the indices from drifting.Table of contentsMore information about antialiased modelsInstructions for antialiasing your own model, using theBlurPoollayerImageNet training and evaluation code. Achieving better consistency, while maintaining or improving accuracy, is an open problem. Help improve the results!(0) PreliminariesPip install this packagepip install antialiased-cnnsOr clone this repository and install requirements (notably, PyTorch)https://github.com/adobe/antialiased-cnns.gitcdantialiased-cnns
pipinstall-rrequirements.txt(1) Loading an antialiased modelThe following loads a pretrained antialiased model, perhaps as a backbone for your application.importantialiased_cnnsmodel=antialiased_cnns.resnet50(pretrained=True,filter_size=4)We also provide weights for antialiasedAlexNet,VGG16(bn),Resnet18,34,50,101,Densenet121, andMobileNetv2(seeexample_usage.py).(2) How to antialias your own architectureTheantialiased_cnnsmodule contains theBlurPoolclass, which does blur+subsampling. Runpip install antialiased-cnnsor copy theantialiased_cnnssubdirectory.MethodologyThe methodology is simple -- first evaluate with stride 1, and then use ourBlurPoollayer to do antialiased downsampling. Make the following architectural changes.importantialiased_cnns# MaxPool --> MaxBlurPoolbaseline=nn.MaxPool2d(kernel_size=2,stride=2)antialiased=[nn.MaxPool2d(kernel_size=2,stride=1),antialiased_cnns.BlurPool(C,stride=2)]# Conv --> ConvBlurPoolbaseline=[nn.Conv2d(Cin,C,kernel_size=3,stride=2,padding=1),nn.ReLU(inplace=True)]antialiased=[nn.Conv2d(Cin,C,kernel_size=3,stride=1,padding=1),nn.ReLU(inplace=True),antialiased_cnns.BlurPool(C,stride=2)]# AvgPool --> BlurPoolbaseline=nn.AvgPool2d(kernel_size=2,stride=2)antialiased=antialiased_cnns.BlurPool(C,stride=2)We assume incoming tensor hasCchannels. Computing a layer at stride 1 instead of stride 2 adds memory and run-time. As such, we typically skip antialiasing at the highest-resolution (early in the network), to prevent large increases.Add antialiasing and then continue trainingIf you already trained a model, and then add antialiasing, you can fine-tune from that old model:antialiased_cnns.copy_params_buffers(old_model,antialiased_model)If this doesn't work, you can just copy the parameters (and not buffers). Adding antialiasing doesn't add any parameters, so the parameter lists are identical. (It does add buffers, so some heuristic is used to match the buffers, which may throw an error.)antialiased_cnns.copy_params(old_model,antialiased_model)(3) ImageNet Evaluation, Results, and Training codeWe observe improvements in bothaccuracy(how often the image is classified correctly) andconsistency(how often two shifts of the same image are classified the same).ACCURACYBaselineAntialiasedDeltaalexnet56.5556.94+0.39vgg1169.0270.51+1.49vgg1369.9371.52+1.59vgg1671.5972.96+1.37vgg1972.3873.54+1.16vgg11_bn70.3872.63+2.25vgg13_bn71.5573.61+2.06vgg16_bn73.3675.13+1.77vgg19_bn74.2475.68+1.44resnet1869.7471.67+1.93resnet3473.3074.60+1.30resnet5076.1677.41+1.25resnet10177.3778.38+1.01resnet15278.3179.07+0.76resnext50_32x4d77.6277.93+0.31resnext101_32x8d79.3179.33+0.02wide_resnet50_278.4778.70+0.23wide_resnet101_278.8578.99+0.14densenet12174.4375.79+1.36densenet16975.6076.73+1.13densenet20176.9077.31+0.41densenet16177.1477.88+0.74mobilenet_v271.8872.72+0.84CONSISTENCYBaselineAntialiasedDeltaalexnet78.1883.31+5.13vgg1186.5890.09+3.51vgg1386.9290.31+3.39vgg1688.5290.91+2.39vgg1989.1791.08+1.91vgg11_bn87.1690.67+3.51vgg13_bn88.0391.09+3.06vgg16_bn89.2491.58+2.34vgg19_bn89.5991.60+2.01resnet1885.1188.36+3.25resnet3487.5689.77+2.21resnet5089.2091.32+2.12resnet10189.8191.97+2.16resnet15290.9292.42+1.50resnext50_32x4d90.1791.48+1.31resnext101_32x8d91.3392.67+1.34wide_resnet50_290.7792.46+1.69wide_resnet101_290.9392.10+1.17densenet12188.8190.35+1.54densenet16989.6890.61+0.93densenet20190.3691.32+0.96densenet16190.8291.66+0.84mobilenet_v286.5087.73+1.23To reduce clutter, extended results (different filter sizes) arehere. Help improve the results!LicensesThis work is licensed under aCreative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.All material is made available underCreative Commons BY-NC-SA 4.0license by Adobe Inc. You canuse, redistribute, and adaptthe material fornon-commercial purposes, as long as you give appropriate credit byciting our paperandindicating any changesthat you've made.The repository builds off the PyTorchexamples repositoryand torchvisionmodels repository. These areBSD-style licensed.Citation, contactIf you find this useful for your research, please consider citing thisbibtex. Please contact Richard Zhang <rizhang at adobe dot com> with any comments or feedback.