Package: torchscan
The very usefulsummarymethod oftf.keras.Modelbut for PyTorch, with more useful information.Quick TourInspecting your PyTorch architectureSimilarly to thetorchsummaryimplementation,torchscanbrings useful module information into readable format. For nested complex architectures, you can use a maximum depth of display as follows:fromtorchvision.modelsimportdensenet121fromtorchscanimportsummarymodel=densenet121().eval().cuda()summary(model,(3,224,224),max_depth=2)which would yield__________________________________________________________________________________________
LayerTypeOutputShapeParam#==========================================================================================densenetDenseNet(-1,1000)0├─featuresSequential(-1,1024,7,7)0|└─conv0Conv2d(-1,64,112,112)9,408|└─norm0BatchNorm2d(-1,64,112,112)257|└─relu0ReLU(-1,64,112,112)0|└─pool0MaxPool2d(-1,64,56,56)0|└─denseblock1_DenseBlock(-1,256,56,56)338,316|└─transition1_Transition(-1,128,28,28)33,793|└─denseblock2_DenseBlock(-1,512,28,28)930,072|└─transition2_Transition(-1,256,14,14)133,121|└─denseblock3_DenseBlock(-1,1024,14,14)2,873,904|└─transition3_Transition(-1,512,7,7)528,385|└─denseblock4_DenseBlock(-1,1024,7,7)2,186,272|└─norm5BatchNorm2d(-1,1024,7,7)4,097├─classifierLinear(-1,1000)1,025,000==========================================================================================Trainableparams:7,978,856
Non-trainableparams:0Totalparams:7,978,856
------------------------------------------------------------------------------------------
Modelsize(params+buffers):30.76Mb
Framework&CUDAoverhead:423.57Mb
TotalRAMusage:454.32Mb
------------------------------------------------------------------------------------------
FloatingPointOperationsonforward:5.74GFLOPs
Multiply-Accumulationsonforward:2.87GMACs
Directmemoryaccessesonforward:2.90GDMAs
__________________________________________________________________________________________Results are aggregated to the selected depth for improved readability.For reference, here are explanations of a few acronyms:FLOPs: floating-point operations (not to be confused with FLOPS which is FLOPs per second)MACs: mutiply-accumulate operations (cf.wikipedia)DMAs: direct memory accesses (many argue that it is more relevant than FLOPs or MACs to compare model inference speeds cf.wikipedia)Additionally, for highway nets (models without multiple branches / skip connections),torchscansupports receptive field estimation.fromtorchvision.modelsimportvgg16fromtorchscanimportsummarymodel=vgg16().eval().cuda()summary(model,(3,224,224),receptive_field=True,max_depth=0)which will add the layer's receptive field (relatively to the last convolutional layer) to the summary.SetupPython 3.6 (or higher) andpip/condaare required to install Torchscan.Stable releaseYou can install the last stable release of the package usingpypias follows:pipinstalltorchscanor usingconda:condainstall-cfrgfmtorchscanDeveloper installationAlternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source:gitclonehttps://github.com/frgfm/torch-scan.git
pipinstall-etorch-scan/.BenchmarkBelow are the results for classification models supported bytorchvisionfor a single image with 3 color channels of size224x224(apart frominception_v3which uses299x299).ModelParams (M)FLOPs (G)MACs (G)DMAs (G)RFalexnet61.11.430.710.72195googlenet6.623.011.511.53--vgg11132.8615.237.617.64150vgg11_bn132.8715.267.637.66150vgg13133.0522.6311.3111.35156vgg13_bn133.0522.6811.3311.37156vgg16138.3630.9615.4715.52212vgg16_bn138.3731.0115.515.55212vgg19143.6739.2819.6319.69268vgg19_bn143.6839.3419.6619.72268resnet1811.693.641.821.84--resnet3421.87.343.673.7--resnet5025.568.214.114.15--resnet10144.5515.667.837.9--resnet15260.1923.111.5611.65--inception_v327.1611.455.735.76--squeezenet1_01.251.640.820.83--squeezenet1_11.240.70.350.36--wide_resnet50_268.8822.8411.4311.51--wide_resnet101_2126.8945.5822.822.95--densenet1217.985.742.872.9--densenet16128.6815.597.797.86--densenet16914.156.813.43.44--densenet20120.018.74.344.39--resnext50_32x4d25.038.514.264.3--resnext101_32x8d88.7932.9316.4816.61--mobilenet_v23.50.630.310.32--shufflenet_v2_x0_51.370.090.040.05--shufflenet_v2_x1_02.280.30.150.15--shufflenet_v2_x1_53.50.60.30.31--shufflenet_v2_x2_07.391.180.590.6--mnasnet0_52.220.220.110.12--mnasnet0_753.170.450.230.24--mnasnet1_04.380.650.330.34--mnasnet1_36.281.080.540.56--The above results were produced using thescripts/benchmark.pyscript.Note: receptive field computation is currently only valid for highway nets.What elseDocumentationThe full package documentation is availableherefor detailed specifications.Example scriptAn example script is provided for you to benchmark torchvision models using the library:pythonscripts/benchmark.pyCreditsThis project is developed and maintained by the repo owner, but the implementation was inspired or helped by the following contributions:Pytorch summary: existing PyTorch porting oftf.keras.Model.summaryTorchstat: another module inspection toolFlops counter Pytorch: operation counter toolTHOP: PyTorch Op counterNumber of operations and memory estimation articles byMatthijs Hollemans, andSicaraPruning Convolutional Neural Networks for Resource Efficient InferenceCitationIf you wish to cite this project, feel free to use thisBibTeXreference:@misc{torchscan2020,title={Torchscan: meaningful module insights},author={François-Guillaume Fernandez},year={2020},month={March},publisher={GitHub},howpublished={\url{https://github.com/frgfm/torch-scan}}}ContributingAny sort of contribution is greatly appreciated!You can find a short guide inCONTRIBUTINGto help grow this project!LicenseDistributed under the Apache 2.0 License. SeeLICENSEfor more information.