Package: ptflops
Flops counting tool for neural networks in pytorch frameworkThis script is designed to compute the theoretical amount of multiply-add operations
in convolutional neural networks. It can also compute the number of parameters and
print per-layer computational cost of a given network.Supported layers:Conv1d/2d/3d (including grouping)ConvTranspose1d/2d/3d (including grouping)BatchNorm1d/2d/3d, GroupNorm, InstanceNorm1d/2d/3d, LayerNormActivations (ReLU, PReLU, ELU, ReLU6, LeakyReLU, GELU)LinearUpsamplePoolings (AvgPool1d/2d/3d, MaxPool1d/2d/3d and adaptive ones)Experimental support:RNN, LSTM, GRU (NLH layout is assumed)RNNCell, LSTMCell, GRUCelltorch.nn.MultiheadAttentiontorchvision.ops.DeformConv2dvisual transformers fromtimmRequirements: Pytorch >= 1.1, torchvision >= 0.3Thanks to @warmspringwinds for the initial version of script.Usage tipsThis tool doesn't take into account some of thetorch.nn.functional.*andtensor.*operations. Therefore unsupported operations are
not contributing to the final complexity estimation. Seeptflops/pytorch_ops.py:FUNCTIONAL_MAPPING,TENSOR_OPS_MAPPINGto check supported ops.ptflopslaunches a given model on a random tensor and estimates amount of computations during inference. Complicated models can have several inputs, some of them could be optional. To construct non-trivial input one can use theinput_constructorargument of theget_model_complexity_info.input_constructoris a function that takes the input spatial resolution as a tuple and returns a dict with named input arguments of the model. Next this dict would be passed to the model as a keyword arguments.verboseparameter allows to get information about modules that don't contribute to the final numbers.ignore_modulesoption forcesptflopsto ignore the listed modules. This can be useful
for research purposes. For instance, one can drop all convolutions from the counting process
specifyingignore_modules=[torch.nn.Conv2d].Install the latest versionFrom PyPI:pipinstallptflopsFrom this repository:pipinstall--upgradegit+https://github.com/sovrasov/flops-counter.pytorch.gitExampleimporttorchvision.modelsasmodelsimporttorchfromptflopsimportget_model_complexity_infowithtorch.cuda.device(0):net=models.densenet161()macs,params=get_model_complexity_info(net,(3,224,224),as_strings=True,print_per_layer_stat=True,verbose=True)print('{:<30}{:<8}'.format('Computational complexity: ',macs))print('{:<30}{:<8}'.format('Number of parameters: ',params))CitationIf ptflops was useful for your paper or tech report, please cite me:@online{ptflops,
  author = {Vladislav Sovrasov},
  title = {ptflops: a flops counting tool for neural networks in pytorch framework},
  year = 2018-2023,
  url = {https://github.com/sovrasov/flops-counter.pytorch},
}BenchmarktorchvisionModelInput ResolutionParams(M)MACs(G)alexnet224x22461.100.72convnext_base224x22488.5915.43densenet121224x2247.982.90efficientnet_b0224x2245.290.41efficientnet_v2_m224x22454.145.43googlenet224x22413.001.51inception_v3224x22427.162.86maxvit_t224x22430.925.48mnasnet1_0224x2244.380.33mobilenet_v2224x2243.500.32mobilenet_v3_large224x2245.480.23regnet_y_1_6gf224x22411.201.65resnet18224x22411.691.83resnet50224x22425.564.13resnext50_32x4d224x22425.034.29shufflenet_v2_x1_0224x2242.280.15squeezenet1_0224x2241.250.84vgg16224x224138.3615.52vit_b_16224x22486.5717.60wide_resnet50_2224x22468.8811.45