22.5 GB input image size in model.summary

username: Sanjayvarma11
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>I am running a deep neural network which takes input as two images of size 3X160X160.I am running model.summary() and in input size i am getting 22.5 gb as input size.I also provided the code used for calculating the input size in model.summary github account also.</p>
<pre><code class="lang-auto">code used for model creation
from torchsummary import summary
import torch
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    """
    Base network that defines helper functions, summary and mapping to device
    """
    def conv2d(self, in_channels, out_channels, kernel_size=(3,3), dilation=1, groups=1, padding=1, bias=False, padding_mode="zeros"):
      return [nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size, groups=groups, dilation=dilation, padding=padding, bias=bias, padding_mode=padding_mode)]

    def separable_conv2d(self, in_channels, out_channels, kernel_size=(3,3), dilation=1, padding=1, bias=False, padding_mode="zeros"):
      return [nn.Conv2d(in_channels=in_channels, out_channels=in_channels, kernel_size=kernel_size, groups=in_channels, dilation=dilation, padding=padding, bias=bias, padding_mode=padding_mode),
              nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=(1,1), bias=bias)]

    def activate(self, l, out_channels, bn=True, dropout=0, relu=True,max_pooling=0):
      if(max_pooling&gt;0):
        l.append(nn.MaxPool2d(2,2))
      if bn:
        l.append(nn.BatchNorm2d(out_channels))
      if dropout&gt;0:
        l.append(nn.Dropout(dropout))
      if relu:
        l.append(nn.ReLU())

      return nn.Sequential(*l)

    def create_conv2d(self, in_channels, out_channels, kernel_size=(3,3), dilation=1, groups=1, padding=1, bias=False, bn=True, dropout=0, relu=True, padding_mode="zeros",max_pooling=0):
      return self.activate(self.conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size, groups=groups, dilation=dilation, padding=padding, bias=bias, padding_mode=padding_mode), out_channels, bn, dropout, relu,max_pooling)

    def create_depthwise_conv2d(self, in_channels, out_channels, kernel_size=(3,3), dilation=1, padding=1, bias=False, bn=True, dropout=0, relu=True, padding_mode="zeros"):
      return self.activate(self.separable_conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size, dilation=dilation, padding=padding, bias=bias, padding_mode=padding_mode),
                 out_channels, bn, dropout, relu)

    def __init__(self, name="Model"):
        super(Net, self).__init__()
        self.trainer = None
        self.name = name

    def summary(self, input_size,input_size1): #input_size=(1, 28, 28)
      summary(self, input_size=[input_size,input_size1])

    def gotrain(self, optimizer, train_loader, test_loader, epochs, statspath,criterion,scheduler=None, batch_scheduler=False, L1lambda=0,use_amp=True):
      self.trainer = ModelTrainer(self, optimizer, train_loader, test_loader, statspath, scheduler, batch_scheduler, L1lambda,criterion,use_amp)
      #print("hello")
      self.trainer.run(epochs)

    def stats(self):
      return self.trainer.stats if self.trainer else None

class depth_model(Net):
  def __init__(self,name="Model",dropout_value=0.0):
    super(depth_model,self).__init__(name)
    #first part of architecture of solving the mask image
    self.conv1=self.create_conv2d(3,32)
    self.conv2=self.create_conv2d(32,64)
    self.conv3=self.create_conv2d(64,128)
    self.conv4=self.create_conv2d(128,128)
    self.conv5=self.create_conv2d(128,1,(1,1),padding=0,relu=False,bn=False)
    #second part of architecture for solving the depth image 
    self.conv7=self.create_conv2d(3,32)
    self.conv8=self.create_conv2d(32,64)
    self.conv9=self.create_conv2d(64,128)
    self.conv10=self.create_conv2d(128,128)
    self.conv11=self.create_conv2d(256,256)
    self.conv12=self.create_conv2d(256,1,(1,1),padding=0,relu=False,bn=False)

  def forward(self,x1,x2):
    #first part for finding mask
    x1=self.conv1(x1)
    x1=self.conv2(x1)
    x1=self.conv3(x1)
    x1=self.conv4(x1)
    self.value=x1
    x1=self.conv5(x1) #first output for mask
    #second part for finding depth 
    x2=self.conv7(x2)
    x2=self.conv8(x2)
    x2=self.conv9(x2)    
    x2=self.conv10(x2)
    #x2=self.upsample(x2)
    #print(x.shape)
    x2=torch.cat((x2,self.value),1)
    #print(x.shape)
    x2=self.conv11(x2)
    x2=self.conv12(x2)
    return x1,x2

calling model

import torch
model = depth_model(name="depth_model") #use load_state_dict if there is a saved model
#model.load_state_dict(torch.load("/content/gdrive/My Drive/depth_model.pt"))
use_cuda = torch.cuda.is_available()
model.device = torch.device("cuda" if use_cuda else "cpu")
model.to(model.device)
model.summary((3,160,160),(3,160,160))



output of model summary

----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1         [-1, 32, 160, 160]             864
       BatchNorm2d-2         [-1, 32, 160, 160]              64
              ReLU-3         [-1, 32, 160, 160]               0
            Conv2d-4         [-1, 64, 160, 160]          18,432
       BatchNorm2d-5         [-1, 64, 160, 160]             128
              ReLU-6         [-1, 64, 160, 160]               0
            Conv2d-7        [-1, 128, 160, 160]          73,728
       BatchNorm2d-8        [-1, 128, 160, 160]             256
              ReLU-9        [-1, 128, 160, 160]               0
           Conv2d-10        [-1, 128, 160, 160]         147,456
      BatchNorm2d-11        [-1, 128, 160, 160]             256
             ReLU-12        [-1, 128, 160, 160]               0
           Conv2d-13          [-1, 1, 160, 160]             128
           Conv2d-14         [-1, 32, 160, 160]             864
      BatchNorm2d-15         [-1, 32, 160, 160]              64
             ReLU-16         [-1, 32, 160, 160]               0
           Conv2d-17         [-1, 64, 160, 160]          18,432
      BatchNorm2d-18         [-1, 64, 160, 160]             128
             ReLU-19         [-1, 64, 160, 160]               0
           Conv2d-20        [-1, 128, 160, 160]          73,728
      BatchNorm2d-21        [-1, 128, 160, 160]             256
             ReLU-22        [-1, 128, 160, 160]               0
           Conv2d-23        [-1, 128, 160, 160]         147,456
      BatchNorm2d-24        [-1, 128, 160, 160]             256
             ReLU-25        [-1, 128, 160, 160]               0
           Conv2d-26        [-1, 256, 160, 160]         589,824
      BatchNorm2d-27        [-1, 256, 160, 160]             512
             ReLU-28        [-1, 256, 160, 160]               0
           Conv2d-29          [-1, 1, 160, 160]             256
================================================================
Total params: 1,073,088
Trainable params: 1,073,088
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 22500.00
Forward/backward pass size (MB): 562.89
Params size (MB): 4.09
Estimated Total Size (MB): 23066.98
----------------------------------------------------------------

</code></pre>
<p>CODE USED IN MODEL.SUMMARY() DOCUMENTATION FOR CALCULATION OF INPUT SIZE</p>
<pre><code class="lang-auto">total_input_size = abs(np.prod(sum(input_size, ()))
                           * batch_size * 4. / (1024 ** 2.))

</code></pre>
<p>So how do i this model get 22500 GB input size.Thank you for reading it.</p>

username: Sanjayvarma11
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> please answer this question sir.</p>

username: ptrblck
cooked: <p>It seems the formula might calculate the input shape as:</p>
<pre><code class="lang-python">input_size = [(3, 160, 160), (3, 160, 160)]
batch_size = 1
total_input_size = abs(np.prod(sum(input_size, ())) * batch_size * 4. / (1024 ** 2.))
print(total_input_size)
&gt; 6116.0 # 6GB
</code></pre>
<p>which is wrong as the memory from multiple inputs should be summed not multiplicated:</p>
<pre><code class="lang-python">print(sum(input_size, ()))
&gt; (3, 160, 160, 3, 160, 160)
</code></pre>
<p>I would recommend to create an issue in the repository and explain the concern there.</p>
<p>PS: I’m not a big fan of tagging certain people, as this might discourage others to post an answer. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: Sanjayvarma11
cooked: <p>Okay sir,I waited for few days but no one replied so i thought you might solve it.Thank you</p>

username: ptrblck
cooked: <p>Sure, I didn’t realize you’ve been waiting for a week and apparently this topic was in my backlog, so forget what I said and please ping me after a while. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>