What is -1 in output shape of a model in torch.summary?

Sanjayvarma11: <h2>I am printing summary of a model using summary module from torch.summary.But in the result of column ‘output shape’ of MNIST dataset i am getting output as</h2>
<pre><code>    Layer (type)               Output Shape         Param #
</code></pre>
<h1>================================================================<br>
Conv2d-1           [-1, 32, 28, 28]             320<br>
Conv2d-2           [-1, 64, 28, 28]          18,496<br>
MaxPool2d-3           [-1, 64, 14, 14]               0<br>
Conv2d-4          [-1, 128, 14, 14]          73,856<br>
Conv2d-5          [-1, 256, 14, 14]         295,168<br>
MaxPool2d-6            [-1, 256, 7, 7]               0<br>
Conv2d-7            [-1, 512, 5, 5]       1,180,160<br>
Conv2d-8           [-1, 1024, 3, 3]       4,719,616<br>
Conv2d-9           [-1, 2048, 1, 1]      18,876,416</h1>
<p>can anyone tell me what is -1 in output shape ??Thanks in advance</p>

ptrblck: <p>The <code>-1</code> in your example would represent an arbitrary number for the batch dimension.<br>
While the other shapes are fixed to the shown values, you can use any batch size for this model.</p>

Sanjayvarma11: <p>Sir but I already set my batch size to 128 them why i am getting -1</p>

Geeks_Sid: <p>you’re printing using torchsummary if i am not wrong. Anyway, it’s just for example. it could also be ? where ? is your batchsize.</p>

Sanjayvarma11: <p>torch.manual_seed(1)<br>
batch_size = 128</p>
<p>kwargs = {‘num_workers’: 1, ‘pin_memory’: True} if use_cuda else {}<br>
train_loader = torch.utils.data.DataLoader(<br>
datasets.MNIST(’…/data’, train=True, download=True,<br>
transform=transforms.Compose([<br>
transforms.ToTensor(),<br>
transforms.Normalize((0.1307,), (0.3081,))<br>
])),<br>
batch_size=batch_size, shuffle=True, **kwargs)<br>
test_loader = torch.utils.data.DataLoader(<br>
datasets.MNIST(’…/data’, train=False, transform=transforms.Compose([<br>
transforms.ToTensor(),<br>
transforms.Normalize((0.1307,), (0.3081,))<br>
])),<br>
batch_size=batch_size, shuffle=True, **kwargs)<br>
It is a code i wrote for mnist dataset</p>

erm: <p>I am not sure that is the case. When using the torch.summary, the batch size is shown by the second position, right? In this example, the second column of the tensor appears to be digits power of two.and the other two correspond to H and W of the images in MNIST.<br>
I do not have the answer, I actually have the same question. Can it be related with some flattening operation instead?<br>
I tested my own architecture with several torch vision datasets and for all of them I obtain this -1 on the first column no matter the bs I choose.<br>
For a new dataset that I am interested on, and for which I needed to write a custom dataset class, I get weird results, and in addition, torch.summary shows that my layers lack that initial -1, and I am trying to get it,  reason why I need to understand where does it come from.</p>

ptrblck: <p>I’m assuming that <code>summary()</code> outputs the tensor shapes in the default format.<br>
For 2-dimensional layers, such as <code>nn.Conv2d</code> and <code>nn.MaxPool2d</code>, the expected shape is given as <code>[batch_size, channels, height, width]</code>.<br>
<code>dim1</code> would therefore correspond to the channels, which are often chosen to be powers of 2 for performance reasons (“good” indexing is easier for powers of 2).</p>
<p>The <code>-1</code> would therefore be the batch dimension, which is flexible in PyTorch. I.e. you don’t have to specify the batch size for your model and it will take variable batches as long as you have enough memory etc.</p>
<aside class="quote no-group" data-username="erm" data-post="6" data-topic="67790">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/erm/48/18230_2.png" class="avatar"> erm:</div>
<blockquote>
<p>I get weird results, and in addition, torch.summary shows that my layers lack that initial -1</p>
</blockquote>
</aside>
<p>Could you post your result and (if possible) your model definition, so that we could have a look?</p>

erm: <pre><code class="lang-auto">#Def the Network and forward prop

dim = 12288 #64*64*3channels
n_h = 7
n_classes = 2 # 2 classes
n_y = 5

class Net(nn.Module): 
    def __init__(self):
        
        super(Net,self).__init__()   
        self.input_layer = nn.Linear(dim,n_h)
        self.hidden_layer = nn.Linear(n_h,n_y) 
        self.output_layer = nn.Linear(n_y,n_classes) 
           
    def forward(self, x):
        x = x.view(-1, dim) #flatten image input
        x = F.relu(self.input_layer(x)) 
        x = F.relu(self.hidden_layer(x))
        x = self.output_layer(x)#no ReLU to the last layer.
        return F.log_softmax(x, dim=1) #convert logits to probabilities that sum up to 1. 

net= Net()
print(net)
summary(Net(), input_size=(1,dim), batch_size=bs, device="cpu") 
net.state_dict()

type or paste code here
</code></pre>
<h2>Net(<br>
(input_layer): Linear(in_features=12288, out_features=7, bias=True)<br>
(hidden_layer): Linear(in_features=7, out_features=5, bias=True)<br>
(output_layer): Linear(in_features=5, out_features=2, bias=True)<br>
)</h2>
<pre><code>    Layer (type)               Output Shape         Param #
</code></pre>
<h1>================================================================<br>
Linear-1                    [32, 7]          86,023<br>
Linear-2                    [32, 5]              40<br>
Linear-3                    [32, 2]              12</h1>
<h2>Total params: 86,075<br>
Trainable params: 86,075<br>
Non-trainable params: 0</h2>
<h2>Input size (MB): 1.50<br>
Forward/backward pass size (MB): 0.00<br>
Params size (MB): 0.33<br>
Estimated Total Size (MB): 1.83</h2>
<p>This is the output when feeding hdf5 dataset with custom dataset class.</p>
<h2>Instead, for MNIST:<br>
Net(<br>
(input_layer): Linear(in_features=784, out_features=7, bias=True)<br>
(hidden_layer): Linear(in_features=7, out_features=5, bias=True)<br>
(output_layer): Linear(in_features=5, out_features=10, bias=True)<br>
)</h2>
<pre><code>    Layer (type)               Output Shape         Param #
</code></pre>
<h1>================================================================<br>
Linear-1                [-1, 32, 7]           5,495<br>
Linear-2                [-1, 32, 5]              40<br>
Linear-3               [-1, 32, 10]              60</h1>
<h2>Total params: 5,595<br>
Trainable params: 5,595<br>
Non-trainable params: 0</h2>
<h2>Input size (MB): 0.10<br>
Forward/backward pass size (MB): 0.01<br>
Params size (MB): 0.02<br>
Estimated Total Size (MB): 0.12</h2>

ptrblck: <p>Thanks for the code!<br>
In your first example you are passing the <code>batch_size=bs</code> directly to <code>sumamry()</code>, which will then print it out.<br>
If you skip it and leave it as <code>-1</code> (default), you’ll get:</p>
<pre><code class="lang-python">----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Linear-1                    [-1, 7]          86,023
            Linear-2                    [-1, 5]              40
            Linear-3                    [-1, 2]              12
================================================================
Total params: 86,075
Trainable params: 86,075
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.05
Forward/backward pass size (MB): 0.00
Params size (MB): 0.33
Estimated Total Size (MB): 0.38
----------------------------------------------------------------
</code></pre>

erm: <p>Oh! thanks a lot, your fast answer saved me a lot of time.</p>