What's the proper way to decipher dimensions?

redtailedhawk: <p>I am trying to get this sorted once and for all. How does Pytorch read dimensions? If I get my size of MNIST, and I see torch.tensor([28, 28]), I read width, height. What does Pytorch read if I feed that into a network (since I have to unsqueeze() it for it to work)? Does it read batch_size: 28, of a 1d tensor of [28] values?</p>
<p>Can someone help me with this or point me to some solid documentation/articles about it?</p>
<pre><code class="lang-auto"># get hypothetical tensor.size()
torch.tensor([28]) # [?]

torch.tensor([1, 16]) # [?, ?]

torch.tensor([12, 1, 6]) # [?, ?, ?]

torch.tensor([32, 1, 12, 12]) # [batch_size, channels, height, width]

torch.tensor([32, 3, 2, 16, 28]) # [batch_size, channels, depth, height, width]
</code></pre>
<p>Is the first dimension always batch_size? Are the last 2 always height x width?</p>
<p>I just can’t find the logic anywhere to understand what the clean systematic way of thinking of this is, or what the design choices were. Your help is greatly appreciated.</p>
<p>Lastly, when I am coming out of a conv layer, and want to pass into a Linear layer, which view() parameter should be -1, and which dimensions (using the terminology above) should be multiplied together?</p>
<p>example</p>
<pre><code class="lang-auto"># x.size() = torch.tensor([32, 21, 12, 12])
# should I flatten it like this?
x = x.view(-1, 21 * 12 * 12)

# or like this:
x = x.view(-1, 32 * 12 * 12)

#or like this:
x = x.view(32, -1)

# or other?
</code></pre>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a>, you’re always great at bringing clarity to these sorts of things…</p>

ptrblck: <p>Each layer specifies the input and target dimensions.<br>
E.g. in the <a href="https://pytorch.org/docs/stable/nn.html?highlight=conv2d#torch.nn.Conv2d">docs</a> of <code>nn.Conv2d</code> you see the input defined as <code>[N, Cin​, Hin​, Win​]</code>.</p>
<p>Generally, all layers usually found in a CNN will accept this shape (pooling, normalization layers etc.).</p>
<p>Your example of the 5-dimensional tensor would be a possible input to <code>nn.Conv3d</code>.</p>
<p>However, you have to be careful when it comes to e.g. RNNs, as the default shape is expected as <code>[seq_len, batch_size, features]</code>. You could use <code>batch_first=True</code> while creating the RNN to use inputs as <code>[batch_size, seq_len, features]</code>.</p>
<p>The design choices were most likely made due to performance reasons.</p>

redtailedhawk: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="66249">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>Each layer specifies the input and target dimensions.<br>
E.g. in the <a href="https://pytorch.org/docs/stable/nn.html?highlight=conv2d#torch.nn.Conv2d" rel="noopener nofollow ugc">docs</a> of <code>nn.Conv2d</code> you see the input defined as <code>[N, Cin​, Hin​, Win​]</code> .</p>
</blockquote>
</aside>
<p>Right, I was reading that earlier. Am I making this more complicated than it is? What about the 3d, 2d and 1d tensors?</p>
<p>For instance, if I get the shape of one of my images, let’s just use MNIST. It’s [1, 28, 28]. To me, I’m reading [channel, height, width] as a human. As an input to a Conv2d, Pytorch just sees it as missing a dimension, and in a Conv1d would read [batch, channels, length]?</p>

ptrblck: <aside class="quote no-group" data-username="redtailedhawk" data-post="3" data-topic="66249">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/redtailedhawk/48/19728_2.png" class="avatar"> redtailedhawk:</div>
<blockquote>
<p>To me, I’m reading [channel, height, width] as a human. As an input to a Conv2d, Pytorch just sees it as missing a dimension, and in a Conv1d would read [batch, channels, length]?</p>
</blockquote>
</aside>
<p>Yes, that is correct.<br>
The input shape would throw an error for <code>nn.Conv2d</code>, while <code>nn.Conv1d</code> would treat dim2 as the sequence length.</p>

redtailedhawk: <p>And in this example:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; m = nn.Linear(20, 30)
&gt;&gt;&gt; input = torch.randn(128, 20)
&gt;&gt;&gt; output = m(input)
&gt;&gt;&gt; print(output.size())
torch.Size([128, 30])
</code></pre>
<p>is the 128 considered batch_size both for input and output?</p>

ptrblck: <p>Yes, dim0 would correspond to the batch size.</p>