Dimensions of an input image

username: Dr_John
cooked: <p>I am new with Pytorch, and will be glad if someone will be able to help me understand the following (and correct me if I am wrong), regarding the meaning of the command x.view in Pytorch first tutorial:</p>
<p><a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html</a></p>
<p>(    def forward(self, x):<br>
x = self.pool(F.relu(self.conv1(x)))<br>
x = self.pool(F.relu(self.conv2(x)))<br>
<strong>x = x.view(-1, 16 * 5 * 5)</strong><br>
x = F.relu(self.fc1(x))<br>
x = F.relu(self.fc2(x))<br>
x = self.fc3(x)<br>
return x)</p>
<ol>
<li>
<p>As far as I understand, an input 256X256 image to a convolutional layer is inserted in its 2D form (i.e. - a 256X256 matrix, or a 256X256X3 in the case of a color image). Nevertheless, when we insert an image to a fully-connected linear layer, we need to first reshape the 2D image into a 1D vector. Is this why we use the command “x = x.view(-1, 16 * 5 * 5)” before inserting x into the fully connected layers?</p>
</li>
<li>
<p>If the input image x would be 3D (e.g. 256X256X256), would the syntax of the given above “forward” function remain the same?</p>
</li>
</ol>
<p>Thanks a lot in advance</p>

username: ptrblck
cooked: <p>Your explanation is right in general.<br>
Just some minor issues:<br>
In PyTorch, images are represented as <code>[channels, height, width]</code>, so a color image would be <code>[3, 256, 256]</code>.<br>
During the training you will get batches of images, so your shape in the forward method will get an additional batch dimension at dim0: <code>[batch_size, channels, height, width]</code>.</p>
<p>The <code>.view()</code> operation gives you a new view on the tensor without copying any data.<br>
This means <code>view</code> is cheap to call and you don’t think too much about potential performance issues.<br>
In your example, <code>x</code> is reshaped into a <code>tensor</code> of shape <code>16*5*5</code> in dim1 and all remaining values in dim0. That’s what the <code>-1</code> stands for.<br>
In the case of a 256 -channel image, the <code>view</code> operation would resume the same, since the spatial dimensions of the image are the same and the channels are defined by the number of kernels of <code>conv2</code>.</p>

username: Dr_John
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="19439">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>ou a new view on the tensor without copying any data.<br>
This means <code>view</code> is cheap to call and you don’t think too much about potential performance issues.<br>
In your example, <code>x</code> is reshaped into a <code>tensor</code> of shape</p>
</blockquote>
</aside>
<p>Thanks a lot ptrbick!</p>

username: Sayak_Paul
cooked: <p>Sorry for revisiting this.</p>
<p>But this particular line seems to suggest the order is <code>n, b, w, h</code> i.e., width comes before height. Probably a mistake but better to clarify.</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/facebookresearch/dino/blob/main/vision_transformer.py#L197">
  <header class="source">

      <a href="https://github.com/facebookresearch/dino/blob/main/vision_transformer.py#L197" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/facebookresearch/dino/blob/main/vision_transformer.py#L197" target="_blank" rel="noopener nofollow ugc">facebookresearch/dino/blob/main/vision_transformer.py#L197</a></h4>



    <pre class="onebox">      <code class="lang-py">
        <ol class="start lines" start="187" style="counter-reset: li-counter 186 ;">
            <li>    patch_pos_embed = nn.functional.interpolate(</li>
            <li>        patch_pos_embed.reshape(1, int(math.sqrt(N)), int(math.sqrt(N)), dim).permute(0, 3, 1, 2),</li>
            <li>        scale_factor=(w0 / math.sqrt(N), h0 / math.sqrt(N)),</li>
            <li>        mode='bicubic',</li>
            <li>    )</li>
            <li>    assert int(w0) == patch_pos_embed.shape[-2] and int(h0) == patch_pos_embed.shape[-1]</li>
            <li>    patch_pos_embed = patch_pos_embed.permute(0, 2, 3, 1).view(1, -1, dim)</li>
            <li>    return torch.cat((class_pos_embed.unsqueeze(0), patch_pos_embed), dim=1)</li>
            <li>
            </li>
<li>def prepare_tokens(self, x):</li>
            <li class="selected">    B, nc, w, h = x.shape</li>
            <li>    x = self.patch_embed(x)  # patch linear embedding</li>
            <li>
            </li>
<li>    # add the [CLS] token to the embed patch tokens</li>
            <li>    cls_tokens = self.cls_token.expand(B, -1, -1)</li>
            <li>    x = torch.cat((cls_tokens, x), dim=1)</li>
            <li>
            </li>
<li>    # add positional encoding to each token</li>
            <li>    x = x + self.interpolate_pos_encoding(x, w, h)</li>
            <li>
            </li>
<li>    return self.pos_drop(x)</li>
        </ol>
      </code>
    </pre>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


username: ptrblck
cooked: <p>I’m not familiar enough with the repository so cannot comment on the correctness or if internal permutations are applied to the tensor.<br>
The standard is <code>[N, C, H, W]</code> for channels-last and <code>[N, H, W, C]</code> for channels last.<br>
Here is a small code snippet which draws a vertical line in the H dimension at W=20:</p>
<pre data-code-wrap="python"><code class="lang-python"># create input image
N, C, H, W = 1, 3, 200, 300
x = torch.zeros(N, C, H, W).to(torch.uint8)
# draw line in H dimension at W=30
x[:, :, :, 20] = 255

img = transforms.ToPILImage()(x[0])
arr = np.array(img)
print(arr.shape)
# (200, 300, 3) # H, W, C
plt.imshow(arr)
</code></pre>
<p>Output:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/e/5eba42c21f672f600bcf5338b0b94abdd79eece2.png" alt="image" data-base62-sha1="dvZWx5o5AM3dfUJYaojtnugH2vM" width="366" height="252"></p>
<p>Given the output and the dimension of the output image, the format in PyTorch is <code>[N, C, H, W]</code> as the default while <code>arr</code> uses the channels-last format as <code>[H, W, C]</code> (the batch dimension is missing as <code>imshow</code> expects a single image).</p>