Pytorch question: reshaping problem

username: lifeblack
cooked: <p>Hello!<br>
I am a newbie to deep learning world.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/1/01b159c8b3ccc0e2396f193c1109e584aea06d4d.png" alt="image" data-base62-sha1="eYrSo6REMxvuvqyQlGarA0lSsl" width="649" height="315"></p>
<p>I don’t know haw to code Conv p1,p2,p3.<br>
SR is lower than one.<br>
would you mind helping me code this?</p>

username: ptrblck
cooked: <p>Which dimension does <code>(SR*b*b)</code> describe?<br>
If it’s the channel dimension, you could create a conv layer with <code>in_channels=SR*b*b</code> and <code>out_channels=b*b</code>.<br>
However, based on the provided dimensions I’m a bit confused what type of input you are dealing with.<br>
Could you describe your use case a bit?</p>

username: lifeblack
cooked: <p>hi!<br>
Thanks for your reply.<br>
Suppose I have a image. At first I should extract patches. then flatten each patch and then take samples. (SR is my sampling rate) now I should find a conv layer that reconstructs my sampled data.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/4/5455d9fea02c208149cded05bb05e20bb01b7c3e.png" data-download-href="https://discuss.pytorch.org/uploads/default/5455d9fea02c208149cded05bb05e20bb01b7c3e" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/5455d9fea02c208149cded05bb05e20bb01b7c3e_2_690x201.png" alt="image" data-base62-sha1="c242HAq2eNnaKuzxyaN1ZuvYmo6" width="690" height="201" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/4/5455d9fea02c208149cded05bb05e20bb01b7c3e_2_690x201.png, https://discuss.pytorch.org/uploads/default/original/3X/5/4/5455d9fea02c208149cded05bb05e20bb01b7c3e.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/4/5455d9fea02c208149cded05bb05e20bb01b7c3e.png 2x" data-dominant-color="F4F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">839×245 35.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
In this figure, b refers to the block size and (h<em>b) and (w</em>b) denotes the height and width of the frame.<br>
First, the original frame is reshaped into a tensor of size h×w×(b·b). Then Conv_phi layer senses each block and outputs a measurement tensor of size h×w×(b·b). The kernel size of Conv_phi layer is 1×1×(b·b). The red parts represent a block and its corresponding measurement vector.</p>
<p>now I don’t know how to code this part <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=12" title=":thinking:" class="emoji" alt=":thinking:" loading="lazy" width="20" height="20"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/9/f/9f051a4cd48a76214f9711c0aac8f59041c361b6.png" alt="image" data-base62-sha1="mGKQIMGuhpr6VkYsm7s3BXkh7am" width="689" height="312"><br>
I should use conv3? conv2d?<br>
I cant code it <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=12" title=":thinking:" class="emoji" alt=":thinking:" loading="lazy" width="20" height="20"></p>

username: ptrblck
cooked: <p>I’m still unsure what <code>n</code> stands for, and the highlighted part doesn’t seem to be drawn in the figure.<br>
However, since <code>SR*b*b</code> represent the channels in the output of <code>Conv_phi</code>, I assume <code>b*b</code> represent the channel dimension of the output of <code>Conv_px</code>.<br>
If that’s the case, <code>nn.Conv2d(SR*b*b, b*b*, kernel_size, stride, padding)</code> should work.</p>

username: lifeblack
cooked: <p>Thanks for your reply.<br>
n stands for the number of hypotheses. Suppose a window that our patch is in center of it. There are n patches inside the window. In other words I have a key frame for each image and I should extract n contiguous patches of the patch in key frame inside the window.<br>
anyway…<br>
I wrote your code but i got an error.<br>
code:</p>
<pre><code class="lang-auto">import torch
import numpy as np
import torch.nn as nn
B,h,w,Sr,b,n=16,32,56,0.1,8,25
kernel_size=(1,1,int(np.ceil(0.1*b*b)))
x=torch.rand(B,h,w,int(np.ceil(0.1*b*b)),n)
conv=nn.Conv2d(int(np.ceil(0.1*b*b)),b*b,kernel_size)
conv(x)
</code></pre>
<p>B: batch size, h=height, w=width , SR: sample rate, b= block size.<br>
My kernel size should be: (1,1,SR* b* b).<br>
out channel=(b*b)</p>
<p>Would you please help me solve this error?</p>
<pre><code class="lang-auto">RuntimeError: expected stride to be a single integer value or a list of 3 values to match the convolution dimensions, but got stride=[1, 1]
</code></pre>

username: ptrblck
cooked: <p>If <code>n</code> is therefore supposed to be a the “depth” on the input, you would have to create the input as <code>[batch_size, channels, depth, height, width]</code>:</p>
<pre><code class="lang-python">x=torch.rand(B,int(np.ceil(0.1*b*b)),n,h,w,)
conv=nn.Conv3d(int(np.ceil(0.1*b*b)),b*b,kernel_size)
</code></pre>
<p>and use a 3D convolution.</p>

username: lifeblack
cooked: <p>Thanks for your reply. You solved my problem but still there is a little problem:</p>
<pre><code class="lang-auto">import torch
import numpy as np
import torch.nn as nn
B,h,w,Sr,b,n=16,32,56,0.1,8,25
kernel_size=(1,1,int(np.ceil(0.1*b*b)))
x=torch.rand(B,int(np.ceil(0.1*b*b)),n,h,w)
print(x.shape)
conv=nn.Conv3d(int(np.ceil(0.1*b*b)),b*b,kernel_size)
conv(x).shape
</code></pre>
<p>the output was:</p>
<pre><code class="lang-auto">torch.Size([16, 7, 25, 32, 56])

torch.Size([16, 64, 25, 32, 50])
</code></pre>
<p>Why 56 is changed to 50?<br>
would you mind explain it or introduce me a source that explains this kind of reshaping stuff ?</p>

username: lifeblack
cooked: <p>I had another problem,too.</p>
<pre><code class="lang-auto">#conv_phi
import torch
import numpy as np
import torch.nn as nn
B,h,w,Sr,b,n=16,32,56,0.1,8,25
kernel_size=(1,1,b*b)
x=torch.rand(B,b*b,h,w)
print(x.shape)
conv=nn.Conv3d(b*b,int(np.ceil(0.1*b*b)),kernel_size)
print(conv)
conv(x).shapepe or paste code here
</code></pre>
<p>error:</p>
<pre><code class="lang-auto">Expected 5-dimensional input for 5-dimensional weight 7 64 1 1 64, but got 4-dimensional input of size [16, 64, 32, 56] instead
</code></pre>
<p>I will very thankful if anybody helps me solve this error and explains how to encounter with such errors. <img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=9" title=":sob:" class="emoji" alt=":sob:"> <img src="https://discuss.pytorch.org/images/emoji/apple/sob.png?v=9" title=":sob:" class="emoji" alt=":sob:"></p>

username: ptrblck
cooked: <p>You are using a kernel size of <code>(1, 1, b*b)</code>, so the width dimension will be reduced since you are not using any padding.</p>
<p>The last error is raised, because based on the information you’ve provided, <a href="https://discuss.pytorch.org/t/pytorch-question-reshaping-problem/80410/7">I assumed that <code>n</code> is the depth dimension</a>, which expects a 5D input tensor as given in my previous post.</p>

username: lifeblack
cooked: <p>Thanks for your help. <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>