Dividing the image into patches

username: Roqyah_bdeen
cooked: <p>I have an image batch with size [10,3,256,832]. I want to create a new tensor from each image by dividing each image in this batch into small windows in which the next window will move like in the convolution operation, I mean there is overlapping between the windows. I want also to control the size of each window and the stride. This pic shows what I mean.<br>
could any one tell me how to do it please<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/b/2b63091c5b1b94e53addb0f244cf455374a45a96.png" data-download-href="https://discuss.pytorch.org/uploads/default/2b63091c5b1b94e53addb0f244cf455374a45a96" title="example"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/b/2b63091c5b1b94e53addb0f244cf455374a45a96.png" alt="example" data-base62-sha1="6bOIgCc1keqZkmRR9CLgKQdAxL0" width="690" height="346" data-dominant-color="F4F4F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">example</span><span class="informations">1245×625 12.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p><code>tensor.unfold</code> should work:</p>
<pre><code class="lang-python">x = torch.arange(7*7).view(7, 7)
print(x)
# tensor([[ 0,  1,  2,  3,  4,  5,  6],
#         [ 7,  8,  9, 10, 11, 12, 13],
#         [14, 15, 16, 17, 18, 19, 20],
#         [21, 22, 23, 24, 25, 26, 27],
#         [28, 29, 30, 31, 32, 33, 34],
#         [35, 36, 37, 38, 39, 40, 41],
#         [42, 43, 44, 45, 46, 47, 48]])

out = x.unfold(0, 3, 1).unfold(1, 3, 1)
out = out.contiguous().view(-1, 3*3)
print(out)
# tensor([[ 0,  1,  2,  7,  8,  9, 14, 15, 16],
#         [ 1,  2,  3,  8,  9, 10, 15, 16, 17],
#         [ 2,  3,  4,  9, 10, 11, 16, 17, 18],
#         [ 3,  4,  5, 10, 11, 12, 17, 18, 19],
#         [ 4,  5,  6, 11, 12, 13, 18, 19, 20],
#         [ 7,  8,  9, 14, 15, 16, 21, 22, 23],
#         [ 8,  9, 10, 15, 16, 17, 22, 23, 24],
#         [ 9, 10, 11, 16, 17, 18, 23, 24, 25],
#         [10, 11, 12, 17, 18, 19, 24, 25, 26],
#         [11, 12, 13, 18, 19, 20, 25, 26, 27],
#         [14, 15, 16, 21, 22, 23, 28, 29, 30],
#         [15, 16, 17, 22, 23, 24, 29, 30, 31],
#         [16, 17, 18, 23, 24, 25, 30, 31, 32],
#         [17, 18, 19, 24, 25, 26, 31, 32, 33],
#         [18, 19, 20, 25, 26, 27, 32, 33, 34],
#         [21, 22, 23, 28, 29, 30, 35, 36, 37],
#         [22, 23, 24, 29, 30, 31, 36, 37, 38],
#         [23, 24, 25, 30, 31, 32, 37, 38, 39],
#         [24, 25, 26, 31, 32, 33, 38, 39, 40],
#         [25, 26, 27, 32, 33, 34, 39, 40, 41],
#         [28, 29, 30, 35, 36, 37, 42, 43, 44],
#         [29, 30, 31, 36, 37, 38, 43, 44, 45],
#         [30, 31, 32, 37, 38, 39, 44, 45, 46],
#         [31, 32, 33, 38, 39, 40, 45, 46, 47],
#         [32, 33, 34, 39, 40, 41, 46, 47, 48]])
</code></pre>

username: Roqyah_bdeen
cooked: <p>Thank you so much for your quick replay.<br>
I have another 2 questions please if you don’t mind, first unfold(0,3,1), 3 is the window size which equals=3X3 so if I need to enlarge the window to 25 pixels, 3 should be 5 right?, 1 is the stride I think, so what is 0?<br>
my second question is how can I do this operation for the 3 channels together, the output should have 3 batches, each batch represents one channel, what if I want to do the same for the 10 images together<br>
I hope that my question is clear<br>
thank you in advance</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="Roqyah_bdeen" data-post="3" data-topic="183726">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/roqyah_bdeen/48/39618_2.png" class="avatar"> Roqyah_bdeen:</div>
<blockquote>
<p>so what is 0?</p>
</blockquote>
</aside>
<p>It’s the dimension as described in the <a href="https://pytorch.org/docs/stable/generated/torch.Tensor.unfold.html#torch-tensor-unfold">docs</a>.</p>
<aside class="quote no-group" data-username="Roqyah_bdeen" data-post="3" data-topic="183726">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/roqyah_bdeen/48/39618_2.png" class="avatar"> Roqyah_bdeen:</div>
<blockquote>
<p>how can I do this operation for the 3 channels together</p>
</blockquote>
</aside>
<p>Specify the spatial dimensions only instead of the batch or channel dimension in the <code>unfold</code> operation.</p>

username: Roqyah_bdeen
cooked: <p>for example, i want to apply this unfolding to an image with only 2 channels, but I got this error message<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/e/de1291feef920d7907fc3a78e9e244665a2c2542.png" data-download-href="https://discuss.pytorch.org/uploads/default/de1291feef920d7907fc3a78e9e244665a2c2542" title="ex"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/e/de1291feef920d7907fc3a78e9e244665a2c2542_2_690x96.png" alt="ex" data-base62-sha1="vGxMFZ6SAoZczn1EzXgwFy3RXnY" width="690" height="96" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/e/de1291feef920d7907fc3a78e9e244665a2c2542_2_690x96.png, https://discuss.pytorch.org/uploads/default/original/3X/d/e/de1291feef920d7907fc3a78e9e244665a2c2542.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/e/de1291feef920d7907fc3a78e9e244665a2c2542.png 2x" data-dominant-color="3B1930"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">ex</span><span class="informations">775×108 71.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I don’t understand why<br>
and how to specify the spatial dimension instead of the batch or channel dimension please? could you provide a line of code to explain it?</p>

username: Roqyah_bdeen
cooked: <p>to explain more the operation I want to do<br>
I have one image with 2 channels size=[10,2,256,382] , the second image has 3 channels with size=[10,3,256,382]. I want to divide both images into patches and I did using the following code:</p>
<pre><code class="lang-auto">image2d=torch.rand([10,2,256,382])
image3d=torch.rand([10,,3,256,382])
win_size=5
stride=1
image2d_patches=image2d.unfold(2,win_size,stride).unfold(3,win_size,stride)
image3d_patches=image3d.unfold(2,win_size,stride).unfold(3,win_size,stride)
</code></pre>
<p>after that, I need to multiply the center of each patch in <code>image3d_patches</code> with its surrounding pixels inside the same patch, and subtract the center of each patch in <code>image2d_patches</code> from each surrounding pixel. I did this step by first create two tensors of the centered pixels for all patches in <code>image2d_patches</code> and <code>image3d_patches</code>:</p>
<pre><code class="lang-auto">center_im2d=image2d_patches[:,:,:,:,math.floor(image2d_patches.shapes[4]/2)]
center_im3d=image3d_patches[:,:,:,:,math.floor(image3d_patches.shapes[4]/2)]

</code></pre>
<p>Finally, I perform the multiplication and subtraction as follows:</p>
<pre><code class="lang-auto">mul_out=center_im3d[i,j,k]*image3d_patches[i,j,k,:] for i in range(image3d_patches.shape[0]) for j in range(image3d_patches.shape[1]) for k in range(image3d_patches.shape[2])]
</code></pre>
<p>The subtraction is performed in the same way, by replacing multiplication with subtraction.<br>
Actually, it works, but it is extremely slow even after I reduced the batch size from 10 to 4, is there any efficient way to perform this operation please , I would very much appreciate your help</p>

username: ptrblck
cooked: <p>I would probably try to get id of the nested for loop and check if one could replace it with a single operation.</p>