Splitting image into small batches

iuryt: <p>Hi,<br>
I am new to Pytorch but would like to do the following.<br>
Lets say I have 37 images with two channels and size <code>256x256</code>.<br>
In other words, my data is the shape <code>(37,2,256,256)</code>.<br>
I want to split each image into <code>(256/stride)x(256/stride)</code> images of size <code>kernel_shape</code> assuming <code>circular/periodic</code> padding. I am assuming that each small batch is independent from each other.</p>
<p>For example, assuming <code>stride=1</code> and <code>kernel_shape=(5,5)</code>, a data like <code>(37,2,256,256)</code> will become <code>(37*256*256,2,5,5)</code>. I initially thought that <code>Conv2D</code> could be used for that, but after seeing examples I realized this is actually for creating convolutional layers.</p>

ptrblck: <p>I think <code>unfold</code> should work. <a href="https://discuss.pytorch.org/t/creating-nonoverlapping-patches-from-3d-data-and-reshape-them-back-to-the-image/51210/6">This post</a> gives you an example.</p>

iuryt: <p>I haven’t tried yet, but I think I understand the code. Thanks!<br>
That solves the part o breaking into patches. Do Pytorch have anything for adding the padding needed for the periodic/circular padding? I want that the every pixel has it’s own patch.<br>
Edit: Just noticed <code>F.pad</code> has option for periodic boundary condition.</p>

iuryt: <p>I was exploring the padding options and</p>
<pre><code class="lang-auto">x = torch.randn(2, 37, 256, 256)+10
S = x.shape

kh, kw = 101, 101  # kernel size
dh, dw = 1, 1  # stride

pad = (S[-1]%kw // 2, S[-1]%kw // 2, S[-2]%kh // 2, S[-2]%kh // 2, 0, 0, 0, 0)
# Pad to multiples of 32
x = F.pad(x, pad)

plt.imshow(x[0,0,:,:])
</code></pre>
<p>Returns<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/d/bd98fb3ecbaa06fab785b1e0f14cddff8c4783f1.png" alt="image" data-base62-sha1="r3g0bXjbO9C2z6XuGnVCr066S0F" width="260" height="252"></p>
<p>But selecting <code>mode="circular"</code> gives <code>NotImplementedError</code>.</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
/home/gridsan/groups/dedalus_group/gfd22/stochastic_closures_repo/src/01-model/ml/cnn.py in &lt;cell line: 11&gt;()
      40 pad = (S[-1]%kw // 2, S[-1]%kw // 2, S[-2]%kh // 2, S[-2]%kh // 2, 0, 0, 0, 0)
     41 # Pad to multiples of 32
---&gt; 42 x = F.pad(x, pad, mode="circular")
     44 plt.imshow(x[0,0,:,:])

File /state/partition1/llgrid/pkg/anaconda/anaconda3-2022b/lib/python3.8/site-packages/torch/nn/functional.py:4397, in _pad(input, pad, mode, value)
   4395         raise NotImplementedError
   4396 else:
-&gt; 4397     raise NotImplementedError("Only 2D, 3D, 4D, 5D padding with non-constant padding are supported for now")

NotImplementedError: Only 2D, 3D, 4D, 5D padding with non-constant padding are supported for now
</code></pre>
<p><code>torch.__version__ = 1.11.0+cu113</code><br>
Not sure I understand this error, because for me, this is a 4D padding.</p>
<p>I tried to perform the padding for each channel and snapshot and got the same error.</p>
<p>Similar code works for <code>numpy.pad</code>.</p>

ptrblck: <p>Circular padding seems to have more limitations as given in the <a href="https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html#torch.nn.functional.pad">docs</a> so you might need to pass 4 values only:</p>
<pre><code class="lang-python">x = torch.randn(2, 37, 256, 256)+10
S = x.shape

kh, kw = 101, 101  # kernel size
dh, dw = 1, 1  # stride

pad = (S[-1]%kw // 2, S[-1]%kw // 2, S[-2]%kh // 2, S[-2]%kh // 2, 0, 0, 0, 0)
y1 = F.pad(x, pad)
print(y1.shape)

pad = (S[-1]%kw // 2, S[-1]%kw // 2, S[-2]%kh // 2, S[-2]%kh // 2)
y2 = F.pad(x, pad, mode="circular")
print(y2.shape)
</code></pre>