Periodic padding

username: Pengchuan_Zhang
cooked: <p>In Pytorch, I cannot find a straightforward possibility to do a convolution (nn.conv2d) with periodic boundary conditions. For example, take the tensor<br>
[[1,2,3],<br>
[4,5,6],<br>
[7,8,9]]<br>
and any 3x3 filter. A convolution with periodic boundary conditions could in principle be done by doing a periodic padding to 5x5<br>
[[9,7,8,9,7],<br>
[3,1,2,3,1],<br>
[6,4,5,6,4],<br>
[9,7,8,9,7],<br>
[3,1,2,3,1]]<br>
and subsequently a convolution with the filter in “padding=0” mode. Unfortunately, I cannot find a padding method that supports this periodic padding.</p>
<p>Is there a simple walkaround?</p>

username: fmassa
cooked: <p>You can use<a href="http://pytorch.org/docs/master/nn.html#torch.nn.ReflectionPad2d"> <code>nn.ReflectionPad2d</code></a> for that, or the functional interface <a href="http://pytorch.org/docs/master/nn.html?highlight=pad#torch.nn.functional.pad"><code>F.pad</code></a></p>

username: Pengchuan_Zhang
cooked: <p><a class="mention" href="/u/fmassa">@fmassa</a>, thank you for replying! However, I don’t think that the “ReflectionPad2d” does what I want. The result given by “ReflectionPad2d” for my example is<br>
[[5,4,5,6,5],<br>
[2,1,2,3,2],<br>
[5,4,5,6,5],<br>
[8,7,8,9,8],<br>
[5,4,5,6,5]],<br>
not what I want for the periodic padding.</p>
<p>Periodic padding is a natural choice for problems with periodic boundary conditions in physics.</p>

username: fmassa
cooked: <p>I see. If you have a function (say in numpy or scipy) that performs this periodic padding for you, you could somewhat easily write an autograd <code>Function</code> that perform this operation.<br>
That will require writing the backward for this operation, but that can be performed by using <code>index_add</code> function.<br>
Something like (<strong>warning</strong>: untested)</p>
<pre><code class="lang-python">from autograd import Function
class PeriodicPadding2d(Function):
    @staticmethod
    def forward(ctx, input, pad):
        output = np.periodicpad(input, pad)  # find the function that performs what you want
        ctx.pad = pad
        ctx.size = input.size()
        ctx.numel = input.numel()
        return output

    @once_differentiable
    @staticmethod
    def backward(ctx, grad_output):
        pad = ctx.pad
        idx = grad_output.new(ctx.size)
        torch.arange(0, ctx.numel, out=idx)
        idx = np.periodicpad(idx, pad)  # or whatever is the function
        grad_input = grad_output.new(ctx.numel).zero_()
        grad_input.index_add_(0, idx, grad_output.view(-1))
        return grad_input.view(ctx.size)
</code></pre>

username: nancy_goyal
cooked: <p>Could you find it? I need the same function for an assignment.</p>

username: azeeshan
cooked: <p>You can use wrap padding in numpy.</p>
<pre><code class="lang-auto">import numpy as np
a = np.array([[1,2,3],[4,5,6],[7,8,9]])
b = np.pad(a, (1,1), 'wrap') 
</code></pre>
<p>b can be used as input for the <code>nn.conv2d</code></p>

username: conv8d
cooked: <p>I think this will create a bottleneck in GPU based implementations</p>

username: alsina
cooked: <p>I came across this same problem and it seems that now it is possible to implement periodic boundary conditions using the functional interface <code>F.pad</code> and <code>mode='circular'</code>.</p>