Spatial Separable Convolution

Tejan_Mehndiratta: <p>Hi,</p>
<p>Can anybody tell me how to code a Spatial Separable Convolution layer?</p>
<p>Thanks.</p>

gtamba: <p>This previous discussion may seem relevant:-</p>
<aside class="quote quote-modified" data-post="13" data-topic="15843">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chirag_kalpesh_shah/48/14945_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-modify-a-conv2d-to-depthwise-separable-convolution/15843/13">How to modify a Conv2d to Depthwise Separable Convolution?</a> 
  </div>
  <blockquote>
    Can you help me in figuring out how do I modify the Depthwise convolution to full convolution? I have them as follows - 
class ConvBNReLU(nn.Sequential):
    def __init__(self, in_planes, out_planes, kernel_size, stride=1, groups=1):
        padding = (kernel_size - 1) // 2
        super(ConvBNReLU, self).__init__(
            nn.Conv2d(in_planes, out_planes, kernel_size, stride, padding, groups=groups, bias=False),
            nn.BatchNorm2d(out_planes),
            nn.ReLU6(inplace=True)
     …
  </blockquote>
</aside>


Tejan_Mehndiratta: <p>Hi,</p>
<p>This is for Depthwise Separable Conv.<br>
I want for Spatial Separable Conv.</p>
<p>Thanks.</p>

gtamba: <p>My bad I misread your post <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_frowning_face.png?v=9" title=":slightly_frowning_face:" class="emoji" alt=":slightly_frowning_face:"></p>
<p>While I’m still here let me try and be a little helpful, but trying to understand the concept of spatial separable convolutions from <a href="https://towardsdatascience.com/a-basic-introduction-to-separable-convolutions-b99ec3102728" rel="noopener nofollow ugc">here</a>:</p>
<p>You need to be able to factor your initial kernel into two different kernels. Since the kernels are real valued floating points learnt from the data, there may not be a solution to decompose every real-valued square NXN matrix into NX1 and 1XN vectors. (<a href="https://math.stackexchange.com/questions/326008/is-it-possible-to-decompose-a-matrix-as-the-product-of-two-vectors" rel="noopener nofollow ugc">Ref</a>)</p>
<p>To have an autograd engine enforce this between two layers would seem very complicated as well even if it were possible, it seems like this is something that would only be possible on the forward pass for pre-determined filters.</p>
<p>But I am unaware if there is a way to do so regardless, just thought this might help</p>

KFrank: <p>Hi Tejan!</p>
<aside class="quote no-group" data-username="Tejan_Mehndiratta" data-post="1" data-topic="121722" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tejan_mehndiratta/48/29857_2.png" class="avatar"> Tejan_Mehndiratta:</div>
<blockquote>
<p>Can anybody tell me how to code a Spatial Separable Convolution layer?</p>
</blockquote>
</aside>
<p>Unless I misunderstand your question, you can use two convolutions in<br>
a row (without an intervening non-linear activation).</p>
<p>So if you want, say, a 5x5 separable convolution (with single channels):</p>
<pre data-code-wrap="python"><code class="lang-python">conv15 = torch.nn.Conv2d (1, 1, (1, 5), bias = False)
conv51 = torch.nn.Conv2d (1, 1, (5, 1), bias = False)
y = conv51 (conv15 (x))
</code></pre>
<p><code>y</code> is now <code>x</code> convolved with a separable 5x5 kernel.  (And as <code>conv15</code><br>
and <code>conv51</code> are trained, their product will, of course, stay separable.)</p>
<p>Best.</p>
<p>K. Frank</p>

Tejan_Mehndiratta: <p>Hi,</p>
<p>This would not be correct. Because in my understanding, what Spatial Separable Conv does is that for e.g.<br>
In the forward pass, there is a 3x3 kernel, then, it would break the kernel into two parts, say, (3x1) and (1x3), and then the convolution process would go on, as usual, 1st (3x1) and then (1x3). Now, in the backward pass, the model should give us a (3x3) kernel, which was our original kernel_size which then <strong>should</strong> be breakable into a (3x1) and (1x3) kernel.</p>
<p>This is my understanding. It might be I am wrong.</p>
<p>Thanks.</p>