3D depthwise separable convolution

Anno: <p>Hi all,</p>
<p>I try to implement a depthwise separable convolution as described in the <a href="https://arxiv.org/abs/1610.02357" rel="noopener nofollow ugc">Xception paper</a> for <strong>3D input data</strong> (batch size, channels, x, y, z). Is the following class correct or am I missing something?</p>
<pre><code class="lang-auto">class DepthwiseSeparableConv(nn.Module):
  def __init__(self, in_channels, out_channels):
    super(DepthwiseSeparableConv, self).__init__()
    self.depthwise = nn.Conv3d(in_channels=in_channels, out_channels=in_channels, kernel_size=3, stride=1, padding=1, dilation=1, groups=in_channels, bias=False, padding_mode='zeros')
    self.pointwise = nn.Conv3d(in_channels=in_channels, out_channels=out_channels, kernel_size=1, stride=1, padding=0, dilation=1, groups=1, bias=False)
    self.bn = nn.BatchNorm3d(num_features=out_channels)

  def forward(self, x):
    out = self.depthwise(x)
    out = self.pointwise(out)
    return self.bn(out)
</code></pre>
<p>I think it might be necessary to additionally add a parameter for the number of kernels according to <a href="https://github.com/seungjunlee96/Depthwise-Separable-Convolution_Pytorch" rel="noopener nofollow ugc">this example</a>. But what would the number of kernels be then? I could not read something about this in the paper.</p>

nexus_hotaru: <p>Hi, depthwise separable convolution is used instead of ordinary convolution. In your code you just take ordinary convolution using Conv3d plus pointwise convolution. Unique feature of depthwise convolution is work on channels separately (1 filter = 1 input channel), but ordinary convolution processes all channels (1 filter = information from all input layers)</p>

nexus_hotaru: <p>Ok, I’ve explored some other code and now I’m not so sure about my commentary…</p>

J_Johnson: <p>When reading the paper, note:</p>
<p>“A [2d] convolution layer attempts to learn filters in a 3D space, with <strong>2 spatial dimensions (width and height) and a channel dimension</strong>; thus a single convolution kernel is tasked with simultaneously mapping cross-channel correlations and spatial correlations.”<br>
<em>emphasis mine</em></p>
<p>They just mean a Conv2d with 3x3  and stride of 1, followed by 1x1 and stride of 1.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/9/39307e35499493710a7d689b4bf755ef482034b7.png" alt="image" data-base62-sha1="89V6UX4CIshwQKOY6nywj8clfj9" width="357" height="236"></p>
<p>In the above case, you’d set the <code>groups</code> argument to 3. For example:</p>
<pre><code class="lang-auto">simp_inception = nn.Sequential(nn.Conv2d(in_channels, out_channels, (3,3), padding=1, groups=3), 
                               nn.Conv2d(out_channels, out_channels, (1,1), groups=3))
</code></pre>
<p>There are multiple variations in the same paper, but that is the general gist of it.</p>
<p>Lastly, I would like to also bring to your attention that Pytorch now has a native Inception_v3 model you can call:</p>
<aside class="onebox allowlistedgeneric" data-onebox-src="https://pytorch.org/hub/pytorch_vision_inception_v3/">
  <header class="source">
      <img src="https://pytorch.org/favicon.ico?" class="site-icon" width="32" height="32">

      <a href="https://pytorch.org/hub/pytorch_vision_inception_v3/" target="_blank" rel="noopener nofollow ugc">PyTorch</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/362;"><img src="https://pytorch.org/assets/images/social-share.jpg" class="thumbnail" width="690" height="362"></div>

<h3><a href="https://pytorch.org/hub/pytorch_vision_inception_v3/" target="_blank" rel="noopener nofollow ugc">Inception_v3</a></h3>

  <p>import torch
model = torch.hub.load('pytorch/vision:v0.10.0', 'inception_v3', pretrained=True)
model.eval()</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>