DepthWise1dConv with causal padding. Need help

ergaster: <h1>
<a name="explanation-1" class="anchor" href="#explanation-1"></a>Explanation</h1>
<p>I want to implement a DepthWise1dConv with causal padding. That is, padding should be applied before the signal starts.</p>
<p>I followed this wonderful <a href="https://github.com/seungjunlee96/Depthwise-Separable-Convolution_Pytorch" rel="noopener nofollow ugc">resource</a> to understand depth-wise convolutions, although explained for the 2D case, I assume it its directly applicable to the 1D case.</p>
<p>From the torch.nn.Conv1d <a href="https://pytorch.org/docs/stable/generated/torch.nn.Conv1d.html" rel="noopener nofollow ugc">documentation</a>, whenever padding is added, it is added to both sides of the input (and it can be observed in the module variable <code>_reversed_padding_repeated_twice</code>. But this is not the desired behavior.</p>
<p>However, whenever I try to set the padding manually by using <code>torch.nn.functional.pad</code> I get an error. What am I missing here?</p>
<h1>
<a name="minimal-reproducible-example-2" class="anchor" href="#minimal-reproducible-example-2"></a>Minimal reproducible example</h1>
<pre><code class="lang-auto">import torch

x = torch.rand(1, 64, 384) # (B, C, L)
conv1d = torch.nn.Conv1d(in_channels=64, out_channels=64, kernel_size=17, padding='same', groups=64)
print(f"{conv1d(x).shape=}, {conv1d._reversed_padding_repeated_twice=}")

p3d = (0, 0, 16, 0, 0, 0)
x = torch.nn.functional.pad(x, p3d)
print(f"{x.shape=}")
conv1d = torch.nn.Conv1d(in_channels=80, out_channels=64, kernel_size=17, padding='valid', groups=80)
conv1d(x)
</code></pre>
<h2>
<a name="output-3" class="anchor" href="#output-3"></a>Output</h2>
<pre><code class="lang-auto">conv1d(x).shape=torch.Size([1, 64, 384]), conv1d._reversed_padding_repeated_twice=[8, 8]
x.shape=torch.Size([1, 80, 384])
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[98], line 10
      8 x = torch.nn.functional.pad(x, p3d)
      9 print(f"{x.shape=}")
---&gt; 10 conv1d = torch.nn.Conv1d(in_channels=80, out_channels=64, kernel_size=17, padding='valid', groups=80)
     11 conv1d(x)

File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/conv.py:300, in Conv1d.__init__(self, in_channels, out_channels, kernel_size, stride, padding, dilation, groups, bias, padding_mode, device, dtype)
    298 padding_ = padding if isinstance(padding, str) else _single(padding)
    299 dilation_ = _single(dilation)
--&gt; 300 super().__init__(
    301     in_channels, out_channels, kernel_size_, stride_, padding_, dilation_,
    302     False, _single(0), groups, bias, padding_mode, **factory_kwargs)

File /opt/conda/lib/python3.10/site-packages/torch/nn/modules/conv.py:92, in _ConvNd.__init__(self, in_channels, out_channels, kernel_size, stride, padding, dilation, transposed, output_padding, groups, bias, padding_mode, device, dtype)
     90     raise ValueError('in_channels must be divisible by groups')
     91 if out_channels % groups != 0:
---&gt; 92     raise ValueError('out_channels must be divisible by groups')
     93 valid_padding_strings = {'same', 'valid'}
     94 if isinstance(padding, str):

ValueError: out_channels must be divisible by groups
</code></pre>
<hr>
<h1>
<a name="in-tensorflow-4" class="anchor" href="#in-tensorflow-4"></a>In tensorflow</h1>
<pre><code class="lang-auto">import tensorflow as tf

x = tf.random.uniform((1, 64, 384))
print(f"{x.shape=}")
causal_pad = tf.keras.layers.ZeroPadding1D((16,0))(x)
print(f"{causal_pad.shape=}")
dw_conv = tf.keras.layers.DepthwiseConv1D(
    kernel_size=17,
    strides=1,
    dilation_rate=1,
    padding='valid',
    use_bias=True,
    depthwise_initializer='glorot_uniform')(causal_pad)
print(f"{dw_conv.shape=}")
</code></pre>
<h2>
<a name="output-5" class="anchor" href="#output-5"></a>Output</h2>
<pre><code class="lang-auto">x.shape=TensorShape([1, 64, 384])
causal_pad.shape=TensorShape([1, 80, 384])
dw_conv.shape=TensorShape([1, 64, 384])
</code></pre>

ptrblck: <p>I’m not deeply familiar with Keras, but based on the shape I assume you are padding the temporal dimension.<br>
In this case this code should work:</p>
<pre><code class="lang-python">import torch

x = torch.rand(1, 64, 384) # (B, C, L)
conv1d = torch.nn.Conv1d(in_channels=64, out_channels=64, kernel_size=17, padding='same', groups=64)
print(f"{conv1d(x).shape=}, {conv1d._reversed_padding_repeated_twice=}")

p3d = (16, 0, 0, 0, 0, 0)
x = torch.nn.functional.pad(x, p3d)
print(f"{x.shape=}")
conv1d = torch.nn.Conv1d(in_channels=64, out_channels=64, kernel_size=17, padding='valid', groups=64)
out = conv1d(x)
print(out.shape)
# torch.Size([1, 64, 384])
</code></pre>