Conv1d layer performance

username: 1ce9b3de5641ab6f51c6
cooked: <p>Hi, I was working with the <code>Conv1d</code> layer and noticed a weird inference speed degradation comparing 2 ways of input propagation through this layer, lets say we have:</p>
<pre><code class="lang-auto">conv_1 = nn.Conv1d(in_channels=1, out_channels=20, kernel_size=(1, 300))
conv_1.weight.data.fill_(0.01)
conv_1.bias.data.fill_(0.01)

conv_2 = nn.Conv1d(in_channels=300, out_channels=20, kernel_size=1)
conv_2.weight.data.fill_(0.01)
conv_2.bias.data.fill_(0.01)

x1 = torch.FloatTensor(np.ones((10, 1, 100000, 300)))
out1 = conv_1(x1).squeeze(3)

x2 = torch.FloatTensor(np.ones((10, 300, 100000)))
out2 = conv_2(x2)

torch.allclose(out1, out2, atol=1e-6)

&gt;&gt;&gt; True
</code></pre>
<p>Then I tried to measure performance speed for <code>conv_1</code> and <code>conv_2</code> and got the following results:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/1/d1940f69d23825e9fe55aa0f9ba1b496f3116107.png" data-download-href="https://discuss.pytorch.org/uploads/default/d1940f69d23825e9fe55aa0f9ba1b496f3116107" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/1/d1940f69d23825e9fe55aa0f9ba1b496f3116107_2_690x74.png" alt="image" data-base62-sha1="tU10y2YXPkBV55vciBhpb3Gqi8v" width="690" height="74" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/1/d1940f69d23825e9fe55aa0f9ba1b496f3116107_2_690x74.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/1/d1940f69d23825e9fe55aa0f9ba1b496f3116107_2_1035x111.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/d/1/d1940f69d23825e9fe55aa0f9ba1b496f3116107.png 2x" data-dominant-color="F5F5F5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1091×118 8.79 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/0/a05d7723ea0428206b930150f91b2af85f257ffe.png" data-download-href="https://discuss.pytorch.org/uploads/default/a05d7723ea0428206b930150f91b2af85f257ffe" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/0/a05d7723ea0428206b930150f91b2af85f257ffe_2_690x77.png" alt="image" data-base62-sha1="mSEDVllLwWEDeZOcaHPmdJKs9tQ" width="690" height="77" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/0/a05d7723ea0428206b930150f91b2af85f257ffe_2_690x77.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/0/a05d7723ea0428206b930150f91b2af85f257ffe_2_1035x115.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/a/0/a05d7723ea0428206b930150f91b2af85f257ffe.png 2x" data-dominant-color="F5F5F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1082×122 8.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Can please someone explain me this almost 2-x performance degradation and if this issue is reproducible or not?</p>
<p>Config:<br>
PyTorch==1.6.0 via pip<br>
Operating System: Ubuntu 18.04.5 LTS<br>
Kernel: Linux 4.15.0-123-generic<br>
CPU: product: Intel(R) Core™ i5-7200U CPU @ 2.50GHz</p>

username: googlebot
cooked: <p>your input tensors are permuted differently (300 element vectors are either contiguous or scattered), so different strategies may be used to obtain the result, in this case mkldnn library does the inner loop, and in second case avx may be unusable.</p>

username: 1ce9b3de5641ab6f51c6
cooked: <p>I still don’t understand, this is weird behavior, the way, how <code>Conv1d</code> ‘supposed’ to be used is a second way when we processing multichannel 1-d inputs, that’s how documentation proposes to use <code>Conv1d</code>, I didn’t even know till recent times that <code>Conv1d</code> can handle 4-d inputs. So why the “correct” way is 2 times slower, or it is not a “correct” way and I’m missing something?</p>

username: googlebot
cooked: <ol>
<li>you shouldn’t see such a difference on CUDA</li>
<li>conv_2 is faster for me (1.8.0a0 with OMP/MKL threading disabled). you may also see a different picture if you change 100000 -&gt; 100</li>
<li>I’ve just seen a related PR: <a href="https://github.com/pytorch/pytorch/pull/48885" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/pull/48885</a>
</li>
<li>in general, performance &amp; best approach may vary a lot depending on shapes</li>
</ol>

username: 1ce9b3de5641ab6f51c6
cooked: <p>Thanks for the answers, your comments are helpful.</p>