Batch-wise F.conv2d Convolution

FrankiePoon: <p>Hi, I am trying to implement a convolution using F.conv2d batch-wise. Below is my current implementation with using a for loop. I was wondering whether there are any ways to avoid using the for loop here?</p>
<pre><code class="lang-auto">import torch.nn.functional as F

B = torch.randn(50, 26)
c = torch.randn(8, 26, 128, 128)
h = torch.randn(8, 1, 50, 1)

for bs in range(c.size(0)):
    # compute Bc
    Bc = torch.matmul(B.unsqueeze(0), torch.reshape(c[bs, :, :, :], (c.size(1), c.size(2) * c.size(3))))
    # compute hBc
    hBc = F.conv2d(Bc.unsqueeze(0), h[bs, :, :, :].unsqueeze(0).flip(2), padding=nPadding).squeeze(0)
    # reshape hBc
    output = torch.reshape(hBc, (1, h.size(2), c.size(2), c.size(3)))
    outputs.append(output)
</code></pre>
<p>Any help would be appreciated. Thanks!</p>

ptrblck: <p>Your padding is undefined and removing it raises a shape mismatch.<br>
Could you also explain what <code>B</code> and <code>c</code> are as it seems <code>Bc</code> is the actual input?<br>
Is the calculation of <code>Bc</code> relevant for the "batch-wise` conv or just a preprocessing step?</p>

FrankiePoon: <p>I defined the padding as following for the convolution</p>
<pre><code class="lang-auto">nPadding = h.size(2)-1
</code></pre>
<p>You are right that Bc is the actual input. However, the matrix multiplication between B and c are performed batch-wise, i.e. each batch c is multiplied by a matrix B. The product Bc then is convolved with h batch-wise. Hope this would be clear.</p>

ptrblck: <p>This this <code>nPadding</code> value I still see:</p>
<pre><code class="lang-python">RuntimeError: shape '[1, 50, 128, 128]' is invalid for input of size 1631718
</code></pre>

FrankiePoon: <aside class="quote no-group" data-username="FrankiePoon" data-post="1" data-topic="177546">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/frankiepoon/48/56313_2.png" class="avatar"> FrankiePoon:</div>
<blockquote>
<pre><code class="lang-auto">import torch.nn.functional as F

B = torch.randn(50, 26)
c = torch.randn(8, 26, 128, 128)
h = torch.randn(8, 1, 50, 1)

for bs in range(c.size(0)):
    # compute Bc
    Bc = torch.matmul(B.unsqueeze(0), torch.reshape(c[bs, :, :, :], (c.size(1), c.size(2) * c.size(3))))
    # compute hBc
    hBc = F.conv2d(Bc.unsqueeze(0), h[bs, :, :, :].unsqueeze(0).flip(2), padding=nPadding).squeeze(0)
    # reshape hBc
    output = torch.reshape(hBc, (1, h.size(2), c.size(2), c.size(3)))
    outputs.append(output)
</code></pre>
</blockquote>
</aside>
<p>I missed one step after convolution, here is the updated code and should work</p>
<pre><code class="lang-auto">import torch.nn.functional as F
import torch



nFrame = 50
B = torch.randn(50, 26)
c = torch.randn(8, 26, 128, 128)
h = torch.randn(8, 1, 50, 1)
nPadding = h.size(2) - 1
outputs = []

for bs in range(c.size(0)):
    # compute Bc
    Bc = torch.matmul(B.unsqueeze(0), torch.reshape(c[bs, :, :, :], (c.size(1), c.size(2) * c.size(3))))
    # compute hBc
    hBc = F.conv2d(Bc.unsqueeze(0), h[bs, :, :, :].unsqueeze(0).flip(2), padding=nPadding).squeeze(0)
    # remove facked indices
    hBc = hBc[:, :nFrame, nPadding:(hBc.size(2) - nPadding)]
    # reshape hBc
    output = torch.reshape(hBc, (1, h.size(2), c.size(2), c.size(3)))
    outputs.append(output)
</code></pre>