Do I Need Different Conv Layers to perform Regression and Classification?

PCPJ: <p>I created a CNN model that takes as input a 3x512x512 image, and reduces it, through convolutions and maxpoolings, to a 1024x32x32 cube. In this cube I am using a 1x1 kernel conv layer to make a binary classification and a (x, y) regression for each of the 32x32 cells.</p>
<p>So I have a Conv2d later like this:</p>
<pre><code class="lang-auto">predConv = nn.Conv2d(1024, 3, kernel_size=1, padding=0)
</code></pre>
<p>My question is: there is any difference between using a single conv layer with 3 kernels (the first for classification, and the last two for regression), and two conv layers, one with 1 kernel (for classification) and another with 2 kernels (for regression), like this:</p>
<pre><code class="lang-auto">predClass = nn.Conv2d(1024, 1, kernel_size=1, padding=0)
predXYCoord = nn.Conv2D(1024, 2, kernel_size=2, padding=0)
</code></pre>
<p>I think that the answer is that there is no difference, but my classification is getting a good accuracy (0.85) and the regression is outputting nonsense values. I am using BCEWithlogitsloss and MSELoss as loss functions.</p>
<p>Thank you very much, and sorry for the silly question.</p>

ptrblck: <p>Both approaches should yield the same result, if you are using the same setup besides the <code>out_channels</code>.<br>
Note that <code>predXYCoord</code> uses <code>kernel_size=2</code>, while the others use a kernel size of 1.</p>
<p>Here is a small example:</p>
<pre><code class="lang-python"># Single layer
input = torch.randn(1, 10, 32, 32, requires_grad=True)
conv1 = nn.Conv2d(10, 3, 3, 1, 1, bias=False)

out1 = conv1(input)

out1[:, :2].mean().backward(retain_graph=True)
out1[:, 2:].mean().backward()

input_grad_reference = input.grad.clone()
conv_grad_reference = conv1.weight.grad.clone()

input.grad.zero_()

# Two layers
conv2 = nn.Conv2d(10, 2, 3, 1, 1, bias=False)
conv3 = nn.Conv2d(10, 1, 3, 1, 1, bias=False)
with torch.no_grad():
    conv2.weight.copy_(conv1.weight[:2])
    conv3.weight.copy_(conv1.weight[2:])
    
out2 = conv2(input)
out3 = conv3(input)

out2.mean().backward()
out3.mean().backward()

# Compare
print((input.grad == input_grad_reference).all())
&gt; tensor(True)
print((conv2.weight.grad == conv_grad_reference[:2]).all())
&gt; tensor(True)
print((conv3.weight.grad == conv_grad_reference[2:]).all())
&gt; tensor(True)
</code></pre>

PCPJ: <p>Thank you very much for your answer. I was 99% sure that they yield the same result, I just could not find the error on my code. But I think have just found it.</p>
<p>I was permuting the output channels after separating them. Like this:</p>
<pre><code class="lang-auto">classification = out[:, 0]  # (N, 32, 32)
regression = out[:, 1:]     # (N, 2, 32, 32)
regression = regression.permute(0, 2, 3, 1).contiguous()  # (N, 32, 32, 2)
</code></pre>
<p>Now I am permuting before separating the classifications regression outputs, and the regression results are something meaningful.</p>
<p>By the way, the kernel_size=2 on my question was a typo, it should be 1 on both layers. Is there any reason why you used kernel_size=3 in your examples?</p>
<p>Thank you again!</p>

ptrblck: <aside class="quote no-group" data-username="PCPJ" data-post="3" data-topic="76217">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/47e85d/48.png" class="avatar"> PCPJ:</div>
<blockquote>
<p>Now I am permuting before separating the classifications regression outputs, and the regression results are something meaningful.</p>
</blockquote>
</aside>
<p>Sounds good! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<aside class="quote no-group" data-username="PCPJ" data-post="3" data-topic="76217">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/47e85d/48.png" class="avatar"> PCPJ:</div>
<blockquote>
<p>Is there any reason why you used kernel_size=3 in your examples?</p>
</blockquote>
</aside>
<p>No, it’s just my default setup for debugging without any particular reason.<br>
I’ve probably seen VGG/AlexNet etc. too often and just write the default conv layer. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>