Why add an extra dimension to convolution layer weights?

Wesley_Neill: <p>In the following sample class from Udacity’s PyTorch class, an additional dimension must be added to the incoming kernel weights, and there is no explanation as to why in the course. I’ve highlighted this fact by the multi-line comment in <code>__init__</code>:</p>
<pre><code class="lang-auto">
class Net(nn.Module):
    """
    Network containing a 4 filter convolutional layer and 2x2 maxpool layer. 
    """
    def __init__(self, weights):
        """
        weights: the kernel values as a tensor (n_kernels, 1, k_height, k_width).
        """
        super(Net, self).__init__()
        
        # Get height and width of kernel
        k_height, k_width = weight.shape[2:]
        
        # define a 4 feature convolutional layer
        self.conv = nn.Conv2d(1, 4, kernel_size=(k_height, k_width), bias=False)
        self.conv.weight = torch.nn.Parameter(weight)
        
        # define a (2x2) pooling layer
        self.pool = nn.MaxPool2d(2,2)
        
    def forward(x):
        conv_x = self.conv(x)
        relu_x = F.relu(conv_x)
        pool_x = self.pool(relu_x)
        return conv_x, relu_x, pool_x
</code></pre>
<p>This is also illustrated in the class notebook with the following code:</p>
<pre><code class="lang-auto">filter_vals = np.array([[-1, -1 , 1, 1]]*4)
filter_1 = filter_vals
filter_2 = -filter_1
filter_3 = filter_1.T
filter_4 = -filter_3

filters = np.array([filter_1, filter_2, filter_3, filter_4])
weights = torch.from_numpy(filters).unsqueeze(1).type(torch.FloatTensor)
</code></pre>
<p>I’m just wondering why the class wasn’t simply designed to take a kernel tensor that has shape (4,4,4). Why did they change it to (4,1,4,4)?</p>

Nikronic: <p>Hi,</p>
<p>Conv2d needs 2D kernels with 1 channel (grayscale mode, 3 in RGB). For having outputs with more than one, you need to run conv2d <code>out_channel</code> times using <code>[1, k, k]</code> size kernels so the result will be like <code>[out_channel, h, w]</code> because all the respones to <code>out_channel</code> different <code>[1, k, k]</code> kernels have been concatenated.</p>
<p>For instance, assume a case that your input image has 10 dimensions <code>[batch_size, 10, h, w]</code> and you want to have 3 as output channel, <code>[batch_size, 3, h, w]</code>. In this case, we need 3 different filters that each has size of <code>[10, k, k]</code>. Each one will create a output with size of <code>[batch_size, 1, h, w]</code> and finally all will be concatenated to have a output <code>[batch, 3, h, w]</code>. So, the kernel size in this case would be <code>[3, 10, k, k]</code>.</p>
<p>This thread may also help:</p><aside class="quote quote-modified" data-post="1" data-topic="84054">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/helaly96/48/31679_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-does-conv2d-actually-work-from-inside-regarding-channel-size-of-filters/84054">How does conv2d actually work from inside regarding channel size of filters?</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    i have a small question that i actually didn’t think about much until  after some time after finishing a small project and reflecting on the code -obv, i am still a beginner- 
anyways 
let’s say we have 
L1=nn.conv2d(in_channels=3,output_channels=10,kernel=.. etc...) 
so basically what this does it preforms convolution2D on the input image which has 3 Channels and will have 10 channels of depth as an output. 
what this means is this layer is preforming (10) filters on the given image. 
my simple…
  </blockquote>
</aside>

<p>Bests</p>

Wesley_Neill: <p>Thank you again for a wonderful, easy to understand explanation!</p>