Convolution input and output channels

liangstein: <p>Hi,<br>
in convolution 2D layer, the input channel number and the output channel number can be different. What does the kernel do with various input and output channel numbers?<br>
For example, if the input channel number is 32 and the output channel number is 1, how does the kernel converts 32 features into 1 feature? What is the kernel matrix like?</p>

ptrblck: <p>In the vanilla convolution each kernel convolves over the whole input volume.<br>
Example: Your input volume has 3 channels (RGB image).<br>
Now you would like to create a ConvLayer for this image. Each kernel in your ConvLayer will use all input channels of the input volume. Let’s assume you would like to use a 3 by 3 kernel. This kernel will have 27 weights and 1 bias, since (W * H * input_Channels = 3 * 3 * 3 = 27 weights).<br>
The number of output channels is the number of different kernels used in your ConvLayer. If you would like to output 64 channels, your layer will have 64 different 3x3 kernels, each with 27 weights and 1 bias.</p>
<p>I hope this makes it a bit clearer.<br>
Have a look at <a href="http://cs231n.github.io/convolutional-networks/" rel="nofollow noopener">Stanford’s CS231n</a> if your would like to dig a bit deeper.</p>

karmus89: <p>Hi <a class="mention" href="/u/liangstein">@liangstein</a>!</p>
<p>I hope this image helps. Its from my lecture notes and shows how a feature map is produced from an image <em>I</em> with a kernel <em>K</em>. With each application of the kernel a dot product is calculated, which effectively is the sum of element-wise multiplications between <em>I</em> and <em>K</em> in an <em>K</em>-sized area within <em>I</em>.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/6/67be2721e8e4b8cd20bc96ceb351ce44d5156ff6.png" data-download-href="https://discuss.pytorch.org/uploads/default/67be2721e8e4b8cd20bc96ceb351ce44d5156ff6" title="kuva"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/6/67be2721e8e4b8cd20bc96ceb351ce44d5156ff6_2_690x210.png" alt="kuva" data-base62-sha1="eNKzXPSsPumqyc5mlZI5uEKYzuC" width="690" height="210" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/6/67be2721e8e4b8cd20bc96ceb351ce44d5156ff6_2_690x210.png, https://discuss.pytorch.org/uploads/default/original/2X/6/67be2721e8e4b8cd20bc96ceb351ce44d5156ff6.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/6/67be2721e8e4b8cd20bc96ceb351ce44d5156ff6.png 2x" data-dominant-color="E3E9D7"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">kuva</span><span class="informations">691×211 14.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>You could say that kernel looks for diagonal features. It then searches the image and finds a perfect matching feature in the lower left corner. Otherwise the kernel is able to identify only parts the feature its looking for. This why the product is called a feature map, as it tells how well a kernel was able to identify a feature in any location of the image it was applied to.</p>

topiaruss: <p>just a tiny nitpick.</p>
<aside class="quote no-group" data-username="karmus89" data-post="3" data-topic="10205">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/karmus89/48/2210_2.png" class="avatar"> karmus89:</div>
<blockquote>
<p>lower left corner</p>
</blockquote>
</aside>
<p>lower <em>right</em> corner ?</p>

bsash: <p>“Each kernel in your ConvLayer will use all input channels of the input volume.”<br>
I am confused with this line. So Convolution of an RGB image (with 3 channels) with a kernel will generate a Grey Scale Image (with 1 Channel) instead of an RGB image (with 3 channels)?</p>
<p>I was thinking Kernel would be applied to each channel as mentioned by <a class="mention" href="/u/karmus89">@karmus89</a> and hence we would have 3 output channels for 3 input channels.<br>
What am I missing?</p>

ptrblck: <p>Your first explanation is right.<br>
Each kernel will produce one activation map.<br>
If you would like to create 3 output channels, you could use 3 separate filters (all working on all input channels) or use grouped convolution (each filter working on a separate input channel).<br>
Have a look at <a href="http://cs231n.github.io/convolutional-networks/#conv" rel="nofollow noopener">Stanford’s CS231n</a> info on convolutions.</p>

Eduard_Xue: <p>Thanks for explaining it in such a simple way!</p>