How to customize Conv2D with desired shape?

TitsHunter: <p>I want to do modify the shape of the convolution layer so that it doesn’t just calculate the up and down direction. It should be something like the <a href="http://openaccess.thecvf.com/content_iccv_2017/html/Dai_Deformable_Convolutional_Networks_ICCV_2017_paper.html" rel="noopener nofollow ugc">deformable convolution</a> but with less flexibility.<br>
For example, I may want to implement a 2D Conv with shape like this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/e/1e1c107b87810ce736dcd7c11a5381ac442fafbd.png" data-download-href="https://discuss.pytorch.org/uploads/default/1e1c107b87810ce736dcd7c11a5381ac442fafbd" title="directional_conv"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/e/1e1c107b87810ce736dcd7c11a5381ac442fafbd.png" alt="directional_conv" data-base62-sha1="4ims0E1lKCNnujOSk04uFgBIlGd" width="504" height="499" data-dominant-color="DEBAC6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">directional_conv</span><span class="informations">587×582 510 Bytes</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Is that possible with built-in functions?</p>

TitsHunter: <p>Say that the standard convolution is like this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/9/f9a649aa4fc81ad5f22f68a9d961452d09c616c0.png" data-download-href="https://discuss.pytorch.org/uploads/default/f9a649aa4fc81ad5f22f68a9d961452d09c616c0" title="standard_conv"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/9/f9a649aa4fc81ad5f22f68a9d961452d09c616c0.png" alt="standard_conv" data-base62-sha1="zCv9V5LHbTSmg4MUOEWD4YWlk1W" width="503" height="499" data-dominant-color="E29EA9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">standard_conv</span><span class="informations">586×582 494 Bytes</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

TitsHunter: <p>I wan to implement something like this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/a/0a022879b295150803bd42baa2247a08bab22f35.png" data-download-href="https://discuss.pytorch.org/uploads/default/0a022879b295150803bd42baa2247a08bab22f35" title="standard_conv1"><img src="https://discuss.pytorch.org/uploads/default/original/3X/0/a/0a022879b295150803bd42baa2247a08bab22f35.png" alt="standard_conv1" data-base62-sha1="1qxoHyo00xppQ1Q4CqdXLk87hVr" width="504" height="500" data-dominant-color="E2A0AA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">standard_conv1</span><span class="informations">586×581 506 Bytes</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Do the images show the stride pattern of each conv kernel or are they kernel layouts with valid weights?</p>
<p>In the latter case, you could zero out all invalid weight positions at the initialization as well as their gradients after each <code>backward</code> call.</p>

TitsHunter: <p>Yes, they are the kernel layouts.<br>
What do you mean ‘zero out their gradients after each backward call’? Do you mean multiplying a mask to the weights at each iteration?</p>

ptrblck: <p>If you have the mask for the current pattern, you could use it to zero out the initial randomly initialized <code>weight</code> matrix as well as the populated gradients after the <code>backward()</code> call.<br>
Each <code>backward</code> will accumulate the gradients in the <code>param.grad</code> attribute.</p>
<p>Let me know, if you get stuck.</p>

TitsHunter: <p>I got it. So this is implemented in the training code in your solution (instead of writen in the model). Thx.</p>