How to build my own convolutional method in Pytorch?

11179: <p>Hello Im new to deeplearning<br>
I want to test something so I want to make own convolution 2d method.<br>
I want to design as follows. kernel size = T</p>
<ol>
<li>Giving input Channel x H x W with kernel also Channel x T x T. Same as before</li>
<li>But each kernel with one multiplication gives output T x T not as a one number.</li>
<li>So It is like kind of projection to surface of TxT<br>
killing channel dimension.</li>
</ol>
<p>How can I modify in Pytorch but still having properties same like autograd or other stuffs.</p>

ptrblck: <p>You could use <code>unfold</code> to create the patches, apply your custom operation using your filters, and <code>fold</code> or reshape the output back.<br>
<a href="https://discuss.pytorch.org/t/patch-making-does-pytorch-have-anything-to-offer/33850/11">This post</a> gives you an example of unfolding the input.</p>

11179: <p>Hello <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"><br>
I’m beginner in DL&amp;Pytorch and got helped so many times via other answers of yours  in this page.<br>
So it’s very glad to see you here first question of mine.</p>
<p>However, in this case unfold would might not be the case.<br>
I am stocked because of  custom operation gives one value through one element-wise multiplication, but what I want is to bring (kernel_size x kernel_size) output of each element-wise multiplication. After multiplying, just sum down in dimension of channel.</p>
<p>I’ve considered view or unfold, but I think it can not help.</p>
<p>This way I will get input size bigger, for example if 3x3 is kernel_size<br>
Input_size will grow 3x224,3x224.(Start from 224*224)</p>
<p>And then calculating new convolution of 3x3 by stride=3 will give us the normal operation does.</p>
<p>The reason I want to do this process is because I want to quantize part of it. Not whole.<br>
Is there a way to solve this?</p>
<p>Thank you.</p>

ptrblck: <p>How would you like to reshape the output?<br>
<code>unfold</code> will create the patches and you could use it as an <code>im2col</code> method.<br>
Once you get all patches you could apply the desired operation. Would you like to apply another convolution with <code>stride=3</code> afterwards?</p>

11179: <p>Thanks for reply,<br>
I got the point of unfold &amp; desired operation through it.<br>
It could bring desired output for me.</p>
<blockquote>
<pre><code>    desired operation    stride=3 conv
</code></pre>
<p>[ B, C, H, W ]   &gt;&gt;      [ B, C*, 3xH , 3xW ]      &gt;&gt;    [ B, C*, H , W ]</p>
</blockquote>
<p>This is example of what I want in case of kernel=3, so I could quantize the weights in desired operation. After stride=3, it is just same with normal convolution.<br>
.</p>
<p>However I want my function to get weights &amp; internally do all work like padding extra like nn.conv2d does, but it is hard for me to build from start what does in nn.conv2d does exactly <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=12" title=":joy:" class="emoji" alt=":joy:" loading="lazy" width="20" height="20"></p>
<p>So what I tried was putting my function to nn.modules.conv.py function, but it did not work also.</p>
<p>So , I’ve been trouble still<br>
Is there no way to just modify nn.conv2d to serve as desired function?<br>
…<br>
…<br>
…<br>
It’s flow for simple case for stride=1, I just wrote it for in case, lack of explanations above.</p>
<blockquote>
<p>for batchs in range(batch):<br>
…for channels in range(channel):<br>
ㄴ…for i in range(move):<br>
ㄴㄴ…for j in range(move):<br>
ㄴㄴㄴ…out [batchs, channels, kernel * i: kernel * i+kernel, kernel * j: kernel * j+kernel]= <br>
ㄴㄴㄴ…torch.sum(input[batchs, :, i:i+kernel, j:j+kernel] * weigt[channels,:,:,:], dim=1)</p>
</blockquote>

ptrblck: <p>You could try to manipulate the native conv2d implementation, but note that you won’t be able to manipulate the backend implementations from e.g. cudnn.</p>
<p>I’m also not sure, if that would be easier than to write out your operations manually.<br>
The <code>padding</code> argument in <code>nn.Conv2d</code> can be simply added to the unfolding approach using <code>F.pad</code>.<br>
<code>stride</code>, <code>kernel_size</code> and <code>dilation</code> are all set in the <code>unfold</code> call.</p>
<p>You could of course write the nested loop as a CPU/CUDA extension and use it in your Python script as explained <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html">here</a>.</p>

11179: <p>Oh, I’ve got it now,<br>
Thank you very much.<br>
You’ve made my day <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>