I am going to create a custom layer that can be pre-processed for each layer

khero: <p>I am trying to create a layer that is executed every batch in a layer using the torch.<br>
Previously, I tried to write it with TensorFlow and Keras, but NumPy was not possible, so I used PyTorch. When I used NumPy in pytroch, it was run with cpu and it was too slow.<br>
So, in the end, I wrote NumPy by replacing it with the function of the torch.<br>
I am currently training, but it is too slow and the accuracy is 10%. (take 5hour per a epoch)<br>
The sequence of the code is as follows:</p>
<pre><code class="lang-auto">class my_custom_layer(nn.Module):
    def __init__(self, D_in, D_out, kernel_size):
        super(my_custom_layer, self).__init__()
        self.BConv_layer = nn.Conv2d(D_in, D_out, kernel_size, padding=(1))
    def forward(self, x):      
        with torch.no_grad(): 
            y = my_function(x)
            #preprocess batch using histogram data
            #make histogram of img -&gt; Partially scaling according to the characteristics of the image -&gt; return scaling factor for result of conv layer

        results = F.conv(x, kernel)
        results = torch.multiply(results, y)
        return results
</code></pre>
<p>Sometimes it runs on the CPU, but most of the time it runs on the GPU.<br>
I will replace this layer with VGG’s conv layer.<br>
Is it correct to write a custom layer like this?</p>

ptrblck: <p>Your custom layer isn’t using the initialized <code>self.BConv_layer</code> and is using undefined arguments, such as <code>kernel</code> and <code>y</code>, instead, so this won’t work.</p>
<p>Besides that, I’m unsure if you need the <code>no_grad</code> block. If <code>my_function</code> isn’t using any parameters, you could probably remove it.<br>
Also, if you can replace all numpy operations with their PyTorch equivalent, you should be able to use the GPU for all operations. If that’s not possible, note that each data transfer to the CPU would be blocking and might yield a slowdown.</p>

khero: <p>I have a question about F.conv.<br>
In general, nn.conv is declared in intial and used in forward. On the other hand, should F.conv be declared in intial before written? if it’s true, how to init F.conv?<br>
and in <code>my_function</code>, there are preprocessing input data using torch’s function. but in my opinion, for <code>my_function</code>, I used no_grad because I thought that the parameters they used were unnecessary for calculating gradients.</p>

shivammehta007: <p><code>F.convXX</code> is a functional form of convolution where you will manually have to put filters and bias tensors and manage their operation during testing and training time, which is useful for custom implementation of functionality and it does not require initialisation in the <code>__init__</code>  while if you are just using it as the generic convolution I suggest you use the one present in <code>nn.ConvXX</code> if you will look at its source code in the back it also calls <code>f.convXX</code> but manages the filters and bias tensors for you and their operations during test and train time and it can make use of features of <code>nn.Module</code> like <code>self.training</code> flag for example.</p>

khero: <p>The reason I used f.conv is to convolution each channel of input with kernel and then multiply it by a scaling factor.<br>
Is this also possible using nn.conv?</p>

shivammehta007: <p>Yes! You can, in forward function you pass the features through a convolution layer of (<code>nn.CovXX</code>) and then multiply it with a scaling factor an example will be like:</p>
<pre><code class="lang-auto">class MyCustomLayer(nn.Module):
    def __init__(self, D_in, D_out, kernel_size):
        super(my_custom_layer, self).__init__()
        self.BConv_layer = nn.Conv2d(D_in, D_out, kernel_size, padding=(1))
    def forward(self, x):

        results = y * self.BConv_layer(x) # where y is your scaling factor
        return results
</code></pre>