How to construct the most efficient convolution in the case of large kernel_size

username: Aslan
cooked: <p>When testing the computational efficiency of convolution with different  kernel sizes, it is found that when the kernel sizes are 3, 5 and 7, the computational efficiency is very high, but when the kernel size is greater than 7, the computational efficiency will be greatly reduced. In addition, when the kernel siezes  are 31, 39, 45, 55, 75, 93, and 101, the efficiency will be higher than their adjacent sizes. What is the reason?</p>
<pre><code class="lang-auto">import numpy as np
import torch
from torch import nn
import time

h = 6115
w = 5490
pic = np.random.randint(0,2,(h,w),dtype=(np.int16))
Max_iters = 5
for kernel_size in range(3,110,2):    

    kernel = torch.ones((1,1,kernel_size,kernel_size))
    neighbor_func = nn.Conv2d(1, 1, kernel_size,padding=int((kernel_size-1)/2),bias = False)
    
    with torch.no_grad():
        for i,j in neighbor_func.named_parameters():
            j.copy_(kernel)
   
    
    tst = time.time()
    for i in range(Max_iters):
        N = np.where(pic == 1,1,0)
        N = np.reshape(N,(1,1,h,w))
        N = torch.from_numpy(N).type(torch.float32)
        with torch.no_grad():
            N = neighbor_func(N)
        N = N.detach().numpy()
        N = np.squeeze(N)
    print(kernel_size,time.time()-tst)  
</code></pre>
<p>I want to use nn.Conv2d as a tool for convolution, which only performs convolution calculation on data without involving gradient calculation and back propagation, so which function can I choose to obtain the highest performance convolution in large kernel_sizes?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/c/5c52e29377a86b992d6442e7d305d0dab245da7e.png" alt="20220527121918" data-base62-sha1="daJvtDifI1CZ2fsiJ8TMei3YnHE" width="506" height="368"></p>

username: eqy
cooked: <p>If you are on CPU, a guess would be that these sizes fit nicely into the shapes that the kernels in the library are well-optimized for. Additionally, you might consider packing the tensor in mkldnn layout <a href="https://pytorch.org/docs/stable/generated/torch.Tensor.to_mkldnn.html#torch.Tensor.to_mkldnn" class="inline-onebox" rel="noopener nofollow ugc">torch.Tensor.to_mkldnn — PyTorch 1.11.0 documentation</a> and seeing if you get a different pattern for the “most efficient” kernel sizes.</p>

username: Aslan
cooked: <p>Thank you for your advice! I try to use the mkldnn. But I don’t know if it’s the right way to use it.</p>
<pre><code class="lang-auto">import numpy as np
import torch
from torch import nn
import time
size_list = [3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51]
h = 6115
w = 5490
pic = np.random.randint(0,2,(h,w),dtype=(np.int16))
Max_iters = 5
for kernel_size in range(3,109,2):    

    kernel = torch.ones((1,1,kernel_size,kernel_size))
    neighbor_func = nn.Conv2d(1, 1, kernel_size,padding=int((kernel_size-1)/2),bias = False)
    
    with torch.no_grad():
        for i,j in neighbor_func.named_parameters():
            j.copy_(kernel)
    
    
    tst = time.time()
    for i in range(Max_iters):
        N = np.where(pic == 1,1,0)
        N = np.reshape(N,(1,1,h,w))
        N = torch.from_numpy(N).type(torch.float32).to_mkldnn()
        with torch.no_grad():
            N = neighbor_func(N)
        N = N.detach().to_dense().numpy()
        N = np.squeeze(N)
    print(kernel_size,time.time()-tst)
</code></pre>
<p>I modify the code and can not get  performance improvements.</p>

username: eqy
cooked: <p>I’m a bit confused about the code is written; why does the numpy conversion need to happen in the hot loop here? That might add a lot of noise/overhead to the part of the computation that is relevant for benchmarking.</p>
<p>I would also check if MKLDNN is installed and being used:<br>
(e.g., <a href="https://gist.github.com/mingfeima/bdfb2db3928ca51b795622b29264ef11" class="inline-onebox" rel="noopener nofollow ugc">BKMs to check whether mkl or mkldnn is enabled on PyTorch · GitHub</a>)</p>

username: Aslan
cooked: <p>This is part of my code and numpy array is used in other parts. I tested the overhead of type conversion. Before using mklnn, it was 0.03s/ time, and then it was 0.09s/ time, which can be negligible compared with the computational cost of convolution. I used torch Backends Mkldnn Is_ Available(), torch.backends.mkl.is_available() and both returned true</p>