Too slow conv2d and AvgPool2d for large kernel sizes

smani: <p>I have an image of size 2000 x 2000 pixels. I need to find the average from all the 200 x 200 windows. When I try in Matlab to find the average I am able to get the results quickly. However, when I use pytorch it takes a very long time/hangs. How can I quickly find the average of all the windows of size 200x200?</p>
<p>The approach I tried in Pytorch is:</p>
<pre><code class="lang-auto">        I = torch.rand(2000,2000)
        pool = nn.AvgPool2d(kernel_size=200, stride=1, padding=100)
        re = pool(I.unsqueeze(0))
</code></pre>

ptrblck: <p>You could push the data and pooling layer to the GPU for a potential speedup.<br>
Note however, that you might fallback to the native im2col implementation for this particular use case, if e.g. cudnn cannot find a fast kernel for this workload.</p>

smani: <p>But when I use the following</p>
<pre><code class="lang-auto">I = I.cuda(0)
</code></pre>
<p>I am getting an error “Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the ‘spawn’ start method”. Could you suggest me the cuda code for the image and layer?</p>

ptrblck: <p>To fix the error you could use the <code>spawn</code> method for multiprocessing as explained <a href="https://pytorch.org/docs/stable/notes/multiprocessing.html#cuda-in-multiprocessing">here</a>.</p>
<p>Usually you don’t need multiprocessing and could simply use the GPU in your script.</p>

smani: <p>Sorry for my limited understanding. I need to do this for all the images inside DataLoader. In that case do I need to use multiprocessing ?</p>

ptrblck: <p>You don’t need to use multiprocessing manually and can simply set the <code>num_workers</code> in your <code>DataLoader</code> to a larger number than 0.<br>
This will use multiprocessing under the hood and each worker will load a complete batch in the background.</p>

smani: <p>I am using the samething, num of workers &gt; 0. But still I dont understand where I am using multiprocessing to get that error.</p>

ptrblck: <p>In that case the error might be raised, if you create <code>CUDATensors</code> in your <code>Dataset</code>.<br>
The vanilla use case would be to create CPU tensors in the dataset, process them, and push them to the GPU inside your <code>DataLoader</code> loop.</p>

smani: <p>What could be the problem here?</p>
<pre><code class="lang-auto">def __getitem__(self, index):
        target = self.lbls[index]
        # read image and convert to PIL image
        I = skimage.io.MultiImage(self.fnArr[index])[-1] 
        I = TF.to_pil_image(I, mode='RGB')
        # apply transformations, including totensor()
        I = self.transform(I)
        # here I need to transfer I to GPU so that I can apply pooling 
        I = I.cuda() # I am getting error here
        re = self.pool(I)

</code></pre>
<p>I need to do pooling to do some preprocessing of the image I.</p>

ptrblck: <p>Don’t push the <code>I</code> tensor to the device in your <code>__getitem__</code> method.<br>
Since each worker in your <code>DataLoader</code> will use an own dataset, you will run into these multiprocessing issues as mentioned before.</p>
<p>Remove <code>I = I.cuda()</code> from <code>__getitem__</code> and move it to the training loop:</p>
<pre><code class="lang-python">for data in loader:
    data = data.cuda()
    ...
</code></pre>