Conv2d is 91x slower than regular C implementation

username: tflahaul
cooked: <p>Hello!</p>
<p>I’m trying to understand why I’m seeing such a performance gap between my regular conv2d C implementation and torch.conv2d on CPU.</p>
<p>A single torch.conv2d call with a 512x512 matrix and a 256x256 kernel as arguments takes 59.159 secs to execute whereas a very basic C implementation runs in only 0.650 sec !</p>
<p>I expected it to be the other way around but no: the regular C implementation is about 91 times faster despite running on a single core and having no particular optimization (I only rearranged the <code>for</code> loops to improve locality and compiled with <code>-O2 -ffastmath</code>).</p>
<p>What are the causes of such a performance gap on such a small example?<br>
<br><br>
For reference, here’s the code:</p>
<ol>
<li>used to benchmark <code>torch.conv2d</code>
</li>
</ol>
<pre><code class="lang-auto">import torch
import time
 
x = torch.rand(1, 1, 512, 512)
k = torch.rand(1, 1, 256, 256)
with torch.inference_mode():
    start = time.time()
    torch.conv2d(x, k)
    end = time.time()
print(end - start)
</code></pre>
<ol start="2">
<li>used to benchmark my implementation (compiled with <code>-O0</code>)</li>
</ol>
<pre><code class="lang-auto">#include &lt;stdio.h&gt;
#include &lt;time.h&gt;
 
int         main(void) {
    struct Tensor   *a = tensor_init_random(512, 512);
    struct Tensor   *b = tensor_init_random(256, 256);
    struct Tensor   *c;
    clock_t     start, end;
 
    start = clock();
    c = conv2d(a, b, 0, 1);
    end = clock();
 
    tensor_free(c);
    printf("%f\n", difftime(end, start)/CLOCKS_PER_SEC);
    return (0);
}
</code></pre>
<ol start="3">
<li>The C implementation of conv2d (compiled with <code>-O2 -ffast-math</code>)</li>
</ol>
<pre><code class="lang-auto">struct Tensor       *conv2d(struct Tensor *tensor, struct Tensor *kernel, 
                            float const bias, uint32_t const stride) {
    __assert_non_zero_stride(stride);
    __assert_conv_kernel_size(tensor, kernel);
    uint32_t const  h = ((tensor-&gt;shape[0] - (kernel-&gt;shape[0] - 1) - 1) / stride) + 1;
    uint32_t const  w = ((tensor-&gt;shape[1] - (kernel-&gt;shape[1] - 1) - 1) / stride) + 1;
    struct Tensor   *out = tensor_init_constant(h, w, bias);
    for (uint32_t i = 0; i &lt; h; i++)
        for (uint32_t k = 0; k &lt; kernel-&gt;shape[0]; k++)
            for (uint32_t l = 0; l &lt; kernel-&gt;shape[1]; l++)
                for (uint32_t j = 0; j &lt; w; j++)
                    out-&gt;data[i * w + j] += kernel-&gt;data[k * kernel-&gt;shape[1] + l] * tensor-&gt;data[(i + k) * tensor-&gt;shape[1] + l + j];
    return (out);
}
</code></pre>
<ol start="3">
<li>Some env informations</li>
</ol>
<pre><code class="lang-bash">PyTorch version: 1.10.0
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 12.0.1 (arm64)
Clang version: 12.0.5 (clang-1205.0.22.9)
Python platform: macOS-12.0.1-arm64-arm-64bit
Python version: 3.9.7
CUDA runtime version: No CUDA
</code></pre>

username: tflahaul
cooked: <p>The same operation runs in around 1.663 sec using tensorflow.<br>
Is Pytorch less optimized for ARM architectures?</p>