In pytorch nn.conv2d,when the input matrix is big enough,the output is different from sub of matrix input

username: yyd199948
cooked: <pre><code class="lang-auto">m = nn.Conv2d(2,4,(1,1))
x=torch.randn(1,2,10,1024)
o1=m(x)
o2=m(x[:,:,:,0:1])
print(o1[:,:,:,0:1].equal(o2))
output:True
x=torch.randn(1,2,10,2000)
o11=m(x)
o22=m(x[:,:,:,0:1])
print(o11[:,:,:,0:1].equal(o22))
output:False
print(o11[:,:,:,0:1].isclose(o22))
</code></pre>
<p>why o11[:,:,:,0:1] is different from o22??</p>

username: ptrblck
cooked: <p>Using your code I get an abs. error of <code>1.1921e-07</code>, which is expected for <code>float32</code> due to its limited floating point precision. These errors are caused by a different order of operation, which might be used depending on the input shape.</p>

username: yyd199948
cooked: <p>thanks for your reply！<br>
Is there someway to make o11[:,:,:,0:1] same as o22？<br>
Is it optimized for conv2d when the input size is big?</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="yyd199948" data-post="3" data-topic="166684">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/96bed5/48.png" class="avatar"> yyd199948:</div>
<blockquote>
<p>Is there someway to make o11[:,:,:,0:1] same as o22？</p>
</blockquote>
</aside>
<p>No, I don’t think there is an easy way to create the same errors unless you write an algorithm with a defined sequence of operations (for this slice). Also note that neither of them is “correct” and both show the expected error to the theoretically ground truth value. If you need more precision, you could use <code>float64</code> instead.</p>
<aside class="quote no-group" data-username="yyd199948" data-post="3" data-topic="166684">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/y/96bed5/48.png" class="avatar"> yyd199948:</div>
<blockquote>
<p>Is it optimized for conv2d when the input size is big?</p>
</blockquote>
</aside>
<p>Yes, I guess so, but it would also depend on the used libraries (in this case MKL might be used as the conv is performed on the CPU) as well as the selected algorithm.</p>

username: yyd199948
cooked: <p>But I found that the results are same in tensorflow，</p>
<pre><code class="lang-auto">input_shape = (10, 10000, 20, 2)
x=tf.random.normal(input_shape)
m=tf.keras.layers.Conv2D(2,(5,1))
y1=m(x)
y2=m(x[:,:,0:1,:])
print(tf.math.reduce_sum(abs(y1[:,:,0:1,:]-y2)))
tf.Tensor(0.0, shape=(), dtype=float32)
</code></pre>

username: yyd199948
cooked: <p>But I found that the results are same in tensorflow，</p>
<pre><code class="lang-auto">input_shape = (10, 10000, 20, 2)
x=tf.random.normal(input_shape)
m=tf.keras.layers.Conv2D(2,(5,1))
y1=m(x)
y2=m(x[:,:,0:1,:])
print(tf.math.reduce_sum(abs(y1[:,:,0:1,:]-y2)))
tf.Tensor(0.0, shape=(), dtype=float32)
</code></pre>
<p>l</p>

username: ptrblck
cooked: <p>That doesn’t contradict my explanation and maybe TF does not switch between different algorithms (for performance gains) for different input shapes.</p>