Elements are not matching

MMehrozKhan10: <p>I have a list of tensors<br>
<code>tw = </code> <code>[tensor([0.7762]),  tensor([0.4388, 0.5389]),  tensor([-0.4876, -0.2691,  0.0869])]</code> and another tensor<br>
<code>tensor([[-0.0297, -0.3764,  0.4388,  0.5389,  2.0634, -1.5948, -0.2823, -0.2109,           0.7762,  1.8073, -0.9201, -1.7119,  0.5853, -0.7324, -1.2306,  0.3180,          -1.2586, -1.5528, -0.4876, -0.4876],         [-0.0297, -0.3764,  0.4388,  0.5389,  2.0634, -1.5948, -0.2823, -0.2109,           0.7762,  1.8073, -0.9201, -1.7119,  0.5853, -0.7324, -1.2306,  0.3180,          -1.2586, -1.5528, -0.4876, -0.4876],         [-0.0297, -0.3764,  0.4388,  0.5389,  2.0634, -1.5948, -0.2823, -0.2109,           0.7762,  1.8073, -0.9201, -1.7119,  0.5853, -0.7324, -1.2306,  0.3180,          -1.2586, -1.5528, -0.4876, -0.4876]])</code>.<br>
I checked <code>sent[0][2] == tw[1][0]</code> and the result was <code>tensor(False)</code>. I do not understand what is problem here. Both data types are same and when I print both of them separately they show same result <code>tensor(0.4388)</code>.</p>

thecho7: <p>We can’t ensure the two floating type values are the same even though they are displayed as the same value.<br>
I give you a material which can help you understand what’s going on in your code.</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://floating-point-gui.de/errors/comparison/">
  <header class="source">
      <img src="https://floating-point-gui.de/favicon.ico" class="site-icon" width="16" height="16">

      <a href="https://floating-point-gui.de/errors/comparison/" target="_blank" rel="noopener nofollow ugc">floating-point-gui.de</a>
  </header>

  <article class="onebox-body">
    

<h3><a href="https://floating-point-gui.de/errors/comparison/" target="_blank" rel="noopener nofollow ugc">The Floating-Point Guide - Comparison</a></h3>

  <p>Explanation of the various pitfalls in comparing floating-point numbers.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


MMehrozKhan10: <p>Thanks <a class="mention" href="/u/thecho7">@thecho7</a> , but when I checks for following example, it works</p>
<pre><code class="lang-auto">t = torch.Tensor([[1, 2, 3], [1,2,4]])
t[0][1] == 2
</code></pre>
<p>result:</p>
<pre><code class="lang-auto">tensor(True)
</code></pre>
<p>Why is it so? I read your link and all I understood was it may or may not be same but how can I work around this?<br>
I can convert to integer type as mentioned in the article but I will pay with performance as I will not be able to use GPU in this case (<a href="https://stackoverflow.com/a/32231011" rel="noopener nofollow ugc">source</a>) .</p>

thecho7: <p>The reason you got the result <code>tensor(True)</code> is because they are integer.<br>
In any case, it is not recommended that comparing two floating types.</p>

MMehrozKhan10: <p>Understood. But<br>
Even if I check <code>t[0][1] == torch.Tensor([2])</code> result is <code>tensor([True])</code>.</p>

ptrblck: <p>As <a class="mention" href="/u/thecho7">@thecho7</a> described: comparing floating point numbers for a bitwise identical representation will fail in most use cases due to the limited floating point precision and you should compare their error against a small <code>eps</code> value defined by the <code>dtype</code>.<br>
As you can see in this example both sums should have the same value, which is not the case due to a different order of operations:</p>
<pre><code class="lang-python">a = torch.randn(100, 100, 100)

s1 = a.sum()
s2 = a.sum(0).sum(0).sum(0)

print((s1 - s2).abs().max())
# tensor(6.1035e-05)
</code></pre>