_saved_result comparison with y

mrityu: <p>Please refer to the below code snippet:</p>
<pre><code class="lang-auto">a = torch.ones(5, requires_grad=True)
y = torch.exp(a)

print(y.grad_fn._saved_result.equal(y)) 
print(y.grad_fn._saved_result is y)     

print(y.data_ptr())
print(y.grad_fn._saved_result.data_ptr())
</code></pre>
<pre><code class="lang-auto">Output:
True
False
94655401114176
94655401114176
</code></pre>
<p>My question is, why the output of <code>print(y.grad_fn._saved_result is y)     </code> is <code>False</code> even though it is evident from the <code>data_ptr()</code> output that both the tensors share the same storage? If they are pointing to the same location, then shouldn’t the comparison with “is” should return <code>True</code>.<br>
Am I missing something here?</p>

VictorieeMan: <p>According to the <a href="https://docs.python.org/3/library/functions.html#print" rel="noopener nofollow ugc">python documentation</a>: All none keyword arguments are converted to strings using str(). This mean that the print() can’t be trusted. What’s happening in the example below might be the reason for your results.</p>
<pre><code class="lang-auto">a = 5
b = "5"
print(a is b)
print(a)
print(b)
</code></pre>
<pre><code class="lang-auto">Output:
False
5
5
</code></pre>
<p>In other words, there is a string conversion happening in the print() function that might be the source for the confusion. If you have a debugger, add a break point and have a look in  the variable explorer to see if there’s any difference.</p>
<p>Edit:<br>
Try adding this to your code when checking</p>
<pre><code class="lang-auto">print(y.data_ptr(),type(y.data_ptr()))
print(y.grad_fn._saved_result.data_ptr(),type(y.grad_fn._saved_result.data_ptr()))
</code></pre>
<p>Then you’ll get more information in the printout that might convey the mystery <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

mrityu: <p>Hey,<br>
The output of the type(…) is the same for both, the output is <code>class&lt;'int'&gt;</code></p>

VictorieeMan: <aside class="quote no-group" data-username="mrityu" data-post="1" data-topic="142837">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mrityu/48/33916_2.png" class="avatar"> mrityu:</div>
<blockquote>
<p><code>print(y.grad_fn._saved_result is</code></p>
</blockquote>
</aside>
<p>Of course, they do. I forgot to check the <a href="https://pytorch.org/docs/stable/generated/torch.Tensor.data_ptr.html" rel="noopener nofollow ugc">docs</a> before I made my edit above. It says that .data_ptr() returns the address of the first element in the tensor it’s attached to.</p>
<p>The detailed answer to your question I believe you can find here: <a href="https://pytorch.org/docs/stable/notes/autograd.html#saved-tensors" class="inline-onebox" rel="noopener nofollow ugc">Autograd mechanics — PyTorch 2.1 documentation</a></p>

my3bikaht: <pre><code class="lang-auto">&gt;&gt;&gt; a = torch.ones(5, requires_grad=True)
&gt;&gt;&gt; y = torch.exp(a)
&gt;&gt;&gt; id(y.grad_fn._saved_result)
140420994735056
&gt;&gt;&gt; id(y)
140420994748480
</code></pre>

mrityu: <blockquote>
<p>Under the hood, to prevent reference cycles, PyTorch has <em>packed</em> the tensor upon saving and <em>unpacked</em> it into a different tensor for reading. Here, the tensor you get from accessing <code>y.grad_fn._saved_result</code> is a different tensor object than <code>x</code> (but they still share the same storage).</p>
</blockquote>
<p>It says that the memory location is shared between the objects, but the comparison using “is” return <code>False</code>. As far as I know, “is” return <code>True</code> if two objects are at the same memory location.</p>

mrityu: <p>What’s the difference between <code>id()</code> and <code>.data_ptr()</code>?</p>

my3bikaht: <p>id is a “pointer” to tensor itself, which is compound object containing other objects, not the data<br>
I think this is similar to (soft / shallow) copy()</p>
<pre><code class="lang-auto">&gt;&gt;&gt; a = [1,2,3]
&gt;&gt;&gt; b = a.copy()
&gt;&gt;&gt; a is b
False
</code></pre>
<p>as you can see a and b are different objects, but</p>
<pre><code class="lang-auto">&gt;&gt;&gt; hex(id(a))
'0x7fb56080a180'
&gt;&gt;&gt; hex(id(b))
'0x7fb560817740'
&gt;&gt;&gt; hex(id(a[0]))
'0x7fb64fa72930'
&gt;&gt;&gt; hex(id(b[0]))
'0x7fb64fa72930'
</code></pre>
<p>they both point to same elements inside</p>