Any way to check if two tensors have the same base

username: Intel_Novel
cooked: <p>OK, here is the example.</p>
<pre><code class="lang-auto">x = torch.randn(4, 4)
y = x.view(2,-1)
</code></pre>
<p>How can I make sure <code>y</code> has the same origin and different metadata comparing to the <code>x</code>, and there was no <code>clone()</code> operation involved, like there would be on:</p>
<pre><code class="lang-auto">x = torch.randn(4, 4)
y = x.clone().view(2,-1)
</code></pre>
<p>or in case on <code>reshape()</code> non contiguous one.</p>

username: LeviViana
cooked: <p>I’m not sure I fully understood your question, but I’ll try to answer:</p>
<pre><code class="lang-python">import torch
x = torch.randn(4, 4)
y = x.view(2,-1)
print(x.data_ptr() == y.data_ptr()) # prints True
y = x.clone().view(2,-1)
print(x.data_ptr() == y.data_ptr()) # prints False
</code></pre>
<p>But it doesn’t work if you are interested in comparing tensor storage. For instance, in the example below, <code>x</code> and <code>y</code> share storage, but they don’t share the same data pointer:</p>
<pre><code class="lang-python">import torch
x = torch.arange(10)
y = x[1::2]
print(x.data_ptr() == y.data_ptr()) # prints False
</code></pre>
<p>The following snippet checks whether two tensors share storage, though, I guess there might be a more efficient way of doing this.</p>
<pre><code class="lang-auto">import torch
def same_storage(x, y):
	x_ptrs = set(e.data_ptr() for e in x.view(-1))
	y_ptrs = set(e.data_ptr() for e in y.view(-1))
	return (x_ptrs &lt;= y_ptrs) or (y_ptrs &lt;= x_ptrs)

x = torch.arange(10)
y = x[1::2]
print(same_storage(x, y)) # prints True
z = y.clone()
print(same_storage(x, z)) # prints False
print(same_storage(y, z)) # prints False
</code></pre>

username: MariosOreo
cooked: <p>Hello <a class="mention" href="/u/leviviana">@LeviViana</a>,</p>
<p>I found that <code>.data_ptr()</code> checks whether the tensors have the same memory address of the first element. What about <code>X is Y</code>, it also checks whether the tensors have the same memory address (if I understand clearly).</p>
<p>In the first code snippet, <code>x</code> and <code>y</code> should share the same memory, but why <code>x.data_ptr() == y.data_ptr()</code> and <code>x is y</code> returns <strong>False</strong>?</p>
<p>Could you give me some ideas?</p>
<pre><code class="lang-auto">x.data_ptr()
94522278737584
id(x)
140148138888576
y.data_ptr()
94522278737584
id(y)
140152047258360
</code></pre>

username: Intel_Novel
cooked: <pre><code class="lang-auto">import torch
x=torch.arange(10)
y=x.view(-1)
z=x.reshape(2,5)
c=x.clone()
d=x[1::2] 
print(x, x.data_ptr())
print(y, y.data_ptr())
print(z, z.data_ptr())
print(c, c.data_ptr())
print(d, d.data_ptr())
print("...")

print(id(x), id(x.data))
print(id(y), id(y.data))
print(id(z), id(z.data))
print(id(c), id(c.data))
print(id(d), id(d.data))
</code></pre>
<p>Returns:</p>
<pre><code class="lang-auto">tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) 73968896
tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) 73968896
tensor([[0, 1, 2, 3, 4],
        [5, 6, 7, 8, 9]]) 73968896
tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) 73969024
tensor([1, 3, 5, 7, 9]) 73968904
...
139879908757576 139879908721648
139879908724240 139879908721648
139879908723736 139879908721648
139879908722440 139879908721648
139879908721936 139879909096904
</code></pre>
<p><code>data_ptr()</code> was something I was looking for. For any variable <code>var</code>, <code>id(var)</code> returns different result, and <code>id(var.data)</code> returns the same location id almost always, but not always.</p>

username: Intel_Novel
cooked: <p>Pytorch is still young. I hoped the storage object may provide me hint for “same storage” idea, but it will not.</p>
<pre><code class="lang-auto">import torch
def same_storage(x, y):    
    return x.storage is y.storage 

x = torch.arange(10)
y = x[1::2]
z = y.clone()
print(same_storage(x, y)) # prints False
print(same_storage(x, z)) # prints False
print(same_storage(y, z)) # prints False
</code></pre>
<p>From <a href="http://docs.python.org" rel="noopener nofollow ugc">docs.python.org</a>:</p>
<blockquote>
<p>“Every object has an identity, a type and a value. An object’s identity never changes once it has been created; you may think of it as the object’s address in memory. The ‘is’ operator compares the identity of two objects; the id() function returns an integer representing its identity.”</p>
</blockquote>
<p>I tested this on CPU, but apparently the problem is maybe because PyTorch uses C and storage objects actually are pointers to memory location handled in C. So if we can use Python to get these memory locations…</p>

username: LeviViana
cooked: <p>Hello <a class="mention" href="/u/mariosoreo">@MariosOreo</a>,</p>
<p>It seems to me that <code>x</code> and <code>y</code> should be treated as different objects. The reason they share storage is for performance (saving memory &amp; not copying the original tensor).</p>
<p>Moreover, the <code>is</code> operator is very restrictive. For instance, <code>[] is []</code> returns <code>False</code>, whereas <code>[] == []</code> returns True. In the first case python tests whether two different empty lists are the same object, which by construction is false. In the second case, python tests whether two list objects have the same value, which is true. Back to our case, I think now it is clear why <code>x is y</code> returns <code>False</code>.</p>

username: kytabyte
cooked: <p><a class="mention" href="/u/leviviana">@LeviViana</a></p>
<p>Your solution is great in check whether a tensor is a subset of another tensor. However, it still has some limitation in checking the case if two different sub-tensors pointing to the same storage.</p>
<pre><code class="lang-python">import torch

t = torch.rand((3,3))
t1 = t[0,:]
t2 = t[:,0]

same_storage(t1, t2)

&gt;&gt; False
</code></pre>
<p>However, if I change <code>t1[0]</code>, <code>t2</code> will change as well.</p>

username: LeviViana
cooked: <p>For a more general case:</p>
<pre><code class="lang-python">import torch
def same_storage(x, y):
    return x.storage().data_ptr() == y.storage().data_ptr()

# It works on your test.
t = torch.rand((3,3))
t1 = t[0,:]
t2 = t[:,0]

print(same_storage(t1, t2)) # prints True
 
x = torch.arange(10)
y = x[1::2]
print(same_storage(x, y)) # prints True
z = y.clone()
print(same_storage(x, z)) # prints False
print(same_storage(y, z)) # prints False

</code></pre>

username: dejanbatanjac
cooked: <p><a class="mention" href="/u/alband">@albanD</a> can you confirm that <code>x.storage().data_ptr()==y.storage().data_ptr()</code> is the correct way to check if two tensors <code>x</code>, and <code>y</code> share the same storage?</p>

username: albanD
cooked: <p>Hey,</p>
<p>While it does work, I am not sure I would ever recommend to use it <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"><br>
If you need to do this check, you are most likely doing something very weird. And you should understand what the storage is in this case to know what you’re doing.</p>

username: dejanbatanjac
cooked: <p><a class="mention" href="/u/alband">@albanD</a> keep with the good work. Thanks.</p>

username: LLlearner
cooked: <p><a class="mention" href="/u/alband">@albanD</a> Why is the var.storage().data_ptr() sometimes different from var.data_ptr()? What is other than the “storage” wrapped in a tensor to make the memory address different? And why is id(x) different from x.data_ptr(). Isn’t it supposed to be the same?<br>
Thanks in advance</p>

username: albanD
cooked: <p>Hi,</p>
<p><code>var.data_ptr()</code> will point to the beginning of the data used by the Tensor. This might not be the beginning of the storage. You can check <code>var.storage_offset()</code> to see the offset in question.</p>
<p><code>id(x)</code> is python giving you a unique id for the python object x. This has nothing to do with the storage for a Tensor.<br>
If we were using the data_ptr() for this, you can see that this id would very quickly not be unique as soon as two Tensors look at the same memory.</p>

username: ado_sar
cooked: <p>Is there something like <a href="https://numpy.org/doc/stable/reference/generated/numpy.shares_memory.html" rel="noopener nofollow ugc">np.shares_memory</a>?</p>