What Parameter.data returns?

ado_sar: <p>In the following snippet:</p>
<pre><code class="lang-auto">from torch.nn import Linear, Parameter
import torch

layer = Parameter(torch.arange(4.))

y = layer.data
z = layer.data

print(id(y))
print(id(z))
print(id(layer.data))

z[0] = 100
print(layer)

print('Is z a view?', z._is_view())

</code></pre>
<p>the output is:</p>
<pre><code class="lang-auto">132203146695440
132203146810368
132203146811088
Parameter containing:
tensor([100.,   1.,   2.,   3.], requires_grad=True)
Is z a view? False
</code></pre>
<p>Can someone explain this output? All of <code>z</code>, <code>y</code>, <code>layer.data</code> are different objects so I was thinking that the accessing <code>layer.data</code>  returns views, which is not the case as shown in the last <code>print</code>.</p>

KFrank: <p>Hi ado!</p>
<aside class="quote no-group" data-username="ado_sar" data-post="1" data-topic="217605">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ado_sar/48/59241_2.png" class="avatar"> ado_sar:</div>
<blockquote>
<p>Can someone explain this output? All of <code>z</code>, <code>y</code>, <code>layer.data</code> are different objects so I was thinking that the accessing <code>layer.data</code> returns views, which is not the case as shown in the last <code>print</code>.</p>
</blockquote>
</aside>
<p>Two comments:</p>
<p><code>.data</code> has been deprecated for public use since at least one major revision ago.  It is<br>
still used internally by pytorch, so it is still around.  Using <code>.data</code> can break how autograd<br>
tracks gradients, so you should avoid using it.  (I think you can access <code>.data</code>, but you<br>
shouldn’t use it to modify the contents of a tensor.)</p>
<p><code>Parameter</code> is a subclass of <code>Tensor,</code>and I’m pretty sure that <code>Parameter.data</code> gets<br>
delegated to <code>Tensor.data</code> without any modification.  However, <code>.data</code> appears to<br>
return some sort of proxy object (that gets created on the fly).  Even though <code>.data</code> looks<br>
like a “property,” python supports things that look like properties but that actually invoke<br>
methods under the hood.</p>
<p>When you call <code>layer.data,</code> a new proxy object gets created, but if you call <code>layer.data</code><br>
again, the new proxy object may or may not reside in the same location in memory,<br>
depending whether the previous proxy object had been freed (typically because it had<br>
no references bound to it such as your <code>y</code> or <code>z</code>).</p>
<p>Try:</p>
<pre data-code-wrap="python"><code class="lang-python">for   _ in range (10):
    print (id (layer.data))
</code></pre>
<p>vs.</p>
<pre data-code-wrap="python"><code class="lang-python">l = []
for  _ in range (10):
    print (id (layer.data))
    l.append (torch.ones (1))   # use up some locations in memory
</code></pre>
<p>The point is that <code>id (layer.data)</code> is not telling you where the actual “data” resides in<br>
memory; it’s telling you where the proxy object – which was created on the fly – resides<br>
in memory.</p>
<p>Such a proxy object is in some ways like a view – it’s a “handle” to some stuff – but it’s<br>
not considered an official pytorch view (perhaps because it doesn’t provide a transformed<br>
view into the underlying stuff).</p>
<p>Note that because its public use is deprecated, how it works and what is does may change<br>
unexpectedly, so I haven’t really tried to understand the details.</p>
<p>Best.</p>
<p>K. Frank</p>

ado_sar: <p>Thanks for the comment <a class="mention" href="/u/kfrank">@KFrank</a> ! It cleared up a lot of my confusion.</p>
<p>With regards to PyTorch views:</p>
<blockquote>
<p>Such a proxy object is in some ways like a view – it’s a “handle” to some stuff – but it’s<br>
not considered an official pytorch view (perhaps because it doesn’t provide a transformed<br>
view into the underlying stuff).</p>
</blockquote>
<p>If I change the way of checking for views (see the last line):</p>
<pre data-code-wrap="python"><code class="lang-python">from torch.nn import Linear, Parameter
import torch

layer = Parameter(torch.arange(4.))

y = layer.data
z = layer.data

print(id(y))
print(id(z))
print(id(layer.data))

z[0] = 100
print(layer)

print('Is z a view?', z.untyped_storage().data_ptr() == y.untyped_storage().data_ptr())
</code></pre>
<p>the output is:</p>
<pre><code class="lang-auto">130347870622480
130347870737408
130347870738128
Parameter containing:
tensor([100.,   1.,   2.,   3.], requires_grad=True)
Is z a view? True
</code></pre>
<p>So maybe <code>tensor._is_view()</code> is not a right way to check for views (contrary to its name)?</p>
<p>Another example (due to <code>.detach()</code> <code>y</code> shares the same storage as <code>x</code>):</p>
<pre data-code-wrap="python"><code class="lang-python">&gt;&gt;&gt; x = torch.arange(3.)
&gt;&gt;&gt; y = x.detach()
&gt;&gt;&gt; x._is_view(), y._is_view()
(False, False)
&gt;&gt;&gt; y.untyped_storage().data_ptr() == x.untyped_storage().data_ptr()
True
</code></pre>
<h3><a name="p-466994-related-sources-1" class="anchor" href="#p-466994-related-sources-1"></a>Related sources</h3>
<ul>
<li><a href="https://pytorch.org/docs/stable/storage.html#untyped-storage-api" class="inline-onebox" rel="noopener nofollow ugc">torch.Storage — PyTorch 2.6 documentation</a></li>
<li><a href="https://pytorch.org/docs/stable/tensor_view.html" class="inline-onebox" rel="noopener nofollow ugc">Tensor Views — PyTorch 2.6 documentation</a></li>
</ul>

KFrank: <p>Hi ado!</p>
<aside class="quote no-group quote-modified" data-username="ado_sar" data-post="3" data-topic="217605">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ado_sar/48/59241_2.png" class="avatar"> ado_sar:</div>
<blockquote>
<pre><code class="lang-auto">print('Is z a view?', z.untyped_storage().data_ptr() == y.untyped_storage().data_ptr())
</code></pre>
<p>…<br>
So maybe <code>tensor._is_view()</code> is not a right way to check for views (contrary to its name)?</p>
</blockquote>
</aside>
<p>By drilling down to <code>.data_ptr()</code>, you are showing that the two higher-level objects do,<br>
ultimately, refer to the same underlying data.</p>
<p>But this isn’t what pytorch mean by “view.”  Things like <code>.reshape()</code> or <code>.permute()</code> let<br>
you look at the same underlying data, but reorganized somehow.  (If, for example, you<br>
wanted the permuted version of a tensor without using a view, you would have to make<br>
a new copy of the tensor’s elements with their order shuffled.  A view, in a sense, does<br>
that shuffling “on the fly,” without the need to copy the elements.)</p>
<p>So a view does refer to the same underlying data, but reorganizes that underlying data<br>
somehow (without copying it).  Furthermore, views are tracked properly by autograd.</p>
<p>(Lower level things like <code>.data</code> and <code>.data_ptr()</code> aren’t tracked properly by autograd,<br>
and if you use them, backpropagation won’t work properly.)</p>
<p>Consider:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'2.6.0+cu126'
&gt;&gt;&gt; t = torch.ones (1, requires_grad = True)
&gt;&gt;&gt; v = t.reshape ((1,))
&gt;&gt;&gt; d = t.data
&gt;&gt;&gt; t
tensor([1.], requires_grad=True)
&gt;&gt;&gt; v
tensor([1.], grad_fn=&lt;ViewBackward0&gt;)
&gt;&gt;&gt; d
tensor([1.])
&gt;&gt;&gt; t._is_view()
False
&gt;&gt;&gt; v._is_view()
True
&gt;&gt;&gt; d._is_view()
False
</code></pre>
<p>Best.</p>
<p>K. Frank</p>