Does indexing a tensor return a copy of it?

sadra-barikbin: <p>Hi, it seems indexing a tensor with <code>slice</code> or <code>int</code> returns a view of that without copying its underlying storage but indexing with another tensor (a <code>Bool</code> or a <code>Long</code> one but not a 0-dim long tensor) or a list returns a copy of the tensor. Am I right? If yes, why is this the case?<br>
This behavior seems not to be documented anywhere in the docs.</p>
<pre><code class="lang-python">original = torch.arange(12).reshape(3, 4)

view = original[2, 1:3]
view = original[torch.tensor(2), 1:3]

copy = original[2, torch.tensor([1, 2])]
copy = original[2, torch.tensor([False, True, True, False])]
copy = original[2, [1, 2]]
</code></pre>
<p>Another question is that is that copy performed right away or upon tensor modification?<br>
And if it is done right away, isn’t better to be done lazily to avoid unnecessary copy in the case user wants to just examine the data?</p>
<pre><code class="lang-python">copy = original[2, torch.tensor([False, True, True, False])] # Is copy performed here?
copy[1] = 100 # or here?</code></pre>

KFrank: <p>Hi Sadra!</p>
<aside class="quote no-group" data-username="sadra-barikbin" data-post="1" data-topic="164905" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sadra-barikbin/48/46599_2.png" class="avatar"> sadra-barikbin:</div>
<blockquote>
<p>Am I right?</p>
</blockquote>
</aside>
<p>Yes.</p>
<p>(You can use <code>.storage().data_ptr()</code> to see if the underlying data of<br>
a tensor has been copied or not.)</p>
<aside class="quote no-group">
<blockquote>
<p>If yes, why is this the case?</p>
</blockquote>
</aside>
<p>With slices you can leave the original data in its original location in memory<br>
and still access it (reasonably) efficiently with strides and offsets.</p>
<p>When you index into a tensor, you are plucking out values from unstructured<br>
locations, so you would need a more complicated view / access approach,<br>
and pytorch prefers the efficiency that making a copy provides for any future<br>
manipulations of the tensor.</p>
<p>(Pytorch does not try to be clever enough to recognize whether the result<br>
of indexing into a tensor could have been obtained by slicing.)</p>
<aside class="quote no-group">
<blockquote>
<p>This behavior seems not to be documented anywhere in the docs.</p>
</blockquote>
</aside>
<p>There is some documentation here and there.  Quoting, for example, from<br>
the <a href="https://pytorch.org/docs/1.13/tensor_view.html" rel="noopener nofollow ugc">Tensor Views documentation:</a></p>
<blockquote>
<p>Note</p>
<p>When accessing the contents of a tensor via indexing, PyTorch follows Numpy behaviors that basic indexing returns views, while advanced indexing returns a copy. Assignment via either basic or advanced indexing is in-place.</p>
</blockquote>
<aside class="quote no-group">
<blockquote>
<p>Another question is that is that copy performed right away or upon tensor modification?</p>
</blockquote>
</aside>
<p>The copy is performed right away – but note the exception to this (mentioned<br>
in the quoted documentation) when you are assigning to an indexed tensor.</p>
<p>Here are some illustrations of what is going on:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; print (torch.__version__)
1.12.0
&gt;&gt;&gt;
&gt;&gt;&gt; original = torch.arange (12).reshape (3, 4)
&gt;&gt;&gt; view = original[2, 1:3]
&gt;&gt;&gt; copy = original[2, [1, 2]]
&gt;&gt;&gt;
&gt;&gt;&gt; original.storage().data_ptr()   # original storage()
2341749571200
&gt;&gt;&gt; view.storage().data_ptr()       # same storage() so it's a view
2341749571200
&gt;&gt;&gt; copy.storage().data_ptr()       # a copy of the original data in new storage() so it's a copy
2341738124416
&gt;&gt;&gt;
&gt;&gt;&gt; # note, assigning to an indexed tensor assigns in-place without a copy
&gt;&gt;&gt; original[2, [1, 2]] = torch.tensor ([99, 666])
&gt;&gt;&gt; original
tensor([[  0,   1,   2,   3],
        [  4,   5,   6,   7],
        [  8,  99, 666,  11]])
&gt;&gt;&gt; original.storage().data_ptr()
2341749571200
&gt;&gt;&gt;
&gt;&gt;&gt; # but indexing twice does cause a copy to be made
&gt;&gt;&gt; original[2, torch.tensor([False, True, True, False])][1] = 100   # Is copy performed here? -- yes
&gt;&gt;&gt; original
tensor([[  0,   1,   2,   3],
        [  4,   5,   6,   7],
        [  8,  99, 666,  11]])
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

sadra-barikbin: <p>Many thanks. Just a little question. In an assignment to an advanced-indexed tensor, how does PyTorch know if the <code>original[advanced_index]</code> is <code>lhs</code> so avoid copying?</p>

KFrank: <p>Hi Sadra!</p>
<aside class="quote no-group" data-username="sadra-barikbin" data-post="3" data-topic="164905" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sadra-barikbin/48/46599_2.png" class="avatar"> sadra-barikbin:</div>
<blockquote>
<p>how does PyTorch know if the <code>original[advanced_index]</code> is <code>lhs</code> so avoid copying?</p>
</blockquote>
</aside>
<p>I have no idea.  I would be grateful if an expert could explain what<br>
python / pytorch techniques are used to achieve this.</p>
<p>To my mind, this is conceptually similar to the distinction in C (and<br>
descendant languages) between “l-values” that are (results of)<br>
expressions that can be assigned to (“left-hand side of =”) and<br>
“r-values” that are normal values (that you might assign to something).<br>
But how python does it I don’t know.</p>
<p>Best.</p>
<p>K. Frank</p>

jakelevi1996: <aside class="quote no-group" data-username="sadra-barikbin" data-post="3" data-topic="164905">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sadra-barikbin/48/46599_2.png" class="avatar"> sadra-barikbin:</div>
<blockquote>
<p>how does PyTorch know if the <code>original[advanced_index]</code> is <code>lhs</code> so avoid copying</p>
</blockquote>
</aside>
<p>I’m a bit late to the party, but I think this is just the <code>__setitem__</code> method, see documentation from the Python " Data model": <a href="https://docs.python.org/3/reference/datamodel.html#object.__setitem__" class="inline-onebox" rel="noopener nofollow ugc">3. Data model — Python 3.13.3 documentation</a></p>
<p>For example:</p>
<pre data-code-wrap="python"><code class="lang-python">class ClassName:
    def __init__(self):
        self.data = dict()

    def __setitem__(self, key, value):
        print("ClassName[%s] = %s" % (key, value))
        self.data[key] = value

c = ClassName()
print(c.data)
# &gt;&gt;&gt; {}
c[1, 2, 3] = 4.56
# &gt;&gt;&gt; ClassName[(1, 2, 3)] = 4.56
print(c.data)
# &gt;&gt;&gt; {(1, 2, 3): 4.56}
</code></pre>