Tensor indexing with another tensor

Riccorl: <p>Hi, I usually index tensors with lists of indices, like</p>
<pre><code class="lang-python">x = torch.as_tensor([[1,2,3,4,5], [6,7,8,9,0]])
index = [[0, 1, 1], [1, 1, 2]]
# tensor([2, 7, 8])
x[index]
</code></pre>
<p>Now I need <code>index</code> to be a tensor object, but doing this, I get an error:</p>
<pre><code class="lang-python">x = torch.as_tensor([[1,2,3,4,5], [6,7,8,9,0]])
index = torch.as_tensor( [[0, 1, 1], [1, 1, 2]])
# IndexError: index 2 is out of bounds for dimension 0 with size 2
x[index]
</code></pre>
<p>I don’t know if it is expected to work differently, but a numpy array behaves like the list:</p>
<pre><code class="lang-python">x = torch.as_tensor([[1,2,3,4,5], [6,7,8,9,0]])
index = [[0, 1, 1], [1, 1, 2]]
index_n = numpy.asarray(index)
index_t = torch.as_tensor(index)

# tensor([2, 7, 8])
x[index]
# tensor([2, 7, 8])
x[index_n]
# IndexError: index 2 is out of bounds for dimension 0 with size 2
x[index_t]
</code></pre>
<p>How can I get the same output using a tensor instead of a list?</p>

Naruto-Sasuke: <pre><code class="lang-python">ind = index[0,:] * x.size(1) + index[1,:]
torch.take(x, ind)
</code></pre>

Riccorl: <p>Thank you for your answer. I realize that my example was too simple, because I need to do this on 3D tensors. My solution for now is:</p>
<pre><code class="lang-python">x = torch.as_tensor(
        [
            [[1, 2, 3, 4, 5], [6, 7, 8, 9, 0], [11, 12, 13, 14, 15]],
            [[16, 17, 18, 19, 20], [21, 22, 23, 24, 25], [26, 27, 28, 29, 30]],
        ]
    )

index = [[0, 0, 1], [1, 2, 0]]
# tensor([[ 6,  7,  8,  9,  0],
#         [11, 12, 13, 14, 15],
#         [16, 17, 18, 19, 20]])
x[index]

index_t = torch.as_tensor(index)
# tensor([[ 6,  7,  8,  9,  0],
#         [11, 12, 13, 14, 15],
#         [16, 17, 18, 19, 20]])
x = x.index_select(0, index_t[0])
x = x[torch.arange(x.shape[0]).unsqueeze(-1), index_t[1].unsqueeze(-1)].squeeze()

</code></pre>
<p>It is a bit convoluted, but it gets the job done. If there is a better way to do it, I will be happy to learn <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Michael_Chen: <p>ind = index[0,:] * x.size(1) + index[1,:]<br>
torch.take(x, ind)</p>

naveenvemy: <aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/41849">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/41849" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/41849" target="_blank" rel="noopener nofollow ugc">After replace `torch.Tensor.__getitem__` with some other implementation, indexing with tensors of one dim behaves unexpectedly.</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-07-22" data-time="11:36:53" data-timezone="UTC">11:36AM - 22 Jul 20 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2020-07-28" data-time="13:34:27" data-timezone="UTC">01:34PM - 28 Jul 20 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/ChieloNewctle" target="_blank" rel="noopener nofollow ugc">
          <img alt="ChieloNewctle" src="https://avatars.githubusercontent.com/u/26654234?v=4" class="onebox-avatar-inline" width="20" height="20">
          ChieloNewctle
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## 🐛 Bug



## To Reproduce

Steps to reproduce the behavior:
```
import<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden"> torch

a = torch.tensor([[1, 2, 3], [4, 5, 6]])
b = torch.tensor([True, True])

print(a[b].shape)

print('__getitem__' in torch.Tensor.__dict__)
f = torch.Tensor.__getitem__
g = lambda *a, **kw: f(*a, **kw)
torch.Tensor.__getitem__ = g
print('__getitem__' in torch.Tensor.__dict__)

print(a[b].shape)

delattr(torch.Tensor, '__getitem__')
print('__getitem__' in torch.Tensor.__dict__)

print(a[b].shape)
```

This code gives:
```
torch.Size([2, 3])
False
True
torch.Size([1, 2, 3])
False
torch.Size([1, 2, 3])
```


## Expected behavior


The code should have outputs like:
```
torch.Size([2, 3])
False
True
torch.Size([2, 3])
False
torch.Size([2, 3])
```

Or it's even okay to have:
```
torch.Size([2, 3])
False
True
torch.Size([1, 2, 3])
False
torch.Size([2, 3])
```

The most annoying problem is, when I undo the modification, indexing is still wrong.

## Environment

```
Collecting environment information...
PyTorch version: 1.3.1
Is debug build: No
CUDA used to build PyTorch: 10.1.243

OS: Ubuntu 18.04.4 LTS
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
CMake version: version 3.10.2

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce GTX 1060 6GB
Nvidia driver version: 450.51.05
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5

Versions of relevant libraries:
[pip3] numpy==1.17.5
[pip3] torch==1.3.1
[pip3] torchvision==0.4.2
[conda] Could not collect
```


## Additional context


If we set `torch.Tensor.__getitem__` with `f`, which is the original `__getitem__` method from `torch._C._TensorBase`, the code behaves well.</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>In the above issue raised in the github repository of pytorch, the person who opened the issue has replaced <code>torch.Tensor.__getitem__</code> with their own lambda.</p>
<p>I made the same change as follows:</p>
<pre><code class="lang-auto">f = torch.Tensor.__getitem__
g = lambda *a, **kw: f(*a, **kw)
torch.Tensor.__getitem__ = g
</code></pre>
<p>And now I tried indexing <code>x</code> with <code>index_t</code>, and got the following output:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/f/afb46db54005619a7defd676811966bb0e261abc.png" alt="image" data-base62-sha1="p4m7FwXGJ5m3npya8nhEXVwArV2" width="680" height="156"></p>

Riccorl: <p>Does it break other tensor functionalities?</p>

naveenvemy: <p><code>__getitem__</code> only affects indexing. But, as far as I have checked, it works as expected for Lists, Numpy Arrays and Tensors. If you find any bugs with this implementation, i’ll be curious to know what that is <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>