Assigning tensor to multiple rows on GPU

SpontaneousDuck: <p>Hello!</p>
<p>So I recently updated to Pytorch 1.11 from 1.9 and the below code started throwing errors (also shown below) on GPU only. When using CPU, this works fine. The weird part of this is I am doing boolean indexing. Integer list indexing also throws the same error. When slicing, this works perfectly. If I modify the tensor I am using to set the rows to be the same shape as the number of rows I am assigning to, it works again. Is there a proper way to use this method to assign multiple rows in a tensor without creating an exactly matching size tensor? I assume this is supposed to work since it works on CPU.</p>
<pre><code class="lang-auto">x = torch.zeros((5,4), device=torch.device('cuda:0'))
x[[False,True,False,True,True]] = torch.tensor([1.0, 1.0, 1.0, 1.0], device=torch.device('cuda:0'), dtype=torch.float32)
</code></pre>
<pre><code class="lang-auto">RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == expandedValue.numel()INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1646755897462/work/aten/src/ATen/native/cuda/Indexing.cu":268, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor: 12 vs 4
</code></pre>

KFrank: <p>Hi Ken!</p>
<aside class="quote no-group quote-modified" data-username="SpontaneousDuck" data-post="1" data-topic="154421" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/spontaneousduck/48/35934_2.png" class="avatar"> SpontaneousDuck:</div>
<blockquote>
<p>So I recently updated to Pytorch 1.11 from 1.9 and the below code started throwing errors<br>
…</p>
<pre><code class="lang-auto">x = torch.zeros((5,4), device=torch.device('cuda:0'))
x[[False,True,False,True,True]] = torch.tensor([1.0, 1.0, 1.0, 1.0], device=torch.device('cuda:0'), dtype=torch.float32)
</code></pre>
<pre><code class="lang-auto">RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == expandedValue.numel()INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1646755897462/work/aten/src/ATen/native/cuda/Indexing.cu":268, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor: 12 vs 4
</code></pre>
</blockquote>
</aside>
<p>This appears to a known issue, but specifically in the context of using<br>
“deterministic” cuda algorithms.  It looks like some work-arounds are<br>
discussed in the relevant (closed?!) github issue:</p>
<aside class="onebox githubissue" data-onebox-src="https://github.com/pytorch/pytorch/issues/68525">
  <header class="source">

      <a href="https://github.com/pytorch/pytorch/issues/68525" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path fill-rule="evenodd" d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/68525" target="_blank" rel="noopener nofollow ugc">Deterministic integer indexing operation fails in indices size check / missing broadcast</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-11-17" data-time="16:13:19" data-timezone="UTC">04:13PM - 17 Nov 21 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2021-12-01" data-time="05:58:11" data-timezone="UTC">05:58AM - 01 Dec 21 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/dbalchev" target="_blank" rel="noopener nofollow ugc">
          <img alt="dbalchev" src="https://avatars.githubusercontent.com/u/3525601?v=4" class="onebox-avatar-inline" width="20" height="20">
          dbalchev
        </a>
      </div>
    </div>

    <div class="labels">
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: cuda
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          triaged
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: advanced indexing
        </span>
        <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">
          module: determinism
        </span>
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">## 🐛 Bug

When I run advanced integer indexing on cuda with deterministic algo<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">rithms I get

```
RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == value.numel()INTERNAL ASSERT FAILED at "/pytorch/aten/src/ATen/native/cuda/Indexing.cu":250, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor21
```

This bug is similar to https://github.com/pytorch/pytorch/issues/61032, but it affects integer indexing instead of boolean indexing.




## To Reproduce

Steps to reproduce the behavior:

1. Run the following script 
    ```python
    import torch

    torch.use_deterministic_algorithms(True)

    x = torch.zeros(5).cuda()
    x[torch.tensor([1, 3]).cuda()] = 2
    print(x)
    ```
1. Get the error
    ```
    Traceback (most recent call last):
      File "Distilled Example.py", line 6, in &lt;module&gt;
        x[torch.tensor([1, 3]).cuda()] = 2
    RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == value.numel()INTERNAL ASSERT FAILED at "/pytorch/aten/src/ATen/native/cuda/Indexing.cu":250, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor21
    ```




## Expected behavior
Not getting a runtime error


## Environment

```
Collecting environment information...
PyTorch version: 1.10.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.10.2
Libc version: glibc-2.26

Python version: 3.7.9 (default, Nov 15 2021, 19:06:29)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.141-67.229.amzn2.x86_64-x86_64-with-debian-buster-sid
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: Tesla K80
Nvidia driver version: 460.73.01
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] mypy==0.910
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.18.0
[pip3] torch==1.10.0
[pip3] torch-scatter==2.0.9
[pip3] torchvision==0.11.0
[conda] Could not collect
```

## Additional context

I originally get this error when I try to run `torchvision` `FasterRCNN` on cuda with deterministic algorithms:

```python
import torch

torch.use_deterministic_algorithms(True)

from torchvision.models.detection.faster_rcnn import fasterrcnn_resnet50_fpn

model = fasterrcnn_resnet50_fpn(pretrained=False, pretrained_backbone=False).cuda()

model(
    torch.zeros(1, 3, 800, 800).cuda(), [{
        'boxes': torch.tensor([[100, 200, 300, 400]]).cuda(), 
        'labels': torch.tensor([1]).cuda(),
    }])
```




cc @ngimel @mruberry @kurtamohler</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I can reproduce your issue, both in version 1.11 and in a recent nightly<br>
(1.13.0.dev20220604), but only if I set:</p>
<pre data-code-wrap="python"><code class="lang-python">torch.use_deterministic_algorithms (True)
</code></pre>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.11.0'
&gt;&gt;&gt; torch.version.cuda
'11.3'
&gt;&gt;&gt; torch.cuda.get_device_name()
'GeForce GTX 1050 Ti'
&gt;&gt;&gt; x = torch.zeros((5,4), device=torch.device('cuda:0'))
&gt;&gt;&gt; x[[False,True,False,True,True]] = torch.tensor([1.0, 1.0, 1.0, 1.0], device=torch.device('cuda:0'), dtype=torch.float32)
&gt;&gt;&gt; x
tensor([[0., 0., 0., 0.],
        [1., 1., 1., 1.],
        [0., 0., 0., 0.],
        [1., 1., 1., 1.],
        [1., 1., 1., 1.]], device='cuda:0')
&gt;&gt;&gt; torch.use_deterministic_algorithms (True)
&gt;&gt;&gt; x[[False,True,False,True,True]] = torch.tensor([1.0, 1.0, 1.0, 1.0], device=torch.device('cuda:0'), dtype=torch.float32)
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
RuntimeError: linearIndex.numel()*sliceSize*nElemBefore == expandedValue.numel()INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1646755903507/work/aten/src/ATen/native/cuda/Indexing.cu":268, please report a bug to PyTorch. number of flattened indices did not match number of elements in the value tensor: 12 vs 4
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

ptrblck: <p>CC <a class="mention" href="/u/eqy">@eqy</a> could you take a look at this failure as you’ve worked on the last fix?<br>
This might be a new issue (previously untested) or the same error popped up again.</p>