Repository: pytorch
Issue: PyTorch bug: Cannot pass gradient through index_add
Labels: high priority,module: autograd,triaged,module: advanced indexing,module: correctness (silent)

### 🐛 Describe the bug

I'm reposting this here on request of @ptrblck from the PyTorch forums:
https://discuss.pytorch.org/t/pytorch-bug-how-to-pass-gradient-through-index-add/140985

--------------------------------------------

I'm trying to backpropogate through a Tensor.index_add operation, but it breaks when index_add broadcasts vectors. Is this intended, or is this a bug in PyTorch?

```python
#The following code does NOT throw an error:

m=torch.zeros(4)
m.requires_grad=True

i=[0,1,2,3]
v=[6,7,8,9]

i=torch.tensor(i).long()
v=torch.tensor(v).float()
v.requires_grad=True

print(m)
m=m.index_add(0,i,v)
print(m)
m.sum().backward()
print(v.grad)
print("DONE")

#The following code throws an error:

m=torch.zeros(4,4)
m.requires_grad=True

i=[0,1,2,3]
v=[6,7,8,9]

i=torch.tensor(i).long()
v=torch.tensor(v).float()
v.requires_grad=True

print(m)
m=m.index_add(0,i,v)
print(m)
m.sum().backward()
print(v.grad)
print("DONE")
```

That code, when run, gives the following output and error:

```python
tensor([0., 0., 0., 0.], requires_grad=True)
tensor([6., 7., 8., 9.], grad_fn=<IndexAddBackward>)
tensor([1., 1., 1., 1.])
DONE
tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]], requires_grad=True)
tensor([[6., 6., 6., 6.],
        [7., 7., 7., 7.],
        [8., 8., 8., 8.],
        [9., 9., 9., 9.]], grad_fn=<IndexAddBackward>)


```

Strangely, it works fine - until we call the second `m.sum().backward()`. Before then, nothing appears to be any more wrong than the one above it.

```
ERROR: RuntimeError: expand(torch.FloatTensor{[4, 4]}, size=[4]): the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (2)
```

Note: I also asked this question at [**Stack Overflow**](https://stackoverflow.com/questions/70612357/pytorch-how-to-pass-gradient-through-index-add), but right now it's unanswered

A bit more context about my use case - I need to map a bunch of pixels from one image onto another image, where some pixels overlap (and in that case I'd like their sum). It's like refracting light - I want parts of the image that are more squished because of a deformation to be brighter. This index_add function is a critical component, and I need it to broadcast across RGB values.

---------------------------------------------

Workaround suggested by @ptrblck:
```python
m=m.index_add(0,i,v.expand_as(m).contiguous())
```



### Versions

```
Collecting environment information...
PyTorch version: 1.10.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.8.0-55-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 10.1.243
GPU models and configuration: GPU 0: GeForce RTX 3090
Nvidia driver version: 460.91.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.4
[pip3] torch==1.10.1+cu113
[pip3] torchaudio==0.10.1+cu113
[pip3] torchvision==0.11.2+cu113
[conda] cudatoolkit               11.3.1               h2bc3f7f_2
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] numpy                     1.20.3                   pypi_0    pypi
[conda] torch                     1.10.1+cu113             pypi_0    pypi
[conda] torchaudio                0.10.1+cu113             pypi_0    pypi
[conda] torchvision               0.11.2+cu113             pypi_0    pypi
```

cc @ezyang @gchanan @zou3519 @bdhirsh @albanD @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7