Repository: pytorch
Issue: The result of doing a dot product between two vectors, using einsum, depends on another unrelated vector
Labels: triaged,module: numerical-reproducibility

### 🐛 Describe the bug

I have two tensors, `x` and `y`. The first has shape `(2, 3, 2)`, the second has shape `(34, 2)`. I use `einsum` to calculate the dot product between each of the six 2-dimensional vectors that lie in the last dimension of `x`, and each of the 34 vectors that lie in the last dimension of `y`. The bug is that the result of the dot product between `x[0, 0]` and `y[0]` changes if we ignore the last vector of `y`, i.e. if we take `y[:33]` instead of `y`. This is undesired behavior (I think).

See here:
```python
from torch import tensor, zeros, einsum

x = zeros((2, 3, 2))
x[0, 0, 0] = 1.0791796445846558
x[0, 0, 1] = 0.30579063296318054
  
y = zeros((34, 2))
y[0, 0] = -0.14987720549106598
y[0, 1] = 0.9887046217918396

# the following two numbers should be equal, but they are not.
# the expressions differ in that the second one uses y[:33]

a = einsum('dhb,nb->ndh', x, y     )[0, 0, 0].item()  # =0.14059218764305115
b = einsum('dhb,nb->ndh', x, y[:33])[0, 0, 0].item()  # =0.14059217274188995

# returns False
a == b
```

I believe this is a minimal example (at least, local minimum). If I take `x` to be 1d or 2d instead of 3d, the bug does not occur. If I take `x` and `y` to have last dimension of size 1, the bug does not occur. If I change any of the non-zero entries of `x` and `y` to value zero, the bug does not occur. If I do a "manual" dot product instead, like this: `(x[None,...]*y[:33,None,None,:]).sum(3)[0,1,0].item()`, the bug does not occur (and we get the value `0.14059217274188995`, equal to `b` above). If I put the tensors on GPU (`.to('cuda')`), the bug does not occur (and again we get the value of `b` above). If I use numpy, the bug does not occur (but we get a different value: `0.14059218275815866`).

### Versions

PyTorch version: 1.11.0+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.22.5
Libc version: glibc-2.26

Python version: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 460.32.03
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.11.0+cu113
[pip3] torchaudio==0.11.0+cu113
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.12.0
[pip3] torchvision==0.12.0+cu113
[conda] Could not collect