Repository: pytorch
Issue: torch.unique() nondeterministic behavior on nan inputs (on GPU)
Labels: high priority,module: cuda,triaged,module: NaNs and Infs,module: determinism,module: sorting and selection

### 🐛 Describe the bug

```python
import torch

def unique(x):
    return torch.unique(x, sorted=False, return_inverse=False, return_counts=True)

s = torch.tensor(0.).cuda()
x = torch.tensor(float('nan')).cuda()

print(unique(s))
print(unique(x)) # <- these two calls have different outputs
print(unique(x)) # <-
```

output:
```
(tensor([0.], device='cuda:0'), tensor([1], device='cuda:0'))
(tensor([0., 0.], device='cuda:0'), tensor([1, 0], device='cuda:0'))
(tensor([0., 0.], device='cuda:0'), tensor([0, 0], device='cuda:0'))
```

Two issues:
* a) the last two results output a tensor with two elements, despite the fact that there's only one unique value (and only one element in the input tensor)
* b) the last two results are different, despite having the same input.

### Versions

A100 aws cluster, pytorch master branch
```
Collecting environment information...
PyTorch version: 1.12.0a0+gite816e17
Is debug build: True
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.20.4
Libc version: glibc-2.27

Python version: 3.9.12 (main, Apr  5 2022, 06:56:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.0-1051-aws-x86_64-with-glibc2.27
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration: GPU 0: A100-SXM4-40GB
Nvidia driver version: 450.119.03
cuDNN version: Probably one of the following:
/usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7.6.5
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn.so.7.6.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.0.5
/usr/local/cuda-11.0/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.12.0a0+git95eea52
[conda] numpy                     1.22.3                   pypi_0    pypi
[conda] torch                     1.12.0a0+git95eea52           dev_0    <develop
```

cc @ezyang @gchanan @zou3519 @ngimel @mruberry @kurtamohler