Repository: pytorch
Issue: simple matrix multiplication yields wrong result on Ampere (3080)
Labels: triaged,module: cublas,module: correctness (silent),module: tf32

## 🐛 Bug

Simple matrix multiplication yields wrong result on Ampere (3080). Loading to `.cpu()` gets the correct result. On two other machines with 2080 and Titan X there's no such problem.

## To Reproduce

Check consistency between `A.cuda()@x.cuda()` and `A.cpu()@x.cpu()` after loading the attached `.pt` tensors.

[x.txt](https://github.com/pytorch/pytorch/files/6262029/x.txt)
[A.txt](https://github.com/pytorch/pytorch/files/6262030/A.txt)

![image](https://user-images.githubusercontent.com/865317/113657906-f8d0cd00-966c-11eb-96fc-24986bcfaf85.png)

## Expected behavior

CUDA result agrees with CPU result.

## Environment

```
Collecting environment information...
PyTorch version: 1.8.1+cu111
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 10.1.0-2ubuntu1~18.04) 10.1.0
Clang version: Could not collect
CMake version: version 3.20.0

Python version: 3.6 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 11.1.105
GPU models and configuration: GPU 0: GeForce RTX 3080
Nvidia driver version: 460.67
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.8.1+cu111
[pip3] torchfile==0.1.0
[pip3] torchvision==0.9.1+cu111
[conda] Could not collect
```

```
nvidia-smi
Tue Apr  6 04:24:28 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.67       Driver Version: 460.67       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 3080    Off  | 00000000:01:00.0 Off |                  N/A |
| 33%   24C    P8    19W / 320W |   1777MiB / 10018MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+
```

## Additional context

<!-- Add any other context about the problem here. -->


cc @csarofeen @ptrblck @xwang233 @zasdfgbnm @ngimel