Repository: pytorch
Issue: Some type combinations of cublas gemm are not supported when they should
Labels: module: cuda,triaged,module: type promotion,module: linear algebra

## 🐛 Bug

`torch.mm/addmm` are calling `cublasGemmEx` under the hood. However, they are type combinations that are claimed to be non-supported by pytorch when they should work fine: 

Example:

`CUBLAS_COMPUTE_32F` algorithm for gemm is claimed to support Half@Half + Float. But fails in pytorch.

See: https://docs.nvidia.com/cuda/cublas/index.html#cublas-GemmEx 

## To Reproduce

```
import torch
A = torch.randn(10, 10).cuda().half()
B = torch.randn(10, 10).cuda().half()
result = torch.cuda.FloatTensor(A.shape[1], B.shape[0])

torch.mm(A, B, out=result)

# =>> RuntimeError: expected scalar type Float but found Half
```

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

Do not throw an error

## Environment

```
Collecting environment information...
PyTorch version: 1.9.0
Is debug build: False
CUDA used to build PyTorch: 11.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (crosstool-NG 1.24.0.133_b0863d8_dirty) 9.3.0
Clang version: Could not collect
CMake version: version 3.10.2
Libc version: glibc-2.27

Python version: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-4.15.0-159-generic-x86_64-with-glibc2.27
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: A100-PCIE-40GB
GPU 1: A100-PCIE-40GB
GPU 2: A100-PCIE-40GB
GPU 3: A100-PCIE-40GB
GPU 4: A100-PCIE-40GB
GPU 5: A100-PCIE-40GB
GPU 6: A100-PCIE-40GB
GPU 7: A100-PCIE-40GB
GPU 8: A100-PCIE-40GB

Nvidia driver version: 460.91.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.1
[pip3] torch==1.9.0
[pip3] torchaudio==0.9.0a0+33b2469
[pip3] torchmetrics==0.6.0
[pip3] torchvision==0.10.0a0
[conda] cudatoolkit               11.3.1               h2bc3f7f_2  
[conda] libblas                   3.9.0            11_linux64_mkl    conda-forge
[conda] libcblas                  3.9.0            11_linux64_mkl    conda-forge
[conda] liblapack                 3.9.0            11_linux64_mkl    conda-forge
[conda] liblapacke                3.9.0            11_linux64_mkl    conda-forge
[conda] magma                     2.5.4                h6103c52_2    conda-forge
[conda] mkl                       2021.3.0           h06a4308_520  
[conda] numpy                     1.21.1           py39hdbf815f_0    conda-forge
[conda] pytorch                   1.9.0           cuda112py39hbeb36f3_1    conda-forge
[conda] pytorch-gpu               1.9.0           cuda112py39h0bbbad9_1    conda-forge
[conda] torchaudio                0.9.0                      py39    pytorch
[conda] torchmetrics              0.6.0                    pypi_0    pypi
[conda] torchvision               0.10.0          py39cuda112hc5182df_0_cuda    conda-forge

```



cc @ngimel @nairbv @mruberry @jianyuh @nikitaved @pearu @walterddr @IvanYashchuk @xwang233 @Lezcano