Wrong results with torch.linalg.inv on batched matrices when using cuda
Labels: module: cuda, triaged, module: linear algebra, module: correctness (silent), module: magma

### 🐛 Describe the bug

When trying to invert multiple matrices at once with an initial batch size dimension, the result is completely off if the tensor to invert is on a cuda device.
Steps to reproduce:
```python
amatrix = torch.rand(5,3,3)
torch.linalg.inv(amatrix[0])
torch.linalg.inv(amatrix[0].cuda())
torch.linalg.inv(amatrix)
torch.linalg.inv(amatrix.cuda())
```
```python
>>> amatrix = torch.rand(5,3,3)
>>> torch.linalg.inv(amatrix[0])
tensor([[ 4.8674, -4.3110, -0.8659],
        [-2.0176,  0.9240,  3.4274],
        [-3.9164,  4.6243,  0.3675]])
>>> torch.linalg.inv(amatrix[0].cuda())
tensor([[ 4.8674, -4.3110, -0.8659],
        [-2.0176,  0.9240,  3.4274],
        [-3.9164,  4.6243,  0.3675]], device='cuda:0')
>>> torch.linalg.inv(amatrix)
tensor([[[ 4.8674, -4.3110, -0.8659],
         [-2.0176,  0.9240,  3.4274],
         [-3.9164,  4.6243,  0.3675]],

        [[ 2.7000, -4.1113,  1.5560],
         [-0.1757,  2.7767, -1.4721],
         [-0.5517, -1.0877,  1.8265]],

        [[ 1.5498, -0.1105, -0.2822],
         [ 1.4577,  1.6909, -1.7640],
         [-2.4122, -0.5478,  2.2266]],

        [[-0.7832, -0.1020,  3.3803],
         [ 5.7228, -1.4989, -7.8944],
         [-7.5880,  3.6522,  9.4335]],

        [[-4.9874,  5.5891,  1.3973],
         [ 2.9066, -1.1185, -2.3003],
         [ 0.6056, -0.9546,  1.0999]]])
>>> torch.linalg.inv(amatrix.cuda())
tensor([[[ -2.2865,   2.0979,   2.0907],
         [ 22.3898,  -5.2104, -35.2302],
         [ -1.3390,  -0.2901,   3.7796]],

        [[ -2.5799,   2.3801,   0.6247],
         [  1.7680,  -0.1084,  -0.6976],
         [ -0.7683,  -0.1366,   1.0912]],

        [[ -0.1224,   1.3583,  -0.1425],
         [  1.5276,   0.5215,  -0.9947],
         [ -0.4010,  -1.0566,   1.1865]],

        [[ -1.5202,   0.4430,   2.8945],
         [  2.5907,   3.1441,  -8.2405],
         [ -2.0677,  -4.8118,  11.4331]],

        [[ -3.8695,   5.0462,  -0.7982],
         [  1.9668,  -0.5551,  -1.2615],
         [ -0.1290,  -0.1451,   1.0779]]], device='cuda:0')
```

### Versions

```bash
[trougnouf@d]: ~/tmp>$ python collect_env.py
Collecting environment information...
PyTorch version: 1.12.1
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Arch Linux (x86_64)
GCC version: (GCC) 12.2.0
Clang version: 14.0.6
CMake version: version 3.24.2
Libc version: glibc-2.36

Python version: 3.10.7 (main, Sep  6 2022, 21:22:27) [GCC 12.2.0] (64-bit runtime)
Python platform: Linux-5.15.70-1-lts-x86_64-with-glibc2.36
Is CUDA available: True
CUDA runtime version: 11.7.99
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1070
Nvidia driver version: 515.76
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.8.5.0
/usr/lib/libcudnn_adv_infer.so.8.5.0
/usr/lib/libcudnn_adv_train.so.8.5.0
/usr/lib/libcudnn_cnn_infer.so.8.5.0
/usr/lib/libcudnn_cnn_train.so.8.5.0
/usr/lib/libcudnn_ops_infer.so.8.5.0
/usr/lib/libcudnn_ops_train.so.8.5.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.971
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.3
[pip3] numpydoc==1.4.0
[pip3] pytorch-msssim==0.2.1
[pip3] torch==1.12.1
[pip3] torchmetrics==0.9.3
[pip3] torchvision==0.13.1a0+b97f035
[conda] Could not collect
```

cc @ngimel @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano