Repository: pytorch
Issue: `test_cholesky_solve` gradgradcheck fails sometimes
Labels: module: autograd,module: tests,triaged,module: linear algebra,module: magma

For fast_mode=False, there may be failures when the seed is set to certain values. Fast mode on the other hand seems to always succeed on CPU (LAPACK), but may still fail on CUDA.
```python
@skipCUDAIfNoMagma
@skipCPUIfNoLapack
@dtypes(torch.float64, torch.complex128)
def test_cholesky_solve_autograd(self, device, dtype):
    def run_test(A_dims, B_dims, upper, fast_mode, atol):
        root = torch.randn(*A_dims, device=device, dtype=dtype).requires_grad_()
        b = torch.randn(*B_dims, device=device, dtype=dtype).requires_grad_()

        def func(root, b, upper):
            if upper:
                A = root.triu()
            else:
                A = root.tril()
            return torch.cholesky_solve(b, A, upper)

        # gradcheck(func, [root, b, upper])
        # TODO(#50743): the following fails with batched grad testing
        gradgradcheck(func, [root, b, upper], atol=atol, check_batched_grad=False, fast_mode=fast_mode)

    nruns = 50
    for fast_mode in (True, False):
        for atol in (1e-3, 1e-1, 1e1):
            nfail = 0
            for i in range(nruns):
                try:
                    torch.manual_seed(i)
                    for (a_size, b_size), upper in itertools.product([((3, 3), (3, 4))],
                                                                    [True, False]):
                        run_test(a_size, b_size, upper, fast_mode, atol)
                except RuntimeError:
                    nfail += 1
            print(f"fast_mode: {fast_mode}, atol: {atol}, number failures: {nfail}/{nruns}")
```
test_cholesky_solve_autograd_cpu_float64
```
fast_mode: True, atol: 0.001, number failures: 0/50
fast_mode: True, atol: 0.1, number failures: 0/50
fast_mode: True, atol: 10.0, number failures: 0/50
fast_mode: False, atol: 0.001, number failures: 4/50
fast_mode: False, atol: 0.1, number failures: 1/50
fast_mode: False, atol: 10.0, number failures: 0/50
```
Though we don't see any failures on CPU for fast_mode=True, there may be failures on CUDA (requires MAGMA):
```
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 979, in wrapper
    method(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 297, in instantiated_test
    raise rte
  File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 292, in instantiated_test
    result = test_fn(self, *args)
  File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 578, in dep_fn
    return fn(slf, device, *args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_device_type.py", line 578, in dep_fn
    return fn(slf, device, *args, **kwargs)
  File "test_linalg.py", line 2623, in test_cholesky_solve_autograd
    run_test(a_size, b_size, upper)
  File "test_linalg.py", line 2618, in run_test
    gradgradcheck(func, [root, b, upper], atol=1e-3, check_batched_grad=False)
  File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_utils.py", line 2093, in gradgradcheck
    return torch.autograd.gradgradcheck(fn, inputs, grad_outputs, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/torch/autograd/gradcheck.py", line 1082, in gradgradcheck
    check_grad_dtypes=check_grad_dtypes, check_batched_grad=check_batched_grad, fast_mode=fast_mode)
  File "/opt/conda/lib/python3.6/site-packages/torch/autograd/gradcheck.py", line 963, in gradcheck
    atol, raise_exception, check_grad_dtypes, nondet_tol):
  File "/opt/conda/lib/python3.6/site-packages/torch/autograd/gradcheck.py", line 853, in fast_gradcheck
    return fail_test(get_notallclose_msg(a, n, i, j, prefix))
  File "/opt/conda/lib/python3.6/site-packages/torch/autograd/gradcheck.py", line 939, in fail_test
    raise RuntimeError(msg)
RuntimeError: Jacobian mismatch for output 1 with respect to input 0,
numerical:tensor(-21779015.0694, device='cuda:0', dtype=torch.float64)
analytical:tensor(-22499604.7940, device='cuda:0', dtype=torch.float64)
```
After making some modifications for gradcheck (fast_mode=False) to print out max element-wise differences on failure, we see that in the vast majority of sampled seeds(89/100), the difference did not exceed <1e-12 (edit: default rtol of 1e-3 is probably playing a role here). However, there were some seeds that you could set to elicit extremely large differences - 
```
The max per-element difference (slow mode) is: 0.7281307610683143.
Fail for random seed: 12
The max per-element difference (slow mode) is: 0.0031403141628736364.
Fail for random seed: 17
The max per-element difference (slow mode) is: 0.052171220384479966.
Fail for random seed: 31
The max per-element difference (slow mode) is: 0.07823810621266603.
Fail for random seed: 43
The max per-element difference (slow mode) is: 0.10340233273689137.
Fail for random seed: 55
The max per-element difference (slow mode) is: 0.013627347102735499.
Fail for random seed: 58
The max per-element difference (slow mode) is: 0.013666869243934343.
Fail for random seed: 64
The max per-element difference (slow mode) is: 0.0017789742510103679.
Fail for random seed: 68
The max per-element difference (slow mode) is: 67408893.5859375.
Fail for random seed: 77
The max per-element difference (slow mode) is: 4.518171817384427.
Fail for random seed: 96
The max per-element difference (slow mode) is: 216.2751303485129.
Fail for random seed: 98
fast_mode: False, atol: 1e-12, number failures: 11/100
```



cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @mruberry @VitalyFedyunin @walterddr @jianyuh @heitorschueroff @IvanYashchuk @xwang233