Repository: pytorch
Issue: Printing a Tensor that is being vmap'ed over in C++ raises an error
Labels: triaged,module: vmap

## 🐛 Bug

`eig_backward` uses in-place operations, however `gradcheck` and `gradgradcheck` are happy.
Once a printing statement is introduced, an error is raised.

## To Reproduce
Simply add the following printing line according to the diff below:

```
diff --git a/torch/csrc/autograd/FunctionsManual.cpp b/torch/csrc/autograd/FunctionsManual.cpp
index 80461fa9a6..a2b6a44da1 100644
--- a/torch/csrc/autograd/FunctionsManual.cpp
+++ b/torch/csrc/autograd/FunctionsManual.cpp
@@ -2098,6 +2098,8 @@ Tensor eig_backward(const std::vector<torch::autograd::Variable> &grads, const T
     auto B = hm * at::matmul(vt, gvortho);
     auto A = at::matmul(B, vt);
 
+    std::cout << gvortho << std::endl;
+
     std::tie(result, std::ignore) = at::solve(A, vt);
   }
   // contribution from eigenvalues
```
Then run the test `pytest test/test_autograd.py -vk eig`.
The error message:
```
...
test/test_autograd.py:2796: in run_test                                                                                                                             [84/1897]
    gradcheck(func, [B])                                                                                                                                                     
torch/testing/_internal/common_utils.py:1936: in gradcheck                                                                                                                   
    return torch.autograd.gradcheck(fn, inputs, **kwargs)                                                                                                                    
torch/autograd/gradcheck.py:495: in gradcheck                                                                                                                                
    test_batched_grad(fail_test, tupled_inputs, o, j)                                                                                                                        
torch/autograd/gradcheck.py:274: in test_batched_grad
    f'While computing batched gradients, got: {ex}\n\n{FAILED_BATCHED_GRAD_MSG}')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

msg = "While computing batched gradients, got: Cannot access data pointer of Tensor that doesn't have storage\n\ngradcheck o...se see\nNOTE: [How to write vmap-compatible ba
ckward formulas]\nin the codebase for an explanation of how to fix this."

    def fail_test(msg):
        if raise_exception:
>           raise RuntimeError(msg)
E           RuntimeError: While computing batched gradients, got: Cannot access data pointer of Tensor that doesn't have storage
E           
E           gradcheck or gradgradcheck failed while testing batched gradient computation.
E           This could have been invoked in a number of ways (via a test that calls
E           gradcheck/gradgradcheck directly or via an autogenerated test).
E           
E           If you are adding a new operator, please file an issue and then use one of the
E           workarounds. The workaround depends on how your test invokes gradcheck/gradgradcheck.
E           If the test
E           - manually invokes gradcheck/gradgradcheck, then call gradcheck/gradgradcheck
E             with `check_batched_grad=False` as a keyword argument.
E           - is OpInfo-based (e.g., in test_ops.py), then modify the OpInfo for the test
E             to have `check_batched_grad=False` and/or `check_batched_gradgrad=False`.
E           - is common_method_invocations-based, then add your test to the denylist
E             EXCLUDE_BATCHED_GRAD_TESTS in test_autograd.py
E           
E           If you're modifying an existing operator that supports batched grad computation,
E           or wish to make a new operator work with batched grad computation, please read
E           the following.
E           
E           To compute batched grads (e.g., jacobians, hessians), we vmap over the backward
E           computation. The most common failure case is if there is a 'vmap-incompatible
E           operation' in the backward pass. Please see
E           NOTE: [How to write vmap-compatible backward formulas]
E           in the codebase for an explanation of how to fix this.

torch/autograd/gradcheck.py:367: RuntimeError

```

## Environment

```
PyTorch version: 1.8.0a0+fb40079                                                                                                                                     [2/1943]
Is debug build: False                                                                                                                                                        
CUDA used to build PyTorch: 11.0                                                                                                                                             
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (crosstool-NG 1.24.0.133_b0863d8_dirty) 9.3.0
Clang version: 11.0.0 (https://github.com/conda-forge/clangdev-feedstock 234a25ce90b797cf04427a62cd668fc985808c23)
CMake version: version 3.5.0

Python version: 3.6 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 11.0.221
GPU models and configuration:
GPU 0: GeForce RTX 2060
GPU 1: GeForce RTX 2060

Nvidia driver version: 460.27.04
cuDNN version: Probably one of the following:
/usr/local/cuda-10.1.243/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-10.1.243/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-10.1.243/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-10.1.243/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-10.1.243/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-10.1.243/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-10.1.243/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
/usr/local/cuda-10.2.89/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-10.2.89/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-10.2.89/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-10.2.89/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-10.2.89/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-10.2.89/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-10.2.89/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.5
[pip3] torch==1.9.0a0+992d251
[conda] magma-cuda110             2.5.2                         1    pytorch
[conda] mkl                       2020.4             h726a3e6_304    conda-forge
[conda] mkl-include               2020.4             h726a3e6_304    conda-forge
[conda] numpy                     1.19.5           py36h2aa4a07_0    conda-forge
[conda] torch                     1.9.0a0+992d251           dev_0    <develop>
```