Repository: pytorch
Issue: copy.deepcopy fails on spectral-norm layers after the first forward
Labels: module: nn,triaged

## 🐛 Bug

After wrapping up an `nn.Module` layer into `torch.nn.utlis_spectral_norm`, it fails to be copied after the first forward pass.

## To Reproduce

```python
import torch
import copy
layer = torch.nn.utils.spectral_norm(torch.nn.Linear(3,3))
_ = copy.deepcopy(layer) # works fine
_ = layer(torch.randn(3,3)) # copy.deepcopy will fail now
_ = copy.deepcopy(layer) # raises the exception
```

This raises the following exception:
```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-f12eb96a0223> in <module>
      4 _ = copy.deepcopy(layer) # works fine
      5 layer(torch.randn(3,3)) # copy.deepcopy will fail now
----> 6 _ = copy.deepcopy(layer) # raises the exception

~/anaconda3/lib/python3.7/copy.py in deepcopy(x, memo, _nil)
    178                     y = x
    179                 else:
--> 180                     y = _reconstruct(x, memo, *rv)
    181 
    182     # If is its own copy, don't memoize.

~/anaconda3/lib/python3.7/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    278     if state is not None:
    279         if deep:
--> 280             state = deepcopy(state, memo)
    281         if hasattr(y, '__setstate__'):
    282             y.__setstate__(state)

~/anaconda3/lib/python3.7/copy.py in deepcopy(x, memo, _nil)
    148     copier = _deepcopy_dispatch.get(cls)
    149     if copier:
--> 150         y = copier(x, memo)
    151     else:
    152         try:

~/anaconda3/lib/python3.7/copy.py in _deepcopy_dict(x, memo, deepcopy)
    238     memo[id(x)] = y
    239     for key, value in x.items():
--> 240         y[deepcopy(key, memo)] = deepcopy(value, memo)
    241     return y
    242 d[dict] = _deepcopy_dict

~/anaconda3/lib/python3.7/copy.py in deepcopy(x, memo, _nil)
    159             copier = getattr(x, "__deepcopy__", None)
    160             if copier:
--> 161                 y = copier(memo)
    162             else:
    163                 reductor = dispatch_table.get(cls)

~/anaconda3/lib/python3.7/site-packages/torch/tensor.py in __deepcopy__(self, memo)
     53             return handle_torch_function(Tensor.__deepcopy__, (self,), self, memo)
     54         if not self.is_leaf:
---> 55             raise RuntimeError("Only Tensors created explicitly by the user "
     56                                "(graph leaves) support the deepcopy protocol at the moment")
     57         if id(self) in memo:

RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment
```

## Expected behavior

The layer should be successfully copied without any exceptions.

## Environment
```
Collecting environment information...
PyTorch version: 1.8.1
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: CentOS Linux release 7.9.2009 (Core) (x86_64)
GCC version: (GCC) 6.4.0
Clang version: Could not collect
CMake version: version 3.19.2

Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.1.243
GPU models and configuration: GPU 0: NVIDIA Tesla V100-SXM2-32GB
Nvidia driver version: 465.19.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.1
[pip3] numpydoc==1.1.0
[pip3] pytorch-lightning==1.0.8
[pip3] torch==1.8.1
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.2.2
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               10.2.89              hfd86e86_1
[conda] mkl                       2021.2.0           h06a4308_296
[conda] mkl-service               2.3.0            py37h27cfd23_1
[conda] mkl_fft                   1.3.0            py37h42c9631_2
[conda] mkl_random                1.2.1            py37ha9443f7_2
[conda] numpy                     1.20.1           py37h93e21f0_0
[conda] numpy-base                1.20.1           py37h7d8b39e_0
[conda] numpydoc                  1.1.0              pyhd3eb1b0_1
[conda] pytorch                   1.8.1           py3.7_cuda10.2_cudnn7.6.5_0    pytorch
[conda] pytorch-lightning         1.0.8                    pypi_0    pypi
[conda] torchsummary              1.5.1                    pypi_0    pypi
[conda] torchvision               0.2.2                      py_3    pytorch
```
(i currently have the old torchvision version due to https://github.com/pytorch/vision/issues/3414, but it does not influence anything)


cc @albanD @mruberry @jbschlosser