[ONNX] Memory leak when exporting a jit model to onnx
Labels: needs reproduction, oncall: jit, module: onnx

## Reproduction

The following code, which repeatedly exports a `torch.jit.script` model with `torch.onnx.export`, has a memory leak. During the export, every tensor parameter in `network` is cloned once and then immediately leaked forever, without ever being collected by the GC. It's not the _underlying buffer_ that's cloned, it's the lightweight `torch.Tensor` wrapper object itself. Still, for long running processes that often export networks in this manner this is a unbounded memory leak that eventually results in OOM errors.

I've reproduced this issue on both Linux and Windows, with pytorch versions `1.10.0` and `1.12.0` respectively.

```python
network = nn.Linear(4, 4)
network = torch.jit.script(network)

path = "network.onnx"
arg = torch.randn(1, 4)

while True:
    if os.path.exists(path):
        os.remove(path)
    torch.onnx.export(network, arg, path)

    # debug tools, these don't affect the behaviour
    gc.collect()
    objgraph.show_growth()
    print([t.shape for t in objgraph.by_type("Tensor")[-4:]])
    print([t.storage().data_ptr() for t in objgraph.by_type("Tensor")[-4:]])
    print(gc.get_referrers(objgraph.by_type("Tensor")[-1]))
```

The final five lines inside the for loop are to debug what happens, they are not neccesary to reproduce the issue.

- `gc.collect()` forces a gc collection cycle, ensuring we're not accidentally counting dead objects
  
- `objgraph.show_growth()` show the total amount of objects that exist for each type for all objects whose amount has increased. From this we can see that we're leaking 2 additional tensors per iteration.

- `print([t.shape ... ])` shows that the tensors we're leaking have shapes `(4,4)` and `(4)`, so they're just the weight and bias of the linear layer.

- `print([t.storage() ... ])` shows that the underlying buffer is always the same, so only the shallow `tensor` class instance is being leaked.

- `print(gc. ... )` shows that nothing is pointing to these newly created objects, so they _should_ be collected.
  

Example output after running for a while:

```
Tensor     1081        +2
[torch.Size([4, 4]), torch.Size([4]), torch.Size([4, 4]), torch.Size([4])]
[2369979263104, 2369977312000, 2369979263104, 2369977312000]
[]
Tensor     1083        +2
[torch.Size([4, 4]), torch.Size([4]), torch.Size([4, 4]), torch.Size([4])]
[2369979263104, 2369977312000, 2369979263104, 2369977312000]
[]
Tensor     1085        +2
[torch.Size([4, 4]), torch.Size([4]), torch.Size([4, 4]), torch.Size([4])]
[2369979263104, 2369977312000, 2369979263104, 2369977312000]
[]
```

## Related issues

#61263 seems closely related but is more about a temporary doubling in memory, this issue is about a permanent memory leak.
#28414 was closed as a duplicate of the previous issue, but better matches this issue.

## PyTorch version info (for Linux)

Collecting environment information...
PyTorch version: 1.10.0
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.10.2
Libc version: glibc-2.17

Python version: 3.7.11 (default, Jul 27 2021, 14:32:16)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.15.0-41-generic-x86_64-with-debian-buster-sid
Is CUDA available: True
CUDA runtime version: 11.3.109
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3080 Ti
GPU 1: NVIDIA GeForce RTX 3080 Ti

Nvidia driver version: 515.48.07
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.2
[pip3] torch==1.10.0
[pip3] torchelastic==0.2.0
[pip3] torchtext==0.11.0
[pip3] torchvision==0.11.0
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               ha36c431_9    nvidia
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.3.0           h06a4308_520  
[conda] mkl-service               2.4.0            py37h7f8727e_0  
[conda] mkl_fft                   1.3.1            py37hd3c417c_0  
[conda] mkl_random                1.2.2            py37h51133e4_0  
[conda] numpy                     1.21.2           py37h20f2e39_0  
[conda] numpy-base                1.21.2           py37h79a1101_0  
[conda] pytorch                   1.10.0          py3.7_cuda11.3_cudnn8.2.0_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchelastic              0.2.0                    pypi_0    pypi
[conda] torchtext                 0.11.0                     py37    pytorch
[conda] torchvision               0.11.0               py37_cu113    pytorch