RAM not free when deleting a model in CPU? worse after inference, is there some cache hidden?
Labels: module: memory usage, triaged

### 🐛 Describe the bug

Hi, all,
After deleting a model (e.g. ResNet50) with `del(model)`, the memory does not seem to be releasing.
Worse, memory grows too much after simple inference, in eval mode, without grads : I understand the model needs to keep some infos about tensors during inference for skip connections, but I guess it should not retain anything after this connection, and be back to same state after initialization?
I checked it's not the case for simple modules like Sequentials of Linears etc., so it may be related to special modules.
Not sure if it's a real bug, but I feel like it's not the output we would expect. If I should ask on the Forum, please tell me.
Manual Garbage Collection does not change anything, so i'm confused where the memory goes. Thanks for the answer

first, some quick code to get RAM usage
```python
import os, psutil
import time
def print_memory_usage(prefix):
    """prints current process RAM consumption in Go"""
    process = psutil.Process(os.getpid())
    print("memory usage ", prefix, process.memory_info().rss / 1e9, "Go")
```

Now the ResNet50 initialization

```python
import torch
from torchvision.models import resnet50
print_memory_usage(prefix="before anything")

model = resnet50(pretrained=True)
model.eval()
for param in model.parameters():
    param.requires_grad_(False)
print_memory_usage(prefix="after model init")

del model
print_memory_usage(prefix="after removing model")
```

```
memory usage  before anything 0.15622144 Go
memory usage  after model init 0.362528768 Go
memory usage  after removing model 0.362561536 Go # not released
```

and worse : memory "explosion" after inference in eval mode and grad-free

```python
import torch
from torchvision.models import resnet50
print_memory_usage(prefix="before anything")

model = resnet50(pretrained=True)
model.eval()
for param in model.parameters():
    param.requires_grad_(False)
print_memory_usage(prefix="after model init")

input_tensor = torch.rand((100, 3, 128, 128))
print_memory_usage(prefix="after init input")

with torch.no_grad():
    model.forward(input_tensor)
print_memory_usage(prefix="after inference")

del model
print_memory_usage(prefix="after removing model")
del input_tensor
print_memory_usage(prefix="after removing input")
```

```
memory usage  before anything 0.155877376 Go
memory usage  after model init 0.362856448 Go
memory usage  after init input 0.382713856 Go
memory usage  after inference 1.41819904 Go # outch, where does it comes from, if it's grad-free?
memory usage  after removing model 1.294467072 Go # memory not released
memory usage  after removing input 1.18964224 Go
```

### Versions

```
Collecting environment information...
PyTorch version: 1.12.0.post2
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.4 (arm64)
GCC version: Could not collect
Clang version: 12.0.5 (clang-1205.0.22.9)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:00:30)  [Clang 11.0.1 ] (64-bit runtime)
Python platform: macOS-12.4-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.920
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.2
[pip3] torch==1.12.0.post2
[pip3] torchaudio==0.12.1
[pip3] torchvision==0.11.3
[conda] numpy                     1.20.3                   pypi_0    pypi
[conda] pytorch                   1.12.0          cpu_py39h0768760_2    conda-forge
[conda] torch                     1.10.2                   pypi_0    pypi
[conda] torchaudio                0.12.1                   pypi_0    pypi
[conda] torchvision               0.11.3                   pypi_0    pypi
```