Repository: pytorch
Issue: torch.zeros memory leak : cpu
Labels: high priority,module: memory usage,triaged

## 🐛 Bug

Hello, please let have some advice or solution of memory leak problem with torch.zeros : 

When I run torch.zeros with thread, as I start and stop thread more,  memory leak happens as follows:

## To Reproduce

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

1. I made simple code to start and stop thread with torch.zeros as follows:
     due to the hardware limitation I am using torch.zeros with cpu without CUDA!!!
```python
#================================= code =============================
from tkinter import *
from tkinter import ttk
import torch
import psutil
import threading

thread = None
function_on = False

def processing():
    img = torch.zeros((1, 3, 448, 448), device='cpu')  # init img
    #print("RAM Used : ", psutil.virtual_memory()[3], ", ",psutil.virtual_memory()[2],"%")

def my_func():
    print("==============================>>torch version:",torch.__version__)
    while function_on:
        processing()
    
def start():
  global function_on
  global thread
  if function_on:
      return
  function_on = True
  thread = threading.Thread(target = my_func)
  thread.start()

      
def end():
  global function_on

  function_on = False
  print("RAM Used : ", psutil.virtual_memory()[3], ", ",psutil.virtual_memory()[2],"%")

    
root = Tk()
button_start = ttk.Button(root,text="start", command=start)
button_end = ttk.Button(root,text="end", command=end)
button_start.grid(row=0, column=0)
button_end.grid(row=1, column=0)

root.mainloop()
#================================= code =============================
```

## Expected behavior
as I click start and stop button several times : memory usage is getting bigger.

==============================>>torch version: 1.8.1+cu102
RAM Used :  1813143552 ,  12.9 %
==============================>>torch version: 1.8.1+cu102
RAM Used :  1833521152 ,  13.1 %
==============================>>torch version: 1.8.1+cu102
RAM Used :  1855455232 ,  13.2 %
==============================>>torch version: 1.8.1+cu102
RAM Used :  1877147648 ,  13.3 %

## Environment

Collecting environment information...
PyTorch version: 1.8.1+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A
OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.10
Python version: 3.7 (64-bit runtime)
Python platform: Linux-5.8.0-53-generic-x86_64-with-debian-bullseye-sid
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.19.5
[conda] blas                      1.0                         mkl  
[conda] mkl                       2020.0                      166  
[conda] mkl-service               2.3.0            py37he904b0f_0  
[conda] mkl_fft                   1.0.15           py37ha843d7b_0  
[conda] mkl_random                1.1.0            py37hd6b4f25_0  
[conda] numpy                     1.19.5                   pypi_0    pypi
[conda] numpydoc                  0.9.2                      py_0  
[conda] torch                     1.8.1                    pypi_0    pypi
[conda] torchvision               0.9.1                    pypi_0    pypi

## Additional context
- inside (infinite loop)thread, there is no memory change(leakage) but everytime after stop the thread it looks like there are no memory cleaning therefore small amount of memory leakage appears. 

<!-- Add any other context about the problem here. -->
I try simple while loop, there was no memory leakage. 
#===================== simple while loop =====================
import torch
import psutil


while True:     
    img = torch.zeros((1, 3, 448, 448), device='cpu')  # init img
    print("RAM Used : ", psutil.virtual_memory()[3], ", ",psutil.virtual_memory()[2],"%")

cc @ezyang @gchanan @zou3519 @bdhirsh @jbschlosser @anjali411 @Varal7