Repository: pytorch
Issue: Memory leak issue still exists in CI 
Labels: module: build,module: ci,triaged

## 🐛 Bug

I checked several CI from one of today's commits, see 
cuda 10.1 nogpu: https://circleci.com/gh/pytorch/pytorch/5100845?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
cuda 10.1: https://circleci.com/gh/pytorch/pytorch/5101775?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
cuda 10.2: https://circleci.com/gh/pytorch/pytorch/5100839?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

They all **displayed passed**. However, if you open "Test" log and search "Leak Summary", you can find those memory leaks from 
```
valgrind --suppressions=/var/lib/jenkins/workspace/aten/tools/valgrind.sup --error-exitcode=1 ./basic '--gtest_filter=-*CUDA'
```

e.g. 
```
Apr 09 21:22:34 ==73453== HEAP SUMMARY:
Apr 09 21:22:34 ==73453==     in use at exit: 707,522,005 bytes in 616,626 blocks
Apr 09 21:22:34 ==73453==   total heap usage: 1,620,959 allocs, 1,004,333 frees, 1,764,490,084 bytes allocated
Apr 09 21:22:34 ==73453== 
Apr 09 21:22:38 ==73453== LEAK SUMMARY:
Apr 09 21:22:38 ==73453==    definitely lost: 72,888 bytes in 6 blocks
Apr 09 21:22:38 ==73453==    indirectly lost: 72 bytes in 3 blocks
Apr 09 21:22:38 ==73453==      possibly lost: 3,279,992 bytes in 24,437 blocks
Apr 09 21:22:38 ==73453==    still reachable: 704,169,053 bytes in 592,180 blocks
Apr 09 21:22:38 ==73453==         suppressed: 0 bytes in 0 blocks
Apr 09 21:22:38 ==73453== Rerun with --leak-check=full to see details of leaked memory
```

## To Reproduce

I checked the latest version of this file. https://github.com/pytorch/pytorch/blob/6d3783a6bccf7054e793e5e617e46747c1d54ce6/aten/src/ATen/test/basic.cpp
It has three tests `BasicTestCPU`, `BasicTestCUDA`, and `FactoryMethodsTest`. 
The first contains cpu code, the second contains cuda code and the third contains both. 

So I did tests like this for **CPU**:
```sh
cd aten/tools
valgrind --suppressions=./valgrind.sup --error-exitcode=1 ../../build/bin/basic --gtest_filter=BasicTest.BasicTestCPU
```
I got 
```
==3843385== 
==3843385== HEAP SUMMARY:
==3843385==     in use at exit: 1,419,660 bytes in 15,506 blocks
==3843385==   total heap usage: 684,285 allocs, 668,779 frees, 83,394,360 bytes allocated
==3843385== 
==3843385== LEAK SUMMARY:
==3843385==    definitely lost: 16 bytes in 2 blocks
==3843385==    indirectly lost: 4 bytes in 1 blocks
==3843385==      possibly lost: 25,864 bytes in 64 blocks
==3843385==    still reachable: 1,393,776 bytes in 15,439 blocks
==3843385==         suppressed: 0 bytes in 0 blocks
==3843385== Rerun with --leak-check=full to see details of leaked memory
==3843385== 
==3843385== For lists of detected and suppressed errors, rerun with: -s
==3843385== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 16 from 2)
```

Then I did this for **GPU**: 
```sh
valgrind --suppressions=./valgrind.sup --error-exitcode=1 ../../build/bin/basic --gtest_filter=BasicTest.BasicTestGPU
```
I got 
```
==3843732== 
==3843732== HEAP SUMMARY:
==3843732==     in use at exit: 999,399 bytes in 15,369 blocks
==3843732==   total heap usage: 280,321 allocs, 264,952 frees, 40,447,163 bytes allocated
==3843732== 
==3843732== LEAK SUMMARY:
==3843732==    definitely lost: 0 bytes in 0 blocks
==3843732==    indirectly lost: 0 bytes in 0 blocks
==3843732==      possibly lost: 0 bytes in 0 blocks
==3843732==    still reachable: 999,399 bytes in 15,369 blocks
==3843732==         suppressed: 0 bytes in 0 blocks
==3843732== Rerun with --leak-check=full to see details of leaked memory
==3843732== 
==3843732== For lists of detected and suppressed errors, rerun with: -s
==3843732== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
```

`FactoryMethodsTest` took too long on my machine so I stopped it. 

The two tests give similar leak/no leak results on my machine with **cuda 10.1 and 10.2**

## Expected behavior

No memory leak and correct CI return code. 

## Environment

 - PyTorch Version (e.g., 1.0): 1.6.0a0+1ffc2d9
 - OS (e.g., Linux): Fedora 29
 - How you installed PyTorch (`conda`, `pip`, source): source
 - Build command you used (if compiling from source): python setup.py develop --user
 - Python version: 3.7.6
 - CUDA/cuDNN version: 10.1 and 10.2
 - GPU models and configuration: RTX 2070 Super
 - Any other relevant information:

## Additional context

This issue may be related to #33471 #34241

For internal reference: 2870960

cc. @ptrblck @ezyang @seemethere 



cc @ezyang @seemethere