CPU Memory Leak

mchawaV: <p>Hello,</p>
<p>I’m trying to experiment different configuration with the A3C code posted on GitHub under the following link:<br>
<a href="https://github.com/MorvanZhou/pytorch-A3C" rel="noopener nofollow ugc">https://github.com/MorvanZhou/pytorch-A3C</a></p>
<p>All my tested are concerned with the script “discrete_A3C.py”<br>
<a href="https://github.com/MorvanZhou/pytorch-A3C/blob/master/discrete_A3C.py" rel="noopener nofollow ugc">https://github.com/MorvanZhou/pytorch-A3C/blob/master/discrete_A3C.py</a></p>
<p>Python Version: 3.6.9<br>
Torch Version: 1.4.0</p>
<p>If I keep everything as in the original code the memory usage is consistent and does not increase over time. Which can be shown by plotting the memory usage graph using the command:</p>
<blockquote>
<p>mprof run --multiprocess discrete_A3C.py</p>
</blockquote>
<p>However, I tried to change the NN architecture to be as follows:</p>
<pre><code class="lang-auto">    def __init__(self, s_dim, a_dim):
        super(Net, self).__init__()
        self.s_dim = s_dim
        self.a_dim = a_dim

        self.fc1 = nn.Linear(s_dim, 128)
        self.fc2 = nn.Linear(128, 128)

        self.pi1 = nn.Linear(128, 64)
        self.pi2 = nn.Linear(64, a_dim)  # Q values for each action are output

        self.v1 = nn.Linear(128, 128)
        self.v2 = nn.Linear(128, 1)
        set_init([self.pi1, self.pi2, self.fc1, self.fc2, self.v1, self.v2])
        self.distribution = torch.distributions.Categorical

    def forward(self, x):
        x = F.relu6(self.fc1(x))
        x = F.relu6(self.fc2(x))

        pi1 = F.relu6(self.pi1(x))
        logits = self.pi2(pi1)

        v1 = F.relu6(self.v1(x))
        values = self.v2(v1)
        return logits, values
</code></pre>
<p>re-running the same command as before however it can be noticed a linear increase of memory usage overtime.</p>
<p>Please check the image below where the figure on the left is from running the original code, and the figure on the right is from the modified architecture.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/8/e8357caa8eb7fef156416362dd30544a3e6605e6.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/e8357caa8eb7fef156416362dd30544a3e6605e6" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/8/e8357caa8eb7fef156416362dd30544a3e6605e6_2_690x184.jpeg" alt="image" data-base62-sha1="x8dmQnwavdSMYbFGrhZewpfRH4q" width="690" height="184" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/8/e8357caa8eb7fef156416362dd30544a3e6605e6_2_690x184.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/8/e8357caa8eb7fef156416362dd30544a3e6605e6_2_1035x276.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/8/e8357caa8eb7fef156416362dd30544a3e6605e6_2_1380x368.jpeg 2x" data-dominant-color="F9F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">3706×991 292 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I tried to debug this myself during the past week but can not come up with any clues, if anyone can provide any insights would be of valuable help.</p>

albanD: <p>Hi,</p>
<p>This looks very similar to <a href="https://github.com/pytorch/pytorch/issues/32284">https://github.com/pytorch/pytorch/issues/32284</a>.<br>
Does it actually run out of memory (like your process gets killed by the OS)? Or does it stabilizes when it gets to full memory?</p>

mchawaV: <p>Yes , it does get out of memory eventually.</p>
<p>I read the link, it seems similar indeed however in my case I’m not using any MaxPool2D and also the leak happens due to architecture change without using new functions.</p>

albanD: <p>Interesting.<br>
Could you give both architectures side by side?<br>
also does running <code>import gc; gc.collect()</code> helps?</p>

mchawaV: <p>I tried to use gc.collect() but it does not help at all.</p>
<p>Below are the two architectures side-by-side and the only modification to the whole code, the left one is the original one where the problem does not exist, the one on the right is the modified one which causes the issue:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/0/900805321dd7fddff0fedcd84722716cac8a43d8.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/900805321dd7fddff0fedcd84722716cac8a43d8" title="image (1)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/0/900805321dd7fddff0fedcd84722716cac8a43d8_2_690x222.jpeg" alt="image (1)" data-base62-sha1="ky9We5CXvmWOqGOopA7cwSQOefe" width="690" height="222" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/0/900805321dd7fddff0fedcd84722716cac8a43d8_2_690x222.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/9/0/900805321dd7fddff0fedcd84722716cac8a43d8.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/9/0/900805321dd7fddff0fedcd84722716cac8a43d8.jpeg 2x" data-dominant-color="F5F6F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image (1)</span><span class="informations">1014×327 84 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

albanD: <p>What is <code>F.relu6</code> here?</p>
<p>Otherwise this should not change anything.</p>

mchawaV: <p>F.relu6 is torch.nn.functional.relu6</p>
<p>documentation can be found here:<br>
<a href="https://pytorch.org/docs/stable/nn.functional.html" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.functional.html</a></p>

albanD: <p>Ha, I didn’t knew this was a thing <img src="https://discuss.pytorch.org/images/emoji/apple/open_mouth.png?v=9" title=":open_mouth:" class="emoji" alt=":open_mouth:"></p>
<p>Trying locally with these two nets do not lead to any issue.<br>
Do you have a small (40/50 lines) code sample I can run locally that will reproduce this?</p>

mchawaV: <p>Are you using CentOS ? , I did a quick test on CentOS and it seems no issue there. But the issue happens on Ubuntu 18.04</p>
<p>You can find the whole code on the github repo in the description , just change the 2 functions I wrote above and launch the script discrete_A3C.py .</p>

albanD: <p>Yes i checked on centos.</p>
<blockquote>
<p>You can find the whole code on the github repo</p>
</blockquote>
<p>Unfortunately I cannot install gym and run the multiprocess training here locally (due to some restrictions on the installs on the machine I use) <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=12" title=":confused:" class="emoji" alt=":confused:" loading="lazy" width="20" height="20"></p>

mchawaV: <p>Ok , I will double check if the issue can be reproduced on CentOS and get back.<br>
Also I found some interesting results after some tinkering that needs confirmation.</p>
<p>Will be back tomorrow. Thanks mate.</p>

mchawaV: <p>The issue exists on CentOSv8.</p>
<p>Can you retry to do the test at your side with the following packages installed ?</p>
<blockquote>
<p>cloudpickle==1.2.2<br>
cycler==0.10.0<br>
future==0.18.2<br>
gym==0.15.4<br>
joblib==0.14.1<br>
kiwisolver==1.1.0<br>
matplotlib==3.1.2<br>
numpy==1.18.1<br>
opencv-python==4.1.2.30<br>
pandas==1.0.0<br>
Pillow==7.0.0<br>
pkg-resources==0.0.0<br>
pyglet==1.3.2<br>
pyparsing==2.4.6<br>
python-dateutil==2.8.1<br>
pytz==2019.3<br>
scikit-learn==0.22.1<br>
scipy==1.4.1<br>
six==1.14.0<br>
sklearn==0.0<br>
torch==1.4.0<br>
torchvision==0.5.0<br>
xlrd==1.2.0</p>
</blockquote>
<p>I doubt that maybe it’s something in my environment that is causing this issue.</p>