Looping over Tensor leaks memory?

username: pascalm
cooked: <p>Hello,</p>
<p>The following code generates 30 million values, which occupy 30’000’000*8 / 1024^2 ~ 229MB. My machine has 16 GB and runs out of memory instantly!</p>
<pre><code class="lang-auto">import torch

data = torch.rand(30000000)

for item in data:
    pass
</code></pre>
<p>I didn’t find an open issue and I didn’t create one.</p>
<p>I’m on arch linux and work in a virtual environment using:</p>
<p>Python 3.7.12 (default, Dec  2 2021, 11:47:57)<br>
[GCC 11.1.0] on linux</p>
<p>and</p>
<pre><code class="lang-auto">$ pip list
Package            Version   Editable project location
------------------ --------- ----------------------------------
astropy            4.3.1
beautifulsoup4     4.10.0
cached-property    1.5.2
certifi            2021.10.8
cffi               1.15.0
charset-normalizer 2.0.7
cryptography       35.0.0
cycler             0.10.0
Cython             0.29.24
decorator          5.1.0
gwdatafind         1.0.4
h5py               3.6.0
idna               3.3
importlib-metadata 4.8.1
Jinja2             3.0.2
kiwisolver         1.3.2
lalsuite           7.0
ligo-segments      1.4.0
lscsoft-glue       2.0.0
Mako               1.1.5
MarkupSafe         2.0.1
matplotlib         3.4.3
mpi4py             3.1.3
mpld3              0.5.5
numpy              1.21.2
Pillow             8.4.0
pip                21.3.1
PyCBC              0.0a8210  /home/pascal/venv_thesis/src/pycbc
pycparser          2.20
pyerfa             2.0.0
pyOpenSSL          21.0.0
pyparsing          2.4.7
python-dateutil    2.8.2
requests           2.26.0
scipy              1.7.1
setuptools         60.0.0
six                1.16.0
soupsieve          2.2.1
torch              1.9.1
tqdm               4.62.3
typing-extensions  3.10.0.2
urllib3            1.26.7
zipp               3.6.0
</code></pre>

username: KFrank
cooked: <p>Hi Fancy!</p>
<aside class="quote no-group" data-username="pascalm" data-post="1" data-topic="141487" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pascalm/48/76741_2.png" class="avatar"> pascalm:</div>
<blockquote>
<p>My machine has 16 GB and runs out of memory instantly!</p>
</blockquote>
</aside>
<p>I can reproduce this on a 32 GB ubuntu machine by doubling the size<br>
of <code>data</code>:</p>
<pre data-code-wrap="python"><code class="lang-python">data = torch.rand(60000000)
</code></pre>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.10.0'
&gt;&gt;&gt; data = torch.rand(60000000)
&gt;&gt;&gt; for item in data:
...     pass
...
Killed
</code></pre>
<pre><code class="lang-plaintext">Python 3.8.3 (default, May 19 2020, 18:47:26)
</code></pre>
<p>top shows memory (%MEM) increasing up to 100% before the python<br>
process is killed.  This happens after a little more than half a minute.</p>
<p>Note, this does not happen with a numpy array:</p>
<pre data-code-wrap="python"><code class="lang-python">data = np.random.rand (60000000)
</code></pre>
<p>(In this case the loop takes a few seconds to complete.)</p>
<p>Best.</p>
<p>K. Frank</p>

username: KFrank
cooked: <p>Hi Fancy!</p>
<aside class="quote no-group" data-username="pascalm" data-post="1" data-topic="141487" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pascalm/48/76741_2.png" class="avatar"> pascalm:</div>
<blockquote>
<p>My machine has 16 GB and runs out of memory instantly!</p>
</blockquote>
</aside>
<p>Furthermore, it looks like the full time and memory are taken up by<br>
creating the iterator (or whatever implements <code>for  item in data:</code>)<br>
rather than by actually executing the loop:</p>
<pre><code class="lang-plaintext">&gt;&gt;&gt; import torch
&gt;&gt;&gt; torch.__version__
'1.10.0'
&gt;&gt;&gt; data = torch.rand (30000000)
&gt;&gt;&gt; for  item in data:
...     break
...
&gt;&gt;&gt; data = torch.rand (60000000)
&gt;&gt;&gt; for  item in data:
...     break
...
Killed
</code></pre>
<p>Best.</p>
<p>K. Frank</p>

username: ptrblck
cooked: <p>I was also able to reproduce the issue and came to the same conclusion as my print statements inside the loop were not shown while the memory increased massively in <code>torch==1.11.0.dev20220108</code>.<br>
<a class="mention" href="/u/pascalm">@pascalm</a> would you mind creating a <a href="https://github.com/pytorch/pytorch/issues">GitHub issue</a> so that we could track and fix this bug, please?</p>

username: pascalm
cooked: <p>Thanks for pointing that out. I reduced it to</p>
<pre><code class="lang-auto">import torch
data = torch.rand (60000000)
it = iter(data)
</code></pre>
<p>I’ll create a github issue.</p>