Pytorch Memory leak with pretrained models

Gul_Zain: <p>I am trying to run a model that requires vgg16 pretrained models. My issue is presented here:</p><aside class="onebox githubissue" data-onebox-src="https://github.com/richzhang/PerceptualSimilarity/issues/87">
  <header class="source">

      <a href="https://github.com/richzhang/PerceptualSimilarity/issues/87" target="_blank" rel="noopener nofollow ugc">github.com/richzhang/PerceptualSimilarity</a>
  </header>

  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/richzhang/PerceptualSimilarity/issues/87" target="_blank" rel="noopener nofollow ugc">CUDA OUT OF MEMORY</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-11-21" data-time="06:45:38" data-timezone="UTC">06:45AM - 21 Nov 21 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/gulzainali98" target="_blank" rel="noopener nofollow ugc">
          <img alt="gulzainali98" src="https://avatars.githubusercontent.com/u/34415593?v=4" class="onebox-avatar-inline" width="20" height="20">
          gulzainali98
        </a>
      </div>
    </div>

    <div class="labels">
    </div>
  </div>
</div>

  <div class="github-row">
    <p class="github-body-container">I am trying to calculate lpips metric on directory of images. i use following co<span class="show-more-container"><a href="" rel="noopener" class="show-more">…</a></span><span class="excerpt hidden">de to load and calculate

```
for i in range(len(ground_truth)):

                p= lpips.im2tensor(lpips.load_image(predictions[i]))
                g= lpips.im2tensor(lpips.load_image(ground_truth[i]))
                if use_gpu:
                    p=p.cuda()
                    g=g.cuda()
                mean_total=mean_total+loss_fn.forward(g,p).mean()
                im_counter=im_counter+1
                del p
                del g
                torch.cuda.empty_cache()
```

However, I am getting cuda out of memory error even with alexnet backbone and A100 40GB GPU!</span></p>
  </div>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>I have narrowed down the problem. Continuous memory increase occurs from line 36 in pretrained_networks file to line 48. (<a href="https://github.com/richzhang/PerceptualSimilarity/blob/master/lpips/pretrained_networks.py#L36" class="inline-onebox" rel="noopener nofollow ugc">PerceptualSimilarity/pretrained_networks.py at master · richzhang/PerceptualSimilarity · GitHub</a>)</p>
<p>This simply contains the layers from pretrained vgg16 network. I have tried with torch.no_grad() as well and I also tried to not inherit nn.Module. I have tried deleting my variables and also not deleting them. Problem persist and even at A100 40GB I get cuda out of memory after just some 200 images. Looking forward to quick response <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <p>In the code snippet posted in the linked issue you are accumulating the loss and are thus also storing the computation graphs which are potentially attached to it:</p>
<pre><code class="lang-python">for i in range(len(ground_truth)):

                p= lpips.im2tensor(lpips.load_image(predictions[i]))
                g= lpips.im2tensor(lpips.load_image(ground_truth[i]))
                if use_gpu:
                    p=p.cuda()
                    g=g.cuda()
                mean_total=mean_total+loss_fn.forward(g,p).mean()
                im_counter=im_counter+1
                del p
                del g
                torch.cuda.empty_cache()
</code></pre>
<p>The linked line of code only uses a subset of the <code>features</code> of the pretrained model, so I don’t think it’s “leaking” memory. Could you describe how you narrowed down the leak to this line of code?</p>

Gul_Zain: <p>I printed usage before and after certain lines narrowing it down to the lines for which i showed github link. But i will try to find a work around for .forward function maybe call detach().cpu().numpy before doing addition</p>

Gul_Zain: <p>Thanks a lot. calling detach().cpu().numpy() solves the problem.</p>