NoiseTunnel: CUDA out of memory when trying out

username: nareshr8
cooked: <p>I was trying out captum. I am getting CUDA out of memory when I try NoiseTunnel option in captum.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/a/2af1c8aede0de607333b2f5a3c87ab496af4de8e.png" data-download-href="https://discuss.pytorch.org/uploads/default/2af1c8aede0de607333b2f5a3c87ab496af4de8e" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/a/2af1c8aede0de607333b2f5a3c87ab496af4de8e_2_690x124.png" alt="image" data-base62-sha1="67U4ycLHIrUpeQsdg2lRrVbCVJA" width="690" height="124" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/a/2af1c8aede0de607333b2f5a3c87ab496af4de8e_2_690x124.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/a/2af1c8aede0de607333b2f5a3c87ab496af4de8e_2_1035x186.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/a/2af1c8aede0de607333b2f5a3c87ab496af4de8e.png 2x" data-dominant-color="F9F7F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1287×233 31.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Can someone help in solving where I am going wrong…</p>

username: ptrblck
cooked: <p>Your device might not have enough memory and thus you are seeing the OOM issue.<br>
Could you explain, what <code>noise_tunnel</code> does and could you try to lower the batch size or number of samples you are passing to this method?</p>

username: nareshr8
cooked: <p>I am using GeForce GTX 1070 Ti, which has 8 GB of memory. I was just trying out the example <a href="https://captum.ai/tutorials/Resnet_TorchVision_Interpret" rel="nofollow noopener">here</a>. I am using just one image.</p>

username: ptrblck
cooked: <p>Could you check the memory usage via <code>nvidia-smi</code> and make sure the device is empty and stop other processes, if necessary?<br>
I haven’t tried the code, so I’m not sure how large the expected memory footprint is.</p>

username: nareshr8
cooked: <p>I saw that it as using just around 300-400 MB before running this line. And on execution, it went out of memory.</p>

username: Narine
cooked: <p><a class="mention" href="/u/nareshr8">@nareshr8</a>, you can reduce n_samples to limit the number of perturbed examples and n_steps to reduce the number of integral approximation steps.<br>
With our recent improvements (the PR got merged a couple of days ago) you should be able to adjust internal_batch_size and run IG for very large number of steps.<br>
<aside class="onebox githubcommit">
  <header class="source">
      <a href="https://github.com/pytorch/captum/commit/455483a34e6012a09fb22346f969072c478de68b" target="_blank" rel="nofollow noopener">github.com/pytorch/captum</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Commit">
    <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M10.86 7c-.45-1.72-2-3-3.86-3-1.86 0-3.41 1.28-3.86 3H0v2h3.14c.45 1.72 2 3 3.86 3 1.86 0 3.41-1.28 3.86-3H14V7h-3.14zM7 10.2c-1.22 0-2.2-.98-2.2-2.2 0-1.22.98-2.2 2.2-2.2 1.22 0 2.2.98 2.2 2.2 0 1.22-.98 2.2-2.2 2.2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/captum/commit/455483a34e6012a09fb22346f969072c478de68b" target="_blank" rel="nofollow noopener">Internal Batching Improvements (#333)</a>
    </h4>

    <div class="github-info">
      <div class="date">
        committed <span class="discourse-local-date" data-format="ll" data-date="2020-04-01" data-time="18:48:26" data-timezone="UTC">06:48PM - 01 Apr 20 UTC</span>
      </div>

      <div class="user">
        <a href="https://github.com/vivekmig" target="_blank" rel="nofollow noopener">
          <img alt="vivekmig" src="https://avatars0.githubusercontent.com/u/11067177?v=4" class="onebox-avatar-inline" width="20" height="20">
          vivekmig
        </a>
        
      </div>

      <div class="lines" title="changed 18 files with 337 additions and 85 deletions">
        <a href="https://github.com/pytorch/captum/commit/455483a34e6012a09fb22346f969072c478de68b" target="_blank" rel="nofollow noopener">
          <span class="added">+337</span>
          <span class="removed">-85</span>
        </a>
      </div>
    </div>

  </div>
</div>


  <div class="github-row">
    <pre class="github-content" style="white-space: normal;">Summary:
This PR improves internal batching to avoid creating an expanded tensor initially and simply recursively compute each subset of steps and...</pre>
  </div>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

username: nareshr8
cooked: <p>Thanks a lot <a class="mention" href="/u/narine">@Narine</a> it worked for me. Thanks.</p>