When using conv3d, a large amount of video memory is occupied

yinjun1131: <p>As mentioned above, do you have any good optimization solutions? Our model used an input source with a size of 1 * 150 * 128 * 128 * 3. When attempting to run it, the first conv3d instantly consumed almost 1g of memory, which is unacceptable for end-to-end deployment. When I try to reduce the duration, its memory usage becomes very low again. What should I do to optimize memory usage? I have tried quantifying fp16 and int8, but they have not worked and instead have increased memory usage。<br>
My model file，</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://accounts.google.com/v3/signin/identifier?continue=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F1FDzaawgUP0PvmOVlYpKM_nQmpqt9S9vy%2Fview%3Fusp%3Ddrive_link&amp;followup=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F1FDzaawgUP0PvmOVlYpKM_nQmpqt9S9vy%2Fview%3Fusp%3Ddrive_link&amp;ifkv=AcMMx-dGEdC_EYtOQcB5gi526BGBukQENOe31bhleAusLQKiVX1HnotfQ74CQ-hcKADrkAco-Md_&amp;osid=1&amp;passive=1209600&amp;service=wise&amp;flowName=GlifWebSignIn&amp;flowEntry=ServiceLogin&amp;dsh=S-2110298726%3A1732256373078754&amp;ddm=1">
  <header class="source">

      <a href="https://accounts.google.com/v3/signin/identifier?continue=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F1FDzaawgUP0PvmOVlYpKM_nQmpqt9S9vy%2Fview%3Fusp%3Ddrive_link&amp;followup=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F1FDzaawgUP0PvmOVlYpKM_nQmpqt9S9vy%2Fview%3Fusp%3Ddrive_link&amp;ifkv=AcMMx-dGEdC_EYtOQcB5gi526BGBukQENOe31bhleAusLQKiVX1HnotfQ74CQ-hcKADrkAco-Md_&amp;osid=1&amp;passive=1209600&amp;service=wise&amp;flowName=GlifWebSignIn&amp;flowEntry=ServiceLogin&amp;dsh=S-2110298726%3A1732256373078754&amp;ddm=1" target="_blank" rel="noopener nofollow ugc">accounts.google.com</a>
  </header>

  <article class="onebox-body">
    

<h3><a href="https://accounts.google.com/v3/signin/identifier?continue=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F1FDzaawgUP0PvmOVlYpKM_nQmpqt9S9vy%2Fview%3Fusp%3Ddrive_link&amp;followup=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F1FDzaawgUP0PvmOVlYpKM_nQmpqt9S9vy%2Fview%3Fusp%3Ddrive_link&amp;ifkv=AcMMx-dGEdC_EYtOQcB5gi526BGBukQENOe31bhleAusLQKiVX1HnotfQ74CQ-hcKADrkAco-Md_&amp;osid=1&amp;passive=1209600&amp;service=wise&amp;flowName=GlifWebSignIn&amp;flowEntry=ServiceLogin&amp;dsh=S-2110298726%3A1732256373078754&amp;ddm=1" target="_blank" rel="noopener nofollow ugc">Google Drive: Sign-in</a></h3>

  <p>Access Google Drive with a Google account (for personal use) or Google Workspace account (for business use).</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


ptrblck: <p>How many channels does the first convolution generate?</p>

yinjun1131: <p>oops ,sorry  ，look like my img was update faile,i will be update img<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/6/66ba610ee4560865f2cf0bf587ac98d9921238fc.png" data-download-href="https://discuss.pytorch.org/uploads/default/66ba610ee4560865f2cf0bf587ac98d9921238fc" title="微信图片_20241123125104"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/6/66ba610ee4560865f2cf0bf587ac98d9921238fc.png" alt="微信图片_20241123125104" data-base62-sha1="eEM151SPH80MSypqGMuNI99XSJC" width="690" height="480" data-dominant-color="222526"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20241123125104</span><span class="informations">717×499 21.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8bbc0992a6b11d02b17631dd430b94b710f70b5f.png" data-download-href="https://discuss.pytorch.org/uploads/default/8bbc0992a6b11d02b17631dd430b94b710f70b5f" title="微信图片_20241123125108"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8bbc0992a6b11d02b17631dd430b94b710f70b5f.png" alt="微信图片_20241123125108" data-base62-sha1="jW9eFGyvSQYpEH9kuaDwGnA4IF1" width="686" height="278"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20241123125108</span><span class="informations">686×278 10 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

yinjun1131: <p>and my input was [1,3,150,128,128]</p>

ptrblck: <p>I cannot reproduce any issues and see the expected memory usage:</p>
<pre data-code-wrap="python"><code class="lang-python">print("{:.2f}MB used".format(torch.cuda.memory_allocated()/ 1024**2))
# 0.00MB used

device = "cuda"
x = torch.randn(1, 3, 150, 128, 128, device=device)
conv = nn.Conv3d(3, 16, [3, 5, 5], stride=1, padding=(4, 2, 2), dilation=(4, 1, 1)).to(device)

print("{:.2f}MB used".format(torch.cuda.memory_allocated()/ 1024**2))
# 28.14MB used

out = conv(x)
print("{:.2f}MB used".format(torch.cuda.memory_allocated()/ 1024**2))
# 178.14MB used
</code></pre>

yinjun1131: <p>I just tried using your method to call only one conv3d separately, and the memory usage is within an acceptable range. However, if I execute the following code, it will estimated to occupy 1g of memory. Is there any good optimization solution for this? I am trying to convert this model into onnx, which only takes up less than 500mb of memory on mobile devices, while the first layer of this model (the following code) uses 1g<br>
Thank you for your reply</p>
<pre><code class="lang-auto">self.Feature_extracter = nn.Sequential(
            nn.Conv3d(3, 16, [3,5,5],stride=1, padding=(4,2,2), dilation=(4,1,1)),
            nn.BatchNorm3d(16),
            nn.ReLU(inplace=True),
            nn.MaxPool3d((1, 2, 2), stride=(1, 2, 2)),

            # self.drop,

            nn.Conv3d(16, 32, [3, 3, 3], stride=1, padding=(4,1,1), dilation=(4,1,1)),
            nn.BatchNorm3d(32),
            nn.ReLU(inplace=True),

            # self.drop,

            nn.Conv3d(32, 64, [3, 3, 3], stride=1, padding=(4,1,1), dilation=(4,1,1)),
            nn.BatchNorm3d(64),
            nn.ReLU(inplace=True),
            nn.MaxPool3d((1, 2, 2), stride=(1, 2, 2)),
            # self.drop,
        )
</code></pre>

yinjun1131: <p>I once tried to disassemble Sequential, but it seems that the memory usage will only be released after the model has finished running. Is there a way to release it in advance?</p>

ptrblck: <p>Intermediate forward activations will be stored to compute the gradients during the backwards pass. If you don’t want to compute gradients, use <code>torch.no_grad()</code> to disable storing intermediates.</p>

yinjun1131: <p>I followed your method and it seems to have worked, saving a little bit of memory, but it still seems to be an unacceptable usage size on mobile devices. May I ask what method can I use to obtain the memory usage situation? Such as how much memory is occupied by that parameter, etc</p>

ptrblck: <p>Could could estimate the memory usage by calculating the number of parameters, forward activations, gradients etc. as described <a href="https://discuss.pytorch.org/t/resnet-50-takes-10-13gb-to-run-with-batch-size-of-96/117402/2">here</a>. You could also use the dispatch mode as described <a href="https://discuss.pytorch.org/t/number-of-activations-without-loading-the-model/202419/2">here</a>.</p>