GPU out of memory during forward pass in inceptionV3

moo3030: <p>I am trying to train an inceptionV3 model with a batch size of 256. The whole dataset does not exceed 5MB and the size of the model is around 45MB, so i have no problem in loading the model and batches of data.<br>
the error i am getting looks like this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/5/c52e3e6ef86b31709e227c3ecd5ac03615b302eb.png" data-download-href="https://discuss.pytorch.org/uploads/default/c52e3e6ef86b31709e227c3ecd5ac03615b302eb" title="Screenshot 2024-02-28 at 5.32.05 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/c/5/c52e3e6ef86b31709e227c3ecd5ac03615b302eb_2_511x499.png" alt="Screenshot 2024-02-28 at 5.32.05 PM" data-base62-sha1="s8l8BiGYfkK8pRWcko4MlCDMSHV" width="511" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/c/5/c52e3e6ef86b31709e227c3ecd5ac03615b302eb_2_511x499.png, https://discuss.pytorch.org/uploads/default/optimized/3X/c/5/c52e3e6ef86b31709e227c3ecd5ac03615b302eb_2_766x748.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/c/5/c52e3e6ef86b31709e227c3ecd5ac03615b302eb.png 2x" data-dominant-color="232323"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2024-02-28 at 5.32.05 PM</span><span class="informations">916×895 133 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The code crashes in the first epoch, first batch and during the first forward pass. Am I doing something wrong or is this an expected behavior. The forward pass takes about 12GB on its own</p>
<p>This is my code:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/d/ad1d159abe0bd6ebbffbce8f9ad9c441ac8874df.png" data-download-href="https://discuss.pytorch.org/uploads/default/ad1d159abe0bd6ebbffbce8f9ad9c441ac8874df" title="Screenshot 2024-02-28 at 4.56.26 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/a/d/ad1d159abe0bd6ebbffbce8f9ad9c441ac8874df_2_690x404.png" alt="Screenshot 2024-02-28 at 4.56.26 PM" data-base62-sha1="oHqUKQ5M22FSSxDY81af5XaSeKP" width="690" height="404" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/a/d/ad1d159abe0bd6ebbffbce8f9ad9c441ac8874df_2_690x404.png, https://discuss.pytorch.org/uploads/default/optimized/3X/a/d/ad1d159abe0bd6ebbffbce8f9ad9c441ac8874df_2_1035x606.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/a/d/ad1d159abe0bd6ebbffbce8f9ad9c441ac8874df_2_1380x808.png 2x" data-dominant-color="272828"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2024-02-28 at 4.56.26 PM</span><span class="informations">1452×852 103 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>(ignore the cpu memory usage, it is not reported correctly)</p>
<p>Thanks!</p>

ptrblck: <p>The OOM might be expected since the forward activations, needed to compute the gradients, could take the majority of the memory as described <a href="https://discuss.pytorch.org/t/resnet-50-takes-10-13gb-to-run-with-batch-size-of-96/117402/2">here</a>.</p>

moo3030: <p>Thanks, I appreciate your help!</p>