Understand the memory allocation visualization

username: WindowsXp
cooked: <p>I’m using <a href="https://docs.pytorch.org/memory_viz" rel="noopener nofollow ugc">https://docs.pytorch.org/memory_viz</a> (thanks for the amazing tool) to debug my OOM issue and finding it confused about the output.<br>
Take the screenshot below as an example, we focus on memory allocation A and B.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/7/37ae69aac9b16a775314bdb5068f566ab9f39432.png" data-download-href="https://discuss.pytorch.org/uploads/default/37ae69aac9b16a775314bdb5068f566ab9f39432" title="Screenshot 2025-06-30 at 11.00.50 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/37ae69aac9b16a775314bdb5068f566ab9f39432_2_690x88.png" alt="Screenshot 2025-06-30 at 11.00.50 PM" data-base62-sha1="7WzWbzvCHbXLOzKt3l933WG3Osq" width="690" height="88" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/37ae69aac9b16a775314bdb5068f566ab9f39432_2_690x88.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/37ae69aac9b16a775314bdb5068f566ab9f39432_2_1035x132.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/7/37ae69aac9b16a775314bdb5068f566ab9f39432_2_1380x176.png 2x" data-dominant-color="CAD0D0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2025-06-30 at 11.00.50 PM</span><span class="informations">2762×356 37.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
As the red box highlighted, there is 22.8 GB memory left after allocation A is allocated.<br>
<img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/0888e1e44bac01fbde0805dbf476b21fbad9a05c_2_690x11.png" alt="Screenshot 2025-06-30 at 11.05.07 PM" data-base62-sha1="1dv5vQM6pB7ofeRf3qQKZGxIvda" width="690" height="11" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/0888e1e44bac01fbde0805dbf476b21fbad9a05c_2_690x11.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/0888e1e44bac01fbde0805dbf476b21fbad9a05c_2_1035x16.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/0888e1e44bac01fbde0805dbf476b21fbad9a05c_2_1380x22.png 2x" data-dominant-color="E2E2E2"><br>
However, after allocation B is allocated, the total memory became 21.2 GB, which seems to contradict the graph as B should be top of A. The y-axis also suggests that after B is allocated, the total memory used is around 22.7 GB. It leads to another question: how to know the memory peak value? For instance, what’s the maximum memory of the graph below? Is it 27.3 GB(inferred from the y-axis) or 25.4 GB(read from the log)?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/4/24596b92554d09bb8c4d205b1d9df25fcaaf1da2.png" data-download-href="https://discuss.pytorch.org/uploads/default/24596b92554d09bb8c4d205b1d9df25fcaaf1da2" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/4/24596b92554d09bb8c4d205b1d9df25fcaaf1da2_2_517x142.png" alt="image" data-base62-sha1="5byLN5sxfO0r7Plvp834v4gYI6u" width="517" height="142" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/4/24596b92554d09bb8c4d205b1d9df25fcaaf1da2_2_517x142.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/4/24596b92554d09bb8c4d205b1d9df25fcaaf1da2_2_775x213.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/4/24596b92554d09bb8c4d205b1d9df25fcaaf1da2_2_1034x284.png 2x" data-dominant-color="F0F3F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2932×810 102 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>It’s worth mentioning that allocation B lives longer than A. I wonder if the <code>total memory used after allocation</code> counts the lowest memory watermark.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/1/b15e1b6abf1d2610a185411235d5368c027fd6ca.png" data-download-href="https://discuss.pytorch.org/uploads/default/b15e1b6abf1d2610a185411235d5368c027fd6ca" title="Screenshot 2025-06-30 at 11.10.43 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b15e1b6abf1d2610a185411235d5368c027fd6ca_2_242x250.png" alt="Screenshot 2025-06-30 at 11.10.43 PM" data-base62-sha1="pj48q01BGFkYvkO2U1qsL12zI1Q" width="242" height="250" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b15e1b6abf1d2610a185411235d5368c027fd6ca_2_242x250.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b15e1b6abf1d2610a185411235d5368c027fd6ca_2_363x375.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/1/b15e1b6abf1d2610a185411235d5368c027fd6ca_2_484x500.png 2x" data-dominant-color="C8B8A5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot 2025-06-30 at 11.10.43 PM</span><span class="informations">728×752 36.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Thanks in advance for your insights.<br>
cc <a class="mention" href="/u/zdevito">@zdevito</a> , <a class="mention" href="/u/aaronenyeshi">@aaronenyeshi</a></p>

username: Programer_Salar
cooked: <p>i did not understand this tools: <a href="https://docs.pytorch.org/memory_viz" rel="noopener nofollow ugc">https://docs.pytorch.org/memory_viz</a></p>