Profiler memory record

saar-eliad: <p>Hi, I want to further understand how PyTorch profiles CPU memory usages.<br>
e.g, see:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/0/300afdfbfdc91a5c62cd3ee6c6e3e23c7a1997c2.png" data-download-href="https://discuss.pytorch.org/uploads/default/300afdfbfdc91a5c62cd3ee6c6e3e23c7a1997c2" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/0/300afdfbfdc91a5c62cd3ee6c6e3e23c7a1997c2.png" alt="image" data-base62-sha1="6R0t5ewxEYnuqYyPHlmwc0CwDEm" width="631" height="500" data-dominant-color="F6F6F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">931×737 28.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
How should I interpret the total memory consumption of this call (which is just the simplest invocation of <code>torch.nn.Linear</code>)<br>
The recorded function consumes <code>-</code> memory. (?).</p>
<p>In the recorded events, seems it is somehow <em>not</em> the first event to get recorded. (?)</p>
<hr>
<pre><code>                 Name    Self CPU %      Self CPU   CPU total %     CPU total  CPU time avg       CPU Mem  Self CPU Mem    # of Calls                           Input Shapes  
</code></pre>
<hr>
<pre><code>          aten::zeros        23.88%     622.000us        25.64%     668.000us     668.000us           4 b           0 b             1                   [[], [], [], [], []]  
          aten::empty         1.19%      31.000us         1.19%      31.000us      31.000us           4 b           4 b             1               [[], [], [], [], [], []]  
          aten::zero_         0.58%      15.000us         0.58%      15.000us      15.000us           0 b           0 b             1                                  [[1]]  
Total function record        23.49%     612.000us        74.36%       1.937ms       1.937ms          -4 b         -60 b             1                                     []  
          aten::empty         1.34%      35.000us         1.34%      35.000us      35.000us          16 b          16 b             1               [[], [], [], [], [], []]  
         aten::linear        12.55%     327.000us        49.52%       1.290ms       1.290ms          40 b           0 b             1              [[1, 10], [10, 10], [10]]  
              aten::t         9.40%     245.000us        14.32%     373.000us     373.000us           0 b           0 b             1                             [[10, 10]]  
      aten::transpose         4.53%     118.000us         4.91%     128.000us     128.000us           0 b           0 b             1                     [[10, 10], [], []]  
     aten::as_strided         0.38%      10.000us         0.38%      10.000us      10.000us           0 b           0 b             1                 [[10, 10], [], [], []]  
          aten::addmm        17.20%     448.000us        22.65%     590.000us     590.000us          40 b          40 b             1      [[10], [1, 10], [10, 10], [], []]  
          aten::empty         0.19%       5.000us         0.19%       5.000us       5.000us           0 b           0 b             1               [[], [], [], [], [], []]  
         aten::expand         4.22%     110.000us         4.45%     116.000us     116.000us           0 b           0 b             1                         [[10], [], []]  
     aten::as_strided         0.23%       6.000us         0.23%       6.000us       6.000us           0 b           0 b             1                     [[10], [], [], []]  
          aten::copy_         0.81%      21.000us         0.81%      21.000us      21.000us           0 b           0 b             1                 [[1, 10], [1, 10], []]  
</code></pre>
<hr>
<p>Self CPU time total: 2.605ms</p>