How to extarct Memory profiling?

username: JUNSUNGKIM99
cooked: <p>Hello I’m using Resnet18, Resnet50 as an AOT bundle now. When the test image is inputted, the inference time and the confidence can be confirmed to be output. Furthermore, I want to check memory usage or memory transfer. What should I do to check? Do I have to implement it directly on the code?</p>

username: jfix
cooked: <p>You should be able to use these structs that are autogenerated and should be included in the main function you write in in the C++ that drives inference:</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/glow/blob/b0de69a7639567bfec9081ab6a8bcb792638cba9/lib/LLVMIRCodeGen/BundleSaver.cpp#L106-L135">
  <header class="source">

      <a href="https://github.com/pytorch/glow/blob/b0de69a7639567bfec9081ab6a8bcb792638cba9/lib/LLVMIRCodeGen/BundleSaver.cpp#L106-L135" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/glow/blob/b0de69a7639567bfec9081ab6a8bcb792638cba9/lib/LLVMIRCodeGen/BundleSaver.cpp#L106-L135" target="_blank" rel="noopener nofollow ugc">pytorch/glow/blob/b0de69a7639567bfec9081ab6a8bcb792638cba9/lib/LLVMIRCodeGen/BundleSaver.cpp#L106-L135</a></h4>



  <pre class="onebox">    <code class="lang-cpp">
      <ol class="start lines" start="106" style="counter-reset: li-counter 105 ;">
          <li>/// Header file common definitions for dynamic API.</li>
          <li>static const char *dynamicApiCommonDefines = R"RAW(</li>
          <li>// Type describing a symbol table entry of a generated bundle.</li>
          <li>struct SymbolTableEntry {</li>
          <li>  // Name of a variable.</li>
          <li>  const char *name;</li>
          <li>  // Offset of the variable inside the memory area.</li>
          <li>  uint64_t offset;</li>
          <li>  // The number of elements inside this variable.</li>
          <li>  uint64_t size;</li>
          <li>  // Variable kind: 1 if it is a mutable variable, 0 otherwise.</li>
          <li>  char kind;</li>
          <li>};</li>
          <li>
          </li>
<li>// Type describing the config of a generated bundle.</li>
          <li>struct BundleConfig {</li>
          <li>  // Size of the constant weight variables memory area.</li>
          <li>  uint64_t constantWeightVarsMemSize;</li>
          <li>  // Size of the mutable weight variables memory area.</li>
          <li>  uint64_t mutableWeightVarsMemSize;</li>
      </ol>
    </code>
  </pre>

  This file has been truncated. <a href="https://github.com/pytorch/glow/blob/b0de69a7639567bfec9081ab6a8bcb792638cba9/lib/LLVMIRCodeGen/BundleSaver.cpp#L106-L135" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>E.g. using the BundleConfig you can understand the total memory usage of the network itself, such as the size of constant weights, etc. Aside from that if you need more info about data transfer then yes you would need to take a look at the actual data you’re passing into the network in the C++ file that drives inference.</p>