Help understanding cpp codegen for inductor

vivekvpandya: <pre><code>    I am studying inductor CPP coregen and I have following questions:

    1) for gen_loops() call the parameter loops can have following kind of list input
       [LoopLevel,Looplevel,...] only for `OuterLoopFusedKernel` right?
        otherwise it will always have
        [LoopLevel] and this loop level itself can have loop in its `inner` member i.e nested loops

    2) At Scheduler level say I have node2 depending on node1 (be it any kind of fusion)
       How at cpp loop level this order is preserved?
    
   3) Also at LoopNest and LoopLevel is it possible to track read and write parameters of kernel?
</code></pre>

EikanWang: <aside class="quote no-group" data-username="vivekvpandya" data-post="1" data-topic="214785">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vivekvpandya/48/72549_2.png" class="avatar"> vivekvpandya:</div>
<blockquote>
<pre><code class="lang-auto"> 1) for gen_loops() call the parameter loops can have following kind of list input
       [LoopLevel,Looplevel,...] only for `OuterLoopFusedKernel` right?
        otherwise it will always have
        [LoopLevel] and this loop level itself can have loop in its `inner` member i.e nested loops
</code></pre>
</blockquote>
</aside>
<p><code>LoopLevel</code> is a general abstraction for <code>for</code> loop and the <code>LoopLevel</code> array is used to generally represent one kernel contains a single or multiple <code>LoopLevel</code> regardless of <code>OuterLoopFusedKernel</code>. Meanwhile, <code>OuterLoopFusedKernel</code> is used to stack different kernels without fusion, that’s why its data filed is a list of <code>LoopNest</code>.</p>
<p>let me give an example here.</p>
<p>Suppose there are two kernels, and the kernel names are <code>kernel0</code> and <code>kernel1</code> accordingly.</p>
<pre data-code-wrap="python"><code class="lang-python"># Kernel 0
for x0 in range(0, 100):
  for y0 in range(0, 1000):
    for z0 in range(0, 1024):
      # Do something for kernel0

# Kernel 1
for x1 in range(0, 100):
  for y1 in range(0, 1000):
    for z1 in range(0, 2048):
      # Do something for kernel0
</code></pre>
<p>Before fusion, the <code>kernel0</code>and <code>kernel1</code> will be represented as a <code>LoopNest</code> which contains 3 <code>LoopLevel</code> and a <code>CppKernel</code>.</p>
<ul>
<li>Kernel0:<pre data-code-wrap="json"><code class="lang-json">LoopNest {
  loops: [LoopLevel_0_for_kernel0, LoopLevel_1_for_kernel0, LoopLevel_2_for_kernel0],
  kernel: CppKernel_for_kernel0
}
</code></pre>
</li>
<li>Kernel1:<pre data-code-wrap="json"><code class="lang-json">LoopNest {
loops: [LoopLevel_0, LoopLevel_1, LoopLevel_2],
kernel: CppKernel_for_kernel1
}
</code></pre>
</li>
</ul>
<p>During the fusion process of the <code>Cpp/OMP</code> backend, the most outer two loops of <code>kernel0</code> and <code>kernel1</code> will be fused and the most inner loop of <code>kernel0</code> and <code>kernel1</code> cannot be fused and will be represented as <code>OuterLoopFusedKernel</code>. The <code>Fused_Kernel0_Kernel1</code> can be viewed as follows.</p>
<pre data-code-wrap="json"><code class="lang-json">LoopNest0: {
  loops: [LoopLevel_2_for_kernel0],
  kernel: CppKernel_for_kernel0
}
LoopNest1: {
  loops: [LoopLevel_2_for_kernel1],
  kernel: CppKernel_for_kernel1
}
OuterLoopFusedKernel: {
  inner: [LoopNest0, LoopNest1]
}
LoopNest: {
  loops: [LoopLevel_0, LoopLevel_1],
  kernel: OuterLoopFusedKernel
}
</code></pre>
<p>So, it means <code>[LoopLevel,Looplevel,...]</code> does not only serve for <code>OuterLoopFusedKernel</code></p>

EikanWang: <aside class="quote no-group" data-username="vivekvpandya" data-post="1" data-topic="214785">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vivekvpandya/48/72549_2.png" class="avatar"> vivekvpandya:</div>
<blockquote>
<pre><code class="lang-auto">At Scheduler level say I have node2 depending on node1 (be it any kind of fusion)
       How at cpp loop level this order is preserved?
</code></pre>
</blockquote>
</aside>
<p>The order is encoded as the index of the <code>LoopLevel</code> array of <code>LoopNest</code>. Say, the most outer loop is <code>LoopNest.loops[0]</code>, while the most inner loop is <code>LoopNest.loops[-1]</code>.</p>

EikanWang: <aside class="quote no-group" data-username="vivekvpandya" data-post="1" data-topic="214785">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vivekvpandya/48/72549_2.png" class="avatar"> vivekvpandya:</div>
<blockquote>
<p><code>3) Also at LoopNest and LoopLevel is it possible to track read and write parameters of kernel?</code></p>
</blockquote>
</aside>
<p>By design, it does not track the read and write parameters because it solely serves for <code>Loop</code> representation. May I know the motivation behind the question?</p>

vivekvpandya: <aside class="quote no-group" data-username="EikanWang" data-post="4" data-topic="214785">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/eikanwang/48/66555_2.png" class="avatar"> EikanWang:</div>
<blockquote>
<p>By design, it does not track the read and write parameters because it solely serves for <code>Loop</code> representation. May I know the motivation behind the question?</p>
</blockquote>
</aside>
<p>so all current (cpu like) HW arch using cpp backend uses OpenMP runtime where mostly once<br>
<code>omp parallel</code> region ends the runtime will automatically add barrier or sync.</p>
<p>However, I am trying for an arch where we don’t have OpenMP support, so I have to decide which are optimal (or just better then syncing after all loop nest) points for sync.</p>

EikanWang: <aside class="quote no-group" data-username="vivekvpandya" data-post="5" data-topic="214785">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vivekvpandya/48/72549_2.png" class="avatar"> vivekvpandya:</div>
<blockquote>
<p>However, I am trying for an arch where we don’t have OpenMP support, so I have to decide which are optimal (or just better then syncing after all loop nest) points for sync.</p>
</blockquote>
</aside>
<p>I think different runtimes should only impact how to generate code for <code>Loop</code> statement as the key concepts should be common among different runtimes like <code>parallelism</code>. The <code>LoopNest</code> and <code>LoopLevel</code> are general to represent the loop.</p>