When does the inductor code run?

username: hohohihi
cooked: <p>Hello, I am new to torch.compile and want to study the inductor.<br>
I wonder when does the inductor’s optimization code run?</p>
<p>for example, there are main.py code here.</p>
<pre><code class="lang-auto">def f(x):
    return torch.sin(x)**2 + torch.cos(x)**2

# first spot of main.py
compiled_f = torch.compile(f, backend='inductor', options={'trace.enabled':True, 'trace.graph_diagram':True})

# second spot of main.py
output_from_compiled_f = compiled_f(some_input)
</code></pre>
<p>Thanks to the torch.compile, my sin^2+cos^2 function’s nodes are merged like,<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/6/e66595015cf635beb73d84e24c90cfe05324a9e1.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/e66595015cf635beb73d84e24c90cfe05324a9e1" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/6/e66595015cf635beb73d84e24c90cfe05324a9e1_2_651x500.jpeg" alt="image" data-base62-sha1="wSbsjJnfWPWKvenf4GJGLvY2g0h" width="651" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/6/e66595015cf635beb73d84e24c90cfe05324a9e1_2_651x500.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/e/6/e66595015cf635beb73d84e24c90cfe05324a9e1.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/6/e66595015cf635beb73d84e24c90cfe05324a9e1.jpeg 2x" data-dominant-color="F6EBEA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">912×700 110 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I wonder when does this node-merge(inductor’s optimization logic) happen?</p>
<p>In this situation, I insert some print() in source code of dynamo and inductor for debug purposes.<br>
for example 1,</p>
<pre><code class="lang-auto">@ torch/_inductor/compile_fx.py
def compile_fx(
    model_: torch.fx.GraphModule,
    example_inputs_: List[torch.Tensor],
    inner_compile: Callable[..., Any] = compile_fx_inner,
    config_patches: Optional[Dict[str, Any]] = None,
    decompositions: Optional[Dict[OpOverload, Callable[..., Any]]] = None,
):
    """Main entrypoint to a compile given FX graph"""
    print("_____________________________ compile_fx.py ")
</code></pre>
<p>for example 2,</p>
<pre><code class="lang-auto">@torch/_inductor/lowering.py 
def make_foreach_pointwise(pw_fn, allow_alpha=False):
    print("_____________________________ lowering.py ")
</code></pre>
<p>In this situation,</p>
<ol>
<li>example 2’s debug print happen when first spot of main.py.</li>
<li>example 1’s debug print happen when second spot of main.py.</li>
</ol>
<p>I am confused that why example 1’s debug print happen when second spot of main.py.<br>
Because i think all the inductor’s optimization could happen when first spot of main.py. (some sort of compile-time of DNN)<br>
So, example 1’s debug print should have happen when the first spot of main.py (i believe this way, but this is not the true).</p>
<p>The location of “example 1’s debug print” is compile_fx which is the ’ “”“Main entrypoint to a compile given FX graph”“” '. This means here is the entrance of inductor logic.</p>
<p>But why inductor (backend compiler) logic is start at the second spot of main.py ? (some sort of run-time of DNN)</p>
<p>My question is summarized as follows.<br>
<strong>Q1. When does the inductor’s optimization logic(i.e., node merge for less memory access etc) activated ?</strong><br>
at the first spot of main.py or at the second spot of main.py?</p>
<p><strong>Q2. What is the sequence of the inductor’s execution?</strong><br>
I traced the source code of torch. And i got following execution sequence information.</p>
<p>(1) torch/_dynamo/eval_frame.py : def optimize → <br>
(2) torch/_dynamo/eval_frame.py : def _optimize_catch_errors → <br>
(3) torch/_dynamo/eval_frame.py : class OptimizeContext(_TorchDynamoContext): → <br>
(4) torch/_dynamo/eval_frame.py : class _TorchDynamoContext: 's def <strong>call</strong> → ???</p>
<p>but i am lost in here. where should i start again ? (for example, torch/<strong>init</strong>’s def compile)<br>
where can i find the caller of inductor optimization code?</p>