Triton installation and requires grad mismatch

Julian_Buchel: <p>Hi,</p>
<p>on the torch.compile page, it says that <code>torchtriton</code> might need to be installed via <code>pip install torchtriton --extra-index-url "https://download.pytorch.org/whl/nightly/cu117"</code></p>
<p>I tried that, but when I wanted to compile my model, torch was complaining that I don’t have a backend installed and that I should check out the triton page of openAI, which I did. This disappeared after I installed the nightly version from openAIs Github.</p>
<p>Why is that? I thought the reason for torchtriton is that one is not dependent on the openAI version. Maybe I misunderstood.</p>
<p>After this, compilation seemed to work, but the model wasn’t faster. I think there are still some issues.<br>
I am getting one of these</p>
<pre><code class="lang-auto">[2023-02-23 11:14:21,084] torch._dynamo.convert_frame: [WARNING] torch._dynamo hit config.cache_size_limit (64)
   function: '&lt;graph break in forward&gt;' (/X/simulator/tiles/torch_analog.py:832)
   reasons:  tensor 'x_input_scaled' requires_grad mismatch. expected requires_grad=0
</code></pre>
<p>But when I inspect the variable during training, requires grad is always set to zero. In general, what does this warning mean?</p>
<p>I am also getting a lot of these</p>
<pre><code class="lang-auto">torch/_functorch/aot_autograd.py:1251: UserWarning: Your compiler for AOTAutograd is returning a a function that doesn't take boxed arguments. Please wrap it with functorch.compile.make_boxed_func or handle the boxed arguments yourself. See https://github.com/pytorch/pytorch/pull/83137#issuecomment-1211320670 for rationale.
</code></pre>
<p>I looked at the issue and it seems that one must pass the args as lists so that the memory can be freed. Is that something that I have to worry about in the user-code?</p>
<p>Thanks!</p>

ptrblck: <p>The <code>pytorch-triton</code> binary will be installed as a dependency when the nightly binaries are installed. Also note that <code>torchtriton</code> is the name of the old binaries, which you shouldn’t install manually.<br>
Unfortunately, it seems the nightly binaries are broken right now and you can track the issue <a href="https://github.com/pytorch/builder/issues/1318">here</a>.</p>

Julian_Buchel: <p>Are the issues I am seeing related to this, or is that just my code that isn’t good enough for compile to work?</p>

ptrblck: <p>The warnings you are seeing might be related to the usage of the old <code>torchtriton</code> binary and I would thus recommend installing the latest nightly and allow PyTorch to pull the needed <code>pytorch-triton</code> dependency.</p>

bdhirsh: <p>Also, those warnings you’re seeing from AOT Autograd should be fixed by <a href="https://github.com/pytorch/pytorch/pull/95521" class="inline-onebox" rel="noopener nofollow ugc">fix spurious aot autograd warning by bdhirsh · Pull Request #95521 · pytorch/pytorch · GitHub</a> <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=12" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:" loading="lazy" width="20" height="20"></p>

Julian_Buchel: <p>Ok, thanks. I will try it out soon.</p>