Custom backward() breakpoint doesn't get hit

Sia_Rezaei: <p>I’m new to Pytorch. I’m running this example: <a href="http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-defining-new-autograd-functions" rel="nofollow noopener">http://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-defining-new-autograd-functions</a></p>
<p>My breakpoints in the forward() function are hit, but not the ones in the backward function. Why?<br>
I’m using PyCharm CE.</p>

purrfegt: <p>had same question. I think the backward() of a custom Function is called in a separate thread. Try adding “print(current_thread())” (don’t forget the import “from threading import current_thread”) in your backward() and your forward() in your custom module (and maybe some other place in the code). Different things are printed. Still don’t know how to have breakpoints in backward() tho.</p>

SimonW: <p>Could you post your code so that me or other people can help you?</p>

purrfegt: <p>the example linked by the original question should do it (edit: meaning not hitting the breakpoint in backward())</p>

Sia_Rezaei: <p>Just use the code in the example.</p>

SimonW: <p>I am not able to reproduce it with pdb. Maybe that is different with PyCharm’s breakpoint. The backward() functions are called from C side so that might be breaking these things. You could use pdb instead, which is also more commonly used…</p>

Andrei_Pokrovsky: <p>One related bit of information is PyCharm has a breakpoint Suspend Policy, which can be set either to thread or application (can be configured by right clicking on breakpoint red knob), but in this case it still doesn’t fix the issue with callback from C API wrapper.</p>
<p>However when combining that with this the solution from this stackoverflow answer, it seems to now work for me:</p>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/34299082/not-working-python-breakpoints-in-c-thread-in-pycharm-or-eclipsepydev" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/538895/alek-kowalczyk" target="_blank" rel="nofollow noopener">
    <img alt="Alek Kowalczyk" src="https://www.gravatar.com/avatar/725efaa3c1ec00184922f289467d9240?s=128&amp;d=identicon&amp;r=PG" class="thumbnail onebox-avatar" width="128" height="128">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/34299082/not-working-python-breakpoints-in-c-thread-in-pycharm-or-eclipsepydev" target="_blank" rel="nofollow noopener">Not working python breakpoints in C thread in pycharm or eclipse+pydev</a>
</h4>

<div class="tags">
  <strong>eclipse, multithreading, debugging, pycharm</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/538895/alek-kowalczyk" target="_blank" rel="nofollow noopener">
    Alek Kowalczyk
  </a>
  on <a href="https://stackoverflow.com/questions/34299082/not-working-python-breakpoints-in-c-thread-in-pycharm-or-eclipsepydev" target="_blank" rel="nofollow noopener">08:47PM - 15 Dec 15</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>So, first thing in your backward function, add the following</p>
<p><code>import pydevd</code><br>
<code>pydevd.settrace(suspend=False, trace_only_current_thread=True)</code></p>

Oliver_Liu: <p>Worked for me. Thank you very much!</p>

amirhf: <p>I’m using vscode and have the same problem. I put a breakpoint() (Python 3.7+) at the beginning of my backward function and then breakpoints work on debug mode. However when running, it doesn’t stop at breakpoint().</p>

rouniuyizu: <p><a class="mention" href="/u/andrei_pokrovsky">@Andrei_Pokrovsky</a> thank YOU for saving me a whole day!</p>
<p>For whoever have the same issue, the solution works on my env: Pycharm 2019.3/Ubuntu 18.04/conda Pytorch 1.3.1/Python 3.6.8 (need to pip install pydevd)</p>

robclouth: <aside class="quote no-group" data-username="Andrei_Pokrovsky" data-post="7" data-topic="6473">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/andrei_pokrovsky/48/4581_2.png" class="avatar"> Andrei_Pokrovsky:</div>
<blockquote>
<p>pydevd</p>
</blockquote>
</aside>
<p>Hey! I’m using vscode on Windows and I can’t get it to stop in the backwards method.<br>
I’ve tried each of:<br>
import pydevd; pydevd.settrace(suspend=True, trace_only_current_thread=True)<br>
import debugpy; debugpy.debug_this_thread()<br>
breakpoint()<br>
import pdb; pdb.set_trace()</p>
<p>with no luck. I’ve tried using pdb, the vscode debugger, PyCharm. None of them stop. Any ideas?</p>

Aris_L: <p>I’m trying to debug into backward hook, breakpoint not working either, vscode on windows, pytorch 1.10. Is backward hook somehow different? Forward hook works fine.</p>

wwpokenet: <p>Same issue. A temporary solution is to set ‘justMyCode’ = false in vscode debug config, and explicitly raise an error in the backward function. Then the program will stop at this exception.</p>

raining_day513: <p>So I’m the first one who really proves that <a class="mention" href="/u/purrfegt">@purrfegt</a> 's comment is correct:</p>
<p>the two threads in {for,back}ward are different for sure:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/5/f5f7fc5a5dd4b74aa782cb1cd4b28c8f92bc3d87.png" data-download-href="https://discuss.pytorch.org/uploads/default/f5f7fc5a5dd4b74aa782cb1cd4b28c8f92bc3d87" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/5/f5f7fc5a5dd4b74aa782cb1cd4b28c8f92bc3d87.png" alt="image" data-base62-sha1="z5WhOkCrACTi4Hox5iRJZ3C0BWD" width="690" height="355" data-dominant-color="363140"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1352×697 123 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Just leave this evidence here in case some future readers might be lazy to prove-read.</p>

raining_day513: <p>You should put this line:</p>
<pre><code class="lang-python">pydevd.settrace(suspend=False, trace_only_current_thread=True)
</code></pre>
<p><strong>inside</strong> your <code>backward</code> function instead of calling it right after your import! No need to set <code>suspend=True</code>. (While you probably don’t care about it anymore, I just prove-read this thread to help future readers.)</p>