Repository: spyder-ide/spyder

Issue:
%run -d [filename] doesn't stop on breakpoints
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [✅] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [✅] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [✅] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [✅] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [✅] Tried basic troubleshooting (if a bug/error)
    * [✅] Restarted Spyder
    * [✅] Reset preferences with ``spyder --reset``
    * [✅] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [✅] Tried the other applicable steps from the Troubleshooting Guide
* [✅] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below

## Problem Description
After running a file with `%run -d [filename]`, Spyder skips breakpoints. It only seems to work if you type the much more verbose `debugfile([filename], post_mortem=True, current_namespace=True)`, which is annoying if you want to pass command line args.


### What steps reproduce the problem?

1. Set breakpoint
2. Type $run -d [filename]
3. Breakpoint is skipped

### What is the expected output? What do you see instead?

It just skips the breakpoints entirely; I would like it if it followed the breakpoints.

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: 5.0.0
* Python version: 3.9.4
* Qt version: 5.9.7
* PyQt version: 5.9.2
* Operating System name/version: Darwin 17.7.0

### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```

# Mandatory:
applaunchservices >=0.1.7     :  0.2.1 (OK)
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  1.6.0 (OK)
cookiecutter >=1.6.0          :  1.7.2 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0               :  7.22.0 (OK)
jedi =0.17.2                  :  0.17.2 (OK)
jsonschema >=3.2.0            :  3.2.0 (OK)
keyring >=17.0.0              :  22.3.0 (OK)
nbconvert >=4.0               :  6.0.7 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
parso =0.7.0                  :  0.7.0 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.8.0 (OK)
pygments >=2.0                :  2.8.1 (OK)
pylint >=1.0                  :  2.7.4 (OK)
pyls >=0.36.2;<1.0.0          :  0.36.2 (OK)
pyls_black >=0.4.6            :  0.4.6 (OK)
pyls_spyder >=0.3.2           :  0.3.2 (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.1.10 (OK)
qtawesome >=0.5.7             :  1.0.2 (OK)
qtconsole >=5.0.3             :  5.0.3 (OK)
qtpy >=1.5.0                  :  1.9.0 (OK)
rtree >=0.8.3                 :  0.9.7 (OK)
setuptools >=39.0.0           :  52.0.0.post20210125 (OK)
sphinx >=0.6.6                :  3.5.3 (OK)
spyder_kernels >=2.0.1;<2.1.0 :  2.0.1 (OK)
textdistance >=4.2.0          :  4.2.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3;<2.0.0      :  1.0.2 (OK)
zmq >=17                      :  20.0.0 (OK)

# Optional:
cython >=0.21                 :  0.29.23 (OK)
matplotlib >=2.0.0            :  3.3.4 (OK)
numpy >=1.7                   :  1.20.1 (OK)
pandas >=1.1.1                :  1.2.4 (OK)
scipy >=0.17.0                :  1.6.2 (OK)
sympy >=0.7.3                 :  1.8 (OK)
```



Discussion:
I'll make my question even clearer - I just want to put things into some mode such that I can type function names into the console, and try different things, and have breakpoints activate when they are reached. Is there any way to do this? Can I somehow put the entire console into debug mode? And have it match Spyder's breakpoints?
I really doubt there's something we can do about it because `debugfile` is required to use our own debugger instead of regular `ipdb`.

@impact27, what do you think?
It would maybe be useful if there were just a different %magic that calls
`debugfile`, but uses the `%run -d filename arg1 arg2 arg3` syntax.
Something like `%sdbg` or something, for "spyder-debug" (and similarly
%srun). Is that something that's possible to do?

Mike


On Thu, Jun 3, 2021 at 8:34 PM Carlos Cordoba ***@***.***>
wrote:

> I really doubt there's something we can do about it because debugfile is
> required to use our own debugger instead of regular ipdb.
>
> @impact27 <https://github.com/impact27>, what do you think?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/spyder-ide/spyder/issues/15780#issuecomment-854272058>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHZFMW7AR2PCIPGM6IZ5JLTRANRXANCNFSM46BWDPKQ>
> .
>

1 - Why do you want to use `%run -d` instead of the `debugfile`? It has the same functionality. For args use the `args` keyword. Also you can set all the options in the settings.
2 - if you want to debug a function from the debugger you need to have a recursive debugger. so from pdb instead of calling `f()`, use the debug command `debug f()`

Also `%run -d` does stop at the breakpoints. Do you have a reproducible example where it doesn't?
Breakpoints don't work with %run -d on my machine and the behavior is perfectly reproducible. If I am interpreting @ccordoba12 's prior comment correctly, that seems to be the standard behavior and has to do with which debugger is being used. If it's working on *your* machine with %run -d, I'd like to know how...

Sure:
 1- Create a file named "test.py" with a single `print(0)`, where you put a breakpoint
 2- in the console, run `%run -d test.py`
 3- Check that it did stop at the breakpoint as expected. 

@ccordoba12  could you check that this works as expected on your machine?

This only works because `%run -d [filename]` automatically sets a breakpoint on the first line regardless of what it is, or if you've set a breakpoint there or not. This doesn't work

```
print(0)
print(1) # breakpoint here
```

No matter what you've set, `%run -d [filename]` will automatically breakpoint before the `print(0)` just because it's line 1, and skip the breakpoint at `print(1)`.
> @ccordoba12 could you check that this works as expected on your machine?

It seems @battaglia01 is right:

![imagen](https://user-images.githubusercontent.com/365293/121224589-5987ac00-c84e-11eb-8731-904bfce65d7e.png)
For this to work we probably need to patch `%run -d` to get breakpoints from Spyder and set them in Pdb. However, that's what we already do in `debugfile`, so it's not something I'd like to put resources into.

Of course, if you'd like to give us a hand with that @battaglia01, you're more than welcome.
Sure, I can try - is there some way to programmatically get Spyder's list of breakpoints from the console?
@impact27, could you point him in the right direction?
I tried to checkout spyder 5.0.0 and spyder-kernels 2.0.1 but `%run -d` still stops on the breakpoints, so I don't know what is going on. Also, I can't think of anything special that `debugfile` does to the debugger compared to `%run -d`. It mostly exists to make sure we call `runfile` from the debugger to use all the spyder options, but otherwise doesn't really do anything special to the debugger.

You can access the debugger breakpoints from the debugger with the `b` command.

If you want more advanced access:
```
shell = get_ipython()
pdb_obj = shell.kernel._pdb_obj
pdb_obj.breaks
```
This only works if the debugger is running, otherwise you will get `None` for `_pdb_obj`
Ok, but how do I get *Spyder*'s breakpoints from the command line (to put
into pdb to begin with), not the ones that pdb already has put into it?

On Wed, Jun 9, 2021, 2:51 AM Quentin Peter ***@***.***> wrote:

> I tried to checkout spyder 5.0.0 and spyder-kernels 2.0.1 but %run -d
> still stops on the breakpoints, so I don't know what is going on. Also, I
> can't think of anything special that debugfile does to the debugger
> compared to %run -d. It mostly exists to make sure we call runfile from
> the debugger to use all the spyder options, but otherwise doesn't really do
> anything special to the debugger.
>
> You can access the debugger breakpoints from the debugger with the b
> command.
>
> If you want more advanced access:
>
> shell = get_ipython()
> pdb_obj = shell.kernel._pdb_obj
> pdb_obj.breaks
>
> This only works if the debugger is running, otherwise you will get None
> for _pdb_obj
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/spyder-ide/spyder/issues/15780#issuecomment-857434146>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHZFMQATF3NUPPCTG74MJTTR4FQHANCNFSM46BWDPKQ>
> .
>

Ah, sorry, the "more advanced access" method gave me what I was looking for
- thank you
Mike


On Wed, Jun 9, 2021 at 3:06 AM Mike Battaglia ***@***.***> wrote:

> Ok, but how do I get *Spyder*'s breakpoints from the command line (to put
> into pdb to begin with), not the ones that pdb already has put into it?
>
> On Wed, Jun 9, 2021, 2:51 AM Quentin Peter ***@***.***>
> wrote:
>
>> I tried to checkout spyder 5.0.0 and spyder-kernels 2.0.1 but %run -d
>> still stops on the breakpoints, so I don't know what is going on. Also, I
>> can't think of anything special that debugfile does to the debugger
>> compared to %run -d. It mostly exists to make sure we call runfile from
>> the debugger to use all the spyder options, but otherwise doesn't really do
>> anything special to the debugger.
>>
>> You can access the debugger breakpoints from the debugger with the b
>> command.
>>
>> If you want more advanced access:
>>
>> shell = get_ipython()
>> pdb_obj = shell.kernel._pdb_obj
>> pdb_obj.breaks
>>
>> This only works if the debugger is running, otherwise you will get None
>> for _pdb_obj
>>
>> —
>> You are receiving this because you were mentioned.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/spyder-ide/spyder/issues/15780#issuecomment-857434146>,
>> or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/AAHZFMQATF3NUPPCTG74MJTTR4FQHANCNFSM46BWDPKQ>
>> .
>>
>

Wait `b` and `pdb_obj.breaks` *should* return the exact same thing. The command line doesn't know anything about spyder. If you want to have access to spyder console, go to View > Panes > Internal console
From there you can get the breakpoints by:
```
from spyder.config.manager import CONF
CONF.get('run', 'breakpoints', {})
```
It looks like once the debugger has been run at least one time, `shell.kernel._pdb_obj` is no longer `None`.

It looks like that code you gave also runs from the regular console (e.g. internal console not required) which is good, because I will probably just need a general API for spyder stuff from the regular console... Is there some documentation for this?
OK to @ccordoba12 and @impact27 , I think I figured this out...

The **very first** time you call `%run -d _____`, it seems to set all the breakpoints correctly, as @impact27 said. But, once you call it **twice**, it doesn't work anymore.

This seems to be a larger problem with `%debug` in general, and I just wrote another issue in #15830. But, now that I'm doing it, I think it's all part of the same thing, which is that Spyder only sets the ipdb breakpoints correctly the first time you type either `debug` or `%run -d ___` or etc from the console, and then doesn't set them correctly afterward.

For instance, this is the file:

```
#!/usr/bin/env python3
# breakpointtest2.py

print("line 1!")
print("line 2 - breakpoint here!")
```

Now let's call `%run -d`:
```
>>> %run -d breakpointtest2.py
*** Blank or comment
*** Blank or comment
*** Blank or comment
*** Blank or comment
NOTE: Enter 'c' at the ipdb>  prompt to continue execution.
> /Volumes/Shared/Projects/Python/breakpointtest2.py(4)<module>()
      1 #!/usr/bin/env python3
      2 # breakpointtest2.py
      3 
----> 4 print("line 1!")
      5 print("line 2 - breakpoint here!")
```
We're pausing at the first line. Before we continue, we will look at the breakpoints and see that we get

```
IPdb [1]: b
Num Type         Disp Enb   Where
1   breakpoint   keep yes   at /Volumes/Shared/Projects/Python/breakpointtest2.py:5
```

They are set correctly. Then if we continue:

```
IPdb [2]: c
line 1!
> /Volumes/Shared/Projects/Python/breakpointtest2.py(5)<module>()
      1 #!/usr/bin/env python3
      2 # breakpointtest2.py
      3 
      4 print("line 1!")
1---> 5 print("line 2 - breakpoint here!")
```
So it **does** successfully get the breakpoint the first time. Now if we run it again:


```
>>> %run -d breakpointtest2.py
NOTE: Enter 'c' at the ipdb>  prompt to continue execution.
> /Volumes/Shared/Projects/Python/breakpointtest2.py(4)<module>()
      1 #!/usr/bin/env python3
      2 # breakpointtest2.py
      3 
----> 4 print("line 1!")
      5 print("line 2 - breakpoint here!")
# ^^ Pausing on first line, So far so good

IPdb [1]: b
# ^^ breakpoints empty?!

IPdb [2]: c
line 1!
line 2 - breakpoint here!
# Runs everything without breaking!
```

So this and the problem in #15830 seem to be equivalent; if you do any debugger thing from the console it seems to only set breakpoints correctly the first time and then doesn't afterward. (Perhaps these two issues should be merged now that this is clear...)
I can reproduce that. I will take a look when I have time. 

One important thing is that Spyder and the kernel might not run on the same machine, so `CONF.get('run', 'breakpoints', {})` has no garantee to return the right stuff hwen run from the ipython kernel. All the code for spyder pdb can be found here: https://github.com/spyder-ide/spyder-kernels/blob/master/spyder_kernels/customize/spyderpdb.py

The breakpoints are set in this file by:
```
pdb_settings = frontend_request().get_pdb_settings()
self.set_spyder_breakpoints(pdb_settings['breakpoints'])
```

Is that what sets Spyder's internal breakpoints? Where is the code in which
Spyder communicates those breakpoints to pdb/ipdb?

On Thu, Jun 10, 2021, 4:17 AM Quentin Peter ***@***.***>
wrote:

> I can reproduce that. I will take a look when I have time.
>
> One important thing is that Spyder and the kernel might not run on the
> same machine, so CONF.get('run', 'breakpoints', {}) has no garantee to
> return the right stuff hwen run from the ipython kernel. All the code for
> spyder pdb can be found here:
> https://github.com/spyder-ide/spyder-kernels/blob/master/spyder_kernels/customize/spyderpdb.py
>
> The breakpoints are set in this file by:
>
> pdb_settings = frontend_request().get_pdb_settings()
> self.set_spyder_breakpoints(pdb_settings['breakpoints'])
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/spyder-ide/spyder/issues/15780#issuecomment-858415372>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHZFMSONMKXTJLGVQTPESLTSBYJDANCNFSM46BWDPKQ>
> .
>

> Is that what sets Spyder's internal breakpoints?

It's the second line of the code @impact27 posted in his last comment.

> Where is the code in which Spyder communicates those breakpoints to pdb/ipdb?

It's in the first line.
`self.starting` is not True when calling from the magic. PR https://github.com/spyder-ide/spyder-kernels/pull/300 fixes this
@impact27 thanks, that makes sense...

Just skimming the code here - what is the point of `self.starting` to begin with? Isn't the point of `preloop()` that it's only called at the very beginning of the debug session anyway (when `ipdb` is first called)? If so, it seems like it's only called when we're literally "starting" the pdb session anyway, so what is the point of the extra internal logic with `self.starting`...?
`preloop` is called each time you stop somewhere. So if I press continue and the debugger stops at a breakpoint, preloop is called again. In this case we don't want the breakpoint set by the user in the debugging session to be removed.
Hi,
Debugger is still not stoping at break point after spyder 5.0.4 and spyder_kernels
edit, upgrading IPython to 7.24.1 solve the issu

Resolving patch:
diff --git a/external-deps/spyder-kernels/spyder_kernels/customize/spyderpdb.py b/external-deps/spyder-kernels/spyder_kernels/customize/spyderpdb.py
--- a/external-deps/spyder-kernels/spyder_kernels/customize/spyderpdb.py
+++ b/external-deps/spyder-kernels/spyder_kernels/customize/spyderpdb.py
@@ -36,7 +36,7 @@
 
 class DebugWrapper(object):
     """
-    Notifies the frontend when debuggging starts/stops
+    Notifies the frontend when debugging starts/stops
     """
     def __init__(self, pdb_obj):
         self.pdb_obj = pdb_obj
@@ -712,6 +712,7 @@ def run(self, cmd, globals=None, locals=None):
 
         globals defaults to __main__.dict; locals defaults to globals.
         """
+        self.starting = True
         with DebugWrapper(self):
             super(SpyderPdb, self).run(cmd, globals, locals)
 
@@ -720,6 +721,7 @@ def runeval(self, expr, globals=None, locals=None):
 
         globals defaults to __main__.dict; locals defaults to globals.
         """
+        self.starting = True
         with DebugWrapper(self):
             super(SpyderPdb, self).runeval(expr, globals, locals)
 
@@ -728,6 +730,7 @@ def runcall(self, *args, **kwds):
 
         Return the result of the function call.
         """
+        self.starting = True
         with DebugWrapper(self):
             super(SpyderPdb, self).runcall(*args, **kwds)
 


Test patch: