Repository: ipython/ipython

Issue:
Improving recursive Pdb
I haven't started looking at it yet, but I'm really suffering a lot from how weak Pdb becomes in recursive mode (`debug stmt`) on Linux at least (readline navigation stops working, etc.) and I'm wondering how hard it would be to IPython-enhance the recursive debugger? Once my suffering increases enough I will do something about it myself but before I go digging I just wanted to check if I'm wasting my time or if there are any low hanging fruit there



Discussion:
@wesm, I'm not sure what you mean by 'recursive mode', sorry.  Could you clarify a bit precisely how you are starting pdb that gives these issues?

I'm talking about using the 'debug' command inside pdb, used for example if you wish to step into an arbitrary statement. You are given the message "ENTERING RECURSIVE DEBUGGER" when you do this. But the problem is that the recursive debugger then ceases to be "interactive". Worse, readline stops working. 

Here's a self-contained example:

```

In [1]: paste
import sys

def set_trace():
    from IPython.core.debugger import Pdb
    try:
        Pdb(color_scheme='Linux').set_trace(sys._getframe().f_back)
    except:
        from pdb import Pdb as OldPdb
        OldPdb().set_trace(sys._getframe().f_back)

def f():
    set_trace()
    a = 5
    return 6

def g():
    hello = 0
    world = 3

f()
## -- End pasted text --
> <ipython-input-1-90fd573b08ff>(13)f()
     12     set_trace()
---> 13     a = 5
     14     return 6

ipdb> debug g()
ENTERING RECURSIVE DEBUGGER
> <string>(1)<module>()
(ipdb>) s
--Call--
> <ipython-input-1-90fd573b08ff>(16)g()
-> def g():
(ipdb>) n
> <ipython-input-1-90fd573b08ff>(17)g()
-> hello = 0
(ipdb>) he  ^A^K^A
```

as you can see after typing `he` I tried to type `C-a C-k` and readline functionality is gone

On Sun, Jan 15, 2012 at 6:11 PM, Wes McKinney
reply@reply.github.com
wrote:

> I'm talking about using the 'debug' command inside pdb, used for example if you wish to step into an arbitrary statement. You are given the message "ENTERING RECURSIVE DEBUGGER" when you do this. But the problem is that the recursive debugger then ceases to be "interactive". Worse, readline stops working.

Ah, OK: I didn't even know such a mode existed!  Today is the first
time I hear of it (a bit embarrassing considering this morning I was
lecturing at Berkeley on the uses of pdb :).  Now I get it; it
shouldn't be too hard, just a matter of ensuring that the recursive
pdb instance that pdb instantiates is hooked up with our
readline-enabled pdb instead of the plain vanilla one.  In case you
want to take a stab at it, the code is in IPython.core.debugger.

Cheers,

f

Crazy, since you've been using Python about 6 years longer than I have I'm really surprised too! I've been using that functionality basically daily for a couple years now-- forensic debugging while testing / fixing pandas bugs.

But if you had ever used it I'm sure that you would have fixed the debugger by now. I will take a crack at it-- looking at IPython.core.debugger and it doesn't look so bad at all. 

OK these are the critical lines, tinkering with this:

```
    def new_do_debug(self, arg):
        sys.settrace(None)
        globals = self.curframe.f_globals
        locals = self.curframe_locals
        p = Pdb(color_scheme='Linux', completekey=self.completekey,
                stdin=self.stdin, stdout=self.stdout)
        p.prompt = "(%s) " % self.prompt.strip()
        print >>self.stdout, "ENTERING RECURSIVE DEBUGGER"
        sys.call_tracing(p.run, (arg, globals, locals))
        print >>self.stdout, "LEAVING RECURSIVE DEBUGGER"
        sys.settrace(self.trace_dispatch)
        self.lastcmd = p.lastcmd
    # copied from bdb.py
    def run(self, cmd, globals=None, locals=None):
        from bdb import BdbQuit
        import types

        if globals is None:
            import __main__
            globals = __main__.__dict__
        if locals is None:
            locals = globals
        self.reset()
        sys.settrace(self.trace_dispatch)
        if not isinstance(cmd, types.CodeType):
            cmd = cmd+'\n'
        try:
            exec cmd in globals, locals
        except BdbQuit:
            pass
        finally:
            self.quitting = 1
            sys.settrace(None)
    do_debug = decorate_fn_with_doc(new_do_debug, OldPdb.do_debug)
```

It must have to do with call_tracing

This is still an issue in IPython 5 and probably 6, i.e. it still happens with prompt-toolkit based IPython.

Resolving patch:
diff --git a/IPython/core/debugger.py b/IPython/core/debugger.py
--- a/IPython/core/debugger.py
+++ b/IPython/core/debugger.py
@@ -545,6 +545,25 @@ def do_longlist(self, arg):
         self.print_list_lines(self.curframe.f_code.co_filename, lineno, last)
     do_ll = do_longlist
 
+    def do_debug(self, arg):
+        """debug code
+        Enter a recursive debugger that steps through the code
+        argument (which is an arbitrary expression or statement to be
+        executed in the current environment).
+        """
+        sys.settrace(None)
+        globals = self.curframe.f_globals
+        locals = self.curframe_locals
+        p = self.__class__(completekey=self.completekey,
+                           stdin=self.stdin, stdout=self.stdout)
+        p.use_rawinput = self.use_rawinput
+        p.prompt = "(%s) " % self.prompt.strip()
+        self.message("ENTERING RECURSIVE DEBUGGER")
+        sys.call_tracing(p.run, (arg, globals, locals))
+        self.message("LEAVING RECURSIVE DEBUGGER")
+        sys.settrace(self.trace_dispatch)
+        self.lastcmd = p.lastcmd
+
     def do_pdef(self, arg):
         """Print the call signature for any callable object.
 


Test patch: