internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ipython/ipython

Issue:
Capturing stdout and stderr from %run seems not to work
When trying to capture stdout out from %run when using %%capture cell magic, the output of %run is still streamed to the cell.  This is also true when the %%capture magic is substituded for the older solution, provided on stack overflow a few years ago by minrk:

http://stackoverflow.com/questions/14571090/ipython-redirecting-output-of-a-python-script-to-a-file-like-bash

Will provide a minimum working example shortly.



Discussion:
So, not the best working example, but here is an example of stdout being printed when it should be captured.  I've used the method as shown in the thread above rather than %%capture magic in this example:

http://nbviewer.ipython.org/github/hugadams/pyparty/blob/master/testing/REGRESSIONTESTS.ipynb

I'll work on a solution, if anyone else starts working on this, can you let me know so as not to duplicate effort.  I'm not too familiar with the ipython source, so would likely not find the canonical solution.

Thanks

I can confirm this for the case of pandas html tables in the file that is %run. 

Thanks.

I am looking at the source code, but don't even know where to begin

I noticed that @minrk is a participant in this thread.  I know you're a core developer, and your commit history leads me to believe that you probably are working on 100 things at once, but if you have any suspicions on where this bug might be occurring, let me know and I can give debugging a shot.

I am not sure it is considered a bug. It was discussed and closed in issue #3742.  What happens is that %%capture will capture stuff which is displayed by the display() function but let output slip through. I personally consider it a bug because it is inconsistent with running .py files. 


Resolving patch:
diff --git a/IPython/core/interactiveshell.py b/IPython/core/interactiveshell.py
--- a/IPython/core/interactiveshell.py
+++ b/IPython/core/interactiveshell.py
@@ -2600,7 +2600,7 @@ def get_cells():
                     # raised in user code.  It would be nice if there were
                     # versions of run_cell that did raise, so
                     # we could catch the errors.
-                    self.run_cell(cell, store_history=False, shell_futures=False)
+                    self.run_cell(cell, silent=True, shell_futures=False)
             except:
                 self.showtraceback()
                 warn('Unknown failure executing file: <%s>' % fname)


Test patch:
Repository: ipython/ipython

Issue:
print in multiprocess does not work until a newline is sent, even after a flush
In the notebook:

`print('blah',end='')` (print without a newline) in a process created with the `multiprocess` module does not appear in the output cell even after `sys.stdout.flush()` is called. If a new line is printed it works fine and it appears immediately.

This makes writing progress bars for parallel code quite inconvenient.

It works fine in the terminal.



Discussion:
@minrk may have some idea about this.

What IPython version and OS? Are you using the notebook or other UI?

Flush is being called by the subprocess, but flush from a subprocess only sends output to the kernel. The kernel then sends output to the frontend via its own call to flush. I think I know how to make the kernel process notice the subprocess flush.


Resolving patch:
diff --git a/IPython/kernel/zmq/iostream.py b/IPython/kernel/zmq/iostream.py
--- a/IPython/kernel/zmq/iostream.py
+++ b/IPython/kernel/zmq/iostream.py
@@ -74,6 +74,13 @@ def _setup_pipe_in(self):
             return
         self._pipe_poller = zmq.Poller()
         self._pipe_poller.register(self._pipe_in, zmq.POLLIN)
+        if IOLoop.initialized():
+            # subprocess flush should trigger flush
+            # if kernel is idle
+            IOLoop.instance().add_handler(self._pipe_in,
+                lambda s, event: self.flush(),
+                IOLoop.READ,
+            )
     
     def _setup_pipe_out(self):
         # must be new context after fork


Test patch:
Repository: ipython/ipython

Issue:
%guiref in qtconsole broken
With the latest git master, %guiref gives the following error in qtconsole:

```
In [1]: %guiref
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-abc9b6d653cc> in <module>()
----> 1 get_ipython().magic(u'guiref')

/Users/takimoto/Library/Python/2.7/lib/python/site-packages/IPython/core/interactiveshell.pyc in magic(self, arg_s)
   2238         magic_name, _, magic_arg_s = arg_s.partition(' ')
   2239         magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2240         return self.run_line_magic(magic_name, magic_arg_s)
   2241 
   2242     #-------------------------------------------------------------------------

/Users/takimoto/Library/Python/2.7/lib/python/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line)
   2159                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2160             with self.builtin_trap:
-> 2161                 result = fn(*args,**kwargs)
   2162             return result
   2163 

/Users/takimoto/Library/Python/2.7/lib/python/site-packages/IPython/core/usage.pyc in page_guiref(arg_s)
    567     """Show a basic reference about the GUI Console."""
    568     from IPython.core import page
--> 569     page.page(gui_reference, auto_html=True)
    570 

TypeError: page() got an unexpected keyword argument 'auto_html'
```

See #5902 for my system info.
May be related with the commit 095e73ab7386af077968a3bb6c28e5d88d9abe60 ?



Discussion:

Resolving patch:
diff --git a/IPython/core/usage.py b/IPython/core/usage.py
--- a/IPython/core/usage.py
+++ b/IPython/core/usage.py
@@ -566,5 +566,5 @@
 def page_guiref(arg_s=None):
     """Show a basic reference about the GUI Console."""
     from IPython.core import page
-    page.page(gui_reference, auto_html=True)
+    page.page(gui_reference)
 
diff --git a/IPython/qt/console/ipython_widget.py b/IPython/qt/console/ipython_widget.py
--- a/IPython/qt/console/ipython_widget.py
+++ b/IPython/qt/console/ipython_widget.py
@@ -233,7 +233,6 @@ def _handle_display_data(self, msg):
         # data. But we need to figure out how to handle this in the GUI.
         if not self._hidden and self._is_from_this_session(msg):
             self.flush_clearoutput()
-            source = msg['content']['source']
             data = msg['content']['data']
             metadata = msg['content']['metadata']
             # In the regular IPythonWidget, we simply print the plain text
diff --git a/IPython/qt/console/rich_ipython_widget.py b/IPython/qt/console/rich_ipython_widget.py
--- a/IPython/qt/console/rich_ipython_widget.py
+++ b/IPython/qt/console/rich_ipython_widget.py
@@ -135,7 +135,6 @@ def _handle_display_data(self, msg):
         """
         if not self._hidden and self._is_from_this_session(msg):
             self.flush_clearoutput()
-            source = msg['content']['source']
             data = msg['content']['data']
             metadata = msg['content']['metadata']
             # Try to use the svg or html representations.


Test patch:
Repository: ipython/ipython

Issue:
Qt ImportDenier Does Not Adhere to PEP302
The Qt ImportDenier is installed by IPython in `IPython/external/qt_loaders.py`.  The find_module method does not respect the [PEP302](http://legacy.python.org/dev/peps/pep-0302/#specification-part-1-the-importer-protocol) signature of `finder.find_module(fullname, path=None)`, where the path is optional.

There is a [StackOverflow](http://stackoverflow.com/questions/23960220/richipythonwidget-import-side-effect-on-pkgutil-module#) question which exposes this issue in practice.



Discussion:

Resolving patch:
diff --git a/IPython/external/qt_loaders.py b/IPython/external/qt_loaders.py
--- a/IPython/external/qt_loaders.py
+++ b/IPython/external/qt_loaders.py
@@ -34,17 +34,17 @@ def forbid(self, module_name):
         sys.modules.pop(module_name, None)
         self.__forbidden.add(module_name)
 
-    def find_module(self, mod_name, pth):
-        if pth:
+    def find_module(self, fullname, path=None):
+        if path:
             return
-        if mod_name in self.__forbidden:
+        if fullname in self.__forbidden:
             return self
 
-    def load_module(self, mod_name):
+    def load_module(self, fullname):
         raise ImportError("""
     Importing %s disabled by IPython, which has
     already imported an Incompatible QT Binding: %s
-    """ % (mod_name, loaded_api()))
+    """ % (fullname, loaded_api()))
 
 ID = ImportDenier()
 sys.meta_path.append(ID)


Test patch:
Repository: ipython/ipython

Issue:
Widget set_css doesn't always work (front-end order of ops.)
Scenario:
Create a couple of ContainerWidgets, set the CSS such that `margin` = `0px`.  Later in the notebook try changing just the `margin-left` to something like `10px`, this works sometimes and fails others.  This is because the CSS dict it re-applied in the front-end and the ordering is the order of the dict keys (which has no predictable notion of order since they are hashed) and sometimes during the reapplication of the dict, `margin` will be set after `margin-left` etc...

Workaround:
Use path independent CSS traits.  i.e. first set margin-left, right, top, bottom to 0px.  Later set margin-left to 10px.

Possible solution:
Use a list to store CSS traits so order is preserved between the back-end and front-end.



Discussion:

Resolving patch:
diff --git a/IPython/html/widgets/widget.py b/IPython/html/widgets/widget.py
--- a/IPython/html/widgets/widget.py
+++ b/IPython/html/widgets/widget.py
@@ -341,7 +341,7 @@ def _send(self, msg):
 
 class DOMWidget(Widget):
     visible = Bool(True, help="Whether the widget is visible.", sync=True)
-    _css = Dict(sync=True) # Internal CSS property dict
+    _css = List(sync=True) # Internal CSS property list: (selector, key, value)
 
     def get_css(self, key, selector=""):
         """Get a CSS property of the widget.
@@ -384,18 +384,17 @@ def set_css(self, dict_or_key, value=None, selector=''):
             of the view that should be styled with common CSS (see 
             `$el_to_style` in the Javascript code).
         """
-        if not selector in self._css:
-            self._css[selector] = {}
-        my_css = self._css[selector]
-        
         if value is None:
             css_dict = dict_or_key
         else:
             css_dict = {dict_or_key: value}
         
         for (key, value) in css_dict.items():
-            if not (key in my_css and value == my_css[key]):
-                my_css[key] = value
+            # First remove the selector/key pair from the css list if it exists.
+            # Then add the selector/key pair and new value to the bottom of the 
+            # list.
+            self._css = [x for x in self._css if not (x[0]==selector and x[1]==key)]
+            self._css += [(selector, key, value)]
         self.send_state('_css')
 
     def add_class(self, class_names, selector=""):


Test patch:
Repository: ipython/ipython

Issue:
Error in Notebook-Generated latex (nbconvert)
When there are `...` in the python output (like in long `__repr__`s), they seem to be replaced with `\ldots` in latex. However, this command is not protected with `{}`. So the command will stick to whatever follows. If that is not a whitespace, you get `\ldotsnextword` and a latex error. I checked this with master (actually a week ago master). and the last release.



Discussion:

Resolving patch:
diff --git a/IPython/nbconvert/filters/latex.py b/IPython/nbconvert/filters/latex.py
--- a/IPython/nbconvert/filters/latex.py
+++ b/IPython/nbconvert/filters/latex.py
@@ -20,7 +20,7 @@
 #-----------------------------------------------------------------------------
 
 LATEX_RE_SUBS = (
-    (re.compile(r'\.\.\.+'), r'\\ldots'),
+    (re.compile(r'\.\.\.+'), r'{\\ldots}'),
 )
 
 # Latex substitutions for escaping latex.


Test patch:
diff --git a/IPython/nbconvert/filters/tests/test_latex.py b/IPython/nbconvert/filters/tests/test_latex.py
--- a/IPython/nbconvert/filters/tests/test_latex.py
+++ b/IPython/nbconvert/filters/tests/test_latex.py
@@ -31,7 +31,7 @@ def test_escape_latex(self):
             (r'How are \you doing today?', r'How are \textbackslash{}you doing today?'),
             (r'\escapechar=`\A\catcode`\|=0 |string|foo', r'\textbackslash{}escapechar=`\textbackslash{}A\textbackslash{}catcode`\textbackslash{}|=0 |string|foo'),
             (r'# $ % & ~ _ ^ \ { }', r'\# \$ \% \& \textasciitilde{} \_ \^{} \textbackslash{} \{ \}'),
-            ('...', r'\ldots'),
+            ('...', r'{\ldots}'),
             ('','')]
 
         for test in tests:
Repository: ipython/ipython

Issue:
qtconsole truncates help
The Ipython shell opened with `qtconsole` option (i.e `ipython qtconsole`) truncates the help message obtained by the `??` magic. i.e

```
In [1]: import string
In [2]: string??
```

Gives:

```

            lineno = len(lines)
        raise ValueError('Invalid placeholder in string: line %d, col %d' %
                         (lineno, colno))

    def substitute(self, *args, **kws):
        if len(args) > 1:
            raise TypeError('Too many positional arguments')
        if not args:
            mapping = kws
        elif kws:
            mapping = _multimap(kws, args[0])
        else:
            mapping = args[0]
        # Helper function for .sub()
        def convert(mo):
            # Check the most common path first.
            named = mo.group('named') or mo.group('braced')
            if named is not None:
                val = mapping[named]
                # We use this idiom instead of str() because the latter will
                # fail if val is a Unicode containing non-ASCII characters.
...
```

This bug is observed on Win32 and OSX. IPython itself (without `qtconsole`) does not exhibit this problem.



Discussion:
I was about to say I couldn't replicate it, but then I tried on Python 2, where the  `string` module is longer. I think it just exceeds the maximum number of lines we keep in the console, so the top bit is cut off before you see it. We could just increase the number of lines we keep, though that would only delay the problem until you do the same with an even longer module.

It would make sense to make it at least as large as the largest module in the standard library. Otherwise maybe some kind of paging solution? But that might require more development time than you have the bandwidth for

We already consider that paging, it just doesn't page _enough_.


Resolving patch:
diff --git a/IPython/qt/console/console_widget.py b/IPython/qt/console/console_widget.py
--- a/IPython/qt/console/console_widget.py
+++ b/IPython/qt/console/console_widget.py
@@ -1472,6 +1472,8 @@ def _event_filter_page_keypress(self, event):
                 self._control.setFocus()
             else:
                 self.layout().setCurrentWidget(self._control)
+                # re-enable buffer truncation after paging
+                self._control.document().setMaximumBlockCount(self.buffer_size)
             return True
 
         elif key in (QtCore.Qt.Key_Enter, QtCore.Qt.Key_Return,
@@ -1903,6 +1905,8 @@ def _page(self, text, html=False):
             if self.paging == 'custom':
                 self.custom_page_requested.emit(text)
             else:
+                # disable buffer truncation during paging
+                self._control.document().setMaximumBlockCount(0)
                 self._page_control.clear()
                 cursor = self._page_control.textCursor()
                 if html:


Test patch:
Repository: ipython/ipython

Issue:
Traceback on Qt console exit
Start the Qt console, and then type `exit` to exit it (closing it from the frontend doesn't exhibit the bug). I see this message, repeated twice:

```
Traceback (most recent call last):
  File "/home/takluyver/.local/lib/python3.4/site-packages/IPython/qt/console/frontend_widget.py", line 812, in _document_contents_change
    self._call_tip()
  File "/home/takluyver/.local/lib/python3.4/site-packages/IPython/qt/console/frontend_widget.py", line 724, in _call_tip
    msg_id = self.kernel_client.inspect(code, cursor_pos)
  File "/home/takluyver/.local/lib/python3.4/site-packages/IPython/kernel/channels.py", line 326, in inspect
    self._queue_send(msg)
  File "/home/takluyver/.local/lib/python3.4/site-packages/IPython/kernel/channels.py", line 171, in _queue_send
    self.ioloop.add_callback(thread_send)
  File "/usr/lib/python3/dist-packages/tornado/ioloop.py", line 732, in add_callback
    raise RuntimeError("IOLoop is closing")
RuntimeError: IOLoop is closing

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@scipy.org

You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    c.Application.verbose_crash=True
```



Discussion:

Resolving patch:
diff --git a/IPython/qt/console/frontend_widget.py b/IPython/qt/console/frontend_widget.py
--- a/IPython/qt/console/frontend_widget.py
+++ b/IPython/qt/console/frontend_widget.py
@@ -716,7 +716,7 @@ def _call_tip(self):
         """ Shows a call tip, if appropriate, at the current cursor location.
         """
         # Decide if it makes sense to show a call tip
-        if not self.enable_calltips:
+        if not self.enable_calltips or not self.kernel_client.shell_channel.is_alive():
             return False
         cursor_pos = self._get_input_buffer_cursor_pos()
         code = self.input_buffer


Test patch:
Repository: ipython/ipython

Issue:
Transparent background for inline plots in IPython notebook
Is there any way of setting transparent background for inline plots in IPython notebook? Currently it seems to be always white, which in my opinion doesn't look good with customized notebook backgrounds.



Discussion:

Resolving patch:
diff --git a/IPython/kernel/zmq/pylab/config.py b/IPython/kernel/zmq/pylab/config.py
--- a/IPython/kernel/zmq/pylab/config.py
+++ b/IPython/kernel/zmq/pylab/config.py
@@ -53,6 +53,7 @@ def _config_changed(self, name, old, new):
         # play nicely with white background in the Qt and notebook frontend
         'figure.facecolor': (1,1,1,0),
         'figure.edgecolor': (1,1,1,0),
+        'axes.facecolor': (1,1,1,0),
         # 12pt labels get cutoff on 6x4 logplots, so use 10pt.
         'font.size': 10,
         # 72 dpi matches SVG/qtconsole


Test patch:
Repository: ipython/ipython

Issue:
Reconnect with password
In case you have a public notebook you will most certainly want to use a password (BTW. why is this not using HTTP basic auth?)

When your notebook server wents down/you do a manual restart you get a reconnect question, though this fails on notebooks with passwords, obviously.

Can we just trigger a normal F5 in this case?



Discussion:
It fails because the cookie secret is random by default, so it doesn't survive across sessions. If the cookie secret is set to a consistent value, refresh after restart works. This should probably be the default behavior.


Resolving patch:
diff --git a/IPython/html/notebookapp.py b/IPython/html/notebookapp.py
--- a/IPython/html/notebookapp.py
+++ b/IPython/html/notebookapp.py
@@ -6,6 +6,7 @@
 
 from __future__ import print_function
 
+import base64
 import errno
 import io
 import json
@@ -357,6 +358,14 @@ def _ip_changed(self, name, old, new):
         help="""The full path to a private key file for usage with SSL/TLS."""
     )
     
+    cookie_secret_file = Unicode(config=True,
+        help="""The file where the cookie secret is stored."""
+    )
+    def _cookie_secret_file_default(self):
+        if self.profile_dir is None:
+            return ''
+        return os.path.join(self.profile_dir.security_dir, 'notebook_cookie_secret')
+    
     cookie_secret = Bytes(b'', config=True,
         help="""The random bytes used to secure cookies.
         By default this is a new random number every time you start the Notebook.
@@ -367,7 +376,26 @@ def _ip_changed(self, name, old, new):
         """
     )
     def _cookie_secret_default(self):
-        return os.urandom(1024)
+        if os.path.exists(self.cookie_secret_file):
+            with io.open(self.cookie_secret_file, 'rb') as f:
+                return f.read()
+        else:
+            secret = base64.encodestring(os.urandom(1024))
+            self._write_cookie_secret_file(secret)
+            return secret
+    
+    def _write_cookie_secret_file(self, secret):
+        """write my secret to my secret_file"""
+        self.log.info("Writing notebook server cookie secret to %s", self.cookie_secret_file)
+        with io.open(self.cookie_secret_file, 'wb') as f:
+            f.write(secret)
+        try:
+            os.chmod(self.cookie_secret_file, 0o600)
+        except OSError:
+            self.log.warn(
+                "Could not set permissions on %s",
+                self.cookie_secret_file
+            )
 
     password = Unicode(u'', config=True,
                       help="""Hashed password to use for web authentication.


Test patch:
Repository: ipython/ipython

Issue:
Command line "ipython trust" option does not work
I'm working with a slideshow template I found on Fernando's github account: https://github.com/fperez/nb-slideshow-template

When I open the notebook, IPython complains that it isn't trusted.  When I close the notebook, kill the server, run `ipython trust my_notebook.ipynb`, and then restart the notebook, I get the same warning about the notebook not being trusted.

I was able to work around this issue using the "Trust this notebook" option in the File dropdown menu on the notebook web interface.



Discussion:
The notebook in question is here: https://bitbucket.org/ngoldbaum/scipy-2014-talk/src/44eb7e13a3ded3e9cc07a68fd496799443d05e81/scipy_presentation.ipynb?at=default

This is a version from before I trusted it.  

Here is a somewhat later version from after I trusted the notebook: https://bitbucket.org/ngoldbaum/scipy-2014-talk/src/e22469b1bda3cf68aed7e5bfeff38b35c36396fb/scipy_presentation.ipynb?at=default

Were you perhaps presenting from a non-default IPython profile?

@minrk yup.  I guess I should have mentioned that in the report, but it didn't occur to me.

The notebook signature key is stored in the profile:

```
ipython trust --profile myprofile [notebook]
```

or

```
ln -s $(ipython locate profile)/security/notebook_secret $(ipython locate profile other_profile)/security/notebook_secret # or cp
```

That makes sense - I guess this bug should be closed then?


Resolving patch:
diff --git a/IPython/nbformat/sign.py b/IPython/nbformat/sign.py
--- a/IPython/nbformat/sign.py
+++ b/IPython/nbformat/sign.py
@@ -263,10 +263,17 @@ class TrustNotebookApp(BaseIPythonApplication):
     description="""Sign one or more IPython notebooks with your key,
     to trust their dynamic (HTML, Javascript) output.
     
+    Trusting a notebook only applies to the current IPython profile.
+    To trust a notebook for use with a profile other than default,
+    add `--profile [profile name]`.
+    
     Otherwise, you will have to re-execute the notebook to see output.
     """
     
-    examples = """ipython trust mynotebook.ipynb and_this_one.ipynb"""
+    examples = """
+    ipython trust mynotebook.ipynb and_this_one.ipynb
+    ipython trust --profile myprofile mynotebook.ipynb
+    """
     
     flags = trust_flags
     


Test patch: