internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ipython/ipython

Issue:
Input parsing issue with %prun
```
In [5]: %prun grouped.agg({'value1' : 'mean', 'value2' : 'var', 'value3' : 'sum'})
  File "<string>", line 1
    'value1' : 'mean', 'value2' : 'var', 'value3' : 'sum'
             ^
SyntaxError: invalid syntax
```



Discussion:
git bisect reveals f5687fc7e0d0c5d373f8e6e26d7908716e7ec9fb as the offender.

The issue is the new var_expand, which uses a DollarFormatter, rather than ItplNS. Formerly, `$` was the only expansion, but since we now use string formatting, the `{name}` syntax is acceptable, and the magic is trying to format the string.  This wasn't an issue, because variables wer unambiguous, as they were not Python syntax, but they are now.

The magic argument transform is proving itself to be in irredeemable mess, and needs to be reconsidered in a much cleaner way.  For instance, this var_expand call should never be made on magics that aren't shell-ish.

Temporary workaround to just disable variable expansion, if you need it:

``` python
get_ipython().var_expand = lambda s, *args, **kwargs: s
```


Resolving patch:
diff --git a/IPython/core/interactiveshell.py b/IPython/core/interactiveshell.py
--- a/IPython/core/interactiveshell.py
+++ b/IPython/core/interactiveshell.py
@@ -2675,7 +2675,12 @@ def var_expand(self, cmd, depth=0, formatter=DollarFormatter()):
         ns = self.user_ns.copy()
         ns.update(sys._getframe(depth+1).f_locals)
         ns.pop('self', None)
-        return formatter.format(cmd, **ns)
+        try:
+            cmd = formatter.format(cmd, **ns)
+        except Exception:
+            # if formatter couldn't format, just let it go untransformed
+            pass
+        return cmd
 
     def mktempfile(self, data=None, prefix='ipython_edit_'):
         """Make a new tempfile and return its filename.


Test patch:
diff --git a/IPython/core/tests/test_interactiveshell.py b/IPython/core/tests/test_interactiveshell.py
--- a/IPython/core/tests/test_interactiveshell.py
+++ b/IPython/core/tests/test_interactiveshell.py
@@ -235,10 +235,24 @@ def test_var_expand(self):
         ip = get_ipython()
         ip.user_ns['f'] = u'Ca\xf1o'
         self.assertEqual(ip.var_expand(u'echo $f'), u'echo Ca\xf1o')
+        self.assertEqual(ip.var_expand(u'echo {f}'), u'echo Ca\xf1o')
+        self.assertEqual(ip.var_expand(u'echo {f[:-1]}'), u'echo Ca\xf1')
+        self.assertEqual(ip.var_expand(u'echo {1*2}'), u'echo 2')
 
         ip.user_ns['f'] = b'Ca\xc3\xb1o'
         # This should not raise any exception:
         ip.var_expand(u'echo $f')
+    
+    def test_bad_var_expand(self):
+        """var_expand on invalid formats shouldn't raise"""
+        ip = get_ipython()
+        
+        # SyntaxError
+        self.assertEqual(ip.var_expand(u"{'a':5}"), u"{'a':5}")
+        # NameError
+        self.assertEqual(ip.var_expand(u"{asdf}"), u"{asdf}")
+        # ZeroDivisionError
+        self.assertEqual(ip.var_expand(u"{1/0}"), u"{1/0}")
 
 
 class TestSafeExecfileNonAsciiPath(unittest.TestCase):
Repository: ipython/ipython

Issue:
Wishlist: Remove hard dependency on ctypes
`IPython.lib.inputhook` has a hard dependency on `ctypes`.

Recently I've been working with Python on a machine without ctypes.  I was able to get a basic IPython terminal by commenting out much of the file.  Obviously this means I won't have any `%gui` magic abilities, but that's not a problem in my situation.

Perhaps instead of this hard dependency we should consider refactoring the class so that InputHookManager was a pluggable configuration option?  Would this be a realistic project?



Discussion:
We certainly won't remove the dependency on ctypes for full functionality, but for basic IPython to start without ctypes is pretty trivial.  Just add to inputhook:

``` python
try:
    import ctypes
except ImportError:
    ctypes = None
    # possibly warn here, because your Python is broken, albeit in a way that's not too rare

# ... in InputHookManager.__init__()
    if ctypes is not None:
        self.PYFUNC = ctypes.PYFUNCTYPE(ctypes.c_int)
```

Then, obviously any actual use of the inputhook machinery would fail (as it should), but basic IPython should start.

I think so; there are a number of environments where in the future we may need this, and ctypes is only required for local, interactive GUI usage.  In those systems most likely people will be using the inline backend to produce figures to be accessed remotely, so making the ctypes dependency only be needed for actualy %gui work is reasonable.


Resolving patch:
diff --git a/IPython/lib/inputhook.py b/IPython/lib/inputhook.py
--- a/IPython/lib/inputhook.py
+++ b/IPython/lib/inputhook.py
@@ -14,10 +14,14 @@
 # Imports
 #-----------------------------------------------------------------------------
 
-import ctypes
+try:
+    import ctypes
+except ImportError:
+    ctypes = None
 import os
 import sys
-import warnings
+
+from IPython.utils.warn import warn
 
 #-----------------------------------------------------------------------------
 # Constants
@@ -98,6 +102,9 @@ class InputHookManager(object):
     """
     
     def __init__(self):
+        if ctypes is None:
+            warn("IPython GUI event loop requires ctypes, %gui will not be available\n")
+            return
         self.PYFUNC = ctypes.PYFUNCTYPE(ctypes.c_int)
         self._apps = {}
         self._reset()
diff --git a/IPython/zmq/parentpoller.py b/IPython/zmq/parentpoller.py
--- a/IPython/zmq/parentpoller.py
+++ b/IPython/zmq/parentpoller.py
@@ -1,5 +1,8 @@
 # Standard library imports.
-import ctypes
+try:
+    import ctypes
+except:
+    ctypes = None
 import os
 import platform
 import time
@@ -54,6 +57,8 @@ def __init__(self, interrupt_handle=None, parent_handle=None):
         """
         assert(interrupt_handle or parent_handle)
         super(ParentPollerWindows, self).__init__()
+        if ctypes is None:
+            raise ImportError("ParentPollerWindows requires ctypes")
         self.daemon = True
         self.interrupt_handle = interrupt_handle
         self.parent_handle = parent_handle


Test patch:
Repository: ipython/ipython

Issue:
%history documentation out of date?
In IPython/core/history.py:722

```
"""Print input history (_i<n> variables), with most recent last.

%history       -> print at most 40 inputs (some may be multi-line)\\
%history n     -> print at most n inputs\\
%history n1 n2 -> print inputs between n1 and n2 (n2 not included)\\
```

...

```
Ranges of history can be indicated using the syntax:
4      : Line 4, current session
4-6    : Lines 4-6, current session
```

...

Does the range syntax supercede the original syntax for history?  Should these lines be removed?

```
%history n     -> print at most n inputs\\
%history n1 n2 -> print inputs between n1 and n2 (n2 not included)\\
```



Discussion:
You're quite right, that's a leftover that should be removed. Would you like to make a pull request for it? Thanks.

sure thing, give me a couple of minutes.


Resolving patch:
diff --git a/IPython/core/history.py b/IPython/core/history.py
--- a/IPython/core/history.py
+++ b/IPython/core/history.py
@@ -717,14 +717,13 @@ def _format_lineno(session, line):
 def magic_history(self, parameter_s = ''):
     """Print input history (_i<n> variables), with most recent last.
 
-    %history       -> print at most 40 inputs (some may be multi-line)\\
-    %history n     -> print at most n inputs\\
-    %history n1 n2 -> print inputs between n1 and n2 (n2 not included)\\
+    %history [-o -p -t -n] [-f filename] [range | -g pattern | -l number]
 
     By default, input history is printed without line numbers so it can be
     directly pasted into an editor. Use -n to show them.
 
-    Ranges of history can be indicated using the syntax:
+    By default, all input history from the current session is displayed.
+    Ranges of history can be indicated using the syntax: 
     4      : Line 4, current session
     4-6    : Lines 4-6, current session
     243/1-5: Lines 1-5, session 243
@@ -772,9 +771,10 @@ def magic_history(self, parameter_s = ''):
     --------
     ::
 
-      In [6]: %hist -n 4 6
+      In [6]: %hist -n 4-6
       4:a = 12
       5:print a**2
+      6:%hist -n 4-6
 
     """
 


Test patch:
Repository: ipython/ipython

Issue:
When our debugger class is used standalone `_oh` key errors are thrown
See [this bug report](https://github.com/inducer/pudb/issues/30) for an example...



Discussion:

Resolving patch:
diff --git a/IPython/core/displayhook.py b/IPython/core/displayhook.py
--- a/IPython/core/displayhook.py
+++ b/IPython/core/displayhook.py
@@ -252,7 +252,11 @@ def flush(self):
             try:
                 del self.shell.user_ns[key]
             except: pass
-        self.shell.user_ns['_oh'].clear()
+        # In some embedded circumstances, the user_ns doesn't have the
+        # '_oh' key set up.
+        oh = self.shell.user_ns.get('_oh', None)
+        if oh is not None:
+            oh.clear()
 
         # Release our own references to objects:
         self._, self.__, self.___ = '', '', ''


Test patch:
Repository: ipython/ipython

Issue:
python3 notebook: TypeError: unorderable types
I got this traceback when starting an python3 notebook on current git head:

```
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tornado/web.py", line 954, in _execute
    getattr(self, self.request.method.lower())(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/tornado/web.py", line 1667, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/IPython/frontend/html/notebook/handlers.py", line 676, in get
    self.finish(jsonapi.dumps(cm.list_profiles()))
  File "/usr/lib/python3/dist-packages/IPython/frontend/html/notebook/clustermanager.py", line 95, in list_profiles
    result.sort()
TypeError: unorderable types: dict() < dict()
```



Discussion:

Resolving patch:
diff --git a/IPython/frontend/html/notebook/clustermanager.py b/IPython/frontend/html/notebook/clustermanager.py
--- a/IPython/frontend/html/notebook/clustermanager.py
+++ b/IPython/frontend/html/notebook/clustermanager.py
@@ -91,8 +91,7 @@ def update_profiles(self):
 
     def list_profiles(self):
         self.update_profiles()
-        result = [self.profile_info(p) for p in self.profiles.keys()]
-        result.sort()
+        result = [self.profile_info(p) for p in sorted(self.profiles.keys())]
         return result
 
     def check_profile(self, profile):


Test patch:
Repository: ipython/ipython

Issue:
Pretty printing exceptions is broken
```
Python 3.2.2 (default, Oct 31 2011, 16:56:14) 
Type "copyright", "credits" or "license" for more information.

IPython 0.12 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
In [1]: ValueError('foo')
Out[1]: ('foo')
```

I would have expected

```
Out[1]: ValueError('foo')
```



Discussion:

Resolving patch:
diff --git a/IPython/lib/pretty.py b/IPython/lib/pretty.py
--- a/IPython/lib/pretty.py
+++ b/IPython/lib/pretty.py
@@ -628,7 +628,7 @@ def _exception_pprint(obj, p, cycle):
             obj.__class__.__name__
         )
     step = len(name) + 1
-    p.begin_group(step, '(')
+    p.begin_group(step, name + '(')
     for idx, arg in enumerate(getattr(obj, 'args', ())):
         if idx:
             p.text(',')


Test patch:
Repository: ipython/ipython

Issue:
Asynchrous issue when using clear_display and print x,y,z
A minor issue

``` python
from IPython.core.display import clear_output
import time
class xxx:
    def __init__(self,color):
        self.color = color
    def _repr_html_(self):
        return '<span style="color:%s">woot</span>'%self.color
for i in range(10):
    x = xxx(['red','green','blue'][(i%3)])
    print 1,2,3
    clear_output(stdout=False, stderr=False, other=True)
    display(x)
    time.sleep(2)
```

This produce a funny looking output like

<pre>
1 2 3
1
 2 3
1
.
.
.
</pre>

Which the raw html is

``` html
<pre> 1 2 3
1</pre>
```

then

``` html
<pre>2 3
1
</pre>
```

While

``` python
print 1,
print 2,
print 3,
```

works just fine. It also works fine if I remove sleep function.



Discussion:

Resolving patch:
diff --git a/IPython/zmq/zmqshell.py b/IPython/zmq/zmqshell.py
--- a/IPython/zmq/zmqshell.py
+++ b/IPython/zmq/zmqshell.py
@@ -59,8 +59,14 @@ class ZMQDisplayPublisher(DisplayPublisher):
     def set_parent(self, parent):
         """Set the parent for outbound messages."""
         self.parent_header = extract_header(parent)
+    
+    def _flush_streams(self):
+        """flush IO Streams prior to display"""
+        sys.stdout.flush()
+        sys.stderr.flush()
 
     def publish(self, source, data, metadata=None):
+        self._flush_streams()
         if metadata is None:
             metadata = {}
         self._validate_data(source, data, metadata)
@@ -75,6 +81,7 @@ def publish(self, source, data, metadata=None):
         )
 
     def clear_output(self, stdout=True, stderr=True, other=True):
+        self._flush_streams()
         content = dict(stdout=stdout, stderr=stderr, other=other)
         self.session.send(
             self.pub_socket, u'clear_output', content,


Test patch:
Repository: ipython/ipython

Issue:
calling .abort on AsyncMapResult  results in traceback
traceback says AsyncMapResult object has no attribute 'client'

@fperez just ran into this doing a live demo, here's the TB:

``` python
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/home/sgeadmin/<ipython-input-45-75c2b5e59cd8> in <module>()
----> 1 amr.abort()

/software/python-2.7.1-release/lib/python2.7/site-packages/IPython/parallel/client/asyncresult.pyc in abort(self)
    196         """abort my tasks."""
    197         assert not self.ready(), "Can't abort, I am already done!"
--> 198         return self.client.abort(self.msg_ids, targets=self._targets, block=True)
    199 
    200     @property

/software/python-2.7.1-release/lib/python2.7/site-packages/IPython/parallel/client/asyncresult.pyc in __getattr__(self, key)
    239         except (error.TimeoutError, KeyError):
    240             raise AttributeError("%r object has no attribute %r"%(
--> 241                     self.__class__.__name__, key))
    242 
    243     # asynchronous iterator:

AttributeError: 'AsyncMapResult' object has no attribute 'client'
```

Fernando said @minrk should be able to fix it before end of class :) (in an hour)



Discussion:

Resolving patch:
diff --git a/IPython/parallel/client/asyncresult.py b/IPython/parallel/client/asyncresult.py
--- a/IPython/parallel/client/asyncresult.py
+++ b/IPython/parallel/client/asyncresult.py
@@ -195,7 +195,7 @@ def __dict__(self):
     def abort(self):
         """abort my tasks."""
         assert not self.ready(), "Can't abort, I am already done!"
-        return self.client.abort(self.msg_ids, targets=self._targets, block=True)
+        return self._client.abort(self.msg_ids, targets=self._targets, block=True)
 
     @property
     def sent(self):


Test patch:
diff --git a/IPython/parallel/tests/test_asyncresult.py b/IPython/parallel/tests/test_asyncresult.py
--- a/IPython/parallel/tests/test_asyncresult.py
+++ b/IPython/parallel/tests/test_asyncresult.py
@@ -19,6 +19,7 @@
 
 from IPython.parallel.error import TimeoutError
 
+from IPython.parallel import error
 from IPython.parallel.tests import add_engines
 from .clienttest import ClusterTestCase
 
@@ -32,7 +33,8 @@ def wait(n):
 
 class AsyncResultTest(ClusterTestCase):
 
-    def test_single_result(self):
+    def test_single_result_view(self):
+        """various one-target views get the right value for single_result"""
         eid = self.client.ids[-1]
         ar = self.client[eid].apply_async(lambda : 42)
         self.assertEquals(ar.get(), 42)
@@ -111,5 +113,13 @@ def test_single_result(self):
         self.assertTrue(isinstance(ar['engine_id'], int))
         self.assertTrue(isinstance(ar.engine_id, int))
         self.assertEquals(ar.engine_id, ar['engine_id'])
+    
+    def test_abort(self):
+        e = self.client[-1]
+        ar = e.execute('import time; time.sleep(1)', block=False)
+        ar2 = e.apply_async(lambda : 2)
+        ar2.abort()
+        self.assertRaises(error.TaskAborted, ar2.get)
+        ar.get()
Repository: ipython/ipython

Issue:
pyin messages should have execution_count
@ivanov keeps bugging me about this one but never opens an Issue for it when I ask, so I'll just assign it to him ;)

It does hightly the issue that has been raised in the past that our message spec may want to be versioned somehow, and folding that versioning into the spec itself (presumably as a field in the header).



Discussion:
Thanks, glad to see @ivanov working on it ;)

i hate you guys.... this kind of public humiliation is _exactly_ the sort of reason why I _would_ start working on it. you know how to push my buttons, expect a PR shortly


Resolving patch:
diff --git a/IPython/zmq/ipkernel.py b/IPython/zmq/ipkernel.py
--- a/IPython/zmq/ipkernel.py
+++ b/IPython/zmq/ipkernel.py
@@ -226,11 +226,11 @@ def record_ports(self, ports):
     # Kernel request handlers
     #---------------------------------------------------------------------------
 
-    def _publish_pyin(self, code, parent):
+    def _publish_pyin(self, code, parent, execution_count):
         """Publish the code request on the pyin stream."""
 
-        self.session.send(self.iopub_socket, u'pyin', {u'code':code},
-                          parent=parent)
+        self.session.send(self.iopub_socket, u'pyin', {u'code':code,
+                          u'execution_count': execution_count}, parent=parent)
 
     def execute_request(self, ident, parent):
 
@@ -271,7 +271,7 @@ def execute_request(self, ident, parent):
         # Re-broadcast our input for the benefit of listening clients, and
         # start computing output
         if not silent:
-            self._publish_pyin(code, parent)
+            self._publish_pyin(code, parent, shell.execution_count)
 
         reply_content = {}
         try:


Test patch:
Repository: ipython/ipython

Issue:
Open a notebook from the command line
`ipython notebook Somefile.ipynb` should start the notebook server, and open a browser tab to that notebook, rather than the dashboard. This should make it possible to double click an ipynb file and have it open in the browser, ready to run.



Discussion:
+1

Ultimately, we want to be able to do things like notebook -> py export and similar, all from the command line.

Here's a [recent thread on IPython-dev list](http://mail.scipy.org/pipermail/ipython-dev/2012-April/009112.html) ("[IPython-dev] Opening Notebooks from the cmd line") which points to some work in progress on this front.

If someone comes across this bug and is intrigued by Fernando's comment above from 6 months ago, that work is under way in [ipython/nbconvert](https://github.com/ipython/nbconvert) repository.


Resolving patch:
diff --git a/IPython/frontend/html/notebook/notebookapp.py b/IPython/frontend/html/notebook/notebookapp.py
--- a/IPython/frontend/html/notebook/notebookapp.py
+++ b/IPython/frontend/html/notebook/notebookapp.py
@@ -257,6 +257,9 @@ class NotebookApp(BaseIPythonApplication):
     # create requested profiles by default, if they don't exist:
     auto_create = Bool(True)
 
+    # file to be opened in the notebook server
+    file_to_run = Unicode('')
+
     # Network related information.
 
     ip = Unicode(LOCALHOST, config=True,
@@ -392,6 +395,10 @@ def parse_command_line(self, argv=None):
         # Kernel should inherit default config file from frontend
         self.kernel_argv.append("--KernelApp.parent_appname='%s'"%self.name)
 
+        if self.extra_args:
+            self.file_to_run = os.path.abspath(self.extra_args[0])
+            self.config.NotebookManager.notebook_dir = os.path.dirname(self.file_to_run)
+
     def init_configurables(self):
         # force Session default to be secure
         default_secure(self.config)
@@ -540,9 +547,20 @@ def start(self):
                 browser = webbrowser.get(self.browser)
             else:
                 browser = webbrowser.get()
-            b = lambda : browser.open("%s://%s:%i%s" % (proto, ip, self.port,
-                                                           self.base_project_url),
-                                         new=2)
+
+            if self.file_to_run:
+                filename, _ = os.path.splitext(os.path.basename(self.file_to_run))
+                for nb in self.notebook_manager.list_notebooks():
+                    if filename == nb['name']:
+                        url = nb['notebook_id']
+                        break
+                else:
+                    url = ''
+            else:
+                url = ''
+            b = lambda : browser.open("%s://%s:%i%s%s" % (proto, ip,
+                                        self.port, self.base_project_url, url),
+                                    new=2)
             threading.Thread(target=b).start()
         try:
             ioloop.IOLoop.instance().start()


Test patch:
Repository: ipython/ipython

Issue:
Parallel controller failing with Pymongo 2.2
[Test failure](https://jenkins.shiningpanda.com/ipython/job/ipython-mac/94/testReport/junit/nose.failure/Failure/runTest/)

The mongodb backend for the controller tries to import from `pymongo.binary`, but this no longer exists in 2.2; it should import from `bson.binary` instead.



Discussion:

Resolving patch:
diff --git a/IPython/parallel/controller/mongodb.py b/IPython/parallel/controller/mongodb.py
--- a/IPython/parallel/controller/mongodb.py
+++ b/IPython/parallel/controller/mongodb.py
@@ -12,7 +12,7 @@
 #-----------------------------------------------------------------------------
 
 from pymongo import Connection
-from pymongo.binary import Binary
+from bson import Binary
 
 from IPython.utils.traitlets import Dict, List, Unicode, Instance
 


Test patch:
Repository: ipython/ipython

Issue:
cache-size issue in qtconsole
When the qtconsole is started with the option --cache-size=0, the return values of the functions become "None".
Version :0.12
OS: Both in Windows Xp and 7

Note: The same problem does not exist in the terminal.



Discussion:
Sorry, which functions are these?


Resolving patch:
diff --git a/IPython/frontend/qt/console/ipython_widget.py b/IPython/frontend/qt/console/ipython_widget.py
--- a/IPython/frontend/qt/console/ipython_widget.py
+++ b/IPython/frontend/qt/console/ipython_widget.py
@@ -214,7 +214,7 @@ def _handle_pyout(self, msg):
         self.log.debug("pyout: %s", msg.get('content', ''))
         if not self._hidden and self._is_from_this_session(msg):
             content = msg['content']
-            prompt_number = content['execution_count']
+            prompt_number = content.get('execution_count', 0)
             data = content['data']
             if data.has_key('text/html'):
                 self._append_plain_text(self.output_sep, True)
diff --git a/IPython/frontend/qt/console/rich_ipython_widget.py b/IPython/frontend/qt/console/rich_ipython_widget.py
--- a/IPython/frontend/qt/console/rich_ipython_widget.py
+++ b/IPython/frontend/qt/console/rich_ipython_widget.py
@@ -71,7 +71,7 @@ def _handle_pyout(self, msg):
         """
         if not self._hidden and self._is_from_this_session(msg):
             content = msg['content']
-            prompt_number = content['execution_count']
+            prompt_number = content.get('execution_count', 0)
             data = content['data']
             if data.has_key('image/svg+xml'):
                 self.log.debug("pyout: %s", msg.get('content', ''))
diff --git a/IPython/zmq/displayhook.py b/IPython/zmq/displayhook.py
--- a/IPython/zmq/displayhook.py
+++ b/IPython/zmq/displayhook.py
@@ -57,8 +57,7 @@ def start_displayhook(self):
 
     def write_output_prompt(self):
         """Write the output prompt."""
-        if self.do_full_cache:
-            self.msg['content']['execution_count'] = self.prompt_count
+        self.msg['content']['execution_count'] = self.prompt_count
 
     def write_format_data(self, format_dict):
         _encode_binary(format_dict)


Test patch: