internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ipython/ipython

Issue:
Include MathJax by default or access the CDN over a secure connection
IPython Notebook loads MathJax over an insecure connection by default. Code running within the IPython Notebook can, by design, execute code on the user's system. This leaves users vulnerable to MITM attacks.

Possible fixes (that I can think of):
- Always access the MathJax CDN over a secure connection (it support HTTPS).
- Custom loader which verifies the code against a local checksum (this would protect against a compromised CDN, but has some obvious disadvantages like complexity or the need to maintain a list of checksums). 
- Ship local MathJax sources (undesirable because it's ~43MB?).

Related: CVE-2014-3429 / https://github.com/ipython/ipython/pull/4845



Discussion:
I think we can make it https by default. Is there any disadvantage to doing that?

We have considered shipping MathJax in components as well. It's not so bad as it seems, because MathJax doesn't need everything - it contains PNG and SVG fallbacks for fonts, which we can probably avoid shipping.

It is https if server is runned over https (universal url that starts with `//:`).


Resolving patch:
diff --git a/IPython/html/notebookapp.py b/IPython/html/notebookapp.py
--- a/IPython/html/notebookapp.py
+++ b/IPython/html/notebookapp.py
@@ -546,7 +546,7 @@ def _mathjax_url_default(self):
                 return url
         
         # no local mathjax, serve from CDN
-        url = u"//cdn.mathjax.org/mathjax/latest/MathJax.js"
+        url = u"https://cdn.mathjax.org/mathjax/latest/MathJax.js"
         self.log.info("Using MathJax from CDN: %s", url)
         return url
     


Test patch:
Repository: ipython/ipython

Issue:
IPython Notebook Server URL now requires "tree" at the end of the URL? (version 2.2)
I've been using a bookmark to go visit my IPython notebook server so I was surprise to get a 404 Server Not Found message after I upgraded to version 2.2. So after some trial and error, I found out I needed to add "tree" to the end of the URL. Was this documented somewhere? I was going to maybe mention this as an issue at their Github site. I did look at the docs, but no where does it mention that you now have to add "tree" at the end of the URL.  Anyone else notice this or is it just me?

Does the documentation need to be updated to reflect this?  Thanks in advance.



Discussion:
I think it should redirect you to to `/tree` if you try to visit the root of the notebook server. It does for me, anyway.

Is your notebook server behind a proxy? It's possible the redirect doesn't work quite right if there's an extra URL prefix (e.g. `/notebook-proxy/` might redirect to `/notebook-proxy/tree/`, but `/notebook/proxy` might not).

I have set up my notebook server at work and I am on a proxy server.

Here is what my notebook config file has:
c.NotebookApp.base_url = '/ipython/'
c.NotebookApp.webapp_settings = {'static_url_prefix':'/ipython/static/'}
c.NotebookApp.ip = 'my_company_ip'
c.NotebookApp.port = 8888

So when I visit:
http://my_company_ip:8888/ipython/
I get the 404 error

But if I visit:
http://my_company_ip:8888/ipython/tree
it is ok

I am using anaconda, so the only thing I did to upgrade was issue the: conda upgrade ipython command

Then that is when this problem started for me.  I did not have this problem when using version 2.1.  Maybe my company changed their proxy server settings so I may have to do some more trial and error like go wireless or connect directly with LAN cable and see if there is a difference.

I suspect it was #6117 that caused the problem. I will investigate.

@minrk 
Thanks.  But is there a work around other than just adding "tree" to the url?


Resolving patch:
diff --git a/IPython/html/tree/handlers.py b/IPython/html/tree/handlers.py
--- a/IPython/html/tree/handlers.py
+++ b/IPython/html/tree/handlers.py
@@ -80,5 +80,5 @@ def get(self, path=''):
     (r"/tree%s" % notebook_path_regex, TreeHandler),
     (r"/tree%s" % path_regex, TreeHandler),
     (r"/tree", TreeHandler),
-    (r"", TreeRedirectHandler),
+    (r"/?", TreeRedirectHandler),
     ]


Test patch:
Repository: ipython/ipython

Issue:
Implement check_pid for .NET
This is a reminder - I added an `IPython.utils.process.check_pid` function to check if a process exists with a given PID. However, we have a `_process_cli` module containing .NET implementations of these things, and I didn't write the check_pid implementation for .NET.



Discussion:
@takluyver  This seems to be pretty easy to do using Process.GetProcessById, and it raises a ValueError (an ArgumentException in C#) if the process doesn't exist. Can I submit a pull request?

@aminb go for it.


Resolving patch:
diff --git a/IPython/utils/_process_cli.py b/IPython/utils/_process_cli.py
--- a/IPython/utils/_process_cli.py
+++ b/IPython/utils/_process_cli.py
@@ -61,3 +61,18 @@ def getoutput(cmd):
     myError = reg.StandardError
     error = myError.ReadToEnd()
     return output
+
+def check_pid(pid):
+    """
+    Check if a process with the given PID (pid) exists
+    """
+    try:
+        System.Diagnostics.Process.GetProcessById(pid)
+        # process with given pid is running
+        return True
+    except System.InvalidOperationException:
+        # process wasn't started by this object (but is running)
+        return True
+    except System.ArgumentException:
+        # process with given pid isn't running
+        return False 
diff --git a/IPython/utils/process.py b/IPython/utils/process.py
--- a/IPython/utils/process.py
+++ b/IPython/utils/process.py
@@ -23,7 +23,7 @@
 if sys.platform == 'win32':
     from ._process_win32 import _find_cmd, system, getoutput, arg_split, check_pid
 elif sys.platform == 'cli':
-    from ._process_cli import _find_cmd, system, getoutput, arg_split
+    from ._process_cli import _find_cmd, system, getoutput, arg_split, check_pid
 else:
     from ._process_posix import _find_cmd, system, getoutput, arg_split, check_pid
 


Test patch:
Repository: ipython/ipython

Issue:
Setting on_trait_change fails if trait name is unicode in python 2 
The failure really occurs in `IPython.utils.traitlets.parse_notifier_name` in which none of the conditional are `True` if run in python2 and a unicode `name` is passed in.

Minimal code to reproduce (run with python 2):

```
python -c "from IPython.utils.traitlets import parse_notifier_name; assert parse_notifier_name(u'foo') is not None"
```

The use case where this came up was calling `.on_trait_change(some_function, 'value')` in a file that did `from __future__ import unicode_literals`.



Discussion:

Resolving patch:
diff --git a/IPython/utils/traitlets.py b/IPython/utils/traitlets.py
--- a/IPython/utils/traitlets.py
+++ b/IPython/utils/traitlets.py
@@ -55,7 +55,7 @@
 from .importstring import import_item
 from IPython.utils import py3compat
 from IPython.utils import eventful
-from IPython.utils.py3compat import iteritems
+from IPython.utils.py3compat import iteritems, string_types
 from IPython.testing.skipdoctest import skip_doctest
 
 SequenceTypes = (list, tuple, set, frozenset)
@@ -133,13 +133,13 @@ def parse_notifier_name(name):
     >>> parse_notifier_name(None)
     ['anytrait']
     """
-    if isinstance(name, str):
+    if isinstance(name, string_types):
         return [name]
     elif name is None:
         return ['anytrait']
     elif isinstance(name, (list, tuple)):
         for n in name:
-            assert isinstance(n, str), "names must be strings"
+            assert isinstance(n, string_types), "names must be strings"
         return name
 
 
@@ -1193,7 +1193,7 @@ def coerce_str(self, obj, value):
     def validate(self, obj, value):
         value = self.coerce_str(obj, value)
 
-        if isinstance(value, str) and py3compat.isidentifier(value):
+        if isinstance(value, string_types) and py3compat.isidentifier(value):
             return value
         self.error(obj, value)
 
@@ -1202,7 +1202,7 @@ class DottedObjectName(ObjectName):
     def validate(self, obj, value):
         value = self.coerce_str(obj, value)
 
-        if isinstance(value, str) and py3compat.isidentifier(value, dotted=True):
+        if isinstance(value, string_types) and py3compat.isidentifier(value, dotted=True):
             return value
         self.error(obj, value)
 


Test patch:
Repository: ipython/ipython

Issue:
Allow scrolling with j, k on IPython's qtconsole's pager
System/version details:

```
{'commit_hash': 'be0cc2b',
 'commit_source': 'installation',
 'default_encoding': 'UTF-8',
 'ipython_path': '/usr/local/lib/python2.7/dist-packages/IPython',
 'ipython_version': '2.3.0',
 'os_name': 'posix',
 'platform': 'Linux-3.2.0-65-generic-x86_64-with-Ubuntu-12.04-precise',
 'sys_executable': '/usr/bin/python',
 'sys_platform': 'linux2',
 'sys_version': '2.7.3 (default, Feb 27 2014, 19:58:35) \n[GCC 4.6.3]'}
```

IPython has a pager, which comes when I type `?<CR>`, for example, or check for a long docstring. On my standard IPython, the pager allows scrolling with j and k (this is the `LESS` pager). On `qtconsole`, it is not calling that pager, but instead present a page with a side scroller, and I can scroll with the keys, but not with j and k, which is a little bit annoying (only a little bit, but enough to make me report an issue!)



Discussion:

Resolving patch:
diff --git a/IPython/qt/console/console_widget.py b/IPython/qt/console/console_widget.py
--- a/IPython/qt/console/console_widget.py
+++ b/IPython/qt/console/console_widget.py
@@ -1490,6 +1490,22 @@ def _event_filter_page_keypress(self, event):
             QtGui.qApp.sendEvent(self._page_control, new_event)
             return True
 
+        # vi/less -like key bindings
+        elif key == QtCore.Qt.Key_J:
+            new_event = QtGui.QKeyEvent(QtCore.QEvent.KeyPress,
+                                        QtCore.Qt.Key_Down,
+                                        QtCore.Qt.NoModifier)
+            QtGui.qApp.sendEvent(self._page_control, new_event)
+            return True
+
+        # vi/less -like key bindings
+        elif key == QtCore.Qt.Key_K:
+            new_event = QtGui.QKeyEvent(QtCore.QEvent.KeyPress,
+                                        QtCore.Qt.Key_Up,
+                                        QtCore.Qt.NoModifier)
+            QtGui.qApp.sendEvent(self._page_control, new_event)
+            return True
+
         return False
 
     def _on_flush_pending_stream_timer(self):


Test patch:
Repository: ipython/ipython

Issue:
Copying from Qt console appends newline in clipboard
IPython 2.1.0 on Windows

As title says, for example if I copy single word in clipboard I have the word plus '\r\n'.



Discussion:

Resolving patch:
diff --git a/IPython/qt/console/frontend_widget.py b/IPython/qt/console/frontend_widget.py
--- a/IPython/qt/console/frontend_widget.py
+++ b/IPython/qt/console/frontend_widget.py
@@ -215,7 +215,10 @@ def copy(self):
         elif self._control.hasFocus():
             text = self._control.textCursor().selection().toPlainText()
             if text:
+                was_newline = text[-1] == '\n'
                 text = self._prompt_transformer.transform_cell(text)
+                if not was_newline: # user doesn't need newline
+                    text = text[:-1]
                 QtGui.QApplication.clipboard().setText(text)
         else:
             self.log.debug("frontend widget : unknown copy target")


Test patch:
Repository: ipython/ipython

Issue:
Markdown with Mathjax fails during HTML export
This Markdown block is failing for us. (extra `<em>` inserted due to `_`)

```
Water that is stored in $t$, $s_t$, must equal the storage content of the previous stage,
$s_{t-1}$, plus a stochastic inflow, $I_t$, minus what is being released in $t$, $r_t$.
With $s_0$ defined as the initial storage content in $t=1$, we have
```

Is this an issue with #6028 or is the inline Mathjax wrong?

I checked the Regex and they are OK. Maybe the problem is at another place? Any pointers?



Discussion:
Where is it failing ? Nbconvert  or live notebook ? Which version of IPython ? looks fine here : 

![capture d ecran 2014-10-17 a 15 40 13](https://cloud.githubusercontent.com/assets/335567/4679579/2da68be8-5603-11e4-97b6-23f17606e86e.png)

It's fine in the notebook, but fails in nbconvert.

indeed

```
<p>Water that is stored in $t$, $s<em>t$, must equal the storage content of the previous stage,
$s</em>{t-1}$, plus a stochastic inflow, $I_t$, minus what is being released in $t$, $r_t$.
With $s_0$ defined as the initial storage content in $t=1$, we have</p>
```

probably the remove-math and replace math of nbconvert are wrong.


Resolving patch:
diff --git a/IPython/nbconvert/filters/markdown.py b/IPython/nbconvert/filters/markdown.py
--- a/IPython/nbconvert/filters/markdown.py
+++ b/IPython/nbconvert/filters/markdown.py
@@ -96,6 +96,7 @@ def parse_latex_environment(self, m):
 @undoc
 class MathInlineGrammar(mistune.InlineGrammar):
     math = re.compile("^\$(.+?)\$")
+    text = re.compile(r'^[\s\S]+?(?=[\\<!\[_*`~$]|https?://| {2,}\n|$)')
 
 @undoc
 class MathInlineLexer(mistune.InlineLexer):


Test patch:
diff --git a/IPython/nbconvert/filters/tests/test_markdown.py b/IPython/nbconvert/filters/tests/test_markdown.py
--- a/IPython/nbconvert/filters/tests/test_markdown.py
+++ b/IPython/nbconvert/filters/tests/test_markdown.py
@@ -86,6 +86,12 @@ def test_markdown2html_math(self):
         for case in cases:
             self.assertIn(case, markdown2html(case))
 
+    def test_markdown2html_math_paragraph(self):
+        # https://github.com/ipython/ipython/issues/6724
+        a = """Water that is stored in $t$, $s_t$, must equal the storage content of the previous stage,
+$s_{t-1}$, plus a stochastic inflow, $I_t$, minus what is being released in $t$, $r_t$.
+With $s_0$ defined as the initial storage content in $t=1$, we have"""
+        self.assertIn(a, markdown2html(a))
 
     @dec.onlyif_cmds_exist('pandoc')
     def test_markdown2rst(self):
Repository: ipython/ipython

Issue:
QTConsole tooltip placement y-padding missing
![shot](https://cloud.githubusercontent.com/assets/7333933/4716268/a2e51450-5906-11e4-832a-ed3252579b6c.png)
with git master 5ef1976

Upper half is before and bottom is after fix. PR in a moment.

Continuing QTC regression issues from https://github.com/ipython/ipython/issues/6726



Discussion:

Resolving patch:
diff --git a/IPython/qt/console/call_tip_widget.py b/IPython/qt/console/call_tip_widget.py
--- a/IPython/qt/console/call_tip_widget.py
+++ b/IPython/qt/console/call_tip_widget.py
@@ -189,8 +189,9 @@ def show_tip(self, tip):
                 horizontal = 'Left'
         pos = getattr(cursor_rect, '%s%s' %(vertical, horizontal))
         point = text_edit.mapToGlobal(pos())
+        point.setY(point.y() + padding)
         if vertical == 'top':
-            point.setY(point.y() - tip_height - padding)
+            point.setY(point.y() - tip_height)
         if horizontal == 'Left':
             point.setX(point.x() - tip_width - padding)
 


Test patch:
Repository: ipython/ipython

Issue:
Unify codemirror highlight mode
Assigning to myself,

Need to unify the name of the variable that represent the default highlight mode
for each cell type and mode the auto highlight methods as high as possible.



Discussion:
@Carreau what is the status of this one?


Resolving patch:
diff --git a/setupbase.py b/setupbase.py
--- a/setupbase.py
+++ b/setupbase.py
@@ -153,7 +153,6 @@ def find_package_data():
         pjoin(components, "es6-promise", "*.js"),
         pjoin(components, "font-awesome", "fonts", "*.*"),
         pjoin(components, "google-caja", "html-css-sanitizer-minified.js"),
-        pjoin(components, "highlight.js", "build", "highlight.pack.js"),
         pjoin(components, "jquery", "jquery.min.js"),
         pjoin(components, "jquery-ui", "ui", "minified", "jquery-ui.min.js"),
         pjoin(components, "jquery-ui", "themes", "smoothness", "jquery-ui.min.css"),


Test patch:
Repository: ipython/ipython

Issue:
Wrapper Kernel -> `do_shutdown` is not called when the Console is shut down.
If I start a wrapper kernel as such: `ipython qtconsole --kernel=my_kernel`, and exit the app via the "X" button, `do_shutdown` is called as expected.
If I start it as `ipython console --kernel=my_kernel` and exit the app via `Ctrl+D`, `y`, it is not called.  

This can be seen by installing @takluyver's bash_kernel and typing a few commands, closing the kernel properly, and reopening.  When doing this with the `console`, the history is not preserved.  



Discussion:
This problem also occurs when exiting the QtConsole using Ctrl+D.

I'm seeing this on ipython-master with `ipython console`. The python kernel is left running:

```
$ ps
  PID TTY          TIME CMD
 5504 pts/0    00:00:00 ps
23656 pts/0    00:00:03 bash
$ ipython console
IPython Console 3.0.0-dev

In [1]: CONTROL+d
Do you really want to exit ([y]/n)? ENTER
$ ps
  PID TTY          TIME CMD
 5513 pts/0    00:00:00 python
 5532 pts/0    00:00:00 ps
23656 pts/0    00:00:03 bash
```

Every time I run `ipython console` there is another python process added. Debugging info added to do_shutdown() in ipkernel is not executed. Isn't the python kernel supposed to shutdown when exiting from console with no other connections?


Resolving patch:
diff --git a/IPython/qt/console/mainwindow.py b/IPython/qt/console/mainwindow.py
--- a/IPython/qt/console/mainwindow.py
+++ b/IPython/qt/console/mainwindow.py
@@ -205,7 +205,7 @@ def close_tab(self,current_tab):
                         for slave in slave_tabs:
                             background(slave.kernel_client.stop_channels)
                             self.tab_widget.removeTab(self.tab_widget.indexOf(slave))
-                        closing_widget.execute("exit")
+                        kernel_manager.shutdown_kernel()
                         self.tab_widget.removeTab(current_tab)
                         background(kernel_client.stop_channels)
                     elif reply == 0: # close Console
diff --git a/IPython/terminal/console/interactiveshell.py b/IPython/terminal/console/interactiveshell.py
--- a/IPython/terminal/console/interactiveshell.py
+++ b/IPython/terminal/console/interactiveshell.py
@@ -413,6 +413,8 @@ def mainloop(self, display_banner=False):
                 # this should not be necessary, but KeyboardInterrupt
                 # handling seems rather unpredictable...
                 self.write("\nKeyboardInterrupt in interact()\n")
+
+        self.client.shell_channel.shutdown()
     
     def _banner1_default(self):
         return "IPython Console {version}\n".format(version=release.version)


Test patch:
Repository: ipython/ipython

Issue:
IPython.lib.editorhooks broken on windows
The hook that this module installs uses the `pipes` modules' `quote` function to quote the filename to open. Sadly on Windows this function still uses single quote which the shell doesn't understand, instead of double quotes, meaning the hook doesn't work.



Discussion:

Resolving patch:
diff --git a/IPython/lib/editorhooks.py b/IPython/lib/editorhooks.py
--- a/IPython/lib/editorhooks.py
+++ b/IPython/lib/editorhooks.py
@@ -8,6 +8,7 @@
 
 import os
 import pipes
+import shlex
 import subprocess
 
 from IPython import get_ipython
@@ -15,7 +16,7 @@
 from IPython.utils import py3compat
 
 
-def install_editor(template, wait=False):
+def install_editor(template_list, wait=False):
     """Installs the editor that is called by IPython for the %edit magic.
 
     This overrides the default editor, which is generally set by your EDITOR
@@ -48,6 +49,9 @@ def call_editor(self, filename, line=0):
             line = 0
         cmd = template.format(filename=pipes.quote(filename), line=line)
         print(">", cmd)
+        # pipes.quote doesn't work right on Windows, but it does after splitting
+        if sys.platform.startswith('win'):
+            cmd = shlex.split(cmd)
         proc = subprocess.Popen(cmd, shell=True)
         if wait and proc.wait() != 0:
             raise TryNext()


Test patch:
Repository: ipython/ipython

Issue:
console config not written
assigning to @minrk who said: "Oh jeez, I don't wanna fix that right now". Marking it two-point-oh.



Discussion:

Resolving patch:
diff --git a/IPython/core/profileapp.py b/IPython/core/profileapp.py
--- a/IPython/core/profileapp.py
+++ b/IPython/core/profileapp.py
@@ -262,6 +262,7 @@ def init_config_files(self):
         apps = [TerminalIPythonApp]
         for app_path in (
             'IPython.kernel.zmq.kernelapp.IPKernelApp',
+            'IPython.terminal.console.app.ZMQTerminalIPythonApp',
             'IPython.qt.console.qtconsoleapp.IPythonQtConsoleApp',
             'IPython.html.notebookapp.NotebookApp',
             'IPython.nbconvert.nbconvertapp.NbConvertApp',


Test patch:
Repository: ipython/ipython

Issue:
Make KernelSpecManager configurable
I would like to be able to write a custom subclass of `KernelSpecManager` to automatically list all the venvs in my system. I started to try and do this:

https://github.com/Cadair/ipython/blob/custom_kernel_manager/IPython/html/notebookapp.py#L614



Discussion:
I'm reluctant to expose this as a piece of API that people can replace like this, because I think we will want to change the interface for the next release (e.g. to do exactly this kind of thing officially) without feeling bound by maintaining API compatibility for people using custom KernelSpecManagers.

For now, I would recommend that people do this by using the hooks virtualenvwrapper exposes to automatically create a corresponding kernel spec when you create a venv, rather than making IPython aware of your virtualenvs.


Resolving patch:
diff --git a/IPython/html/notebookapp.py b/IPython/html/notebookapp.py
--- a/IPython/html/notebookapp.py
+++ b/IPython/html/notebookapp.py
@@ -616,11 +616,22 @@ def _mathjax_url_changed(self, name, old, new):
     def _kernel_spec_manager_default(self):
         return KernelSpecManager(ipython_dir=self.ipython_dir)
 
+
+    kernel_spec_manager_class = DottedObjectName('IPython.kernel.kernelspec.KernelSpecManager',
+            config=True,
+            help="""
+            The kernel spec manager class to use. Should be a subclass
+            of `IPython.kernel.kernelspec.KernelSpecManager`.
+
+            The Api of KernelSpecManager is provisional and might change
+            without warning between this version of IPython and the next stable one.
+            """)
+
     trust_xheaders = Bool(False, config=True,
         help=("Whether to trust or not X-Scheme/X-Forwarded-Proto and X-Real-Ip/X-Forwarded-For headers"
               "sent by the upstream reverse proxy. Necessary if the proxy handles SSL")
     )
-    
+
     info_file = Unicode()
 
     def _info_file_default(self):
@@ -690,6 +701,9 @@ def init_kernel_argv(self):
     def init_configurables(self):
         # force Session default to be secure
         default_secure(self.config)
+        kls = import_item(self.kernel_spec_manager_class)
+        self.kernel_spec_manager = kls(ipython_dir=self.ipython_dir)
+
         kls = import_item(self.kernel_manager_class)
         self.kernel_manager = kls(
             parent=self, log=self.log, kernel_argv=self.kernel_argv,


Test patch: