Repository: ipython/ipython

Issue:
Kernel/Interrupt Kernel does not terminate stuck subprocesses in the notebook
When a subprocess is run from the notebook, if it gets stuck the kernel will get locked waiting for it. Selecting Kernel/Interrupt from the menu does not terminate the subprocess, but rather leaves the kernel in an unstable, "partially locked" state, where other cells do not execute. The only resolution is to restart the kernel.

This occurred for me on Windows - I do not know if it also happens on Unix.

To demonstrate, start a notebook and enter !python in a cell. The process will lock as it is waiting for interactive input. As there is no way to provide that input, the kernel must be restarted to continue.



Discussion:
duplicate of #514

Thanks, I hadn't spotted the duplicate. Having said that, t#514 is discussing a much more complex scenario, involving actually interacting with subprocesses (and it seems to be Unix based, as it's about pty-style interaction). For my requirements, a simple means of killing a rogue subprocess would do. Consider something as simple as `!sleep 50000`, where just being able to kill the sleep is all you want. (Maybe Ctrl-C works for this on Unix, but it doesn't on Windows).

Sorry, I see what you mean now. Reopening as a separate issue - interrupt not interrupting subprocesses on Windows.

I'm not sure this is limited to subprocesses. Try executing `input()` or `raw_input()` and then clicking the interrupt button--the kernel hangs and has to be restarted.

@arijun on What OS? interrupting input and raw_input raise KeyboardInterrupt here (OS X).

Sorry, windows. That's why I thought it was likely the same issue @pfmoore had, since that also happened on windows.

Ah, crap. I know what that bug is. I think it's a libzmq (or pyzmq) bug that prevents it from handling interrupts properly while polling on zmq sockets. It's nothing in IPython. _sigh_

I think I just got bitten by this and I'll need to restart the kernel, meaning I've just lost a lot of data…

I was using `pdb` to debug a function. I re-ran the cell without first quitting `pdb`, and now I can't interrupt anything.

Here's a minimal example that reproduces this:

``` python
def test():
    import pdb; pdb.set_trace()  # XXX BREAKPOINT
    return 0

test()
```

Run this cell twice in a row.

This same issue happens for me in Unix as well word for word.  

"When a subprocess is run from the notebook, if it gets stuck the kernel will get locked waiting for it. Selecting Kernel/Interrupt from the menu does not terminate the subprocess, but rather leaves the kernel in an unstable, "partially locked" state, where other cells do not execute. The only resolution is to restart the kernel."

Thanks for the nice example of a pdb hang, wmayner. But sInce pdb doesn't run in a subprocess, I opened a separate issue for pdb: #10516 
Printing too much data, let's say accidentally printing a gigantic numpy array, can make the kernel completely unresponsive and impossible to to terminate
Has a solution been found for this issue yet? i just ran a machine learning model that took 14hr to complete and now my kernel is stuck and doesnt execute cells. if i restart, i have to run the model again for 14hrs. So is there any solution?
haven't tried it, but this seems like it could help: http://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/limit_output/readme.html
If a specific subprocess has got stuck, you can probably find it in the task manager and forcibly kill it that way. Hopefully that lets the kernel continue.
no, the issue is that the kernel spams the webserver to death or something. killing the webserver kills the kernel afaik
I'm dealing with a stuck notebook too: interrupt, restart, reconnect - none of them do anything. The `[*]` indicators remain next to cells as if they are queued to run but no cells get executed. 

The behavior began after running a cell containing:
```
filedir = "20161214_rooftest"

!ls -RC $filedir
```

Which is strange because I have analogous cells elsewhere that run successfully. I'm not sure how/if `ls` could get stuck but otherwise my situation seems to match this issue. 
Is there any solution to this . Kernal cannot be interrupted .
For me it's happening with GridSearchCV in sklearn . 
There was a process named conda.exe in Task manager. I killed that process and I was successfully able to interrupt the kernel
Interrupt is still broken. I have to restart and reload my imports every time.
same problem in jupyter lab on python 3.7 kernel

same problem in Jupyter Notebook and I can't find the process named conda.exe in Task manager. Any updates on the solution yet?
**Not a solution**
Sometimes trying to reconnect to the kernel helps in this case
Observing the same, in Windows 10
Did anyone succeed on that? I am getting crazy
> There was a process named conda.exe in Task manager. I killed that process and I was successfully able to interrupt the kernel

@ahmedrao How????
This problem has existed for six years and still no solution. 
> This problem has existed for six years and still no solution.

six years without any solution, just restart the kernel
Having the same problem increasingly frequently, almost to the point where the notebooks are becoming unusable which is a real shame. On Anaconda 3.7 and the cells just hang with the asterisk, and I am unable to interrupt the kernel.
Mark Same Issue
Have always had this problem especially with dbg and input. 
Windows 10; Notebook server 5.7.8; Python 3.6.6.; Conda 4.7.5
Have learned that I basically cannot reliably debug Notebooks :(
yep, the problem still exists. Is there any way to over come this ?? I dont want to run my notebook all over again , because it takes too long to get to where I'm !!
Up!
This problem has been a pain for me for years now every time I use pdb and forget to quit before I re-run the cell.
I created a bounty on BountySource. Maybe this will finally be fixed if we can gather enough money.
https://www.bountysource.com/issues/44958889-hang-after-running-pdb-in-a-cell-kernel-interrupt-doesn-t-help
For the process issue specifically, on Windows specifically, here's a theory (still untested):

1. Process is run via `IPython.utils._process_win32.system`, which calls `_system_body`, which calls `p.wait()` on the `subprocess.Popen` object.
2. Windows `subprocess.Popen.wait()` has a known issue where it is not interruptible: https://bugs.python.org/issue28168

If that's the cause, switching to busy looping every 100ms or so would probably make it interruptible, or if not then taking the approach in the patch.

Resolving patch:
diff --git a/IPython/utils/_process_win32.py b/IPython/utils/_process_win32.py
--- a/IPython/utils/_process_win32.py
+++ b/IPython/utils/_process_win32.py
@@ -18,10 +18,12 @@
 import os
 import sys
 import ctypes
+import time
 
 from ctypes import c_int, POINTER
 from ctypes.wintypes import LPCWSTR, HLOCAL
-from subprocess import STDOUT
+from subprocess import STDOUT, TimeoutExpired
+from threading import Thread
 
 # our own imports
 from ._process_common import read_no_interrupt, process_handler, arg_split as py_arg_split
@@ -93,15 +95,29 @@ def _find_cmd(cmd):
 def _system_body(p):
     """Callback for _system."""
     enc = DEFAULT_ENCODING
-    for line in read_no_interrupt(p.stdout).splitlines():
-        line = line.decode(enc, 'replace')
-        print(line, file=sys.stdout)
-    for line in read_no_interrupt(p.stderr).splitlines():
-        line = line.decode(enc, 'replace')
-        print(line, file=sys.stderr)
 
-    # Wait to finish for returncode
-    return p.wait()
+    def stdout_read():
+        for line in read_no_interrupt(p.stdout).splitlines():
+            line = line.decode(enc, 'replace')
+            print(line, file=sys.stdout)
+
+    def stderr_read():
+        for line in read_no_interrupt(p.stderr).splitlines():
+            line = line.decode(enc, 'replace')
+            print(line, file=sys.stderr)
+
+    Thread(target=stdout_read).start()
+    Thread(target=stderr_read).start()
+
+    # Wait to finish for returncode. Unfortunately, Python has a bug where
+    # wait() isn't interruptible (https://bugs.python.org/issue28168) so poll in
+    # a loop instead of just doing `return p.wait()`.
+    while True:
+        result = p.poll()
+        if result is None:
+            time.sleep(0.01)
+        else:
+            return result
 
 
 def system(cmd):
@@ -116,9 +132,7 @@ def system(cmd):
 
     Returns
     -------
-    None : we explicitly do NOT return the subprocess status code, as this
-    utility is meant to be used extensively in IPython, where any return value
-    would trigger :func:`sys.displayhook` calls.
+    int : child process' exit code.
     """
     # The controller provides interactivity with both
     # stdin and stdout


Test patch:
diff --git a/IPython/utils/tests/test_process.py b/IPython/utils/tests/test_process.py
--- a/IPython/utils/tests/test_process.py
+++ b/IPython/utils/tests/test_process.py
@@ -15,13 +15,19 @@
 #-----------------------------------------------------------------------------
 
 import sys
+import signal
 import os
+import time
+from _thread import interrupt_main  # Py 3
+import threading
+from unittest import SkipTest
 
 import nose.tools as nt
 
 from IPython.utils.process import (find_cmd, FindCmdError, arg_split,
                                    system, getoutput, getoutputerror,
                                    get_output_error_code)
+from IPython.utils.capture import capture_output
 from IPython.testing import decorators as dec
 from IPython.testing import tools as tt
 
@@ -107,6 +113,49 @@ def test_system_quotes(self):
         status = system('%s -c "import sys"' % python)
         self.assertEqual(status, 0)
 
+    def assert_interrupts(self, command):
+        """
+        Interrupt a subprocess after a second.
+        """
+        if threading.main_thread() != threading.current_thread():
+            raise nt.SkipTest("Can't run this test if not in main thread.")
+
+        # Some tests can overwrite SIGINT handler (by using pdb for example),
+        # which then breaks this test, so just make sure it's operating
+        # normally.
+        signal.signal(signal.SIGINT, signal.default_int_handler)
+
+        def interrupt():
+            # Wait for subprocess to start:
+            time.sleep(0.5)
+            interrupt_main()
+
+        threading.Thread(target=interrupt).start()
+        start = time.time()
+        try:
+            result = command()
+        except KeyboardInterrupt:
+            # Success!
+            pass
+        end = time.time()
+        self.assertTrue(
+            end - start < 2, "Process didn't die quickly: %s" % (end - start)
+        )
+        return result
+
+    def test_system_interrupt(self):
+        """
+        When interrupted in the way ipykernel interrupts IPython, the
+        subprocess is interrupted.
+        """
+        def command():
+            return system('%s -c "import time; time.sleep(5)"' % python)
+
+        status = self.assert_interrupts(command)
+        self.assertNotEqual(
+            status, 0, "The process wasn't interrupted. Status: %s" % (status,)
+        )
+
     def test_getoutput(self):
         out = getoutput('%s "%s"' % (python, self.fname))
         # we can't rely on the order the line buffered streams are flushed
@@ -131,7 +180,7 @@ def test_getoutput_error(self):
         out, err = getoutputerror('%s "%s"' % (python, self.fname))
         self.assertEqual(out, 'on stdout')
         self.assertEqual(err, 'on stderr')
-    
+
     def test_get_output_error_code(self):
         quiet_exit = '%s -c "import sys; sys.exit(1)"' % python
         out, err, code = get_output_error_code(quiet_exit)
@@ -142,3 +191,5 @@ def test_get_output_error_code(self):
         self.assertEqual(out, 'on stdout')
         self.assertEqual(err, 'on stderr')
         self.assertEqual(code, 0)
+
+