Repository: ipython/ipython

Issue:
ipython console ignoring some stream messages?
In JuliaLang/IJulia.jl#153, @cbecker commented that `ipython console` does not seem to get standard output (which is sent to the console via `stream` messages according to the message spec).  I confirmed this:

```
$ ipython console --profile julia 
Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
Type "copyright", "credits" or "license" for more information.

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

IPython profile: julia
Starting kernel event loops.

In [1]: 

In [1]: ccall(:printf, Cint, (Ptr{Uint8},), "Hello world!\n")
Out[1]: 13
```

It works (prints `"Hello world!"`) in both the `qtconsole` and `notebook` front-ends, so it seems like it may be a bug in the `console` front-end.



Discussion:
Something weird is happening in the messages being sent. I get all the outputs after the first:

```
In [1]: ccall(:printf, Cint, (Ptr{Uint8},), "Hello world!\n")
Out[1]: 13

In [2]: ccall(:printf, Cint, (Ptr{Uint8},), "Hello world!\n")
Out[2]: 13
Hello world!

In [3]: ccall(:printf, Cint, (Ptr{Uint8},), "Hello world!\n")
Out[3]: 13
Hello world!
```

No stream message arrives from the first execution.

So stream messages are definitely handled properly. But when I inspect the traffic, I see stream messages arriving after status-idle. This shouldn't ever happen - the idle status message should always be the last message on the IOPub channel. It looks unique to ccall, because `println("hello")` always works correctly. How is ccall output captured?

Julia provides a way to redirect all standard output (and standard error) to a "pipe" object, including stdio from external C routines.  A background task blocks on this pipe: whenever output is written, it wakes up and sends a `stream` message to the front-end, then sleeps for 100ms (to allow output to accumulate).  At the end of an execute request, before the `idle` message is sent, however, we flush stdio and send anything in the buffer to the front-end immediately.

Is it possible that the ZMQ messages are just arriving out of order?

Either that, or it is some side-effect of stdio buffering which makes the output not show up in the pipe until after execution finished. (cc: @loladiro).

In any case, Julia can certainly send I/O after the `idle` status is sent, because the user can start an asynchronous background task.  For example,

```
@async for i = 1:10
    println(i)
    sleep(1)
end
```

prints the integers from `1` to `10`, once per second, running in the background (in a green thread), so the stream messages are sent after the execution "finishes".  This works fine in the notebook.


Resolving patch:
diff --git a/IPython/terminal/console/interactiveshell.py b/IPython/terminal/console/interactiveshell.py
--- a/IPython/terminal/console/interactiveshell.py
+++ b/IPython/terminal/console/interactiveshell.py
@@ -159,6 +159,8 @@ def run_cell(self, cell, store_history=True):
           should be set to False.
         """
         if (not cell) or cell.isspace():
+            # pressing enter flushes any pending display
+            self.handle_iopub()
             return
 
         if cell.strip() == 'exit':
@@ -180,7 +182,6 @@ def run_cell(self, cell, store_history=True):
             except Empty:
                 # display intermediate print statements, etc.
                 self.handle_iopub(msg_id)
-                pass
         
         # after all of that is done, wait for the execute reply
         while self.client.is_alive():
@@ -222,26 +223,22 @@ def handle_execute_reply(self, msg_id, timeout=None):
             self.execution_count = int(content["execution_count"] + 1)
 
 
-    def handle_iopub(self, msg_id):
-        """ Method to process subscribe channel's messages
+    def handle_iopub(self, msg_id=''):
+        """Process messages on the IOPub channel
 
            This method consumes and processes messages on the IOPub channel,
            such as stdout, stderr, pyout and status.
            
-           It only displays output that is caused by the given msg_id
+           It only displays output that is caused by this session.
         """
         while self.client.iopub_channel.msg_ready():
             sub_msg = self.client.iopub_channel.get_msg()
             msg_type = sub_msg['header']['msg_type']
             parent = sub_msg["parent_header"]
-            if (not parent) or msg_id == parent['msg_id']:
+            
+            if parent.get("session", self.session_id) == self.session_id:
                 if msg_type == 'status':
-                    state = self._execution_state = sub_msg["content"]["execution_state"]
-                    # idle messages mean an individual sequence is complete,
-                    # so break out of consumption to allow other things to take over.
-                    if state == 'idle':
-                        break
-
+                    self._execution_state = sub_msg["content"]["execution_state"]
                 elif msg_type == 'stream':
                     if sub_msg["content"]["name"] == "stdout":
                         print(sub_msg["content"]["data"], file=io.stdout, end="")


Test patch: