Q:

Waiting for output from a subprocess which does not terminate

I need to run a subprocess from my script.  The subprocess is an interactive (shell-like) application, to which I issue commands through the subprocess' stdin. 
After I issue a command, the subprocess outputs the result to stdout and then waits for the next command (but does not terminate).   
For example:
from subprocess import Popen, PIPE
p = Popen(args = [...], stdin = PIPE, stdout = PIPE, stderr = PIPE, shell = False)
# Issue a command:
p.stdin.write('command\n')
# *** HERE: get the result from p.stdout ***
# CONTINUE with the rest of the script once there is not more data in p.stdout
# NOTE that the subprocess is still running and waiting for the next command
# through stdin. 

My problem is getting the result from p.stdout.  The script needs to get the output while there is new data in p.stdout; but once there is no more data, I want to continue with the script. 
The subprocess does not terminate, so I cannot use communicate() (which waits for the process to terminate). 
I tried reading from p.stdout after issuing the command, like this:  
res = p.stdout.read()

But the subprocess is not fast enough, and I just get empty result. 
I thought about polling p.stdout in a loop until I get something, but then how do I know I got everything?  And it seems wasteful anyway. 
Any suggestions? 

A:

After investigating several options I reached two solutions:

Setting the subprocess' stdout stream to be non blocking by using the fcntl module. 
Using a thread to collect the subprocess' output to a proxy queue, and then reading the queue from the main thread. 

I describe both solutions (and the problem and its origin) in this post.