Repository: psychopy/psychopy

Issue:
import of iohub overrides stdout resulting in swallowed print outputs in IPython notebooks
look at the gory details here:
http://nbviewer.ipython.org/url/www.onerussian.com/tmp/iohub-swallows-stdout.ipynb
making it difficult to teach psychopy using ipython notebooks ;)



Discussion:
@isolver Is iohub reassigning sys.stdout to something else?

It is, but only for the iohub server process, not the experiment / psychopy process. 

Looking at the linked "gory" details, it seems that ipython is now also overriding stdout to use "IPython.kernel.zmq.iostream.OutStream" so that subprocess prints get printed from the ipython server process via the notebook display web interface api or something like that. 

My guess is that this /interaction/ has been done in a somewhat recent version of ipython, since people like Richard have not mentioned an issue with printing from psychopy in ipython when using iohub before.

Regardless, how to fix will need to be looked into and may not be a 10 minute job; been so long since I did that code I have no idea really. Seems like it will require changing the iohub code so that stdout / stderr is not being piped by iohub process anymore; iohub currently does this for: 
1. some simple IPC before the UDP interface is established when the iohub server is first started
2. printing to the psychopy process stderr from the separate iohub process. 

As a guess, 1. should be not too hard to change. 2. will require more effort / thought; perhaps switching things so that the iohub process can only send txt to the psychopy log and not print it out.

I'll try and look into this over the next week or so; if anyone else has time to, and finds a fix before I post any update, please let me know.

Thanks.

Here is a hack fix that can be done in ipython to get around the issue of iohub and ipython both redirecting sys.stdout. 

Before importing iohub add:

``` python
    import sys

    #print "works"

    ipy_stdout = sys.stdout
```

Then do your iohub import stuff and reassign sys.stdout to the ipython zmq connection object 'ipy_stdout ' created above:

``` python
    from psychopy.iohub import launchHubServer

    #print "Would not Work. ;)"
    # io = launchHubServer()
    # io.getEvents()

    sys.stdout = ipy_stdout

    #print "still works"
    #io.getEvents()
```

Printing will work in ipython and iohub server device event access seems fine. Let me know if any issues are not addressed by this patch.

I'll look into detecting if the python env. is running within ipython and then I can do this fix within the iohub code itself so an end user will not have to do anything special. For now, adding 2 lines of code at the start of an ipython session is not too big of an issue I hope.

such a workaround I am already using ;)  but imho iohub shouldn't provide "ipython specific" workaround but rather return "original" sys.stdout assignment back regardless what was that original assignment was (i.e. similar to the proposed workaround).  or am I missing a bigger picture?

>  imho iohub shouldn't provide "ipython specific" workaround but rather
> return "original" sys.stdout assignment back regardless what was that
> original assignment was (i.e. similar to the proposed workaround). or am I
> missing a bigger picture?
> 
> Since outside ipython there are no (known) issues with the current iohub
> stdout mod, which was presumably put there for a reason, I think the "If it
> isn't broken, don't fix it." rule is the safest route for now. So, for the
> short term, IMO the ipython conditional logic makes most sense.

Longer term, when there is a good level of confidence that doing the
ipython stdout fix is not adding bugs to non ipython env's when iohub is
being used, I agree that having a common code base makes most sense.


this issue would re-appear in any scenario folks would like to provide alternative handling to stdout, which noone actually forbidden, e.g.

``` python
import sys
import StringIO
sys.stdout = StringIO.StringIO()
my_stdout = sys.stdout
import psychopy.iohub
print "old stdout ", my_stdout
print sys.stdout
```

but got intrigued what is actually going on ;)  apparently this all due to `reload(sys)` to overcome absence of setdefaultencoding (which gets removed from sys deliberately by site.py)  while process is already running... see eg  http://uucode.com/blog/2007/03/23/shut-up-you-dummy-7-bit-python/

bleh... I am really not sure which one of the two (removing setdefaultencoding) or using reload is the ugliest hack here ;)     that functionality was added a while back:

```
commit 5b8cd288210c0483c0a9bb97469a366709fc0c9b
Author: Marc-André Lemburg <mal@egenix.com>
Date:   Wed Jun 7 09:12:09 2000 +0000

    Marc-Andre Lemburg <mal@lemburg.com>:
    Added support to set the default encoding of strings
    at startup time to the values defined by the C locale.

    The sys.setdefaultencoding() API is deleted after having
    set up the encoding, so that user code cannot subsequentely
    change the setting. This effectively means that only site.py
    may alter the default setting.
```

and setdefaultencoding  was completely removed in v3.2a2~92 (so I expect iohub code break there with python3) since utf-8 is "the default"

So -- why not then to guard around that reload at least for restoring stdout/stderr? (who knows what awkard other side-effects though would be left ;) )


Resolving patch:
diff --git a/psychopy/iohub/util/fix_encoding.py b/psychopy/iohub/util/fix_encoding.py
--- a/psychopy/iohub/util/fix_encoding.py
+++ b/psychopy/iohub/util/fix_encoding.py
@@ -38,11 +38,20 @@ def fix_default_encoding():
   if sys.getdefaultencoding() == 'utf-8':
     return False
 
-  # Regenerate setdefaultencoding.
-  reload(sys)
-  # Module 'sys' has no 'setdefaultencoding' member
-  # pylint: disable=E1101
-  sys.setdefaultencoding('utf-8')
+  # setdefaultencoding is removed by site.py to prevent change of the
+  # default encoding.  Let's import "vanilla" sys and invoke it
+  # we need to remove current sys from known, so __import__ is in effect
+  sys_modules = sys.modules
+  our_sys = sys_modules.pop('sys')
+  try:
+    vanilla_sys = __import__('sys')
+    # Module 'sys' has no 'setdefaultencoding' member
+    # pylint: disable=E1101
+    vanilla_sys.setdefaultencoding('utf-8')
+    assert(sys.getdefaultencoding() == 'utf-8')
+  finally:
+    sys_modules['sys'] = our_sys
+
   for attr in dir(locale):
     if attr[0:3] != 'LC_':
       continue


Test patch: