Repository: ipython/ipython

Issue:
Incompatible Qt imports when running inprocess_qtconsole
There is in inconsistency in how Qt is loaded during the [inprocess_qtconsole.py](https://github.com/ipython/ipython/blob/master/docs/examples/frontend/inprocess_qtconsole.py) example. 

The problem is that `IPython.external.qt.py` and `IPython.external.qt.qt_for_kernel.py` use different logic for choosing between PySide and PyQt4 -- If no `QT_API` environment variable is set and both libraries are installed, the former module loads `PySide` while the latter loads `PyQt4`. 

Both helper modules are imported in the `inprocess_qtconsole` script (from `IPython.lib.guispport` and `IPython.frontend.qt...`) This crashes python.

The only added functionality that `qt_for_kernel.py` seems to provide is apparent support for PyQt4 API v1. However, I suspect this is incompatible with the code in `IPython.frontend.qt`. If this is the case (and API v1 isn't _really_ supported), then the easiest solution is to only use `qt.py`. If V1 is really meant to be supported, `qt.py` should still probably be the only access point to Qt, but the relevant logic from `qt_for_kernel.py` should be ported over.

I can provide a PR with these changes, if desired

cc @MinRK @epatters @mspacek

Incompatible Qt imports when running inprocess_qtconsole
There is in inconsistency in how Qt is loaded during the [inprocess_qtconsole.py](https://github.com/ipython/ipython/blob/master/docs/examples/frontend/inprocess_qtconsole.py) example. 

The problem is that `IPython.external.qt.py` and `IPython.external.qt.qt_for_kernel.py` use different logic for choosing between PySide and PyQt4 -- If no `QT_API` environment variable is set and both libraries are installed, the former module loads `PySide` while the latter loads `PyQt4`. 

Both helper modules are imported in the `inprocess_qtconsole` script (from `IPython.lib.guispport` and `IPython.frontend.qt...`) This crashes python.

The only added functionality that `qt_for_kernel.py` seems to provide is apparent support for PyQt4 API v1. However, I suspect this is incompatible with the code in `IPython.frontend.qt`. If this is the case (and API v1 isn't _really_ supported), then the easiest solution is to only use `qt.py`. If V1 is really meant to be supported, `qt.py` should still probably be the only access point to Qt, but the relevant logic from `qt_for_kernel.py` should be ported over.

I can provide a PR with these changes, if desired

cc @MinRK @epatters @mspacek



Discussion:
There's an important difference between qt for the frontend and qt for the backend.  The frontend is for use in IPython.frontend.qt, which really does depend on v2 (v1 is not and will not be supported).  The backend, on the other hand, is for compatibility with _user_ code, where it is important to support the _default_ behavior of PyQt (v1).  When these were written, there was no chance of the kernel and the frontend being in the same process (since inprocess kernels miss most of the point of the whole IPython architecture).

Since the frontend does require v2, the inprocess kernel should actually be forced to follow the plain `qt` logic, since it is the more restrictive case.

I think the actual fix to introduce that is simply enforce a fallback on qt from qt_for_kernel if qt has been imported.
That should be solved by adding a simple extra condition to check whether qt has been imported already, and should take precedence [here](https://github.com/ipython/ipython/blob/master/IPython/external/qt_for_kernel.py#L45):

``` python
    if QT_API is None and not 'IPython.external.qt' in sys.modules:
```

What about the case where qt_from_kernel is imported before qt? Couldn't `qt.py` could still re-import a potentially incompatible Qt binding in that scenario? Furthermore, user code could also import a Qt binding. At a minimum, an equivalent test would need to be added to `qt.py` (and probably better to check for `PyQt4.QtGui` and/or `PySide.QtGui` in sys.modules)

IMO, the logic that chooses which binding to load is mixed with the transparent import logic, and duplicated between qt.py and qt_for_kernel.py. What about some factory functions in a separate module `qt_loaders.py` that import and return modules from PyQt4 (v1/v2)/PySide? Then qt.py and qt_for_kernel.py could focus on what binding they prefer to load, and leave it to `qt_loader.py` to conjour QtGui et al., or raise an informative ImportError?

> What about the case where qt_from_kernel is imported before qt?

The in-process qt kernel is the only case where that can happen, so it should just be a matter of ensuring that qt is imported first.

If you want to clean up the import code, go ahead (it's a mess), just note that the actual logic / priorities for the frontend and the kernel shouldn't change.

There's an important difference between qt for the frontend and qt for the backend.  The frontend is for use in IPython.frontend.qt, which really does depend on v2 (v1 is not and will not be supported).  The backend, on the other hand, is for compatibility with _user_ code, where it is important to support the _default_ behavior of PyQt (v1).  When these were written, there was no chance of the kernel and the frontend being in the same process (since inprocess kernels miss most of the point of the whole IPython architecture).

Since the frontend does require v2, the inprocess kernel should actually be forced to follow the plain `qt` logic, since it is the more restrictive case.

I think the actual fix to introduce that is simply enforce a fallback on qt from qt_for_kernel if qt has been imported.
That should be solved by adding a simple extra condition to check whether qt has been imported already, and should take precedence [here](https://github.com/ipython/ipython/blob/master/IPython/external/qt_for_kernel.py#L45):

``` python
    if QT_API is None and not 'IPython.external.qt' in sys.modules:
```

What about the case where qt_from_kernel is imported before qt? Couldn't `qt.py` could still re-import a potentially incompatible Qt binding in that scenario? Furthermore, user code could also import a Qt binding. At a minimum, an equivalent test would need to be added to `qt.py` (and probably better to check for `PyQt4.QtGui` and/or `PySide.QtGui` in sys.modules)

IMO, the logic that chooses which binding to load is mixed with the transparent import logic, and duplicated between qt.py and qt_for_kernel.py. What about some factory functions in a separate module `qt_loaders.py` that import and return modules from PyQt4 (v1/v2)/PySide? Then qt.py and qt_for_kernel.py could focus on what binding they prefer to load, and leave it to `qt_loader.py` to conjour QtGui et al., or raise an informative ImportError?

> What about the case where qt_from_kernel is imported before qt?

The in-process qt kernel is the only case where that can happen, so it should just be a matter of ensuring that qt is imported first.

If you want to clean up the import code, go ahead (it's a mess), just note that the actual logic / priorities for the frontend and the kernel shouldn't change.


Resolving patch:
diff --git a/IPython/external/qt.py b/IPython/external/qt.py
--- a/IPython/external/qt.py
+++ b/IPython/external/qt.py
@@ -7,75 +7,17 @@
 """
 
 import os
-from IPython.utils.version import check_version
-# Available APIs.
-QT_API_PYQT = 'pyqt'
-QT_API_PYSIDE = 'pyside'
 
-def prepare_pyqt4():
-    # For PySide compatibility, use the new-style string API that automatically
-    # converts QStrings to Unicode Python strings. Also, automatically unpack
-    # QVariants to their underlying objects.
-    import sip
-    sip.setapi('QString', 2)
-    sip.setapi('QVariant', 2)
+from IPython.external.qt_loaders import (load_qt, QT_API_PYSIDE,
+                                         QT_API_PYQT)
 
-# Select Qt binding, using the QT_API environment variable if available.
-QT_API = os.environ.get('QT_API')
+QT_API = os.environ.get('QT_API', None)
+if QT_API not in [QT_API_PYSIDE, QT_API_PYQT, None]:
+    raise RuntimeError("Invalid Qt API %r, valid values are: %r, %r" %
+                       (QT_API, QT_API_PYSIDE, QT_API_PYQT))
 if QT_API is None:
-    pyside_found = False
-    try:
-        import PySide
-        if not check_version(PySide.__version__, '1.0.3'):
-            # old PySide, fallback on PyQt
-            raise ImportError
-        # we can't import an incomplete pyside and pyqt4
-        # this will cause a crash in sip (#1431)
-        # check for complete presence before importing
-        import imp
-        imp.find_module("QtCore", PySide.__path__)
-        imp.find_module("QtGui", PySide.__path__)
-        imp.find_module("QtSvg", PySide.__path__)
-        pyside_found = True
-        from PySide import QtCore, QtGui, QtSvg
-        QT_API = QT_API_PYSIDE
-    except ImportError:
-        try:
-            prepare_pyqt4()
-            import PyQt4
-            from PyQt4 import QtCore, QtGui, QtSvg
-            if pyside_found:
-                print "WARNING: PySide installation incomplete and PyQt4 " \
-                      "present.\nThis will likely crash, please install " \
-                      "PySide completely, remove PySide or PyQt4 or set " \
-                      "the QT_API environment variable to pyqt or pyside"
-            if not check_version(QtCore.PYQT_VERSION_STR, '4.7'):
-                # PyQt 4.6 has issues with null strings returning as None
-                raise ImportError
-            QT_API = QT_API_PYQT
-        except ImportError:
-            raise ImportError('Cannot import PySide >= 1.0.3 or PyQt4 >= 4.7')
-
-elif QT_API == QT_API_PYQT:
-    # Note: This must be called *before* PyQt4 is imported.
-    prepare_pyqt4()
-
-# Now peform the imports.
-if QT_API == QT_API_PYQT:
-    from PyQt4 import QtCore, QtGui, QtSvg
-    if not check_version(QtCore.PYQT_VERSION_STR, '4.7'):
-        raise ImportError("IPython requires PyQt4 >= 4.7, found %s"%QtCore.PYQT_VERSION_STR)
-
-    # Alias PyQt-specific functions for PySide compatibility.
-    QtCore.Signal = QtCore.pyqtSignal
-    QtCore.Slot = QtCore.pyqtSlot
-
-elif QT_API == QT_API_PYSIDE:
-    import PySide
-    if not check_version(PySide.__version__, '1.0.3'):
-        raise ImportError("IPython requires PySide >= 1.0.3, found %s"%PySide.__version__)
-    from PySide import QtCore, QtGui, QtSvg
-
+    api_opts = [QT_API_PYSIDE, QT_API_PYQT]
 else:
-    raise RuntimeError('Invalid Qt API %r, valid values are: %r or %r' %
-                       (QT_API, QT_API_PYQT, QT_API_PYSIDE))
+    api_opts = [QT_API]
+
+QtCore, QtGui, QtSvg, QT_API = load_qt(api_opts)
diff --git a/IPython/external/qt_for_kernel.py b/IPython/external/qt_for_kernel.py
--- a/IPython/external/qt_for_kernel.py
+++ b/IPython/external/qt_for_kernel.py
@@ -4,6 +4,9 @@
 
 Import Priority:
 
+if Qt4 has been imported anywhere else:
+   use that
+
 if matplotlib has been imported and doesn't support v2 (<= 1.0.1):
     use PyQt4 @v1
 
@@ -33,56 +36,49 @@
 
 from IPython.utils.warn import warn
 from IPython.utils.version import check_version
+from IPython.external.qt_loaders import (load_qt, QT_API_PYSIDE,
+                                         QT_API_PYQT, QT_API_PYQT_DEFAULT,
+                                         loaded_api)
 
-matplotlib = sys.modules.get('matplotlib')
-if matplotlib and not check_version(matplotlib.__version__, '1.0.2'):
-    # 1.0.1 doesn't support pyside or v2, so stick with PyQt @v1,
-    # and ignore everything else
-    from PyQt4 import QtCore, QtGui
-else:
-    # ask QT_API ETS variable *first*
-    QT_API = os.environ.get('QT_API', None)
-    if QT_API is None:
-        # QT_API not set, ask matplotlib if it was imported (e.g. `pylab=qt`)
-        if matplotlib:
-            mpqt = matplotlib.rcParams.get('backend.qt4', None)
-        else:
-            mpqt = None
-        if mpqt is None:
-            # matplotlib not imported or had nothing to say.
-            try:
-                # default to unconfigured PyQt4
-                from PyQt4 import QtCore, QtGui
-            except ImportError:
-                # fallback on PySide
-                try:
-                    from PySide import QtCore, QtGui
-                except ImportError:
-                    raise ImportError('Cannot import PySide or PyQt4')
-        elif mpqt.lower() == 'pyqt4':
-            # import PyQt4 unconfigured
-            from PyQt4 import QtCore, QtGui
-        elif mpqt.lower() == 'pyside':
-            from PySide import QtCore, QtGui
-        else:
-            raise ImportError("unhandled value for backend.qt4 from matplotlib: %r"%mpqt)
-    else:
-        # QT_API specified, use PySide or PyQt+v2 API from external.qt
-        # this means ETS is likely to be used, which requires v2
-        try:
-            from IPython.external.qt import QtCore, QtGui
-        except ValueError as e:
-            if 'API' in str(e):
-                # PyQt4 already imported, and APIv2 couldn't be set
-                # Give more meaningful message, and warn instead of raising
-                warn("""
-    Assigning the ETS variable `QT_API=pyqt` implies PyQt's v2 API for
-    QString and QVariant, but PyQt has already been imported
-    with v1 APIs.  You should unset QT_API to work with PyQt4
-    in its default mode.
-""")
-                # allow it to still work
-                from PyQt4 import QtCore, QtGui
-            else:
-                raise
+#Constraints placed on an imported matplotlib
+def matplotlib_options(mpl):
+    if mpl is None:
+        return
+    mpqt = mpl.rcParams.get('backend.qt4', None)
+    if mpqt is None:
+        return None
+    if mpqt.lower() == 'pyside':
+        return [QT_API_PYSIDE]
+    elif mpqt.lower() == 'pyqt4':
+        return [QT_API_PYQT_DEFAULT]
+    raise ImportError("unhandled value for backend.qt4 from matplotlib: %r" %
+                      mpqt)
+
+def get_options():
+    """Return a list of acceptable QT APIs, in decreasing order of
+    preference
+    """
+    #already imported Qt somewhere. Use that
+    loaded = loaded_api()
+    if loaded is not None:
+        return [loaded]
+
+    mpl = sys.modules.get('matplotlib', None)
+
+    if mpl is not None and not check_version(mpl.__version__, '1.0.2'):
+        #1.0.1 only supports PyQt4 v1
+        return [QT_API_PYQT_DEFAULT]
+
+    if os.environ.get('QT_API', None) is None:
+        #no ETS variable. Ask mpl, then use either
+        return matplotlib_options(mpl) or [QT_API_PYQT_DEFAULT, QT_API_PYSIDE]
+
+    #ETS variable present. Will fallback to external.qt
+    return None
+
+api_opts = get_options()
+if api_opts is not None:
+    QtCore, QtGui, QtSvg, QT_API = load_qt(api_opts)
 
+else: # use ETS variable
+    from IPython.external.qt import QtCore, QtGui, QtSvg, QT_API
diff --git a/IPython/external/qt_loaders.py b/IPython/external/qt_loaders.py
new file mode 100644
--- /dev/null
+++ b/IPython/external/qt_loaders.py
@@ -0,0 +1,258 @@
+"""
+This module contains factory functions that attempt
+to return Qt submodules from the various python Qt bindings.
+
+It also protects against double-importing Qt with different
+bindings, which is unstable and likely to crash
+
+This is used primarily by qt and qt_for_kernel, and shouldn't
+be accessed directly from the outside
+"""
+import sys
+from functools import partial
+
+from IPython.utils.version import check_version
+
+# Available APIs.
+QT_API_PYQT = 'pyqt'
+QT_API_PYQTv1 = 'pyqtv1'
+QT_API_PYQT_DEFAULT = 'pyqtdefault' # don't set SIP explicitly
+QT_API_PYSIDE = 'pyside'
+
+
+class ImportDenier(object):
+    """Import Hook that will guard against bad Qt imports
+    once IPython commits to a specific binding
+    """
+
+    def __init__(self):
+        self.__forbidden = None
+
+    def forbid(self, module_name):
+        sys.modules.pop(module_name, None)
+        self.__forbidden = module_name
+
+    def find_module(self, mod_name, pth):
+        if pth:
+            return
+        if mod_name == self.__forbidden:
+            return self
+
+    def load_module(self, mod_name):
+        raise ImportError("""
+    Importing %s disabled by IPython, which has
+    already imported an Incompatible QT Binding: %s
+    """ % (mod_name, loaded_api()))
+
+ID = ImportDenier()
+sys.meta_path.append(ID)
+
+
+def commit_api(api):
+    """Commit to a particular API, and trigger ImportErrors on subsequent
+       dangerous imports"""
+
+    if api == QT_API_PYSIDE:
+        ID.forbid('PyQt4')
+    else:
+        ID.forbid('PySide')
+
+
+def loaded_api():
+    """Return which API is loaded, if any
+
+    If this returns anything besides None,
+    importing any other Qt binding is unsafe.
+
+    Returns
+    -------
+    None, 'pyside', 'pyqt', or 'pyqtv1'
+    """
+    if 'PyQt4.QtCore' in sys.modules:
+        if qtapi_version() == 2:
+            return QT_API_PYQT
+        else:
+            return QT_API_PYQTv1
+    elif 'PySide.QtCore' in sys.modules:
+        return QT_API_PYSIDE
+    return None
+
+
+def has_binding(api):
+    """Safely check for PyQt4 or PySide, without importing
+       submodules
+
+       Parameters
+       ----------
+       api : str [ 'pyqtv1' | 'pyqt' | 'pyside' | 'pyqtdefault']
+            Which module to check for
+
+       Returns
+       -------
+       True if the relevant module appears to be importable
+    """
+    # we can't import an incomplete pyside and pyqt4
+    # this will cause a crash in sip (#1431)
+    # check for complete presence before importing
+    module_name = {QT_API_PYSIDE: 'PySide',
+                   QT_API_PYQT: 'PyQt4',
+                   QT_API_PYQTv1: 'PyQt4',
+                   QT_API_PYQT_DEFAULT: 'PyQt4'}
+    module_name = module_name[api]
+
+    import imp
+    try:
+        #importing top level PyQt4/PySide module is ok...
+        mod = __import__(module_name)
+        #...importing submodules is not
+        imp.find_module('QtCore', mod.__path__)
+        imp.find_module('QtGui', mod.__path__)
+        imp.find_module('QtSvg', mod.__path__)
+
+        #we can also safely check PySide version
+        if api == QT_API_PYSIDE:
+            return check_version(mod.__version__, '1.0.3')
+        else:
+            return True
+    except ImportError:
+        return False
+
+
+def qtapi_version():
+    """Return which QString API has been set, if any
+
+    Returns
+    -------
+    The QString API version (1 or 2), or None if not set
+    """
+    try:
+        import sip
+    except ImportError:
+        return
+    try:
+        return sip.getapi('QString')
+    except ValueError:
+        return
+
+
+def can_import(api):
+    """Safely query whether an API is importable, without importing it"""
+    if not has_binding(api):
+        return False
+
+    current = loaded_api()
+    if api == QT_API_PYQT_DEFAULT:
+        return current in [QT_API_PYQT, QT_API_PYQTv1, None]
+    else:
+        return current in [api, None]
+
+
+def import_pyqt4(version=2):
+    """
+    Import PyQt4
+
+    Parameters
+    ----------
+    version : 1, 2, or None
+      Which QString/QVariant API to use. Set to None to use the system
+      default
+
+    ImportErrors rasied within this function are non-recoverable
+    """
+    # The new-style string API (version=2) automatically
+    # converts QStrings to Unicode Python strings. Also, automatically unpacks
+    # QVariants to their underlying objects.
+    import sip
+
+    if version is not None:
+        sip.setapi('QString', version)
+        sip.setapi('QVariant', version)
+
+    from PyQt4 import QtGui, QtCore, QtSvg
+
+    if not check_version(QtCore.PYQT_VERSION_STR, '4.7'):
+        raise ImportError("IPython requires PyQt4 >= 4.7, found %s" %
+                          QtCore.PYQT_VERSION_STR)
+
+    # Alias PyQt-specific functions for PySide compatibility.
+    QtCore.Signal = QtCore.pyqtSignal
+    QtCore.Slot = QtCore.pyqtSlot
+
+    # query for the API version (in case version == None)
+    version = sip.getapi('QString')
+    api = QT_API_PYQTv1 if version == 1 else QT_API_PYQT
+    return QtCore, QtGui, QtSvg, api
+
+
+def import_pyside():
+    """
+    Import PySide
+
+    ImportErrors raised within this function are non-recoverable
+    """
+    from PySide import QtGui, QtCore, QtSvg
+    return QtCore, QtGui, QtSvg, QT_API_PYSIDE
+
+
+def load_qt(api_options):
+    """
+    Attempt to import Qt, given a preference list
+    of permissible bindings
+
+    It is safe to call this function multiple times.
+
+    Parameters
+    ----------
+    api_options: List of strings
+        The order of APIs to try. Valid items are 'pyside',
+        'pyqt', and 'pyqtv1'
+
+    Returns
+    -------
+
+    A tuple of QtCore, QtGui, QtSvg, QT_API
+    The first three are the Qt modules. The last is the
+    string indicating which module was loaded.
+
+    Raises
+    ------
+    ImportError, if it isn't possible to import any requested
+    bindings (either becaues they aren't installed, or because
+    an incompatible library has already been installed)
+    """
+    loaders = {QT_API_PYSIDE: import_pyside,
+               QT_API_PYQT: import_pyqt4,
+               QT_API_PYQTv1: partial(import_pyqt4, version=1),
+               QT_API_PYQT_DEFAULT: partial(import_pyqt4, version=None)
+               }
+
+    for api in api_options:
+
+        if api not in loaders:
+            raise RuntimeError(
+                "Invalid Qt API %r, valid values are: %r, %r, %r, %r" %
+                (api, QT_API_PYSIDE, QT_API_PYQT,
+                 QT_API_PYQTv1, QT_API_PYQT_DEFAULT))
+
+        if not can_import(api):
+            continue
+
+        #cannot safely recover from an ImportError during this
+        result = loaders[api]()
+        api = result[-1]  # changed if api = QT_API_PYQT_DEFAULT
+        commit_api(api)
+        return result
+    else:
+        raise ImportError("""
+    Could not load requested Qt binding. Please ensure that
+    PyQt4 >= 4.7 or PySide >= 1.0.3 is available,
+    and only one is imported per session.
+
+    Currently-imported Qt library:   %r
+    PyQt4 installed:                 %s
+    PySide >= 1.0.3 installed:       %s
+    Tried to load:                   %r
+    """ % (loaded_api(),
+           has_binding(QT_API_PYQT),
+           has_binding(QT_API_PYSIDE),
+           api_options))


Test patch: