Repository: vispy/vispy

Issue:
Vispy wrongly detects Spyder as Jupyter notebook
https://github.com/vispy/vispy/blob/a6ec64bb7b56739b463e3c16245a31940ca4f5f7/vispy/app/application.py#L131


```python
Python 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)
Type "copyright", "credits" or "license" for more information.

IPython 7.8.0 -- An enhanced Interactive Python.

ip = get_ipython()

ip.has_trait('kernel')
Out[2]: True

ip
Out[3]: <ipykernel.zmqshell.ZMQInteractiveShell at 0x7f8c103ec2b0>

ip.config
Out[4]: 
{'InteractiveShell': {'xmode': 'Plain'},
 'IPCompleter': {'use_jedi': False, 'greedy': False},
 'IPKernelApp': {'exec_lines': ['',
   "import sys;sys.argv = [''];del sys",
   'get_ipython().kernel._load_autoreload_magic()',
   'get_ipython().kernel._load_wurlitzer()',
   "get_ipython().kernel._set_mpl_backend('qt5', False)"],
  'connection_file': '/run/user/1000/jupyter/kernel-7c84aada0ff3.json'},
 'InlineBackend': {'rc': {'figure.figsize': (6.0, 4.0),
   'figure.dpi': 72,
   'font.size': 10,
   'figure.subplot.bottom': 0.125,
   'figure.facecolor': 'white',
   'figure.edgecolor': 'white'}},
 'ZMQInteractiveShell': {'autocall': 0, 'banner1': ''}}
```

![image](https://user-images.githubusercontent.com/90008/64825685-225a6d00-d58c-11e9-9a06-b2740b3209c7.png)

```python
vispy.__version__
Out[6]: '0.6.1'
```

@ccordoba12 any ideas?

@jni because I think you are using vispy these days????


Discussion:
The workaround is to have 
```
import vispy
vispy.use('pyqt5')
```
but that only happens if you remember to use this before starting any kind of application.
Potentially use `get_ipython().kernel.app` ???

<details> <summary>On Spyder</summary>

```python
kernel = get_ipython().kernel

kernel
Out[9]: <spyder_kernels.console.kernel.SpyderKernel at 0x7f8c0ee309e8>

kernel.app
Out[11]: <PyQt5.QtWidgets.QApplication at 0x7f8c0c0bb3a8>
```
</details>

<details> <summary> Jupyter Notebook </summary>

![image](https://user-images.githubusercontent.com/90008/64827302-a8c57d80-d591-11e9-8341-a8f7c28c9c1f.png)

</details>
We use vispy at napari but we use %gui=qt in notebooks, and I presume also in Spyder. (I have the equivalent command in my ipython config.)
@hmaarrfk In your original post, was that ipython output or spyder?

On my linux system in ipython:

```
In [3]: ip                                                                                                                                                                                                                                                    
Out[3]: <IPython.terminal.interactiveshell.TerminalInteractiveShell at 0x7f7b02118dd8>
```

On same linux system with jupyter notebook and jupyter lab:

```
<ipykernel.zmqshell.ZMQInteractiveShell at 0x7f7f8cd46080>
```

Is there a jupyter dev that we can ask for the most reliable way of doing this? Maybe the ipywidgets folks know, I'll ping their gitter.

Edit: https://stackoverflow.com/a/39662359
That's all from Spyder. Spyder, like the jupyter software, builds ontop of the ipython kernel.


I saw those posts, their suggestions end up converging to what you did, but that doesn't seem to detect the difference too well.
Maybe `%gui qt` is the only reliable way to specify the backend.

That said, I thought that is what selecting the graphics backend from the settings in Spyder did. Maybe that is a bad assumption
Does `%gui qt` work in spyder with vispy? I wouldn't expect it to unless it is changing that kernel trait. I don't *think* there is anything in vispy that is checking the ipython configuration for what graphics backend it was told to use. The `%gui qt` magic command, if I remember correctly, is hooking a Qt event loop in to IPython in a background thread so you can run Qt applications while still having access to the REPL.

Edit: ...work in spyder with vispy to set the default vispy backend?
> @ccordoba12 any ideas?

This is a limitation of the Jupyter architecture: there's no way to tell what frontend (notebook or Spyder) is requesting an evaluation in the kernel. Besides, since several frontends can be connected to same kernel (e.g. Spyder can easily create a console for a kernel started by the notebook), things are even harder because each request should be handled separately.

However, you could check for the presence of several environment variables that are only present in our kernels ([this](https://github.com/spyder-ide/spyder/blob/59eded5422a0c00f6f384600f82f16ee1dfd7a40/spyder/plugins/ipythonconsole/utils/kernelspec.py#L97-L124) is the list of env vars we send to the kernel) to give better support for Spyder.

That will fail for a Spyder console connected to a notebook kernel and also for our third-party [notebook plugin](https://github.com/spyder-ide/spyder-notebook), but it'll work for every console started normally in Spyder.
> Does %gui qt work in spyder with vispy?

Yes. Spyder uses the [qtconsole](https://github.com/jupyter/qtconsole) package, which provides an interface very similar to IPython's terminal version but developed in Qt, which can evaluate any command IPython (or more specifically `ipykernel`) can run.
Pinging @impact27 and @mwcraig about this:

* @impact27, do you think we could improve the detection of a kernel being called by Spyder using the new Comms architecture you developed?
* @mwcraig, I don't know if you're using a similar procedure to what I described above in VPython or you came up with a better solution to distinguish between notebook and Spyder there.
I was more wondering if doing it caused vispy to say "ah we should use Qt instead of webgl" even though vispy was not called in any way. I don't think vispy checks anything that `%gui qt` is affecting.
Yep, we are just checking for an environment variable with the string "SPYDER" in it in vpython.
>     * @impact27, do you think we could improve the detection of a kernel being called by Spyder using the new Comms architecture you developed?

I am not sure I understand what the problem is. can't you just call `get_ipython().kernel.__class__`?
With Jupyter notebook:
```
get_ipython().kernel.__class__
ipykernel.ipkernel.IPythonKernel
```
With spyder:

```get_ipython().kernel.__class__
Out[4]: spyder_kernels.console.kernel.SpyderKernel```
If the kernel is not a `SpyderKernel`, the comms will not be connected, and you can't communicate. So you need to know that you are using spyder to use the comm.
@impact27 The issue, if you can consider it one, is that vispy defaults to a specific webgl backend if it detects it is being used in a notebook. The way it does this was based on stackoverflow answers which said `get_ipython().kernel` should only exist for a notebook. However, that assumed the only alternative was an ipython terminal (where kernel isn't defined) or that `get_ipython` doesn't exist in which case it is being run by the normal python interpreter.

So when importing vispy in a spyder environment, vispy will default to this webgl backend because it sees that a jupyter kernel is being used. As far as the current checks go it only checks for the existence of a kernel, not what type of kernel it is.

I think the best long term solution is to do an isinstance check right after the `kernel` check. It will be `IPythonKernel` in a notebook, otherwise don't assume it is a notebook. @hmaarrfk Do you have time to make a PR for this?

Short term workaround is to explicitly say what backend you want to use as @hmaarrfk pointed out above.
> I am not sure I understand what the problem is. can't you just call get_ipython().kernel.__class__?

The thing is Spyder can connect to an IPythonKernel without problems. I mean, you can start a kernel in a terminal with

    ipython kernel

then pass its connection file to our "Connect to an existing kernel" dialog and keep working with it just fine.

But your solution will work for all kernels started by Spyder directly (which I guess are most kernels started by our users anyway).
It seems the correct way is to understand what `%gui qt` and check for the existence of that state. That magic seems applicable even in the case of somebody using a notebook to launch local QT applications. (I believe that is what @jni is doing with napari). Second, it would be to understand if there is any difference between using the `%gui qt` magic and setting the graphics backend in spyder.
The comms could work if spyder registers a target to open comms and sends back informations when a comm is opened. This is the opposite way as to what is happening in spyder and you wouldn't be sure that a frontend is already connected when you send the message.
> Second, it would be to understand if there is any difference between using the %gui qt magic and setting the graphics backend in spyder.

We `%matplotlib <backend>` to set backends in Spyder instead of `%gui <backend>`. That's because we want to give users an out of the box experience for Matplotlib.
hmm it seems that both `%gui qt` and `%matplotlib qt` create a variable `app`. This `app` seems to depend on what graphics backend you choose to have. Maybe that is the variable we should be checking for? Note that if you simply run the `%gui` magic, it seems to close the app. I'm not a QT expert, so I'm not too sure what variable to check to know if hte app is still running or not

<details>

```
Python 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)
Type "copyright", "credits" or "license" for more information.

IPython 7.8.0 -- An enhanced Interactive Python.


%gui tk

get_ipython().kernel.app
Out[2]: <tkinter.Tk object .>

%gui qt
ERROR:root:Cannot activate multiple GUI eventloops

%gui

get_ipython().kernel.app
Out[5]: <tkinter.Tk object .>

%gui qt

get_ipython().kernel.app
Out[7]: <PyQt5.QtWidgets.QApplication at 0x7f8891e90ee8>

app = get_ipython().kernel.app

app.destroyed
Out[9]: <bound PYQT_SIGNAL destroyed of QApplication object at 0x7f8891e90ee8>

%gui

%gui tk

app = get_ipython().kernel.app

app
Out[13]: <tkinter.Tk object .>
```
</details>
That's an interesting idea. However, on my ipython terminal with same version of python and ipython from conda-forge (on Ubuntu) my `get_ipython()` doesn't have a kernel. Or is your above output from a spyder session?
yeah, sorry, it is from a spyder session.
Ok I think I like this. Just tested the `%gui qt` stuff in a notebook just to make sure and the behavior is similar. Even if you do `%gui notebook` or `%gui ipympl` it doesn't create the `app` attribute because I'm guessing it isn't needed (we're already using the javascript/browser event loop). So if we check that a kernel attribute exists that means we are running on some "remote" kernel. If that `kernel` has an `app` attribute then that means the user or some other system (spyder) has explicitly set what GUI backend to use (for matplotlib or other GUI tools) and vispy can safely assume it shouldn't use the webgl widget/backend.

If this `app` has a different behavior/existence when multiple frontends are connected to the same kernel then we can deal with that at a later time. At that point people should be explicitly setting their backends anyway.
Maybe we need to add a secondary check:

First check for the kernel, then check if the kernel has an active graphical backend, through the app attribute
I guess the problem with checking `get_ipython().kernel.app` is that `kernel` doesnt' exist for ipython consoles.

That said, they can still use the `%gui qt` line magic, and likely should to get hooked into the event loop.

One could check `get_ipython().active_eventloop`, which seems to have the following values:

| Environment | Configuration  | get_ipython().active_eventloop | Usecase  |
|-------------|----------------|--------------------------------|----------|
| console     | none           | None                           | djhoese   |
| console     | `%gui qt`      | 'qt'                           | djhoese   |
| spyder      | inline backend | 'inline'                       | hmaarrfk |
| spyder      | qt5            | 'qt5'                          | hmaarrfk |
| notebook    | none           | None                           | djhoese   |
| notebook    | `%gui qt`      | 'qt'                           | jni      |

I mostly put the name of the interested parties next to them so that they may chime in the testing phase should it be necessary.


Resolving patch:
diff --git a/vispy/app/_detect_eventloop.py b/vispy/app/_detect_eventloop.py
new file mode 100644
--- /dev/null
+++ b/vispy/app/_detect_eventloop.py
@@ -0,0 +1,148 @@
+# Taken from Matplotlib to automatically detect any event loop currently
+# in use.
+# This code is copyright of Matplotlib, and their license is inlcuded at the
+# bottom of this file.
+
+import sys
+import os
+
+
+def _get_running_interactive_framework():
+    """
+    Return the interactive framework whose event loop is currently running, if
+    any, or "headless" if no event loop can be started, or None.
+    Returns
+    -------
+    Optional[str]
+        One of the following values: "qt5", "qt4", "gtk3", "wx", "tk",
+        "macosx", "headless", ``None``.
+    """
+    QtWidgets = (sys.modules.get("PyQt5.QtWidgets")
+                 or sys.modules.get("PySide2.QtWidgets"))
+    if QtWidgets and QtWidgets.QApplication.instance():
+        return "qt5"
+    QtGui = (sys.modules.get("PyQt4.QtGui")
+             or sys.modules.get("PySide.QtGui"))
+    if QtGui and QtGui.QApplication.instance():
+        return "qt4"
+    Gtk = sys.modules.get("gi.repository.Gtk")
+    if Gtk and Gtk.main_level():
+        return "gtk3"
+    wx = sys.modules.get("wx")
+    if wx and wx.GetApp():
+        return "wx"
+    tkinter = sys.modules.get("tkinter")
+    if tkinter:
+        for frame in sys._current_frames().values():
+            while frame:
+                if frame.f_code == tkinter.mainloop.__code__:
+                    return "tk"
+                frame = frame.f_back
+    if 'matplotlib.backends._macosx' in sys.modules:
+        if sys.modules["matplotlib.backends._macosx"].event_loop_is_running():
+            return "macosx"
+    if sys.platform.startswith("linux") and not os.environ.get("DISPLAY"):
+        return "headless"
+    return None
+
+"""
+License agreement for matplotlib versions 1.3.0 and later
+=========================================================
+
+1. This LICENSE AGREEMENT is between the Matplotlib Development Team
+("MDT"), and the Individual or Organization ("Licensee") accessing and
+otherwise using matplotlib software in source or binary form and its
+associated documentation.
+
+2. Subject to the terms and conditions of this License Agreement, MDT
+hereby grants Licensee a nonexclusive, royalty-free, world-wide license
+to reproduce, analyze, test, perform and/or display publicly, prepare
+derivative works, distribute, and otherwise use matplotlib
+alone or in any derivative version, provided, however, that MDT's
+License Agreement and MDT's notice of copyright, i.e., "Copyright (c)
+2012- Matplotlib Development Team; All Rights Reserved" are retained in
+matplotlib  alone or in any derivative version prepared by
+Licensee.
+
+3. In the event Licensee prepares a derivative work that is based on or
+incorporates matplotlib or any part thereof, and wants to
+make the derivative work available to others as provided herein, then
+Licensee hereby agrees to include in any such work a brief summary of
+the changes made to matplotlib .
+
+4. MDT is making matplotlib available to Licensee on an "AS
+IS" basis.  MDT MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
+IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, MDT MAKES NO AND
+DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
+FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF MATPLOTLIB
+WILL NOT INFRINGE ANY THIRD PARTY RIGHTS.
+
+5. MDT SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF MATPLOTLIB
+ FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR
+LOSS AS A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING
+MATPLOTLIB , OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF
+THE POSSIBILITY THEREOF.
+
+6. This License Agreement will automatically terminate upon a material
+breach of its terms and conditions.
+
+7. Nothing in this License Agreement shall be deemed to create any
+relationship of agency, partnership, or joint venture between MDT and
+Licensee.  This License Agreement does not grant permission to use MDT
+trademarks or trade name in a trademark sense to endorse or promote
+products or services of Licensee, or any third party.
+
+8. By copying, installing or otherwise using matplotlib ,
+Licensee agrees to be bound by the terms and conditions of this License
+Agreement.
+
+License agreement for matplotlib versions prior to 1.3.0
+========================================================
+
+1. This LICENSE AGREEMENT is between John D. Hunter ("JDH"), and the
+Individual or Organization ("Licensee") accessing and otherwise using
+matplotlib software in source or binary form and its associated
+documentation.
+
+2. Subject to the terms and conditions of this License Agreement, JDH
+hereby grants Licensee a nonexclusive, royalty-free, world-wide license
+to reproduce, analyze, test, perform and/or display publicly, prepare
+derivative works, distribute, and otherwise use matplotlib
+alone or in any derivative version, provided, however, that JDH's
+License Agreement and JDH's notice of copyright, i.e., "Copyright (c)
+2002-2011 John D. Hunter; All Rights Reserved" are retained in
+matplotlib  alone or in any derivative version prepared by
+Licensee.
+
+3. In the event Licensee prepares a derivative work that is based on or
+incorporates matplotlib  or any part thereof, and wants to
+make the derivative work available to others as provided herein, then
+Licensee hereby agrees to include in any such work a brief summary of
+the changes made to matplotlib.
+
+4. JDH is making matplotlib  available to Licensee on an "AS
+IS" basis.  JDH MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
+IMPLIED.  BY WAY OF EXAMPLE, BUT NOT LIMITATION, JDH MAKES NO AND
+DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
+FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF MATPLOTLIB
+WILL NOT INFRINGE ANY THIRD PARTY RIGHTS.
+
+5. JDH SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF MATPLOTLIB
+ FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR
+LOSS AS A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING
+MATPLOTLIB , OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF
+THE POSSIBILITY THEREOF.
+
+6. This License Agreement will automatically terminate upon a material
+breach of its terms and conditions.
+
+7. Nothing in this License Agreement shall be deemed to create any
+relationship of agency, partnership, or joint venture between JDH and
+Licensee.  This License Agreement does not grant permission to use JDH
+trademarks or trade name in a trademark sense to endorse or promote
+products or services of Licensee, or any third party.
+
+8. By copying, installing or otherwise using matplotlib,
+Licensee agrees to be bound by the terms and conditions of this License
+Agreement.
+"""
diff --git a/vispy/app/application.py b/vispy/app/application.py
--- a/vispy/app/application.py
+++ b/vispy/app/application.py
@@ -16,6 +16,7 @@
 from .backends import CORE_BACKENDS, BACKEND_NAMES, BACKENDMAP, TRIED_BACKENDS
 from .. import config
 from .base import BaseApplicationBackend as ApplicationBackend  # noqa
+from ._detect_eventloop import _get_running_interactive_framework
 from ..util import logger
 from ..ext import six
 
@@ -128,7 +129,17 @@ def is_notebook(self):
             # 'get_ipython' is available in globals when running from
             # IPython/Jupyter
             ip = get_ipython()
-            return ip.has_trait('kernel')
+            if ip.has_trait('kernel'):
+                # There doesn't seem to be an easy way to detect the frontend
+                # That said, if using a kernel, the user can choose to have an
+                # event loop, we therefore make sure the event loop isn't
+                # specified before assuming it is a notebook
+                # https://github.com/vispy/vispy/issues/1708
+                # https://github.com/ipython/ipython/issues/11920
+                return _get_running_interactive_framework() is None
+            else:
+                # `jupyter console` is used
+                return False
         except NameError:
             return False
 


Test patch: