Repository: napari/napari

Issue:
`update_console` example is broken
## 🐛 Bug

The `update_console` example: https://github.com/napari/napari/blob/main/examples/update_console.py does not acutally filly the ipython console with variables.

Also discussed here: https://napari.zulipchat.com/#narrow/stream/212875-general/topic/ipython-variables-from-script

## To Reproduce

Steps to reproduce the behavior:

1. Run the example
2. try to use some of the locals in the napari ipython console


## Expected behavior
Locals are there
<!-- A clear and concise description of what you expected to happen. -->

## Environment

```
napari: 0.4.14
Platform: Linux-5.15.15-76051515-generic-x86_64-with-glibc2.34
System: Pop!_OS 21.10
Python: 3.9.10 | packaged by conda-forge | (main, Jan 30 2022, 18:04:04) [GCC 9.4.0]
Qt: 5.12.9
PyQt5: 5.12.3
NumPy: 1.22.1
SciPy: 1.7.3
Dask: 2022.01.1
VisPy: 0.9.4

OpenGL:
- GL version: 4.6 (Compatibility Profile) Mesa 21.2.2
- MAX_TEXTURE_SIZE: 16384

Screens:
- screen 1: resolution 1920x1080, scale 1.0

Plugins:
- console: 0.0.4
- scikit-image: 0.4.14
- svg: 0.1.5
```
## Additional context

I believe that this is due to the lazy loading of the console. In particular here:

https://github.com/napari/napari/blob/bf8682ec9e1ba50e40f3d078a68360903ce58a06/napari/viewer.py#L78-L81

the check for `_console` will always return `None` here as the console hasn't been initialized. You can forcibly init the console by accessing the

`_qt_viewer.console` property, but that's not ideal because of the usage of a private variable. So i suggest changing the beahvior of `update_console` to:

1. `update_console` defaults to intiatlizing the console if one doesn't exist
2.  add a kwarg to brign the current behavior of not creating one

I think this is a sensible default because if you are updating the console then you probably want one to exist if it doesn't already.


Discussion:
Oh man - also running that example generates two separate napari windows. I'm pretty sure that thats not right either?
Rather than having a public `update_console()`, imho we should make efforts to ensure that the console is prepopulated with the right variables. How this could/should look like:

- console gets launched
- we traverse the stack until we find the highest level where the viewer is present (since the viewer can be present in many lower parts of the stack e.g. because it gets injected and handed to plugins)
- inject the locals at that level
@jni that's definitely what I was expecting to happen when I first tried this so :+1: from me. Though it may also be nice to be able to specify exactly what you want

Resolving patch:
diff --git a/examples/update_console.py b/examples/update_console.py
--- a/examples/update_console.py
+++ b/examples/update_console.py
@@ -2,9 +2,8 @@
 Update console
 ==============
 
-Display one shapes layer ontop of one image layer using the add_shapes and
-add_image APIs. When the window is closed it will print the coordinates of
-your shapes.
+Display one shapes layer on top of one image layer using the add_shapes and
+add_image APIs.
 
 .. tags:: historical
 """
@@ -19,7 +18,7 @@
 
 # add the image
 photographer = data.camera()
-image_layer = napari.view_image(photographer, name='photographer')
+image_layer = viewer.add_image(photographer, name='photographer')
 
 # create a list of polygons
 polygons = [
diff --git a/napari/_qt/qt_viewer.py b/napari/_qt/qt_viewer.py
--- a/napari/_qt/qt_viewer.py
+++ b/napari/_qt/qt_viewer.py
@@ -1,12 +1,13 @@
 from __future__ import annotations
 
 import logging
+import sys
 import traceback
 import typing
 import warnings
 from pathlib import Path
 from typing import TYPE_CHECKING, List, Optional, Sequence, Tuple, Type, Union
-from weakref import WeakSet
+from weakref import WeakSet, ref
 
 from qtpy.QtCore import QCoreApplication, QObject, Qt
 from qtpy.QtGui import QGuiApplication
@@ -183,6 +184,7 @@ def __init__(
         self._viewerButtons = None
         self._key_map_handler = KeymapHandler()
         self._key_map_handler.keymap_providers = [self.viewer]
+        self._console_backlog = []
         self._console = None
 
         self._dockLayerList = None
@@ -437,6 +439,92 @@ def _create_performance_dock_widget(self):
             )
         return None
 
+    def _weakref_if_possible(self, obj):
+        """Create a weakref to obj.
+
+        Parameters
+        ----------
+        obj : object
+            Cannot create weakrefs to many Python built-in datatypes such as
+            list, dict, str.
+
+            From https://docs.python.org/3/library/weakref.html: "Objects which
+            support weak references include class instances, functions written
+            in Python (but not in C), instance methods, sets, frozensets, some
+            file objects, generators, type objects, sockets, arrays, deques,
+            regular expression pattern objects, and code objects."
+
+        Returns
+        -------
+        weakref or object
+            Returns a weakref if possible.
+        """
+        try:
+            newref = ref(obj)
+        except TypeError:
+            newref = obj
+        return newref
+
+    def _unwrap_if_weakref(self, value):
+        """Return value or if that is weakref the object referenced by value.
+
+        Parameters
+        ----------
+        value : object or weakref
+            No-op for types other than weakref.
+
+        Returns
+        -------
+        unwrapped: object or None
+            Returns referenced object, or None if weakref is dead.
+        """
+        unwrapped = value() if isinstance(value, ref) else value
+        return unwrapped
+
+    def add_to_console_backlog(self, variables):
+        """Save variables for pushing to console when it is instantiated.
+
+        This function will create weakrefs when possible to avoid holding on to
+        too much memory unnecessarily.
+
+        Parameters
+        ----------
+        variables : dict, str or list/tuple of str
+            The variables to inject into the console's namespace. If a dict, a
+            simple update is done. If a str, the string is assumed to have
+            variable names separated by spaces. A list/tuple of str can also
+            be used to give the variable names. If just the variable names are
+            give (list/tuple/str) then the variable values looked up in the
+            callers frame.
+        """
+        if isinstance(variables, (str, list, tuple)):
+            if isinstance(variables, str):
+                vlist = variables.split()
+            else:
+                vlist = variables
+            vdict = {}
+            cf = sys._getframe(2)
+            for name in vlist:
+                try:
+                    vdict[name] = eval(name, cf.f_globals, cf.f_locals)
+                except:  # noqa: E722
+                    print(
+                        f'Could not get variable {name} from '
+                        f'{cf.f_code.co_name}'
+                    )
+        elif isinstance(variables, dict):
+            vdict = variables
+        else:
+            raise TypeError('variables must be a dict/str/list/tuple')
+        # weakly reference values if possible
+        new_dict = {k: self._weakref_if_possible(v) for k, v in vdict.items()}
+        self.console_backlog.append(new_dict)
+
+    @property
+    def console_backlog(self):
+        """List: items to push to console when instantiated."""
+        return self._console_backlog
+
     @property
     def console(self):
         """QtConsole: iPython console terminal integrated into the napari GUI."""
@@ -452,6 +540,16 @@ def console(self):
                     self.console.push(
                         {'napari': napari, 'action_manager': action_manager}
                     )
+                    for i in self.console_backlog:
+                        # recover weak refs
+                        self.console.push(
+                            {
+                                k: self._unwrap_if_weakref(v)
+                                for k, v in i.items()
+                                if self._unwrap_if_weakref(v) is not None
+                            }
+                        )
+                    self._console_backlog = []
             except ModuleNotFoundError:
                 warnings.warn(
                     trans._(
diff --git a/napari/viewer.py b/napari/viewer.py
--- a/napari/viewer.py
+++ b/napari/viewer.py
@@ -86,6 +86,7 @@ def update_console(self, variables):
             callers frame.
         """
         if self.window._qt_viewer._console is None:
+            self.window._qt_viewer.add_to_console_backlog(variables)
             return
         self.window._qt_viewer.console.push(variables)
 


Test patch:
diff --git a/napari/_tests/test_advanced.py b/napari/_tests/test_advanced.py
--- a/napari/_tests/test_advanced.py
+++ b/napari/_tests/test_advanced.py
@@ -180,6 +180,56 @@ def test_update_console(make_napari_viewer):
         del viewer.window._qt_viewer.console.shell.user_ns[k]
 
 
+@pytest.mark.filterwarnings("ignore::DeprecationWarning:jupyter_client")
+def test_update_lazy_console(make_napari_viewer, capsys):
+    """Test updating the console with local variables,
+    before console is instantiated."""
+    viewer = make_napari_viewer()
+    view = viewer.window._qt_viewer
+
+    a = 4
+    b = 5
+    viewer.update_console(["a", "b"])
+
+    x = np.arange(5)
+    viewer.update_console("x")
+
+    viewer.update_console("missing")
+    captured = capsys.readouterr()
+    assert 'Could not get' in captured.out
+    with pytest.raises(TypeError):
+        viewer.update_console(x)
+
+    # Create class objects that will have weakrefs
+    class Foo:
+        pass
+
+    obj1 = Foo()
+    obj2 = Foo()
+    viewer.update_console({'obj1': obj1, 'obj2': obj2})
+    del obj1
+
+    # Check viewer in console
+    assert view.console.kernel_client is not None
+    assert 'viewer' in view.console.shell.user_ns
+    assert view.console.shell.user_ns['viewer'] == viewer
+
+    # Check backlog is cleared
+    assert len(view.console_backlog) == 0
+
+    assert 'a' in view.console.shell.user_ns
+    assert view.console.shell.user_ns['a'] == a
+    assert 'b' in view.console.shell.user_ns
+    assert view.console.shell.user_ns['b'] == b
+    assert 'x' in view.console.shell.user_ns
+    assert view.console.shell.user_ns['x'] is x
+    assert 'obj1' not in view.console.shell.user_ns
+    assert 'obj2' in view.console.shell.user_ns
+    assert view.console.shell.user_ns['obj2'] == obj2
+    del viewer.window._qt_viewer.console.shell.user_ns['obj2']
+    del viewer.window._qt_viewer.console.shell.user_ns['x']
+
+
 def test_changing_display_surface(make_napari_viewer):
     """Test adding 3D surface and changing its display."""
     viewer = make_napari_viewer()