Repository: napari/napari

Issue:
Proxy wrapper should be removed when passing object back to napari functions
## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->
There is a possibility to generate Proxy future warnings using only public API, because the proxy is not removed when passing object bac to napari function (which uses private API). 


## To Reproduce

Steps to reproduce the behavior:

The sample scenario is to remove one layer to add it later. 

1.
```python

import numpy as np
from napari_plugin_engine import napari_hook_implementation

from napari import Viewer, run, plugins
from qtpy.QtWidgets import QWidget, QPushButton, QHBoxLayout




class RemoveAdd(QWidget):
    def __init__(self, viewer: Viewer):
        super().__init__()
        self.viewer = viewer
        self.layer = None
        self.btn = QPushButton("Remove/add", self)
        self.btn.clicked.connect(self._click)
        layout = QHBoxLayout(self.btn)
        self.setLayout(layout)

    def _click(self):
        if self.layer is None:
            self.layer = self.viewer.layers[-1]
            del self.viewer.layers[-1]
        else:
            # self.viewer.add_layer(self.layer.__wrapped__)  # This line works correctly
            self.viewer.add_layer(self.layer)
            self.layer = None

@napari_hook_implementation
def napari_experimental_provide_dock_widget():
    return RemoveAdd

plugin = plugins.plugin_manager.register(globals(), name="Test Widget")


viewer = Viewer()
viewer.add_image(np.random.random((20, 20)))
run()
```
2. open Test Widget RemoveAdd plugin. 
3. click "Remove/add" button twice.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

No warnings

<!-- A clear and concise description of what you expected to happen. -->

## Environment

This commit [`d3d090f` (#3777)](https://github.com/napari/napari/pull/3777/commits/d3d090fca3ffe8688602fa2dfb177d6da8f27521)
Running on last main cause exceptions connected with wrapping non-napari objects. 



## Additional context

<!-- Add any other context about the problem here. -->
Generated warnings:
```
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:84: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:87: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:89: FutureWarning: Private attribute access ('Image._empty') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/base.py:126: FutureWarning: Private attribute access ('Image._transforms') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/base.py:127: FutureWarning: Private attribute access ('Image._dims_displayed') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/base.py:140: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/base.py:145: FutureWarning: Private attribute access ('Image._data_to_world') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/base.py:146: FutureWarning: Private attribute access ('Image._dims_displayed') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:107: FutureWarning: Private attribute access ('Image._data_view') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:114: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:118: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:127: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:128: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_vispy/layers/image.py:133: FutureWarning: Private attribute access ('Image._empty') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_qt/layer_controls/qt_image_controls.py:52: FutureWarning: Private attribute access ('EmitterGroup._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_qt/layer_controls/qt_image_controls.py:244: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_qt/layer_controls/qt_image_controls.py:256: FutureWarning: Private attribute access ('Image._ndisplay') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/components/viewer_model.py:317: FutureWarning: Private attribute access ('Image._slice_dims') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/components/layerlist.py:53: FutureWarning: Private attribute access ('Image._on_selection') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/utils/context/_layerlist_context.py:47: FutureWarning: Private attribute access ('Image._type_string') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/utils/context/_layerlist_context.py:39: FutureWarning: Private attribute access ('Image._type_string') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/utils/context/_layerlist_context.py:43: FutureWarning: Private attribute access ('Image._type_string') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_qt/qt_viewer.py:960: FutureWarning: Private attribute access ('Image._displayed_axes') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_qt/qt_viewer.py:965: FutureWarning: Private attribute access ('Image._update_draw') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
/home/czaki/Projekty/napari/napari/_qt/containers/_layer_delegate.py:99: FutureWarning: Private attribute access ('Image._type_string') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14!
```



Discussion:

Resolving patch:
diff --git a/napari/utils/_proxies.py b/napari/utils/_proxies.py
--- a/napari/utils/_proxies.py
+++ b/napari/utils/_proxies.py
@@ -1,9 +1,11 @@
 import re
+import sys
 import warnings
-from typing import Any, Callable, Generic, TypeVar
+from typing import Any, Callable, Generic, TypeVar, Union
 
 import wrapt
 
+from ..utils import misc
 from ..utils.translations import trans
 
 _T = TypeVar("_T")
@@ -41,11 +43,19 @@ class PublicOnlyProxy(wrapt.ObjectProxy, Generic[_T]):
     __wrapped__: _T
 
     def __getattr__(self, name: str):
-        if name.startswith('_'):
+        if name.startswith("_"):
+            # allow napari to access private attributes and get an non-proxy
+            frame = sys._getframe(1) if hasattr(sys, "_getframe") else None
+            if frame.f_code.co_filename.startswith(misc.ROOT_DIR):
+                return super().__getattr__(name)
+
+            typ = type(self.__wrapped__).__name__
             warnings.warn(
                 trans._(
-                    "Private attribute access in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14",
+                    "Private attribute access ('{typ}.{name}') in this context (e.g. inside a plugin widget or dock widget) is deprecated and will be unavailable in version 0.4.14",
                     deferred=True,
+                    name=name,
+                    typ=typ,
                 ),
                 category=FutureWarning,
                 stacklevel=2,
@@ -53,9 +63,10 @@ def __getattr__(self, name: str):
             # name = f'{type(self.__wrapped__).__name__}.{name}'
             # raise AttributeError(
             #     trans._(
-            #         "Private attribute access ('{name}') not allowed in this context.",
+            #         "Private attribute access ('{typ}.{name}') not allowed in this context.",
             #         deferred=True,
             #         name=name,
+            #         typ=typ,
             #     )
             # )
         return self.create(super().__getattr__(name))
@@ -70,8 +81,13 @@ def __dir__(self):
         return [x for x in dir(self.__wrapped__) if not _SUNDER.match(x)]
 
     @classmethod
-    def create(cls, obj: Any) -> 'PublicOnlyProxy':
-        if callable:
+    def create(cls, obj: Any) -> Union['PublicOnlyProxy', Any]:
+        # restrict the scope of this proxy to napari objects
+        if not getattr(obj, '__module__', '').startswith('napari'):
+            return obj
+        if isinstance(obj, PublicOnlyProxy):
+            return obj  # don't double-wrap
+        if callable(obj):
             return CallablePublicOnlyProxy(obj)
         return PublicOnlyProxy(obj)
 


Test patch:
diff --git a/napari/_qt/_tests/test_plugin_widgets.py b/napari/_qt/_tests/test_plugin_widgets.py
--- a/napari/_qt/_tests/test_plugin_widgets.py
+++ b/napari/_qt/_tests/test_plugin_widgets.py
@@ -1,3 +1,5 @@
+from unittest.mock import patch
+
 import pytest
 from napari_plugin_engine import napari_hook_implementation
 from qtpy.QtWidgets import QWidget
@@ -191,5 +193,8 @@ def test_inject_viewer_proxy(make_napari_viewer):
     viewer = make_napari_viewer()
     wdg = _instantiate_dock_widget(Widg3, viewer)
     assert isinstance(wdg.viewer, PublicOnlyProxy)
-    with pytest.warns(FutureWarning):
-        wdg.fail()
+
+    # simulate access from outside napari
+    with patch('napari.utils.misc.ROOT_DIR', new='/some/other/package'):
+        with pytest.warns(FutureWarning):
+            wdg.fail()
diff --git a/napari/utils/_tests/test_proxies.py b/napari/utils/_tests/test_proxies.py
--- a/napari/utils/_tests/test_proxies.py
+++ b/napari/utils/_tests/test_proxies.py
@@ -1,5 +1,9 @@
+from unittest.mock import patch
+
+import numpy as np
 import pytest
 
+from napari.components.viewer_model import ViewerModel
 from napari.utils._proxies import PublicOnlyProxy, ReadOnlyWrapper
 
 
@@ -25,7 +29,14 @@ class TestClass:
         tc_read_only.x = 5
 
 
-def test_PublicOnlyProxy():
+@pytest.fixture
+def patched_root_dir():
+    """Simulate a call from outside of napari"""
+    with patch('napari.utils.misc.ROOT_DIR', new='/some/other/package'):
+        yield
+
+
+def test_PublicOnlyProxy(patched_root_dir):
     class X:
         a = 1
         _b = 'nope'
@@ -49,17 +60,51 @@ def __getitem__(self, key):
     assert isinstance(proxy, Tester)
     with pytest.warns(FutureWarning) as e:
         proxy._private
-        assert 'Private attribute access' in str(e[0].message)
+    assert 'Private attribute access' in str(e[0].message)
 
     with pytest.warns(FutureWarning) as e:
         # works on sub-objects too
         proxy.x._b
-        assert 'Private attribute access' in str(e[0].message)
+    assert 'Private attribute access' in str(e[0].message)
 
     with pytest.warns(FutureWarning) as e:
         # works on sub-items too
         proxy[0]._b
-        assert 'Private attribute access' in str(e[0].message)
+    assert 'Private attribute access' in str(e[0].message)
 
     assert '_private' not in dir(proxy)
     assert '_private' in dir(t)
+
+
+def test_public_proxy_limited_to_napari(patched_root_dir):
+    """Test that the recursive public proxy goes no farther than napari."""
+    viewer = ViewerModel()
+    viewer.add_points(None)
+    pv = PublicOnlyProxy(viewer)
+    assert not isinstance(pv.layers[0].data, PublicOnlyProxy)
+
+
+def test_array_from_proxy_objects(patched_root_dir):
+    """Test that the recursive public proxy goes no farther than napari."""
+    viewer = ViewerModel()
+    viewer.add_points(None)
+    pv = PublicOnlyProxy(viewer)
+    assert isinstance(np.array(pv.dims.displayed, dtype=int), np.ndarray)
+
+
+def test_receive_return_proxy_object():
+    """Test that an"""
+    viewer = ViewerModel()
+    viewer.add_image(np.random.random((20, 20)))
+    pv = PublicOnlyProxy(viewer)
+
+    # simulates behavior from outside of napari
+    with patch('napari.utils.misc.ROOT_DIR', new='/some/other/package'):
+        # the recursion means this layer will be a Proxy Object on __getitem__
+        layer = pv.layers[-1]
+        assert isinstance(layer, PublicOnlyProxy)
+        # remove and add it back, should be fine
+        add_layer = getattr(pv, 'add_layer')
+
+    add_layer(layer)
+    assert len(viewer.layers) == 2