Repository: napari/napari

Issue:
Pyramid level calculation for multiscale is incorrect
## 🐛 Bug

When using pyramids in Napari, the level chosen appears to be incorrect. This can cause the application to slow down or crash in certain instances.  Specifically, a lower resolution level should often be chosen than what is actual picked.  The main issue appears to be that the algorithm assumes the layer image fills the entire field of view when picking the pyramid level.  I think this may also be a contributing factor into why the multiscale images are slower than they should be in napari.

## To Reproduce

Steps to reproduce the behavior:

1.  Create pyramid view (I'm using a 2D pyramid) with multiple levels (say 1kx1k with 4 levels)
2.  Zoom out 
3.  Observe the level chosen (it is in the toolbar).  For an image smaller than the display, it will never go below 0 even if you zoom way out.

Issue I'm running into currently that crashes napari:
I have a large mosaic image that is not square.  The image has dimensions of [3072, 70656].  When I zoom out, napari picks the zoom level to be 1 (scale is [2,2]).  This is because the y-axis is being compared to the whole view window, even though it is only taking up a small area of the window. The x-axis at level 1 is now greater than the max texture size and napari crashes.

## Expected behavior

Napari should NOT use the display window size by itself when calculating the level, but should calculate the extent of the layer being displayed and use that instead.  Then it can pick the level of the multiscale that most closely matches the what needs to be displayed.  To put in other words, if we zoom out so that the 10kx10k image now only takes up 100px x 100px on the display, we should pick the level that is closest to the 100x100 pixels instead of using the size of the display.

napari/layers/base/base.py Layer._update_draw function:
The parameter shape_threshold needs to be cropped to match the extent of the layer being displayed before being passed to compute_multiscale_level.  It looks like the code crops the size of the layer to only what is being displayed, but then doesn't crop the shape_threshold to the layer as well.


## Environment

napari: 0.4.16
Platform: Linux-4.18.0-372.9.1.el8.x86_64-x86_64-with-glibc2.28
Python: 3.9.0 (default, Nov 15 2020, 14:28:56) [GCC 7.3.0]
Qt: 5.15.2
PyQt5: 5.15.6
NumPy: 1.22.3
SciPy: 1.8.0
Dask: 2022.05.0
VisPy: 0.10.0

OpenGL:
- GL version: 4.6.0 NVIDIA 515.48.07
- MAX_TEXTURE_SIZE: 32768

Screens:
- screen 1: resolution 3840x2160, scale 1.0

Plugins:
- console: 0.0.4
- napari-svg: 0.1.6
- scikit-image: 0.4.16

 - Any other relevant information:

## Additional context




Discussion:
Good point @michael-sutherland and I believe I've encountered it before myself. Agree this needs fixing.
Thanks for the write-up here. I'm going to assign myself to this issue because there's some strong overlap with the async slicing redesign I'm working on - we need the non-experimental multi-scale data lookup to work as intended for some of that work to be useful.

I'll start by writing up some test cases based on the great description above.

Thanks @michael-sutherland!

Resolving patch:
diff --git a/napari/layers/base/base.py b/napari/layers/base/base.py
--- a/napari/layers/base/base.py
+++ b/napari/layers/base/base.py
@@ -1559,28 +1559,27 @@ def _update_draw(
         )
 
         # find the maximal data-axis-aligned bounding box containing all four
-        # canvas corners
+        # canvas corners and round them to ints
         data_bbox = np.stack(
             [np.min(data_corners, axis=0), np.max(data_corners, axis=0)]
         )
-        # round and clip the bounding box values
         data_bbox_int = np.stack(
             [np.floor(data_bbox[0]), np.ceil(data_bbox[1])]
         ).astype(int)
-        displayed_extent = self.extent.data[:, displayed_axes]
-        data_bbox_clipped = np.clip(
-            data_bbox_int, displayed_extent[0], displayed_extent[1]
-        )
 
         if self._ndisplay == 2 and self.multiscale:
             level, scaled_corners = compute_multiscale_level_and_corners(
-                data_bbox_clipped,
+                data_bbox_int,
                 shape_threshold,
                 self.downsample_factors[:, displayed_axes],
             )
-            corners = np.zeros((2, self.ndim))
-            corners[:, displayed_axes] = scaled_corners
-            corners = corners.astype(int)
+            corners = np.zeros((2, self.ndim), dtype=int)
+            # The corner_pixels attribute stores corners in the data
+            # space of the selected level. Using the level's data
+            # shape only works for images, but that's the only case we
+            # handle now and downsample_factors is also only on image layers.
+            max_coords = np.take(self.data[level].shape, displayed_axes)
+            corners[:, displayed_axes] = np.clip(scaled_corners, 0, max_coords)
             display_shape = tuple(
                 corners[1, displayed_axes] - corners[0, displayed_axes]
             )
@@ -1594,6 +1593,11 @@ def _update_draw(
                 self.refresh()
 
         else:
+            # The stored corner_pixels attribute must contain valid indices.
+            displayed_extent = self.extent.data[:, displayed_axes]
+            data_bbox_clipped = np.clip(
+                data_bbox_int, displayed_extent[0], displayed_extent[1]
+            )
             corners = np.zeros((2, self.ndim), dtype=int)
             corners[:, displayed_axes] = data_bbox_clipped
             self.corner_pixels = corners
diff --git a/napari/layers/utils/layer_utils.py b/napari/layers/utils/layer_utils.py
--- a/napari/layers/utils/layer_utils.py
+++ b/napari/layers/utils/layer_utils.py
@@ -501,7 +501,7 @@ def compute_multiscale_level(
     # Scale shape by downsample factors
     scaled_shape = requested_shape / downsample_factors
 
-    # Find the highest resolution level allowed
+    # Find the highest level (lowest resolution) allowed
     locations = np.argwhere(np.all(scaled_shape > shape_threshold, axis=1))
     if len(locations) > 0:
         level = locations[-1][0]


Test patch:
diff --git a/napari/_vispy/_tests/test_vispy_multiscale.py b/napari/_vispy/_tests/test_vispy_multiscale.py
--- a/napari/_vispy/_tests/test_vispy_multiscale.py
+++ b/napari/_vispy/_tests/test_vispy_multiscale.py
@@ -156,6 +156,39 @@ def test_image_screenshot_zoomed(make_napari_viewer):
     )
 
 
+@skip_on_win_ci
+@skip_local_popups
+def test_multiscale_zoomed_out(make_napari_viewer):
+    """See https://github.com/napari/napari/issues/4781"""
+    # Need to show viewer to ensure that pixel_scale and physical_size
+    # get set appropriately.
+    viewer = make_napari_viewer(show=True)
+    shapes = [(3200, 3200), (1600, 1600), (800, 800)]
+    data = [np.empty(s) for s in shapes]
+    layer = viewer.add_image(data, multiscale=True)
+    qt_viewer = viewer.window._qt_viewer
+    # Canvas size is in screen pixels.
+    qt_viewer.canvas.size = (1600, 1600)
+    # The camera rect is (left, top, width, height) in scene coordinates.
+    # In this case scene coordinates are the same as data/world coordinates
+    # the layer is 2D and data-to-world is identity.
+    # We pick a camera rect size that is much bigger than the data extent
+    # to simulate being zoomed out in the viewer.
+    camera_rect_size = 34000
+    camera_rect_center = 1599.5
+    camera_rect_start = camera_rect_center - (camera_rect_size / 2)
+    qt_viewer.view.camera.rect = (
+        camera_rect_start,
+        camera_rect_start,
+        camera_rect_size,
+        camera_rect_size,
+    )
+
+    qt_viewer.on_draw(None)
+
+    assert layer.data_level == 2
+
+
 @skip_on_win_ci
 @skip_local_popups
 def test_5D_multiscale(make_napari_viewer):
diff --git a/napari/layers/image/_tests/test_multiscale.py b/napari/layers/image/_tests/test_multiscale.py
--- a/napari/layers/image/_tests/test_multiscale.py
+++ b/napari/layers/image/_tests/test_multiscale.py
@@ -432,3 +432,57 @@ def test_multiscale_data_protocol():
     assert layer.data.dtype == float
     assert layer.data.shape == shapes[0]
     assert isinstance(layer.data[0], np.ndarray)
+
+
+@pytest.mark.parametrize(
+    ('corner_pixels_world', 'exp_level', 'exp_corner_pixels_data'),
+    (
+        ([[5, 5], [15, 15]], 0, [[5, 5], [15, 15]]),
+        # Multiscale level selection uses > rather than >= so use -1 and 21
+        # instead of 0 and 20 to ensure that the FOV is big enough.
+        ([[-1, -1], [21, 21]], 1, [[0, 0], [10, 10]]),
+        ([[-11, -11], [31, 31]], 2, [[0, 0], [5, 5]]),
+    ),
+)
+def test_update_draw_variable_fov_fixed_canvas_size(
+    corner_pixels_world, exp_level, exp_corner_pixels_data
+):
+    shapes = [(20, 20), (10, 10), (5, 5)]
+    data = [np.zeros(s) for s in shapes]
+    layer = Image(data, multiscale=True)
+    canvas_size_pixels = (10, 10)
+
+    layer._update_draw(
+        scale_factor=1,
+        corner_pixels_displayed=np.array(corner_pixels_world),
+        shape_threshold=canvas_size_pixels,
+    )
+
+    assert layer.data_level == exp_level
+    np.testing.assert_equal(layer.corner_pixels, exp_corner_pixels_data)
+
+
+@pytest.mark.parametrize(
+    ('canvas_size_pixels', 'exp_level', 'exp_corner_pixels_data'),
+    (
+        ([16, 16], 0, [[0, 0], [20, 20]]),
+        ([8, 8], 1, [[0, 0], [10, 10]]),
+        ([4, 4], 2, [[0, 0], [5, 5]]),
+    ),
+)
+def test_update_draw_variable_canvas_size_fixed_fov(
+    canvas_size_pixels, exp_level, exp_corner_pixels_data
+):
+    shapes = [(20, 20), (10, 10), (5, 5)]
+    data = [np.zeros(s) for s in shapes]
+    layer = Image(data, multiscale=True)
+    corner_pixels_world = np.array([[0, 0], [20, 20]])
+
+    layer._update_draw(
+        scale_factor=1,
+        corner_pixels_displayed=corner_pixels_world,
+        shape_threshold=canvas_size_pixels,
+    )
+
+    assert layer.data_level == exp_level
+    np.testing.assert_equal(layer.corner_pixels, exp_corner_pixels_data)