Repository: napari/napari

Issue:
Add cadenced runs for benchmarking slicing
## Description

The goal of this task is to ensure that any existing benchmarking tools are running as expected. And that the output of those tasks can be easily used to detect performance changes in napari.

## Motivation

Slicing is a core part of napari and it's critical that any changes to the related code does not substantially negatively impact performance of slicing. Many of the existing ASV benchmarks measure the time and memory used during layer initialization and slicing, which could be affect by this project.

## Output

The key output of this task is to have some way of easily detecting when the performance of napari changes at some regular interval (e.g. daily, weekly) based on the existing ASV benchmarks. This could be as simple as using `asv publish` after running `asv continuous` in the existing GitHub workflow to generate static HTML that can be downloaded and rendered locally. Alternatively, the report could be entirely text based as long as it's easy to find any changes in performance.

We should also have some way to trigger a full benchmark run for a PR related to async slicing that also generates a similar report, but that could be done in a follow-up PR. This task can also be broken down as needed.

## Notes
need to add workflow runs for benchmarking

Jaime can support

Relevant previous PRs
- https://github.com/napari/napari/pull/4554
- https://github.com/napari/napari/pull/4656


<hr>

### KCP update:

Issues with the `Benchmark` workflow have been resolved in https://github.com/napari/napari/pull/5083

Unfortunately I introduced a regression which I'm fixing in https://github.com/napari/napari/pull/5246


Discussion:

Resolving patch:
diff --git a/napari/benchmarks/benchmark_qt_slicing.py b/napari/benchmarks/benchmark_qt_slicing.py
--- a/napari/benchmarks/benchmark_qt_slicing.py
+++ b/napari/benchmarks/benchmark_qt_slicing.py
@@ -48,7 +48,12 @@ def __getitem__(self, item: str):
 
 
 class AsyncImage2DSuite:
+    """TODO: these benchmarks are skipped. Remove the NotImplementedError in
+    setup to enable.
+    """
+
     params = get_image_params()
+    timeout = 300
 
     def setup(self, latency, dataname):
         shape = SAMPLE_PARAMS[dataname]['shape']
@@ -64,6 +69,7 @@ def setup(self, latency, dataname):
         )
 
         self.layer = Image(self.data)
+        raise NotImplementedError
 
     def time_create_layer(self, *args):
         """Time to create an image layer."""
@@ -79,7 +85,12 @@ def time_refresh(self, *args):
 
 
 class QtViewerAsyncImage2DSuite:
+    """TODO: these benchmarks are skipped. Remove the NotImplementedError in
+    setup to enable.
+    """
+
     params = get_image_params()
+    timeout = 300
 
     def setup(self, latency, dataname):
         shape = SAMPLE_PARAMS[dataname]['shape']
@@ -102,6 +113,7 @@ def setup(self, latency, dataname):
 
         self.viewer = napari.Viewer()
         self.viewer.add_image(self.data)
+        raise NotImplementedError
 
     def time_z_scroll(self, *args):
         layers_to_scroll = 4
@@ -115,6 +127,10 @@ def teardown(self, *args):
 
 
 class QtViewerAsyncPointsSuite:
+    """TODO: these benchmarks are skipped. Remove the NotImplementedError in
+    setup to enable.
+    """
+
     n_points = [2**i for i in range(12, 18)]
     params = n_points
 
@@ -128,6 +144,7 @@ def setup(self, n_points):
         self.viewer.add_image(self.empty_image)
         self.point_data = np.random.randint(512, size=(n_points, 3))
         self.viewer.add_points(self.point_data)
+        raise NotImplementedError
 
     def time_z_scroll(self, *args):
         for z in range(self.empty_image.shape[0]):
@@ -138,10 +155,15 @@ def teardown(self, *args):
 
 
 class QtViewerAsyncPointsAndImage2DSuite:
+    """TODO: these benchmarks are skipped. Remove the NotImplementedError in
+    setup to enable.
+    """
+
     n_points = [2**i for i in range(12, 18, 2)]
     chunksize = [256, 512, 1024]
     latency = [0.05 * i for i in range(0, 3)]
     params = (n_points, latency, chunksize)
+    timeout = 600
 
     def setup(self, n_points, latency, chunksize):
         store = SlowMemoryStore(load_delay=latency)
@@ -160,6 +182,7 @@ def setup(self, n_points, latency, chunksize):
         self.viewer.add_image(self.image_data)
         self.point_data = np.random.randint(512, size=(n_points, 3))
         self.viewer.add_points(self.point_data)
+        raise NotImplementedError
 
     def time_z_scroll(self, *args):
         for z in range(self.image_data.shape[0]):


Test patch: