Repository: holoviz/holoviews
Pull request #6251 (2024-05-27 18:48:30)

Move environment variables into Pixi.toml
It seems better to set the environment variables in `pixi.toml` so that the local run mimics CI more. 


Also _sneaking_ in some other changes:
- Remove commented out dash
- Move custom unfixable to unsafe fixes. The main point still stands: I don't want formatters to automatically remove unused variables and imports. Could set pre-commit to unsafe fixes at some point. 

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/holoviz/holoviews/pull/6251?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 17.43%. Comparing base [(`d458f60`)](https://app.codecov.io/gh/holoviz/holoviews/commit/d458f604ecf07bd264bef0dab788c742d30e88c9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) to head [(`1bf50d8`)](https://app.codecov.io/gh/holoviz/holoviews/commit/1bf50d8dde8e9318a04a6a886a7917ab387d0db3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).


<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##             main    #6251       +/-   ##
===========================================
- Coverage   88.42%   17.43%   -71.00%     
===========================================
  Files         323      323               
  Lines       67629    67607       -22     
===========================================
- Hits        59800    11786    -48014     
- Misses       7829    55821    +47992     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/holoviz/holoviews/pull/6251?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/holoviz/holoviews/pull/6251?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 17.43%. Comparing base [(`d458f60`)](https://app.codecov.io/gh/holoviz/holoviews/commit/d458f604ecf07bd264bef0dab788c742d30e88c9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) to head [(`2d4a4a7`)](https://app.codecov.io/gh/holoviz/holoviews/commit/2d4a4a7768742726bd4aeb3f77c5110559030dc1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).


<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##             main    #6251       +/-   ##
===========================================
- Coverage   88.42%   17.43%   -71.00%     
===========================================
  Files         323      323               
  Lines       67629    67607       -22     
===========================================
- Hits        59800    11786    -48014     
- Misses       7829    55821    +47992     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/holoviz/holoviews/pull/6251?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).


Patch:
diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml
index bca34dfa8..660cbf350 100644
--- a/.github/workflows/docs.yaml
+++ b/.github/workflows/docs.yaml
@@ -39,13 +39,7 @@ jobs:
     outputs:
       tag: ${{ steps.vars.outputs.tag }}
     env:
-      MPLBACKEND: "Agg"
       GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      MOZ_HEADLESS: 1
-      PANEL_EMBED: "true"
-      PANEL_EMBED_JSON: "true"
-      PANEL_EMBED_JSON_PREFIX: "json"
-      DASK_DATAFRAME__QUERY_PLANNING: false
     steps:
       - uses: holoviz-dev/holoviz_tasks/pixi_install@pixi
         with:
diff --git a/holoviews/core/options.py b/holoviews/core/options.py
index 1bb4a7a12..35f7b4e79 100644
--- a/holoviews/core/options.py
+++ b/holoviews/core/options.py
@@ -1253,7 +1253,7 @@ class Store:
         parameterized_class = (isinstance(obj,type)
                                and  issubclass(obj,param.Parameterized))
         info = None
-        if parameterized_object or parameterized_class:
+        if InfoPrinter.store and (parameterized_object or parameterized_class):
             info = InfoPrinter.info(obj, ansi=ansi, backend=backend,
                                     visualization=visualization,
                                     pattern=pattern, elements=elements)
diff --git a/pixi.toml b/pixi.toml
index bf8e0bd0c..bf8954c53 100644
--- a/pixi.toml
+++ b/pixi.toml
@@ -8,6 +8,10 @@ check-latest-packages = 'python scripts/check_latest_packages.py'
 download-data = 'python scripts/download_data.py'
 install = 'python -m pip install --no-deps --disable-pip-version-check -e .'
 
+[activation.env]
+DASK_DATAFRAME__QUERY_PLANNING = "False"
+PYTHONIOENCODING = "utf-8"
+
 [environments]
 test-39 = ["py39", "test-core", "test", "example", "test-example", "test-unit-task"]
 test-310 = ["py310", "test-core", "test", "example", "test-example", "test-unit-task"]
@@ -70,18 +74,19 @@ xarray = ">=0.10.4"
 # =================== TESTS ===================
 # =============================================
 [feature.test-core.dependencies]
+psutil = "*"
 pytest = "*"
 pytest-cov = "*"
 pytest-github-actions-annotate-failures = "*"
 pytest-rerunfailures = "*"
+pytest-xdist = "*"
 
 [feature.test-unit-task.tasks] # So it is not showing up in the test-ui environment
-test-unit = 'pytest holoviews/tests'
+test-unit = 'pytest holoviews/tests -n logical --dist loadgroup'
 
 [feature.test.dependencies]
 cftime = "*"
 contourpy = "*"
-# dash >=1.16
 dask-core = "*"
 datashader = ">=0.11.1"
 ffmpeg = "*"
@@ -103,9 +108,7 @@ tsdownsample = "*" # currently not available on Windows
 test-example = 'pytest -n logical --dist loadscope --nbval-lax examples'
 
 [feature.test-example.dependencies]
-psutil = "*"
 nbval = "*"
-pytest-xdist = "*"
 
 [feature.test-ui]
 channels = ["pyviz/label/dev", "microsoft", "conda-forge"]
@@ -125,6 +128,9 @@ depends_on = ["_install-ui"]
 channels = ["pyviz/label/dev", "rapidsai", "conda-forge"]
 platforms = ["linux-64"]
 
+[feature.test-gpu.activation.env]
+NUMBA_CUDA_LOW_OCCUPANCY_WARNINGS = "0"
+
 [feature.test-gpu.dependencies]
 cuda-version = "12.2.*"
 cudf = "24.04.*"
@@ -133,7 +139,7 @@ cuspatial = "*"
 rmm = { version = "*", channel = "rapidsai" }
 
 [feature.test-gpu.tasks]
-test-gpu = { cmd = "pytest holoviews/tests --gpu", env = { NUMBA_CUDA_LOW_OCCUPANCY_WARNINGS = '0' } }
+test-gpu = "pytest holoviews/tests --gpu"
 
 [feature.numpy2]
 channels = ["pyviz/label/dev", "conda-forge/label/numpy_rc", "numba/label/dev", "conda-forge"]
@@ -178,6 +184,13 @@ pooch = "*"
 python-kaleido = "*"
 selenium = "*"
 
+[feature.doc.activation.env]
+MOZ_HEADLESS = "1"
+MPLBACKEND = "Agg"
+PANEL_EMBED = "true"
+PANEL_EMBED_JSON = "true"
+PANEL_EMBED_JSON_PREFIX = "json"
+
 [feature.doc.tasks]
 _docs-generate-rst = 'nbsite generate-rst --org holoviz --project-name holoviews'
 _docs-refmanual = 'python ./doc/generate_modules.py holoviews -d ./doc/reference_manual -n holoviews -e tests'
diff --git a/pyproject.toml b/pyproject.toml
index a1d48ea35..319326779 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -160,7 +160,7 @@ ignore = [
     "RUF005", # Consider {expr} instead of concatenation
     "RUF012", # Mutable class attributes should use `typing.ClassVar`
 ]
-unfixable = [
+extend-unsafe-fixes = [
     "F401", # Unused imports
     "F841", # Unused variables
 ]


Test patch:
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
index 1a8340ad1..d18cf85c8 100644
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -34,9 +34,7 @@ defaults:
 
 env:
   DISPLAY: ":99.0"
-  PYTHONIOENCODING: "utf-8"
   GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-  DASK_DATAFRAME__QUERY_PLANNING: false
   COV: "--cov=./holoviews --cov-report=xml"
 
 jobs:
diff --git a/holoviews/tests/conftest.py b/holoviews/tests/conftest.py
index c0b202f58..abc573291 100644
--- a/holoviews/tests/conftest.py
+++ b/holoviews/tests/conftest.py
@@ -79,10 +79,10 @@ def _plotting_backend(backend):
     if not hv.extension._loaded:
         hv.extension(backend)
     hv.renderer(backend)
-    prev_backend = hv.Store.current_backend
-    hv.Store.current_backend = backend
+    curent_backend = hv.Store.current_backend
+    hv.Store.set_current_backend(backend)
     yield
-    hv.Store.current_backend = prev_backend
+    hv.Store.set_current_backend(curent_backend)
 
 
 @pytest.fixture
@@ -125,3 +125,17 @@ def serve_hv(page, port):  # noqa: F811
         return page
 
     return serve_and_return_page
+
+
+@pytest.fixture(autouse=True)
+def reset_store():
+    _custom_options = {k: {} for k in hv.Store._custom_options}
+    _options = hv.Store._options.copy()
+    current_backend = hv.Store.current_backend
+    renderers = hv.Store.renderers.copy()
+    yield
+    hv.Store._custom_options = _custom_options
+    hv.Store._options = _options
+    hv.Store._weakrefs = {}
+    hv.Store.renderers = renderers
+    hv.Store.set_current_backend(current_backend)
diff --git a/holoviews/tests/core/data/test_imageinterface.py b/holoviews/tests/core/data/test_imageinterface.py
index 71630c05f..6d9b10048 100644
--- a/holoviews/tests/core/data/test_imageinterface.py
+++ b/holoviews/tests/core/data/test_imageinterface.py
@@ -2,6 +2,7 @@ import datetime as dt
 from unittest import SkipTest
 
 import numpy as np
+import pytest
 
 from holoviews import HSV, RGB, Curve, Dataset, Dimension, Image, Table
 from holoviews.core.data.interface import DataError
@@ -465,17 +466,9 @@ class RGBElement_ImageInterfaceTests(BaseRGBElementInterfaceTests):
 
     __test__ = True
 
+    @pytest.mark.xfail(reason="Raises DataError in ImageInterface")
     def test_reduce_to_single_values(self):
-        try:
-            super().test_reduce_to_single_values()
-        except DataError:
-            msg = (
-                "RGB element can't run with this command: "
-                "'pytest holoviews/tests -k test_reduce_to_single_values'"
-                "but runs fine with 'pytest holoviews/tests/core/'"
-            )
-            raise SkipTest(msg)
-
+        super().test_reduce_to_single_values()
 
 class BaseHSVElementInterfaceTests(InterfaceTests):
 
diff --git a/holoviews/tests/core/test_dimensioned.py b/holoviews/tests/core/test_dimensioned.py
index fe4179c1c..f65060dea 100644
--- a/holoviews/tests/core/test_dimensioned.py
+++ b/holoviews/tests/core/test_dimensioned.py
@@ -29,17 +29,6 @@ class CustomBackendTestCase(LoggingComparisonTestCase):
         self.register_custom(ExampleElement, 'backend_2', ['plot_custom2'])
         Store.set_current_backend('backend_1')
 
-    def tearDown(self):
-        super().tearDown()
-        Store._weakrefs = {}
-        Store._options.pop('backend_1')
-        Store._options.pop('backend_2')
-        Store._custom_options.pop('backend_1')
-        Store._custom_options.pop('backend_2')
-        Store.set_current_backend(self.current_backend)
-        Store.renderers.pop('backend_1')
-        Store.renderers.pop('backend_2')
-
     @classmethod
     def register_custom(cls, objtype, backend, custom_plot=None, custom_style=None):
         if custom_style is None:
diff --git a/holoviews/tests/core/test_options.py b/holoviews/tests/core/test_options.py
index 3ba86ad92..e9aa318ff 100644
--- a/holoviews/tests/core/test_options.py
+++ b/holoviews/tests/core/test_options.py
@@ -1,5 +1,4 @@
 import contextlib
-import os
 import pickle
 from unittest import SkipTest
 
@@ -301,7 +300,6 @@ class TestStoreInheritanceDynamic(ComparisonTestCase):
 
     def tearDown(self):
         Store.options(val=self.store_copy)
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
         super().tearDown()
 
     def initialize_option_tree(self):
@@ -504,7 +502,6 @@ class TestStoreInheritance(ComparisonTestCase):
 
     def tearDown(self):
         Store.options(val=self.store_copy)
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
         super().tearDown()
 
     def lookup_options(self, obj, group):
@@ -574,7 +571,6 @@ class TestOptionsMethod(ComparisonTestCase):
 
     def tearDown(self):
         Store.options(val=self.store_copy)
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
         super().tearDown()
 
     def lookup_options(self, obj, group):
@@ -627,7 +623,6 @@ class TestOptsMethod(ComparisonTestCase):
 
     def tearDown(self):
         Store.options(val=self.store_copy)
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
         super().tearDown()
 
     def lookup_options(self, obj, group):
@@ -733,7 +728,6 @@ class TestOptionTreeFind(ComparisonTestCase):
     def tearDown(self):
         Options._option_groups = self.original_option_groups
         Store.options(val=self.original_options)
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
 
     def test_optiontree_find1(self):
         self.assertEqual(self.options.find('MyType').options('group').options,
@@ -814,7 +808,6 @@ class TestCrossBackendOptions(ComparisonTestCase):
         Store.options(val=self.store_mpl, backend='matplotlib')
         Store.options(val=self.store_bokeh, backend='bokeh')
         Store.current_backend = 'matplotlib'
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
 
         if self.plotly_options is not None:
             Store._options['plotly'] = self.plotly_options
@@ -965,6 +958,7 @@ class TestLookupOptions(ComparisonTestCase):
             self.assertNotIn("muted_alpha", options_matplotlib.keys())
 
 
+@pytest.mark.usefixtures("bokeh_backend")
 class TestCrossBackendOptionSpecification(ComparisonTestCase):
     """
     Test the style system can style a single object across backends.
@@ -989,7 +983,6 @@ class TestCrossBackendOptionSpecification(ComparisonTestCase):
         Store.options(val=self.store_mpl, backend='matplotlib')
         Store.options(val=self.store_bokeh, backend='bokeh')
         Store.current_backend = 'matplotlib'
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
 
         if self.plotly_options is not None:
             Store._options['plotly'] = self.plotly_options
@@ -1121,8 +1114,6 @@ class TestCrossBackendOptionSpecification(ComparisonTestCase):
 
 class TestCrossBackendOptionPickling(TestCrossBackendOptions):
 
-    cleanup = ['test_raw_pickle.pkl', 'test_pickle_mpl_bokeh.pkl']
-
     def setUp(self):
         super().setUp()
         self.raw = Image(np.random.rand(10,10))
@@ -1131,19 +1122,24 @@ class TestCrossBackendOptionPickling(TestCrossBackendOptions):
         Store.current_backend = 'bokeh'
         StoreOptions.set_options(self.raw, style={'Image':{'cmap':'Purple'}})
 
-    def tearDown(self):
-        super().tearDown()
-        for f in self.cleanup:
-            try:
-                os.remove(f)
-            except Exception:
-                pass
+    def _create_tmp_path(self):
+        # Remove this when we use pytest
+        import tempfile
+        from pathlib import Path
+
+        tmp_dir = tempfile.TemporaryDirectory()
+        tmp_path = Path(tmp_dir.name)
+
+        return tmp_dir, tmp_path
+
 
     def test_raw_pickle(self):
         """
         Test usual pickle saving and loading (no style information preserved)
         """
-        fname= 'test_raw_pickle.pkl'
+        tmp_dir, tmp_path = self._create_tmp_path()  # Pytest remove
+
+        fname= tmp_path / 'test_raw_pickle.pkl'
         with open(fname,'wb') as handle:
             pickle.dump(self.raw, handle)
         self.clear_options()
@@ -1160,11 +1156,15 @@ class TestCrossBackendOptionPickling(TestCrossBackendOptions):
         bokeh_opts = Store.lookup_options('bokeh', img, 'style').options
         self.assertEqual(bokeh_opts, {})
 
+        tmp_dir.cleanup()  # Pytest remove
+
     def test_pickle_mpl_bokeh(self):
         """
         Test pickle saving and loading with Store (style information preserved)
         """
-        fname = 'test_pickle_mpl_bokeh.pkl'
+        tmp_dir, tmp_path = self._create_tmp_path()  # Pytest remove
+
+        fname = tmp_path / 'test_pickle_mpl_bokeh.pkl'
         with open(fname,'wb') as handle:
             Store.dump(self.raw, handle)
         self.clear_options()
@@ -1180,3 +1180,5 @@ class TestCrossBackendOptionPickling(TestCrossBackendOptions):
         Store.current_backend = 'bokeh'
         bokeh_opts = Store.lookup_options('bokeh', img, 'style').options
         self.assertEqual(bokeh_opts, {'cmap':'Purple'})
+
+        tmp_dir.cleanup()  # Pytest remove
diff --git a/holoviews/tests/ipython/test_displayhooks.py b/holoviews/tests/ipython/test_displayhooks.py
index cf1a7ba3e..d8b20b5c2 100644
--- a/holoviews/tests/ipython/test_displayhooks.py
+++ b/holoviews/tests/ipython/test_displayhooks.py
@@ -16,7 +16,6 @@ class TestDisplayHooks(IPTestCase):
         Store.display_formats = self.format
 
     def tearDown(self):
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
         self.ip.run_line_magic("unload_ext", "holoviews.ipython")
         del self.ip
         Store.display_hooks = self.backup
diff --git a/holoviews/tests/ipython/test_magics.py b/holoviews/tests/ipython/test_magics.py
index ec7055616..cdda43f09 100644
--- a/holoviews/tests/ipython/test_magics.py
+++ b/holoviews/tests/ipython/test_magics.py
@@ -1,17 +1,14 @@
-from unittest import SkipTest
+import pytest
+from pyviz_comms import CommManager
 
 import holoviews as hv
 from holoviews.core.options import Store
+from holoviews.operation import Compositor
 
 try:
-    from holoviews import ipython            # noqa (Import test)
     from holoviews.ipython import IPTestCase
 except ImportError:
-    raise SkipTest("Required dependencies not satisfied for testing magics")
-
-from pyviz_comms import CommManager
-
-from holoviews.operation import Compositor
+    pytest.skip("IPython required to test IPython magics", allow_module_level=True)
 
 
 class ExtensionTestCase(IPTestCase):
@@ -20,10 +17,9 @@ class ExtensionTestCase(IPTestCase):
         super().setUp()
         self.ip.run_line_magic("load_ext", "holoviews.ipython")
         for renderer in Store.renderers.values():
-            renderer.comm_manager = CommManager
+            renderer.comm_manager = CommManager  # TODO: Should set it back
 
     def tearDown(self):
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
         self.ip.run_line_magic("unload_ext", "holoviews.ipython")
         del self.ip
         super().tearDown()
@@ -53,6 +49,7 @@ class TestOptsMagic(ExtensionTestCase):
             Store.lookup_options('matplotlib',
                                  self.get_object('mat1'), 'style').options.get('cmap',None),'hot')
 
+    @pytest.mark.flaky(reruns=3)
     def test_cell_opts_style_dynamic(self):
 
         self.cell("dmap = DynamicMap(lambda X: Curve(np.random.rand(5,2), name='dmap'), kdims=['x'])"
@@ -95,6 +92,7 @@ class TestOptsMagic(ExtensionTestCase):
                                  self.get_object('mat1'), 'plot').options.get('show_title',True),False)
 
 
+    @pytest.mark.flaky(reruns=3)
     def test_cell_opts_plot_dynamic(self):
 
         self.cell("dmap = DynamicMap(lambda X: Image(np.random.rand(5,5), name='dmap'), kdims=['x'])"
diff --git a/holoviews/tests/plotting/bokeh/test_callbacks.py b/holoviews/tests/plotting/bokeh/test_callbacks.py
index 29d7c63a7..1d6e523c8 100644
--- a/holoviews/tests/plotting/bokeh/test_callbacks.py
+++ b/holoviews/tests/plotting/bokeh/test_callbacks.py
@@ -127,6 +127,7 @@ class TestCallbacks(CallbackTestCase):
         self.assertEqual(data['x'], np.arange(10))
         self.assertEqual(data['y'], np.arange(10, 20))
 
+    @pytest.mark.flaky(reruns=3)
     def test_callback_cleanup(self):
         stream = PointerX(x=0)
         dmap = DynamicMap(lambda x: Curve([x]), streams=[stream])
diff --git a/holoviews/tests/plotting/bokeh/test_curveplot.py b/holoviews/tests/plotting/bokeh/test_curveplot.py
index 5a585e55f..0ef1b5b8e 100644
--- a/holoviews/tests/plotting/bokeh/test_curveplot.py
+++ b/holoviews/tests/plotting/bokeh/test_curveplot.py
@@ -32,6 +32,7 @@ class TestCurvePlot(TestBokehPlot):
     def test_batched_curve_subscribers_correctly_linked(self):
         # Checks if a stream callback is created to link batched plot
         # to the stream
+        Callback._callbacks.clear()  # Reset to not be sensitive to other test
         posx = PointerX()
         opts = {'NdOverlay': dict(legend_limit=0),
                 'Curve': dict(line_color=Cycle(values=['red', 'blue']))}
diff --git a/holoviews/tests/plotting/bokeh/test_server.py b/holoviews/tests/plotting/bokeh/test_server.py
index 5686bfbff..418170a35 100644
--- a/holoviews/tests/plotting/bokeh/test_server.py
+++ b/holoviews/tests/plotting/bokeh/test_server.py
@@ -80,6 +80,7 @@ class TestBokehServerSetup(ComparisonTestCase):
 
 
 
+@pytest.mark.flaky(reruns=3)
 class TestBokehServer(ComparisonTestCase):
 
     def setUp(self):
@@ -128,7 +129,6 @@ class TestBokehServer(ComparisonTestCase):
         self.assertEqual(cb.streams, [stream])
         assert 'rangesupdate' in plot.state._event_callbacks
 
-    @pytest.mark.flaky(reruns=3)
     def test_launch_server_with_complex_plot(self):
         dmap = DynamicMap(lambda x_range, y_range: Curve([]), streams=[RangeXY()])
         overlay = dmap * HLine(0)
diff --git a/holoviews/tests/plotting/matplotlib/test_renderer.py b/holoviews/tests/plotting/matplotlib/test_renderer.py
index 0c8f55aa2..569a3f02d 100644
--- a/holoviews/tests/plotting/matplotlib/test_renderer.py
+++ b/holoviews/tests/plotting/matplotlib/test_renderer.py
@@ -150,6 +150,7 @@ class MPLRendererTest(ComparisonTestCase):
         self.assertEqual(obj.widget_location, 'top')
         self.assertEqual(obj.widget_type, 'individual')
 
+    @pytest.mark.filterwarnings('ignore:Attempted to send message over Jupyter Comm:UserWarning')
     def test_render_dynamicmap_with_dims(self):
         dmap = DynamicMap(lambda y: Curve([1, 2, y]), kdims=['y']).redim.range(y=(0.1, 5))
         obj, _ = self.renderer._validate(dmap, None)
@@ -164,6 +165,7 @@ class MPLRendererTest(ComparisonTestCase):
         (_, y) = artist.get_data()
         self.assertEqual(y[2], 3.1)
 
+    @pytest.mark.filterwarnings('ignore:Attempted to send message over Jupyter Comm:UserWarning')
     def test_render_dynamicmap_with_stream(self):
         stream = Stream.define('Custom', y=2)()
         dmap = DynamicMap(lambda y: Curve([1, 2, y]), kdims=['y'], streams=[stream])
@@ -178,6 +180,7 @@ class MPLRendererTest(ComparisonTestCase):
         (_, y) = artist.get_data()
         self.assertEqual(y[2], 3)
 
+    @pytest.mark.filterwarnings('ignore:Attempted to send message over Jupyter Comm:UserWarning')
     def test_render_dynamicmap_with_stream_dims(self):
         stream = Stream.define('Custom', y=2)()
         dmap = DynamicMap(lambda x, y: Curve([x, 1, y]), kdims=['x', 'y'],
diff --git a/holoviews/tests/plotting/plotly/test_renderer.py b/holoviews/tests/plotting/plotly/test_renderer.py
index 3dfce0870..4e25d32d6 100644
--- a/holoviews/tests/plotting/plotly/test_renderer.py
+++ b/holoviews/tests/plotting/plotly/test_renderer.py
@@ -1,9 +1,9 @@
 """
 Test cases for rendering exporters
 """
-
 import panel as pn
 import param
+import pytest
 from panel.widgets import DiscreteSlider, FloatSlider, Player
 from pyviz_comms import CommManager
 
@@ -92,6 +92,7 @@ class PlotlyRendererTest(ComparisonTestCase):
         self.assertEqual(obj.widget_location, 'top')
         self.assertEqual(obj.widget_type, 'individual')
 
+    @pytest.mark.filterwarnings('ignore:Attempted to send message over Jupyter Comm:UserWarning')
     def test_render_dynamicmap_with_dims(self):
         dmap = DynamicMap(lambda y: Curve([1, 2, y]), kdims=['y']).redim.range(y=(0.1, 5))
         obj, _ = self.renderer._validate(dmap, None)
@@ -105,6 +106,7 @@ class PlotlyRendererTest(ComparisonTestCase):
         y = plot.handles['fig']['data'][0]['y']
         self.assertEqual(y[2], 3.1)
 
+    @pytest.mark.filterwarnings('ignore:Attempted to send message over Jupyter Comm:UserWarning')
     def test_render_dynamicmap_with_stream(self):
         stream = Stream.define('Custom', y=2)()
         dmap = DynamicMap(lambda y: Curve([1, 2, y]), kdims=['y'], streams=[stream])
@@ -118,6 +120,7 @@ class PlotlyRendererTest(ComparisonTestCase):
         y = plot.handles['fig']['data'][0]['y']
         self.assertEqual(y[2], 3)
 
+    @pytest.mark.filterwarnings('ignore:Attempted to send message over Jupyter Comm:UserWarning')
     def test_render_dynamicmap_with_stream_dims(self):
         stream = Stream.define('Custom', y=2)()
         dmap = DynamicMap(lambda x, y: Curve([x, 1, y]), kdims=['x', 'y'],
diff --git a/holoviews/tests/util/test_help.py b/holoviews/tests/util/test_help.py
index 9b1975168..4b7945b8a 100644
--- a/holoviews/tests/util/test_help.py
+++ b/holoviews/tests/util/test_help.py
@@ -9,3 +9,15 @@ def test_help_pattern(capsys):
     hv.help(hv.Curve, pattern='border')
     captured = capsys.readouterr()
     assert '\x1b[43;1;30mborder\x1b[0m' in captured.out
+
+
+@pytest.mark.usefixtures("bokeh_backend")
+def test_help_pattern_no_ipython(capsys):
+    info_store = hv.core.pprint.InfoPrinter.store
+    try:
+        hv.core.pprint.InfoPrinter.store = None
+        hv.help(hv.Curve)
+        captured = capsys.readouterr()
+        assert captured.out.startswith('Help on class Curve')
+    finally:
+        hv.core.pprint.InfoPrinter.store = info_store
diff --git a/holoviews/tests/util/test_utils.py b/holoviews/tests/util/test_utils.py
index 4355ebcb4..0d170965b 100644
--- a/holoviews/tests/util/test_utils.py
+++ b/holoviews/tests/util/test_utils.py
@@ -93,7 +93,6 @@ class TestOptsUtil(LoggingComparisonTestCase):
     def tearDown(self):
         Store.current_backend = self.backend
         Store.options(val=self.store_copy)
-        Store._custom_options = {k:{} for k in Store._custom_options.keys()}
         super().tearDown()
 
     def test_opts_builder_repr(self):