Repository: holoviz/holoviews
Pull request #6238 (2024-05-18 08:09:21)

Numpy 2.0 compability

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) Report
Attention: Patch coverage is `0%` with `11 lines` in your changes are missing coverage. Please review.
> Project coverage is 17.43%. Comparing base [(`945e179`)](https://app.codecov.io/gh/holoviz/holoviews/commit/945e1798fa7c7b97a1135e8db78eac3b994541c1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) to head [(`f2739ab`)](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).

| [Files](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) | Patch % | Lines |
|---|---|---|
| [...oloviews/tests/element/test\_comparisondimension.py](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?src=pr&el=tree&filepath=holoviews%2Ftests%2Felement%2Ftest_comparisondimension.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz#diff-aG9sb3ZpZXdzL3Rlc3RzL2VsZW1lbnQvdGVzdF9jb21wYXJpc29uZGltZW5zaW9uLnB5) | 0.00% | [6 Missing :warning: ](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) |
| [holoviews/tests/core/test\_dimensions.py](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?src=pr&el=tree&filepath=holoviews%2Ftests%2Fcore%2Ftest_dimensions.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz#diff-aG9sb3ZpZXdzL3Rlc3RzL2NvcmUvdGVzdF9kaW1lbnNpb25zLnB5) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) |
| [holoviews/core/sheetcoords.py](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?src=pr&el=tree&filepath=holoviews%2Fcore%2Fsheetcoords.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz#diff-aG9sb3ZpZXdzL2NvcmUvc2hlZXRjb29yZHMucHk=) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz) |

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##             main    #6238       +/-   ##
===========================================
- Coverage   88.39%   17.43%   -70.97%     
===========================================
  Files         323      323               
  Lines       67620    67606       -14     
===========================================
- Hits        59770    11784    -47986     
- Misses       7850    55822    +47972     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/holoviz/holoviews/pull/6238?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=holoviz).


Patch:
diff --git a/.github/workflows/nightly_lock.yaml b/.github/workflows/nightly_lock.yaml
index c6fda817a..9b93b85a2 100644
--- a/.github/workflows/nightly_lock.yaml
+++ b/.github/workflows/nightly_lock.yaml
@@ -4,6 +4,9 @@ on:
   schedule:
     - cron: "0 0 * * *"
 
+env:
+  PACKAGE: "holoviews"
+
 jobs:
   pixi_lock:
     name: Pixi lock
@@ -12,11 +15,11 @@ jobs:
     steps:
       - uses: holoviz-dev/holoviz_tasks/pixi_lock@pixi
       - name: Upload lock-file to S3
+        if: "!github.event.pull_request.head.repo.fork"
         env:
           AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           AWS_DEFAULT_REGION: "eu-west-1"
-          PACKAGE: "holoviews"
         run: |
           zip $(date +%Y-%m-%d).zip pixi.lock pixi.toml
           aws s3 cp ./$(date +%Y-%m-%d).zip s3://assets.holoviz.org/lock/$PACKAGE/
diff --git a/examples/README.md b/examples/README.md
index c8a747e9d..7e958bc6b 100644
--- a/examples/README.md
+++ b/examples/README.md
@@ -9,7 +9,6 @@ This directory contains all the notebooks built as part of the
 - `gallery`: Examples shown on the [gallery page](https://holoviews.org/gallery/index.html).
 - `getting_started`: Notebooks used in the [getting started](https://holoviews.org/getting_started/index.html) guide.
 - `reference`: Notebooks shown in the website [reference gallery](https://holoviews.org/reference/index.html)
-- `topics`: Notebooks shown in the [showcase](https://holoviews.org/reference/showcase/index.html)
 - `user_guide`: Notebooks used in the [user guide](https://holoviews.org/user_guide/index.html).
 
 ## Contributing to examples
diff --git a/examples/user_guide/Deploying_Bokeh_Apps.ipynb b/examples/user_guide/Deploying_Bokeh_Apps.ipynb
index 139df4592..6b8bdfd6c 100644
--- a/examples/user_guide/Deploying_Bokeh_Apps.ipynb
+++ b/examples/user_guide/Deploying_Bokeh_Apps.ipynb
@@ -789,7 +789,7 @@
    "metadata": {},
    "source": [
     "If instead we want to deploy this we could add `.servable` as discussed before or use `pn.serve`. Note however that when using `pn.serve` all sessions will share the same state therefore it is best to \n",
-    "wrap the creation of the app in a function which we can then provide to `pn.serve`. For more detail on deploying Panel applications also see the [Panel server deployment guide](https://panel.holoviz.org/user_guide/Server_Deployment.html).\n",
+    "wrap the creation of the app in a function which we can then provide to `pn.serve`. For more detail on deploying Panel applications also see the [Panel server deployment guide](https://panel.holoviz.org/how_to/server/index.html).\n",
     "\n",
     "Now we can reimplement the same example using Bokeh allowing us to compare and contrast the approaches:"
    ]
diff --git a/holoviews/core/sheetcoords.py b/holoviews/core/sheetcoords.py
index a6e8d163c..690a55d70 100644
--- a/holoviews/core/sheetcoords.py
+++ b/holoviews/core/sheetcoords.py
@@ -365,7 +365,7 @@ class Slice(np.ndarray):
         else:
             slicespec=Slice._boundsspec2slicespec(bounds.lbrt(),sheet_coordinate_system)
         # Using numpy.int32 for legacy reasons
-        a = np.array(slicespec, dtype=np.int32, copy=False).view(cls)
+        a = np.asarray(slicespec, dtype=np.int32).view(cls)
         return a
 
 
diff --git a/holoviews/core/util.py b/holoviews/core/util.py
index c2ff670d5..2e2732bfb 100644
--- a/holoviews/core/util.py
+++ b/holoviews/core/util.py
@@ -50,6 +50,9 @@ _NP_SAMPLE_SIZE = 1_000_000
 _PANDAS_ROWS_LARGE = 1_000_000
 _PANDAS_SAMPLE_SIZE = 1_000_000
 
+numpy_version = Version(Version(np.__version__).base_version)
+NUMPY_GE_200 = numpy_version >= Version("2")
+
 pandas_version = Version(pd.__version__)
 try:
     if pandas_version >= Version('1.3.0'):
diff --git a/pixi.toml b/pixi.toml
index 7e6a08d23..bf8e0bd0c 100644
--- a/pixi.toml
+++ b/pixi.toml
@@ -15,6 +15,7 @@ test-311 = ["py311", "test-core", "test", "example", "test-example", "test-unit-
 test-312 = ["py312", "test-core", "test", "example", "test-example", "test-unit-task"]
 test-ui = ["py312", "test-core", "test", "test-ui"]
 test-core = ["py312", "test-core", "test-unit-task"]
+test-numpy = ["py312", "test-core", "test-unit-task", "numpy2", "test-example"]
 test-gpu = ["py311", "test-core", "test", "test-gpu"]
 docs = ["py311", "example", "doc"]
 build = ["py311", "build"]
@@ -134,6 +135,39 @@ rmm = { version = "*", channel = "rapidsai" }
 [feature.test-gpu.tasks]
 test-gpu = { cmd = "pytest holoviews/tests --gpu", env = { NUMBA_CUDA_LOW_OCCUPANCY_WARNINGS = '0' } }
 
+[feature.numpy2]
+channels = ["pyviz/label/dev", "conda-forge/label/numpy_rc", "numba/label/dev", "conda-forge"]
+
+[feature.numpy2.dependencies]
+numpy = "2.*"
+numba = { version = "0.60.*", channel = "numba/label/dev" }
+llvmlite = { version = "*", channel = "numba/label/dev" }
+
+# test dependencies
+cftime = "*"
+contourpy = "*"
+dask-core = "*"
+datashader = ">=0.11.1"
+ffmpeg = "*"
+ibis-sqlite = "*"
+nbconvert = "*"
+networkx = "*"
+pillow = "*"
+scipy = ">=1.10" # Python 3.9 + Windows downloads 1.9
+selenium = "*"
+shapely = "*"
+# spatialpandas = "*"
+xarray = ">=0.10.4"
+xyzservices = "*"
+
+# Examples (removed duplicates)
+# netcdf4 = "*"
+notebook = "*"
+pooch = "*"
+# pyarrow = "*"
+# scikit-image = "*"
+streamz = ">=0.5.0"
+
 # =============================================
 # =================== DOCS ====================
 # =============================================
diff --git a/pyproject.toml b/pyproject.toml
index ed94c5e2d..fb9b916db 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -90,7 +90,7 @@ filterwarnings = [
     # 2023-01: Sqlalchemy 2.0 warning:
     "ignore: Deprecated API features detected:DeprecationWarning:ibis.backends.base.sql.alchemy", # https://github.com/ibis-project/ibis/issues/5048
     # 2023-03: Already handling the nested sequence
-    "ignore:Creating an ndarray from ragged nested sequences:numpy.VisibleDeprecationWarning:holoviews.core.data.spatialpandas",
+    "ignore:Creating an ndarray from ragged nested sequences::holoviews.core.data.spatialpandas",
     # 2023-09: Dash needs to update their code to use the comm module and pkg_resources
     "ignore:The `.+?` class has been deprecated:DeprecationWarning:dash._jupyter",
     "ignore:pkg_resources is deprecated as an API:DeprecationWarning:dash.dash",


Test patch:
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
index b16390f64..1a8340ad1 100644
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@@ -85,7 +85,10 @@ jobs:
         run: |
           MATRIX=$(jq -nsc '{
               "os": ["ubuntu-latest", "macos-latest", "windows-latest"],
-              "environment": ["test-39", "test-312"]
+              "environment": ["test-39", "test-312"],
+              "include": [
+                { "os": "ubuntu-latest", "environment": "test-numpy" }
+              ]
           }')
           echo "MATRIX=$MATRIX" >> $GITHUB_ENV
       - name: Set test matrix with 'full' option
@@ -93,7 +96,10 @@ jobs:
         run: |
           MATRIX=$(jq -nsc '{
               "os": ["ubuntu-latest", "macos-latest", "windows-latest"],
-              "environment": ["test-39", "test-310", "test-311", "test-312"]
+              "environment": ["test-39", "test-310", "test-311", "test-312"],
+              "include": [
+                { "os": "ubuntu-latest", "environment": "test-numpy" }
+              ]
           }')
           echo "MATRIX=$MATRIX" >> $GITHUB_ENV
       - name: Set test matrix with 'downstream' option
diff --git a/examples/conftest.py b/examples/conftest.py
index 22dced713..588cabb54 100644
--- a/examples/conftest.py
+++ b/examples/conftest.py
@@ -1,6 +1,7 @@
 import os
 import platform
 import sys
+from importlib.util import find_spec
 
 import bokeh
 import pandas as pd
@@ -62,6 +63,22 @@ if system == "Windows" and os.environ.get("GITHUB_RUN_ID"):
         "user_guide/Plotting_with_Matplotlib.ipynb",
     ]
 
+# 2024-05: Numpy 2.0
+if find_spec("datashader") is None:
+    collect_ignore_glob += [
+        "reference/elements/matplotlib/ImageStack.ipynb",
+        "reference/elements/plotly/ImageStack.ipynb",
+        "user_guide/15-Large_Data.ipynb",
+        "user_guide/16-Streaming_Data.ipynb",
+        "user_guide/Linked_Brushing.ipynb",
+        "user_guide/Network_Graphs.ipynb",
+    ]
+
+if find_spec("scikit-image"):
+    collect_ignore_glob += [
+        "user_guide/Network_Graphs.ipynb",
+    ]
+
 
 def pytest_runtest_makereport(item, call):
     """
diff --git a/holoviews/tests/core/test_dimensions.py b/holoviews/tests/core/test_dimensions.py
index 06f77066b..7b79e1bce 100644
--- a/holoviews/tests/core/test_dimensions.py
+++ b/holoviews/tests/core/test_dimensions.py
@@ -5,6 +5,7 @@ import numpy as np
 import pandas as pd
 
 from holoviews.core import Dimension, Dimensioned
+from holoviews.core.util import NUMPY_GE_200
 from holoviews.element.comparison import ComparisonTestCase
 
 from ..utils import LoggingComparisonTestCase
@@ -243,7 +244,10 @@ class DimensionCloneTest(ComparisonTestCase):
 class DimensionDefaultTest(ComparisonTestCase):
 
     def test_validate_default_against_values(self):
-        msg = r"Dimension\('A'\) default 1\.1 not found in declared values: \[0, 1\]"
+        if NUMPY_GE_200:
+            msg = r"Dimension\('A'\) default 1\.1 not found in declared values: \[np\.int64\(0\), np\.int64\(1\)\]"
+        else:
+            msg = r"Dimension\('A'\) default 1\.1 not found in declared values: \[0, 1\]"
         with self.assertRaisesRegex(ValueError, msg):
             Dimension('A', values=[0, 1], default=1.1)
 
diff --git a/holoviews/tests/element/test_comparisondimension.py b/holoviews/tests/element/test_comparisondimension.py
index 1234ce27c..3bdfb04c4 100644
--- a/holoviews/tests/element/test_comparisondimension.py
+++ b/holoviews/tests/element/test_comparisondimension.py
@@ -2,6 +2,7 @@
 Test cases for Dimension and Dimensioned object comparison.
 """
 from holoviews.core import Dimension, Dimensioned
+from holoviews.core.util import NUMPY_GE_200
 from holoviews.element.comparison import ComparisonTestCase
 
 
@@ -74,7 +75,11 @@ class DimensionsComparisonTestCase(ComparisonTestCase):
         try:
             self.assertEqual(self.dimension4, self.dimension8)
         except AssertionError as e:
-            self.assertEqual(str(e), "Dimension parameter 'values' mismatched: [] != ['a', 'b']")
+            if NUMPY_GE_200:
+                msg = "Dimension parameter 'values' mismatched: [] != [np.str_('a'), np.str_('b')]"
+            else:
+                msg = "Dimension parameter 'values' mismatched: [] != ['a', 'b']"
+            self.assertEqual(str(e), msg)
 
     def test_dimension_comparison_types_unequal(self):
         try:
diff --git a/holoviews/tests/test_streams.py b/holoviews/tests/test_streams.py
index 17cb93bc3..2bc957f67 100644
--- a/holoviews/tests/test_streams.py
+++ b/holoviews/tests/test_streams.py
@@ -11,7 +11,7 @@ from panel.widgets import IntSlider
 
 import holoviews as hv
 from holoviews.core.spaces import DynamicMap
-from holoviews.core.util import Version
+from holoviews.core.util import NUMPY_GE_200, Version
 from holoviews.element import Curve, Histogram, Points, Polygons, Scatter
 from holoviews.element.comparison import ComparisonTestCase
 from holoviews.streams import *  # noqa (Test all available streams)
@@ -1421,6 +1421,7 @@ class TestExprSelectionStream(ComparisonTestCase):
 
 
     def test_selection_expr_stream_polygon_index_cols(self):
+        # TODO: Should test both spatialpandas and shapely
         # Create SelectionExpr on element
         try: import shapely # noqa
         except ImportError:
@@ -1444,10 +1445,12 @@ class TestExprSelectionStream(ComparisonTestCase):
         self.assertIsNone(expr_stream.bbox)
         self.assertIsNone(expr_stream.selection_expr)
 
+        fmt = lambda x: list(map(np.str_, x)) if NUMPY_GE_200 else x
+
         expr_stream.input_streams[2].event(index=[0, 1])
         self.assertEqual(
             repr(expr_stream.selection_expr),
-            repr(dim('cat').isin(['a', 'b']))
+            repr(dim('cat').isin(fmt(['a', 'b'])))
         )
         self.assertEqual(expr_stream.bbox, None)
         self.assertEqual(len(events), 1)
@@ -1456,7 +1459,7 @@ class TestExprSelectionStream(ComparisonTestCase):
         expr_stream.input_streams[0].event(bounds=(0, 0, 4, 1))
         self.assertEqual(
             repr(expr_stream.selection_expr),
-            repr(dim('cat').isin(['a', 'b']))
+            repr(dim('cat').isin(fmt(['a', 'b'])))
         )
         self.assertEqual(len(events), 1)
 
@@ -1464,7 +1467,7 @@ class TestExprSelectionStream(ComparisonTestCase):
         expr_stream.input_streams[1].event(geometry=np.array([(0, 0), (4, 0), (4, 2), (0, 2)]))
         self.assertEqual(
             repr(expr_stream.selection_expr),
-            repr(dim('cat').isin(['a', 'b', 'c']))
+            repr(dim('cat').isin(fmt(['a', 'b', 'c'])))
         )
         self.assertEqual(len(events), 2)
 
@@ -1472,7 +1475,7 @@ class TestExprSelectionStream(ComparisonTestCase):
         expr_stream.input_streams[2].event(index=[1, 2])
         self.assertEqual(
             repr(expr_stream.selection_expr),
-            repr(dim('cat').isin(['b', 'c']))
+            repr(dim('cat').isin(fmt(['b', 'c'])))
         )
         self.assertEqual(expr_stream.bbox, None)
         self.assertEqual(len(events), 3)