Repository: plotly/plotly.py

Issue:
DatetimeProperties.to_pydatetime is deprecated
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/_plotly_utils/basevalidators.py:105: FutureWarning:

The behavior of DatetimeProperties.to_pydatetime is deprecated, in a future version this will return a Series containing python datetime objects instead of an ndarray. To retain the old behavior, call `np.array` on the result




Discussion:
Same happens here!

It's a pandas issue. To remove the warning try downgrading to 2.0.3 instead of using the new 2.1.0
After upgrading to the latest Pandas, I'm now seeing this `FutureWarning` in a bunch of my plots made with Plotly Express that has datetimes. You can suppress the warning in the meantime, but if the behaviour is going to change, Plotly will need to make an appropriate change before this happens.

See the deprecation note at the top here: https://pandas.pydata.org/docs/dev/reference/api/pandas.Series.dt.to_pydatetime.html

It's unfortunate that the recommended approach for restoring the functionality of returning an numpy array is just to wrap the result in `np.array`, because the `FutureWarning` will still persist until whenever the behaviour change is implemented. Perhaps an alternative way of generating an array of datetimes could be used instead of using `Series.dt.to_pydatetime`?
hey @alexcjohnson how thorny do you think this one is? is there a simple change to a different way of producing numpy arrays that could be swapped over? if so I could be up for submitting a PR.
Hmph yeah frustrating that their recommendation does not silence the warning. We could do the same thing [Pandas does in their own code](https://github.com/pandas-dev/pandas/pull/52459/files#diff-9268174bfb15f08ef2267375665a85fecf201999902542f6fc9c0d3fadfb4553R968-R971) and filter the warning out while wrapping in `np.asarray`. Feels a little risky to me, what if we end up filtering out something else that still matters? I don't see another good option though. Or we could go for the [lower-level thing Pandas added to their code](https://github.com/pandas-dev/pandas/pull/52459/files#diff-9268174bfb15f08ef2267375665a85fecf201999902542f6fc9c0d3fadfb4553R973) ie use the private `ser._values.to_pydatetime()` which apparently is not deprecated, but that feels even more dangerous.
The first of those options feels like the best one to me. By using the `warnings.catch_warnings()` context manager to do the suppressing, it would be localising the risk of a false positive being suppressed to a very small surface area.

Here's a [proof of concept](https://github.com/plotly/plotly.py/commit/3c402a28f9b3039a50d57b0002ddedfdebd992b1) for `_plotly_utils/basevalidators.py`. There's also [two other modules](https://github.com/search?q=repo%3Aplotly%2Fplotly.py%20to_pydatetime&type=code) I see using `Series.dt.to_pydatetime()`, but they also look fairly low risk to me. 


Thanks @ned2 - let's go with that. Your POC looks great (but `nd` -> `np`?), and let's include the other modules you found as well!

Resolving patch:
diff --git a/packages/python/plotly/_plotly_utils/basevalidators.py b/packages/python/plotly/_plotly_utils/basevalidators.py
--- a/packages/python/plotly/_plotly_utils/basevalidators.py
+++ b/packages/python/plotly/_plotly_utils/basevalidators.py
@@ -7,6 +7,7 @@
 import io
 import re
 import sys
+import warnings
 
 from _plotly_utils.optional_imports import get_module
 
@@ -100,7 +101,11 @@ def copy_to_readonly_numpy_array(v, kind=None, force_numeric=False):
         elif v.dtype.kind == "M":
             # Convert datetime Series/Index to numpy array of datetimes
             if isinstance(v, pd.Series):
-                v = v.dt.to_pydatetime()
+                with warnings.catch_warnings():
+                    warnings.simplefilter("ignore", FutureWarning)
+                    # Series.dt.to_pydatetime will return Index[object]
+                    # https://github.com/pandas-dev/pandas/pull/52459
+                    v = np.array(v.dt.to_pydatetime())
             else:
                 # DatetimeIndex
                 v = v.to_pydatetime()
@@ -109,7 +114,13 @@ def copy_to_readonly_numpy_array(v, kind=None, force_numeric=False):
         if dtype.kind in numeric_kinds:
             v = v.values
         elif dtype.kind == "M":
-            v = [row.dt.to_pydatetime().tolist() for i, row in v.iterrows()]
+            with warnings.catch_warnings():
+                warnings.simplefilter("ignore", FutureWarning)
+                # Series.dt.to_pydatetime will return Index[object]
+                # https://github.com/pandas-dev/pandas/pull/52459
+                v = [
+                    np.array(row.dt.to_pydatetime()).tolist() for i, row in v.iterrows()
+                ]
 
     if not isinstance(v, np.ndarray):
         # v has its own logic on how to convert itself into a numpy array
diff --git a/packages/python/plotly/plotly/io/_json.py b/packages/python/plotly/plotly/io/_json.py
--- a/packages/python/plotly/plotly/io/_json.py
+++ b/packages/python/plotly/plotly/io/_json.py
@@ -1,6 +1,7 @@
 import json
 import decimal
 import datetime
+import warnings
 from pathlib import Path
 
 from plotly.io._utils import validate_coerce_fig_to_dict, validate_coerce_output_type
@@ -535,7 +536,11 @@ def clean_to_json_compatible(obj, **kwargs):
                 return np.ascontiguousarray(obj.values)
             elif obj.dtype.kind == "M":
                 if isinstance(obj, pd.Series):
-                    dt_values = obj.dt.to_pydatetime().tolist()
+                    with warnings.catch_warnings():
+                        warnings.simplefilter("ignore", FutureWarning)
+                        # Series.dt.to_pydatetime will return Index[object]
+                        # https://github.com/pandas-dev/pandas/pull/52459
+                        dt_values = np.array(obj.dt.to_pydatetime()).tolist()
                 else:  # DatetimeIndex
                     dt_values = obj.to_pydatetime().tolist()
 


Test patch:
diff --git a/packages/python/plotly/plotly/tests/test_io/test_to_from_plotly_json.py b/packages/python/plotly/plotly/tests/test_io/test_to_from_plotly_json.py
--- a/packages/python/plotly/plotly/tests/test_io/test_to_from_plotly_json.py
+++ b/packages/python/plotly/plotly/tests/test_io/test_to_from_plotly_json.py
@@ -8,6 +8,7 @@
 import datetime
 import re
 import sys
+import warnings
 from pytz import timezone
 from _plotly_utils.optional_imports import get_module
 
@@ -194,7 +195,13 @@ def to_str(v):
     if isinstance(datetime_array, list):
         dt_values = [to_str(d) for d in datetime_array]
     elif isinstance(datetime_array, pd.Series):
-        dt_values = [to_str(d) for d in datetime_array.dt.to_pydatetime().tolist()]
+        with warnings.catch_warnings():
+            warnings.simplefilter("ignore", FutureWarning)
+            # Series.dt.to_pydatetime will return Index[object]
+            # https://github.com/pandas-dev/pandas/pull/52459
+            dt_values = [
+                to_str(d) for d in np.array(datetime_array.dt.to_pydatetime()).tolist()
+            ]
     elif isinstance(datetime_array, pd.DatetimeIndex):
         dt_values = [to_str(d) for d in datetime_array.to_pydatetime().tolist()]
     else:  # numpy datetime64 array