Repository: dask/dask

Issue:
[FEA] Add official mechanism to check if query-planning is enabled in ``dask.dataframe``
The value of the `"dataframe.query-planning"` config currently dictates whether or not `dask.dataframe` will use dask-expr under the hood. However, after `dask.dataframe` is first imported, the value of the `"dataframe.query-planning"` config becomes somewhat useless.

For example, a down-stream library may wish to warn/raise if `dask.dataframe` is already imported with query-planning enabled, but the value of the `"dataframe.query-planning"` config will typically be `None` whether the module was already imported or not.

One way to mitigate this problem is for `dask.dataframe.__init__` to always convert `None` values into `True` (using the config system). This way, a down-stream library (or user) can interpret `None` as a confirmation that the query-planning config can still be modified. An alternative solution is to tell users to inspect `sys.modules` themselves to confirm that `dask.dataframe` is missing.

Whether or not we introduce a simple way for users to check if the query-planning config is already established (i.e. whether `dask.dataframe` was imported), it still seems problematic that the user **can** change the `"query-planning"` config after the fact. This problem is certainly tricky to resolve, but it would be nice if the user could be warned that their config will not be accommodated.

Discussion:
rjzamora: cc @fjetter @ayushdg 
github-actions[bot]: ## Unit Test Results
_See [test report](https://dask.github.io/dask/test_report.html) for an extended history of previous test failures. This is useful for diagnosing flaky tests._

     15 files  ± 0       15 suites  ±0   3h 23m 45s :stopwatch: - 1m 51s
 13 124 tests + 1   12 159 :white_check_mark: +1     931 :zzz: ±0  34 :x: ±0 
162 532 runs  +12  142 464 :white_check_mark: +9  20 034 :zzz: +3  34 :x: ±0 

For more details on these failures, see [this check](https://github.com/dask/dask/runs/26326031005).

Results for commit 0a8a6e1c. ± Comparison against base commit f6e0690e.

[test-results]:data:application/gzip;base64,H4sIAC1ocGYC/03MTQ7CIBCG4as0rF0wA4PFyxjaThNifwyFlfHuUlTa5fcMvC8x+ok3cWuALo3Yko/HGlJw0a/LvhFxp3yN5a4A9X/ft9T35RGQPfDhnxmtgkqj81MmdfzkENaQSWYJaSlpg6Twt2taoza64jeNUqpq53aBc7pf59nHPIR0rTMMPXeWgZBNh2SI25bQjMR2wCsoiSDeH5Z4itcbAQAA


Patch:
diff --git a/dask/dataframe/__init__.py b/dask/dataframe/__init__.py
index 029eef5c5..1a3e128f6 100644
--- a/dask/dataframe/__init__.py
+++ b/dask/dataframe/__init__.py
@@ -5,19 +5,37 @@ import warnings
 
 from packaging.version import Version
 
+# The "dataframe.query-planning" config can only be processed once
+DASK_EXPR_ENABLED: bool | None = None
+
 
 def _dask_expr_enabled() -> bool:
     import pandas as pd
 
     import dask
 
+    global DASK_EXPR_ENABLED
+
     use_dask_expr = dask.config.get("dataframe.query-planning")
+    if DASK_EXPR_ENABLED is not None:
+        if (use_dask_expr is True and DASK_EXPR_ENABLED is False) or (
+            use_dask_expr is False and DASK_EXPR_ENABLED is True
+        ):
+            warnings.warn(
+                "The 'dataframe.query-planning' config is now set to "
+                f"{use_dask_expr}, but query planning is already "
+                f"{'enabled' if DASK_EXPR_ENABLED else 'disabled'}. "
+                "The query-planning config can only be changed before "
+                "`dask.dataframe` is first imported!"
+            )
+        return DASK_EXPR_ENABLED
+
     if (
         use_dask_expr is False
         or use_dask_expr is None
         and Version(pd.__version__).major < 2
     ):
-        return False
+        return (DASK_EXPR_ENABLED := False)
     try:
         import dask_expr  # noqa: F401
     except ImportError:
@@ -29,10 +47,10 @@ This will raise in a future version.
 """
         if use_dask_expr is None:
             warnings.warn(msg, FutureWarning)
-            return False
+            return (DASK_EXPR_ENABLED := False)
         else:
             raise ImportError(msg)
-    return True
+    return (DASK_EXPR_ENABLED := True)
 
 
 try:


Test patch:
diff --git a/dask/dataframe/tests/test_dataframe.py b/dask/dataframe/tests/test_dataframe.py
index 639e0e0c5..900a20688 100644
--- a/dask/dataframe/tests/test_dataframe.py
+++ b/dask/dataframe/tests/test_dataframe.py
@@ -6476,3 +6476,13 @@ def test_enforce_runtime_divisions():
             RuntimeError, match="`enforce_runtime_divisions` failed for partition 1"
         ):
             ddf.enforce_runtime_divisions().compute()
+
+
+def test_query_planning_config_warns():
+    # Make sure dd._dask_expr_enabled() warns if the current
+    # "dataframe.query-planning" config conflicts with the
+    # global dd.DASK_EXPR_ENABLED setting.
+    with dask.config.set({"dataframe.query-planning": not DASK_EXPR_ENABLED}):
+        expect = "enabled" if dd.DASK_EXPR_ENABLED else "disabled"
+        with pytest.warns(match=f"query planning is already {expect}"):
+            dd._dask_expr_enabled()