Repository: dask/dask
Issue category: Feature Request

[FEA] Add official mechanism to check if query-planning is enabled in ``dask.dataframe``
The value of the `"dataframe.query-planning"` config currently dictates whether or not `dask.dataframe` will use dask-expr under the hood. However, after `dask.dataframe` is first imported, the value of the `"dataframe.query-planning"` config becomes somewhat useless.

For example, a down-stream library may wish to warn/raise if `dask.dataframe` is already imported with query-planning enabled, but the value of the `"dataframe.query-planning"` config will typically be `None` whether the module was already imported or not.

One way to mitigate this problem is for `dask.dataframe.__init__` to always convert `None` values into `True` (using the config system). This way, a down-stream library (or user) can interpret `None` as a confirmation that the query-planning config can still be modified. An alternative solution is to tell users to inspect `sys.modules` themselves to confirm that `dask.dataframe` is missing.

Whether or not we introduce a simple way for users to check if the query-planning config is already established (i.e. whether `dask.dataframe` was imported), it still seems problematic that the user **can** change the `"query-planning"` config after the fact. This problem is certainly tricky to resolve, but it would be nice if the user could be warned that their config will not be accommodated.


Functions to edit:
dask/dataframe/__init__.py:_dask_expr_enabled

Patch:
diff --git a/dask/dataframe/__init__.py b/dask/dataframe/__init__.py
index 029eef5c54a..1a3e128f6ca 100644
--- a/dask/dataframe/__init__.py
+++ b/dask/dataframe/__init__.py
@@ -5,19 +5,37 @@
 
 from packaging.version import Version
 
+# The "dataframe.query-planning" config can only be processed once
+DASK_EXPR_ENABLED: bool | None = None
+
 
 def _dask_expr_enabled() -> bool:
     import pandas as pd
 
     import dask
 
+    global DASK_EXPR_ENABLED
+
     use_dask_expr = dask.config.get("dataframe.query-planning")
+    if DASK_EXPR_ENABLED is not None:
+        if (use_dask_expr is True and DASK_EXPR_ENABLED is False) or (
+            use_dask_expr is False and DASK_EXPR_ENABLED is True
+        ):
+            warnings.warn(
+                "The 'dataframe.query-planning' config is now set to "
+                f"{use_dask_expr}, but query planning is already "
+                f"{'enabled' if DASK_EXPR_ENABLED else 'disabled'}. "
+                "The query-planning config can only be changed before "
+                "`dask.dataframe` is first imported!"
+            )
+        return DASK_EXPR_ENABLED
+
     if (
         use_dask_expr is False
         or use_dask_expr is None
         and Version(pd.__version__).major < 2
     ):
-        return False
+        return (DASK_EXPR_ENABLED := False)
     try:
         import dask_expr  # noqa: F401
     except ImportError:
@@ -29,10 +47,10 @@ def _dask_expr_enabled() -> bool:
 """
         if use_dask_expr is None:
             warnings.warn(msg, FutureWarning)
-            return False
+            return (DASK_EXPR_ENABLED := False)
         else:
             raise ImportError(msg)
-    return True
+    return (DASK_EXPR_ENABLED := True)
 
 
 try:

Test patch: diff --git a/dask/dataframe/tests/test_dataframe.py b/dask/dataframe/tests/test_dataframe.py
index 639e0e0c5ea..900a2068835 100644
--- a/dask/dataframe/tests/test_dataframe.py
+++ b/dask/dataframe/tests/test_dataframe.py
@@ -6476,3 +6476,13 @@ def test_enforce_runtime_divisions():
             RuntimeError, match="`enforce_runtime_divisions` failed for partition 1"
         ):
             ddf.enforce_runtime_divisions().compute()
+
+
+def test_query_planning_config_warns():
+    # Make sure dd._dask_expr_enabled() warns if the current
+    # "dataframe.query-planning" config conflicts with the
+    # global dd.DASK_EXPR_ENABLED setting.
+    with dask.config.set({"dataframe.query-planning": not DASK_EXPR_ENABLED}):
+        expect = "enabled" if dd.DASK_EXPR_ENABLED else "disabled"
+        with pytest.warns(match=f"query planning is already {expect}"):
+            dd._dask_expr_enabled()