Repository: sktime/sktime
Pull request #5551 (2023-11-13 21:03:22)

[MNT] [Dependabot](deps-dev): Update pytest-xdist requirement from <3.4,>=3.3 to >=3.3,<3.5
Updates the requirements on [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst">pytest-xdist's changelog</a>.</em></p>
<blockquote>
<h1>pytest-xdist 3.4.0 (2023-11-11)</h1>
<h2>Features</h2>
<ul>
<li>
<p><code>[#963](https://github.com/pytest-dev/pytest-xdist/issues/963) &lt;https://github.com/pytest-dev/pytest-xdist/issues/963&gt;</code>_: Wait for workers to finish reporting when test run stops early.</p>
<p>This makes sure that the results of in-progress tests are displayed.
Previously these reports were being discarded, losing information about the
test run.</p>
</li>
<li>
<p><code>[#965](https://github.com/pytest-dev/pytest-xdist/issues/965) &lt;https://github.com/pytest-dev/pytest-xdist/issues/965&gt;</code>_: Added support for Python 3.12.</p>
</li>
</ul>
<h1>pytest-xdist 3.3.1 (2023-05-19)</h1>
<h2>Bug Fixes</h2>
<ul>
<li>
<p><code>[#907](https://github.com/pytest-dev/pytest-xdist/issues/907) &lt;https://github.com/pytest-dev/pytest-xdist/issues/907&gt;</code>_: Avoid remote calls during startup as <code>execnet</code> by default does not ensure remote affinity with the
main thread and might accidentally schedule the pytest worker into a non-main thread, which breaks numerous frameworks,
for example <code>asyncio</code>, <code>anyio</code>, <code>PyQt/PySide</code>, etc.</p>
<p>A more safe correction will require thread affinity in <code>execnet</code> (<code>pytest-dev/execnet#96 &lt;https://github.com/pytest-dev/execnet/issues/96&gt;</code>__).</p>
</li>
</ul>
<h1>pytest-xdist 3.3.0 (2023-05-12)</h1>
<h2>Features</h2>
<ul>
<li><code>[#555](https://github.com/pytest-dev/pytest-xdist/issues/555) &lt;https://github.com/pytest-dev/pytest-xdist/issues/555&gt;</code>_: Improved progress output when collecting nodes to be less verbose.</li>
</ul>
<h1>pytest-xdist 3.2.1 (2023-03-12)</h1>
<h2>Bug Fixes</h2>
<ul>
<li><code>[#884](https://github.com/pytest-dev/pytest-xdist/issues/884) &lt;https://github.com/pytest-dev/pytest-xdist/issues/884&gt;</code>_: Fixed hang in <code>worksteal</code> scheduler.</li>
</ul>
<h1>pytest-xdist 3.2.0 (2023-02-07)</h1>
<p>Improved Documentation</p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/a6b56112f9b686aed7a354e0d50ecc26ad2d9dfe"><code>a6b5611</code></a> Update CHANGELOG</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/64f9388f4164e9f64d98a7580933e958ea4a09d5"><code>64f9388</code></a> Add support for Python 3.12</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/230ba6ad1057574c9f3d42a97f890788cd9ec6c3"><code>230ba6a</code></a> Properly wait for workers when test run terminates early (<a href="https://redirect.github.com/pytest-dev/pytest-xdist/issues/963">#963</a>)</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/93ca202b400eecc62919bcd9b0bab001e3f3f7ef"><code>93ca202</code></a> fix typo index -&gt; instead</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/8f3d1ad92f02a251bcae5dc648a70c3f30ece364"><code>8f3d1ad</code></a> [pre-commit.ci] pre-commit autoupdate</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/ab3d6a9ee5aa4145fde339823ab1605c02f28ba4"><code>ab3d6a9</code></a> Merge pull request <a href="https://redirect.github.com/pytest-dev/pytest-xdist/issues/955">#955</a> from pytest-dev/pre-commit-ci-update-config</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/a5210d41034b2f48f2df05d2b62b0fb3495f346f"><code>a5210d4</code></a> [pre-commit.ci] pre-commit autoupdate</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/0b6b9c3c3858d05c353a14b29c58c8c3f63ae437"><code>0b6b9c3</code></a> Merge pull request <a href="https://redirect.github.com/pytest-dev/pytest-xdist/issues/954">#954</a> from pytest-dev/pre-commit-ci-update-config</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/838434c269a225bef167fa4314df905b97b9e958"><code>838434c</code></a> Install requirements.txt while building docs</li>
<li><a href="https://github.com/pytest-dev/pytest-xdist/commit/b85a4717f96e2b40910aaa289428671779cb6ecb"><code>b85a471</code></a> Add docs/requirements.txt</li>
<li>Additional commits viewable in <a href="https://github.com/pytest-dev/pytest-xdist/compare/v3.3.0...v3.4.0">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>


Patch:
diff --git a/pyproject.toml b/pyproject.toml
index dd7fe360f..91438fffa 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -200,7 +200,7 @@ regression = [
 transformations = [
   'esig<0.10,>=0.9.7; python_version < "3.11"',
   "filterpy<1.5,>=1.4.5",
-  "holidays>=0.29,<0.36",
+  "holidays>=0.29,<0.37",
   "mne<1.6,>=1.5",
   'numba<0.59,>=0.53; python_version < "3.12"',
   "pycatch22<0.4.4,>=0.4",
@@ -245,7 +245,7 @@ tests = [
   "pytest-cov<4.2,>=4.1",
   "pytest-randomly<3.16,>=3.15",
   "pytest-timeout>=2.1,<2.3",
-  "pytest-xdist<3.4,>=3.3",
+  "pytest-xdist>=3.3,<3.5",
 ]
 
 # CI related soft dependency sets - not for users of sktime, only for developers
diff --git a/sktime/base/_meta.py b/sktime/base/_meta.py
index 78901df76..6e49c88e6 100644
--- a/sktime/base/_meta.py
+++ b/sktime/base/_meta.py
@@ -97,29 +97,31 @@ class _HeterogenousMetaEstimator:
 
     def _get_params(self, attr, deep=True, fitted=False):
         if fitted:
-            method = "_get_fitted_params"
-            methodd = "get_fitted_params"
+            private_method = "_get_fitted_params"
+            public_method = "get_fitted_params"
             deepkw = {}
         else:
-            method = "get_params"
-            methodd = "get_params"
+            private_method = "get_params"
+            public_method = "get_params"
             deepkw = {"deep": deep}
 
-        out = getattr(super(), method)(**deepkw)
+        out = getattr(super(), private_method)(**deepkw)
         if deep and hasattr(self, attr):
             estimators = getattr(self, attr)
             estimators = [(x[0], x[1]) for x in estimators]
             out.update(estimators)
             for name, estimator in estimators:
                 # checks estimator has the method we want to call
-                cond1 = hasattr(estimator, methodd)
+                cond1 = hasattr(estimator, public_method)
                 # checks estimator is fitted if calling get_fitted_params
                 is_fitted = hasattr(estimator, "is_fitted") and estimator.is_fitted
                 # if we call get_params and not get_fitted_params, this is True
                 cond2 = not fitted or is_fitted
                 # check both conditions together
                 if cond1 and cond2:
-                    for key, value in getattr(estimator, methodd)(**deepkw).items():
+                    for key, value in getattr(estimator, public_method)(
+                        **deepkw
+                    ).items():
                         out[f"{name}__{key}"] = value
         return out
 
diff --git a/sktime/datatypes/_vectorize.py b/sktime/datatypes/_vectorize.py
index 5d6d73e3b..f8755d435 100644
--- a/sktime/datatypes/_vectorize.py
+++ b/sktime/datatypes/_vectorize.py
@@ -511,6 +511,7 @@ class VectorizedDF:
 
             - "None": executes loop sequentally, simple list comprehension
             - "loky", "multiprocessing" and "threading": uses ``joblib.Parallel`` loops
+            - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``
             - "dask": uses ``dask``, requires ``dask`` package in environment
             - "dask_lazy": same as "dask", but returns delayed object instead
 
diff --git a/sktime/forecasting/base/_base.py b/sktime/forecasting/base/_base.py
index 46d226617..136c43a8d 100644
--- a/sktime/forecasting/base/_base.py
+++ b/sktime/forecasting/base/_base.py
@@ -115,7 +115,8 @@ class BaseForecaster(BaseEstimator):
             backend to use for parallelization when broadcasting/vectorizing, one of
 
             - "None": executes loop sequentally, simple list comprehension
-            - "loky", "multiprocessing" and "threading": uses ``joblib`` ``Parallel``
+            - "loky", "multiprocessing" and "threading": uses ``joblib.Parallel``
+            - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``
             - "dask": uses ``dask``, requires ``dask`` package in environment
         """,
         "backend:parallel:params": """
@@ -124,12 +125,20 @@ class BaseForecaster(BaseEstimator):
             Valid keys depend on the value of ``backend:parallel``:
 
             - "None": no additional parameters, ``backend_params`` is ignored
-            - "loky", "multiprocessing" and "threading":
-              any valid keys for ``joblib.Parallel`` can be passed here,
-              e.g., ``n_jobs``, with the exception of ``backend`` which is directly
-              controlled by ``backend:parallel``
-            - "dask": any valid keys for ``dask.compute``
-              can be passed, e.g., ``scheduler``
+            - "loky", "multiprocessing" and "threading": default ``joblib`` backends
+              any valid keys for ``joblib.Parallel`` can be passed here, e.g.,
+              ``n_jobs``, with the exception of ``backend`` which is directly
+              controlled by ``backend``.
+              If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+              will default to ``joblib`` defaults.
+            - "joblib": custom and 3rd party ``joblib`` backends,
+              e.g., ``spark``. Any valid keys for ``joblib.Parallel``
+              can be passed here, e.g., ``n_jobs``,
+            ``backend`` must be passed as a key of ``backend_params`` in this case.
+              If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+              will default to ``joblib`` defaults.
+            - "dask": any valid keys for ``dask.compute`` can be passed,
+              e.g., ``scheduler``
         """,
     }
 
diff --git a/sktime/forecasting/compose/_reduce.py b/sktime/forecasting/compose/_reduce.py
index 1fa634987..9f179fc26 100644
--- a/sktime/forecasting/compose/_reduce.py
+++ b/sktime/forecasting/compose/_reduce.py
@@ -1584,68 +1584,31 @@ def _create_fcst_df(target_date, origin_df, fill=None):
     -------
     A pandas dataframe or series
     """
-    oi = origin_df.index
-    if not isinstance(oi, pd.MultiIndex):
-        if isinstance(origin_df, pd.Series):
-            if fill is None:
-                template = pd.Series(np.zeros(len(target_date)), index=target_date)
-            else:
-                template = pd.Series(fill, index=target_date)
-            template.name = origin_df.name
-            return template
-        else:
-            if fill is None:
-                template = pd.DataFrame(
-                    np.zeros((len(target_date), len(origin_df.columns))),
-                    index=target_date,
-                    columns=origin_df.columns.to_list(),
-                )
-            else:
-                template = pd.DataFrame(
-                    fill, index=target_date, columns=origin_df.columns.to_list()
-                )
-            return template
+    if not isinstance(target_date, ForecastingHorizon):
+        ix = pd.Index(target_date)
+        fh = ForecastingHorizon(ix, is_relative=False)
     else:
-        idx = origin_df.index.to_frame(index=False)
-        instance_names = idx.columns[0:-1].to_list()
-        time_names = idx.columns[-1]
-        idx = idx[instance_names].drop_duplicates()
-
-        timeframe = pd.DataFrame(target_date, columns=[time_names])
-        target_frame = idx.merge(timeframe, how="cross")
-        if hasattr(target_date, "freq"):
-            freq_inferred = target_date.freq
-            mi = (
-                target_frame.groupby(instance_names, as_index=True)
-                .apply(
-                    lambda df: df.drop(instance_names, axis=1)
-                    .set_index(time_names)
-                    .asfreq(freq_inferred)
-                )
-                .index
-            )
-        else:
-            mi = (
-                target_frame.groupby(instance_names, as_index=True)
-                .apply(lambda df: df.drop(instance_names, axis=1).set_index(time_names))
-                .index
-            )
+        fh = target_date.to_absolute()
 
-        if fill is None:
-            template = pd.DataFrame(
-                np.zeros((len(target_date) * idx.shape[0], len(origin_df.columns))),
-                index=mi,
-                columns=origin_df.columns.to_list(),
-            )
-        else:
-            template = pd.DataFrame(
-                fill,
-                index=mi,
-                columns=origin_df.columns.to_list(),
-            )
+    index = fh.get_expected_pred_idx(origin_df)
+
+    if isinstance(origin_df, pd.Series):
+        columns = [origin_df.name]
+    else:
+        columns = origin_df.columns.to_list()
+
+    if fill is None:
+        values = 0
+    else:
+        values = fill
+
+    res = pd.DataFrame(values, index=index, columns=columns)
+
+    if isinstance(origin_df, pd.Series) and not isinstance(index, pd.MultiIndex):
+        res = res.iloc[:, 0]
+        res.name = origin_df.name
 
-        template = template.astype(origin_df.dtypes.to_dict())
-        return template
+    return res
 
 
 def _coerce_col_str(X):
diff --git a/sktime/forecasting/model_evaluation/_functions.py b/sktime/forecasting/model_evaluation/_functions.py
index 155caabd8..613360937 100644
--- a/sktime/forecasting/model_evaluation/_functions.py
+++ b/sktime/forecasting/model_evaluation/_functions.py
@@ -413,6 +413,7 @@ def evaluate(
 
         - "None": executes loop sequentally, simple list comprehension
         - "loky", "multiprocessing" and "threading": uses ``joblib.Parallel`` loops
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``
         - "dask": uses ``dask``, requires ``dask`` package in environment
         - "dask_lazy": same as "dask",
           but changes the return to (lazy) ``dask.dataframe.DataFrame``.
@@ -435,12 +436,18 @@ def evaluate(
         Valid keys depend on the value of ``backend``:
 
         - "None": no additional parameters, ``backend_params`` is ignored
-        - "loky", "multiprocessing" and "threading":
-            any valid keys for ``joblib.Parallel`` can be passed here,
-            e.g., ``n_jobs``, with the exception of ``backend``
-            which is directly controlled by ``backend``
+        - "loky", "multiprocessing" and "threading": default ``joblib`` backends
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          with the exception of ``backend`` which is directly controlled by ``backend``.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``.
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          ``backend`` must be passed as a key of ``backend_params`` in this case.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
         - "dask": any valid keys for ``dask.compute`` can be passed,
-            e.g., ``scheduler``
+          e.g., ``scheduler``
 
     Returns
     -------
diff --git a/sktime/forecasting/model_selection/_tune.py b/sktime/forecasting/model_selection/_tune.py
index 7b70b6d1e..fd0fcd585 100644
--- a/sktime/forecasting/model_selection/_tune.py
+++ b/sktime/forecasting/model_selection/_tune.py
@@ -464,6 +464,7 @@ class ForecastingGridSearchCV(BaseGridSearch):
 
         - "None": executes loop sequentally, simple list comprehension
         - "loky", "multiprocessing" and "threading": uses ``joblib.Parallel`` loops
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``
         - "dask": uses ``dask``, requires ``dask`` package in environment
 
         Recommendation: Use "dask" or "loky" for parallel evaluate.
@@ -498,10 +499,16 @@ class ForecastingGridSearchCV(BaseGridSearch):
         Valid keys depend on the value of ``backend``:
 
         - "None": no additional parameters, ``backend_params`` is ignored
-        - "loky", "multiprocessing" and "threading":
-            any valid keys for ``joblib.Parallel`` can be passed here,
-            e.g., ``n_jobs``, with the exception of ``backend``
-            which is directly controlled by ``backend``
+        - "loky", "multiprocessing" and "threading": default ``joblib`` backends
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          with the exception of ``backend`` which is directly controlled by ``backend``.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``.
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          ``backend`` must be passed as a key of ``backend_params`` in this case.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
         - "dask": any valid keys for ``dask.compute`` can be passed, e.g., ``scheduler``
 
     Attributes
@@ -796,6 +803,7 @@ class ForecastingRandomizedSearchCV(BaseGridSearch):
 
         - "None": executes loop sequentally, simple list comprehension
         - "loky", "multiprocessing" and "threading": uses ``joblib.Parallel`` loops
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``
         - "dask": uses ``dask``, requires ``dask`` package in environment
 
         Recommendation: Use "dask" or "loky" for parallel evaluate.
@@ -830,10 +838,16 @@ class ForecastingRandomizedSearchCV(BaseGridSearch):
         Valid keys depend on the value of ``backend``:
 
         - "None": no additional parameters, ``backend_params`` is ignored
-        - "loky", "multiprocessing" and "threading":
-            any valid keys for ``joblib.Parallel`` can be passed here,
-            e.g., ``n_jobs``, with the exception of ``backend``
-            which is directly controlled by ``backend``
+        - "loky", "multiprocessing" and "threading": default ``joblib`` backends
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          with the exception of ``backend`` which is directly controlled by ``backend``.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``.
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          ``backend`` must be passed as a key of ``backend_params`` in this case.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
         - "dask": any valid keys for ``dask.compute`` can be passed, e.g., ``scheduler``
 
     Attributes
@@ -1055,6 +1069,7 @@ class ForecastingSkoptSearchCV(BaseGridSearch):
 
         - "None": executes loop sequentally, simple list comprehension
         - "loky", "multiprocessing" and "threading": uses ``joblib.Parallel`` loops
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``
         - "dask": uses ``dask``, requires ``dask`` package in environment
 
         Recommendation: Use "dask" or "loky" for parallel evaluate.
@@ -1085,10 +1100,16 @@ class ForecastingSkoptSearchCV(BaseGridSearch):
         Valid keys depend on the value of ``backend``:
 
         - "None": no additional parameters, ``backend_params`` is ignored
-        - "loky", "multiprocessing" and "threading":
-            any valid keys for ``joblib.Parallel`` can be passed here,
-            e.g., ``n_jobs``, with the exception of ``backend``
-            which is directly controlled by ``backend``
+        - "loky", "multiprocessing" and "threading": default ``joblib`` backends
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          with the exception of ``backend`` which is directly controlled by ``backend``.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``.
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          ``backend`` must be passed as a key of ``backend_params`` in this case.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
         - "dask": any valid keys for ``dask.compute`` can be passed, e.g., ``scheduler``
 
     Attributes
diff --git a/sktime/transformations/base.py b/sktime/transformations/base.py
index 94ed57546..126ce81f4 100644
--- a/sktime/transformations/base.py
+++ b/sktime/transformations/base.py
@@ -166,7 +166,8 @@ class BaseTransformer(BaseEstimator):
             backend to use for parallelization when broadcasting/vectorizing, one of
 
             - "None": executes loop sequentally, simple list comprehension
-            - "loky", "multiprocessing" and "threading": uses ``joblib`` ``Parallel``
+            - "loky", "multiprocessing" and "threading": uses ``joblib.Parallel``
+            - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``
             - "dask": uses ``dask``, requires ``dask`` package in environment
         """,
         "backend:parallel:params": """
@@ -175,10 +176,18 @@ class BaseTransformer(BaseEstimator):
             Valid keys depend on the value of ``backend:parallel``:
 
             - "None": no additional parameters, ``backend_params`` is ignored
-            - "loky", "multiprocessing" and "threading":
-              any valid keys for ``joblib.Parallel`` can be passed here,
-              e.g., ``n_jobs``, with the exception of ``backend`` which is directly
-              controlled by ``backend:parallel``
+            - "loky", "multiprocessing" and "threading": default ``joblib`` backends
+              any valid keys for ``joblib.Parallel`` can be passed here, e.g.,
+              ``n_jobs``, with the exception of ``backend`` which is directly
+              controlled by ``backend``.
+              If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+              will default to ``joblib`` defaults.
+            - "joblib": custom and 3rd party ``joblib`` backends,
+              e.g., ``spark``. Any valid keys for ``joblib.Parallel``
+              can be passed here, e.g., ``n_jobs``,
+            ``backend`` must be passed as a key of ``backend_params`` in this case.
+              If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+              will default to ``joblib`` defaults.
             - "dask": any valid keys for ``dask.compute``
               can be passed, e.g., ``scheduler``
         """,
diff --git a/sktime/utils/parallel.py b/sktime/utils/parallel.py
index ab839145c..6a8fb4090 100644
--- a/sktime/utils/parallel.py
+++ b/sktime/utils/parallel.py
@@ -38,6 +38,7 @@ def parallelize(fun, iter, meta=None, backend=None, backend_params=None):
 
         - "None": executes loop sequentally, simple list comprehension
         - "loky", "multiprocessing" and "threading": uses ``joblib`` ``Parallel`` loops
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``
         - "dask": uses ``dask``, requires ``dask`` package in environment
         - "dask_lazy": same as ``"dask"``, but returns delayed object instead of list
 
@@ -46,9 +47,16 @@ def parallelize(fun, iter, meta=None, backend=None, backend_params=None):
         Valid keys depend on the value of ``backend``:
 
         - "None": no additional parameters, ``backend_params`` is ignored
-        - "loky", "multiprocessing" and "threading":
+        - "loky", "multiprocessing" and "threading": default ``joblib`` backends
           any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
-          with the exception of ``backend`` which is directly controlled by ``backend``
+          with the exception of ``backend`` which is directly controlled by ``backend``.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
+        - "joblib": custom and 3rd party ``joblib`` backends, e.g., ``spark``.
+          any valid keys for ``joblib.Parallel`` can be passed here, e.g., ``n_jobs``,
+          ``backend`` must be passed as a key of ``backend_params`` in this case.
+          If ``n_jobs`` is not passed, it will default to ``-1``, other parameters
+          will default to ``joblib`` defaults.
         - "dask": any valid keys for ``dask.compute`` can be passed, e.g., ``scheduler``
     """
     if meta is None:
@@ -72,6 +80,7 @@ backend_dict = {
     "loky": "joblib",
     "multiprocessing": "joblib",
     "threading": "joblib",
+    "joblib": "joblib",
     "dask": "dask",
     "dask_lazy": "dask",
 }
@@ -92,7 +101,29 @@ def _parallelize_joblib(fun, iter, meta, backend, backend_params):
     from joblib import Parallel, delayed
 
     par_params = backend_params.copy()
-    par_params["backend"] = backend
+    if "backend" not in par_params:
+        # if user selects custom joblib backend but does not specify backend explicitly,
+        # raise a ValueError
+        if backend == "joblib":
+            raise ValueError(
+                '"joblib" was selected as first layer parallelization backend, '
+                "but no backend string was "
+                '"passed in the backend parameters dict, e.g., "spark". '
+                "Please specify a backend to joblib as a key-value pair "
+                "in the backend_params arg or the backend:parallel:params config "
+                'when using "joblib". '
+                'For clarity, "joblib" should only be used for two-layer '
+                "backend dispatch, where the first layer is joblib, "
+                "and the second layer is a custom backend of joblib, e.g., spark."
+                "For first-party joblib backends, please use the backend string "
+                'of sktime directly, e.g., by specifying "multiprocessing" or "loky".'
+            )
+        # in all other cases, we ensure the backend parameter is one of
+        # "loky", "multiprocessing" or "threading", as passed via backend
+        else:
+            par_params["backend"] = backend
+    elif backend != "joblib":
+        par_params["backend"] = backend
 
     if "n_jobs" not in par_params:
         par_params["n_jobs"] = -1


Test patch:
diff --git a/sktime/forecasting/compose/tests/test_reduce_global.py b/sktime/forecasting/compose/tests/test_reduce_global.py
index c332178dd..e2a78d278 100644
--- a/sktime/forecasting/compose/tests/test_reduce_global.py
+++ b/sktime/forecasting/compose/tests/test_reduce_global.py
@@ -106,6 +106,7 @@ def y_dict():
     # Create integer index data
     y_numeric = y_train.copy()
     y_numeric.index = pd.to_numeric(y_numeric.index)
+    y_numeric.index.names = [None]  # setting None to cover "no index name" case
     y_dict["y_numeric"] = y_numeric
 
     return y_dict
@@ -139,7 +140,7 @@ def check_eval(test_input, expected):
         ),
         (
             "y_train",
-            [None],
+            ["Period"],
         ),
         (
             "y_numeric",
@@ -186,7 +187,7 @@ def test_recursive_reduction(y, index_names, y_dict):
         ),
         (
             "y_train",
-            [None],
+            ["Period"],
         ),
         (
             "y_numeric",
@@ -232,7 +233,7 @@ def test_direct_reduction(y, index_names, y_dict):
         ),
         (
             "y_train",
-            [None],
+            ["Period"],
         ),
         (
             "y_numeric",