Repository: sktime/sktime

Issue:
[ENH] metrics `__call__` to be dispatchable to `evaluate` or `evaluate_by_index`
@kdekker-private had the following idea to give users an option to dispatch metric calls, i.e., calls to metric objects (which are callable via `__call__`) to `evaluate` or `evaluate_by_index` given a `by_index` argument, see below.

I think that is a very nice idea. Opening a new issue for discussion (first) and then potential implementation.

---

_Originally posted by @kdekker-private in https://github.com/sktime/sktime/issues/6688#issuecomment-2242591511_

---

              No problem. I think the evaluation is most intuitively called via the metrics class instance instead via the metrics constructor. I can image you might want to reuse the metrics evaluating by index level and evaluation by timepoints.
The argument _evaluate_by_timepoints_ is used below to allow evaluation by timepoint.

Use case 0: (Default) Evaluate by index level.

```python
import numpy as np
from sktime.performance_metrics.forecasting import MeanAbsoluteError
from sktime.utils._testing.hierarchical import _make_hierarchical

y_true = _make_hierarchical(
                n_columns=2,
                hierarchy_levels = (2, 4),
                max_timepoints=20,
                min_timepoints=20,
            )
y_pred = _make_hierarchical(
                n_columns=2,
                hierarchy_levels = (2, 4),
                max_timepoints=20,
                min_timepoints=20,
            )
mae = MeanAbsoluteError()
mae(y_true, y_pred, evaluate_by_timepoints = False)  # evaluate_by_timepoints should have False as default value and doesn't need to be defined by the user.
```

Use case 1: Evaluate by timepoints and show raw values for dependent variables.

```python
import numpy as np
from sktime.performance_metrics.forecasting import MeanAbsoluteError
from sktime.utils._testing.hierarchical import _make_hierarchical

y_true = _make_hierarchical(
                n_columns=2,
                hierarchy_levels = (2, 4),
                max_timepoints=20,
                min_timepoints=20,
            )
y_pred = _make_hierarchical(
                n_columns=2,
                hierarchy_levels = (2, 4),
                max_timepoints=20,
                min_timepoints=20,
            )
mae = MeanAbsoluteError()
mae(y_true, y_pred, evaluate_by_timepoints = True, multioutput = "raw_values")
```

Use case 2: Evaluate by timepoints and average (weighted) value for dependent variables.

```python
import numpy as np
from sktime.performance_metrics.forecasting import MeanAbsoluteError
from sktime.utils._testing.hierarchical import _make_hierarchical

y_true = _make_hierarchical(
                n_columns=2,
                hierarchy_levels = (2, 4),
                max_timepoints=20,
                min_timepoints=20,
            )
y_pred = _make_hierarchical(
                n_columns=2,
                hierarchy_levels = (2, 4),
                max_timepoints=20,
                min_timepoints=20,
            )
mae = MeanAbsoluteError()
mae(y_true, y_pred, evaluate_by_timepoints = True, multioutput = "uniform_average")
```
            

Discussion:
benHeid: @fkiraly thanks for the review. I added a new test and removed the get_test_params. I have added them since a lot of the metrics do only have one test set, leading to failing tests...
fkiraly: current failures likely due to incomplete renames, or some metrics missing the new param.
benHeid: > current failures likely due to incomplete renames, or some metrics missing the new param.

I have overseen some classes where I needed to add the `by_index` param. The remaining failures seems to be related to TSFresh.

Patch:
diff --git a/.all-contributorsrc b/.all-contributorsrc
index ad4440829..b779b3f97 100644
--- a/.all-contributorsrc
+++ b/.all-contributorsrc
@@ -3261,11 +3261,12 @@
     },
     {
       "login": "TanviPooranmal",
-      "name": "Tanvi Pooranmal Meena"
+      "name": "Tanvi Pooranmal Meena",
       "avatar_url": "https://avatars.githubusercontent.com/TanviPooranmal",
       "profile": "https://github.com/TanviPooranmal",
       "contributions": [
-        "code"
+        "code",
+        "doc"
       ]
     },
     {
@@ -3315,6 +3316,15 @@
         "code"
       ]
     },
+    {
+      "login": "lenaklosik",
+      "name": "Lena Klosik",
+      "avatar_url": "https://avatars.githubusercontent.com/lenaklosik",
+      "profile": "https://github.com/lenaklosik",
+      "contributions": [
+        "code"
+      ]
+    },
     {
       "login": "marcosfelt",
       "name": "Kobi Felton",
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index a27677447..aa74e10e8 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -6,7 +6,7 @@ Contributors
 <!-- Please add your badges to .all-contibutorsrc -->
 
 <!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
-[![All Contributors](https://img.shields.io/badge/all_contributors-328-orange.svg)](#contributors)
+[![All Contributors](https://img.shields.io/badge/all_contributors-329-orange.svg)](#contributors)
 <!-- ALL-CONTRIBUTORS-BADGE:END -->
 
 This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
@@ -340,15 +340,16 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
     <tr>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/Taise228"><img src="https://avatars.githubusercontent.com/u/95762401?s=400&v=4?s=100" width="100px;" alt="Taisei Yamamoto"/><br /><sub><b>Taisei Yamamoto</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=Taise228" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://thayeylolu.github.io/portfolio/"><img src="https://avatars.githubusercontent.com/u/13348874?v=4?s=100" width="100px;" alt="Taiwo Owoseni"/><br /><sub><b>Taiwo Owoseni</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=thayeylolu" title="Code">💻</a></td>
+      <td align="center" valign="top" width="11.11%"><a href="https://github.com/TanviPooranmal"><img src="https://avatars.githubusercontent.com/TanviPooranmal?s=100" width="100px;" alt="Tanvi Pooranmal Meena"/><br /><sub><b>Tanvi Pooranmal Meena</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=TanviPooranmal" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/lnthach"><img src="https://avatars0.githubusercontent.com/u/7788363?v=4?s=100" width="100px;" alt="Thach Le Nguyen"/><br /><sub><b>Thach Le Nguyen</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=lnthach" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=lnthach" title="Tests">⚠️</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/mathco-wf"><img src="https://avatars.githubusercontent.com/mathco-wf?s=100" width="100px;" alt="TheMathcompay Widget Factory Team"/><br /><sub><b>TheMathcompay Widget Factory Team</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=mathco-wf" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/tombh"><img src="https://avatars.githubusercontent.com/u/160835?s=80&v=4?s=100" width="100px;" alt="Thomas Buckley-Houston"/><br /><sub><b>Thomas Buckley-Houston</b></sub></a><br /><a href="https://github.com/sktime/sktime/issues?q=author%3Atombh" title="Bug reports">🐛</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/xxl4tomxu98"><img src="https://avatars.githubusercontent.com/u/62292177?s=40&v=4?s=100" width="100px;" alt="Tom Xu"/><br /><sub><b>Tom Xu</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=xxl4tomxu98" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=xxl4tomxu98" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/tpvasconcelos"><img src="https://avatars.githubusercontent.com/u/17701527?v=4?s=100" width="100px;" alt="Tomas P. de Vasconcelos"/><br /><sub><b>Tomas P. de Vasconcelos</b></sub></a><br /><a href="https://github.com/sktime/sktime/issues?q=author%3Atpvasconcelos" title="Bug reports">🐛</a> <a href="https://github.com/sktime/sktime/commits?author=tpvasconcelos" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/tch"><img src="https://avatars3.githubusercontent.com/u/184076?v=4?s=100" width="100px;" alt="Tomasz Chodakowski"/><br /><sub><b>Tomasz Chodakowski</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=tch" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=tch" title="Documentation">📖</a> <a href="https://github.com/sktime/sktime/issues?q=author%3Atch" title="Bug reports">🐛</a></td>
-      <td align="center" valign="top" width="11.11%"><a href="http://www.timeseriesclassification.com"><img src="https://avatars1.githubusercontent.com/u/9594042?v=4?s=100" width="100px;" alt="Tony Bagnall"/><br /><sub><b>Tony Bagnall</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=TonyBagnall" title="Code">💻</a> <a href="#business-TonyBagnall" title="Business development">💼</a> <a href="https://github.com/sktime/sktime/commits?author=TonyBagnall" title="Documentation">📖</a> <a href="#design-TonyBagnall" title="Design">🎨</a> <a href="#eventOrganizing-TonyBagnall" title="Event Organizing">📋</a> <a href="#fundingFinding-TonyBagnall" title="Funding Finding">🔍</a> <a href="#ideas-TonyBagnall" title="Ideas, Planning, & Feedback">🤔</a> <a href="#projectManagement-TonyBagnall" title="Project Management">📆</a> <a href="#question-TonyBagnall" title="Answering Questions">💬</a> <a href="https://github.com/sktime/sktime/pulls?q=is%3Apr+reviewed-by%3ATonyBagnall" title="Reviewed Pull Requests">👀</a> <a href="#talk-TonyBagnall" title="Talks">📢</a> <a href="#data-TonyBagnall" title="Data">🔣</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="11.11%"><a href="http://www.timeseriesclassification.com"><img src="https://avatars1.githubusercontent.com/u/9594042?v=4?s=100" width="100px;" alt="Tony Bagnall"/><br /><sub><b>Tony Bagnall</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=TonyBagnall" title="Code">💻</a> <a href="#business-TonyBagnall" title="Business development">💼</a> <a href="https://github.com/sktime/sktime/commits?author=TonyBagnall" title="Documentation">📖</a> <a href="#design-TonyBagnall" title="Design">🎨</a> <a href="#eventOrganizing-TonyBagnall" title="Event Organizing">📋</a> <a href="#fundingFinding-TonyBagnall" title="Funding Finding">🔍</a> <a href="#ideas-TonyBagnall" title="Ideas, Planning, & Feedback">🤔</a> <a href="#projectManagement-TonyBagnall" title="Project Management">📆</a> <a href="#question-TonyBagnall" title="Answering Questions">💬</a> <a href="https://github.com/sktime/sktime/pulls?q=is%3Apr+reviewed-by%3ATonyBagnall" title="Reviewed Pull Requests">👀</a> <a href="#talk-TonyBagnall" title="Talks">📢</a> <a href="#data-TonyBagnall" title="Data">🔣</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/onyekaugochukwu"><img src="https://avatars.githubusercontent.com/u/92909501?v=4?s=100" width="100px;" alt="Ugochukwu Onyeka"/><br /><sub><b>Ugochukwu Onyeka</b></sub></a><br /><a href="#business-onyekaugochukwu" title="Business development">💼</a> <a href="https://github.com/sktime/sktime/commits?author=onyekaugochukwu" title="Documentation">📖</a> <a href="#ideas-onyekaugochukwu" title="Ideas, Planning, & Feedback">🤔</a> <a href="#mentoring-onyekaugochukwu" title="Mentoring">🧑‍🏫</a> <a href="#projectManagement-onyekaugochukwu" title="Project Management">📆</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/utsavcoding"><img src="https://avatars3.githubusercontent.com/u/55446385?v=4?s=100" width="100px;" alt="Utsav Kumar Tiwari"/><br /><sub><b>Utsav Kumar Tiwari</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=utsavcoding" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=utsavcoding" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/vagechirkov"><img src="https://avatars.githubusercontent.com/vagechirkov?s=100" width="100px;" alt="Valerii Chirkov"/><br /><sub><b>Valerii Chirkov</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=vagechirkov" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=vagechirkov" title="Documentation">📖</a></td>
@@ -357,9 +358,9 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/vedantag17"><img src="https://avatars.githubusercontent.com/u/118207011?v=4?s=100" width="100px;" alt="Vedant Agrawal"/><br /><sub><b>Vedant Agrawal</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=vedantag17" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/jgyfutub"><img src="https://avatars.githubusercontent.com/u/97391064?s=400&v=4?s=100" width="100px;" alt="Vedant Pandey"/><br /><sub><b>Vedant Pandey</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=jgyfutub" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/VjayRam"><img src="https://avatars.githubusercontent.com/u/76694566?v=4?s=100" width="100px;" alt="Vijay Ram Enaganti"/><br /><sub><b>Vijay Ram Enaganti</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=VjayRam" title="Documentation">📖</a></td>
-      <td align="center" valign="top" width="11.11%"><a href="https://github.com/victordremov"><img src="https://avatars.githubusercontent.com/u/32140716?s=100" width="100px;" alt="Viktor Dremov"/><br /><sub><b>Viktor Dremov</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=victordremov" title="Code">💻</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="11.11%"><a href="https://github.com/victordremov"><img src="https://avatars.githubusercontent.com/u/32140716?s=100" width="100px;" alt="Viktor Dremov"/><br /><sub><b>Viktor Dremov</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=victordremov" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/ViktorKaz"><img src="https://avatars0.githubusercontent.com/u/33499138?v=4?s=100" width="100px;" alt="ViktorKaz"/><br /><sub><b>ViktorKaz</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=ViktorKaz" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=ViktorKaz" title="Documentation">📖</a> <a href="#design-ViktorKaz" title="Design">🎨</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/vnicholson1"><img src="?s=100" width="100px;" alt="Vincent Nicholson"/><br /><sub><b>Vincent Nicholson</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=vnicholson1" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/VyomkeshVyas"><img src="?s=100" width="100px;" alt="Vyomkesh Vyas"/><br /><sub><b>Vyomkesh Vyas</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=VyomkeshVyas" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=VyomkeshVyas" title="Documentation">📖</a> <a href="#example-VyomkeshVyas" title="Examples">💡</a> <a href="https://github.com/sktime/sktime/commits?author=VyomkeshVyas" title="Tests">⚠️</a></td>
@@ -368,9 +369,9 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="11.11%"><a href="https://www.linkedin.com/in/templierw/"><img src="https://github.com/templierw.png?s=100" width="100px;" alt="William Templier"/><br /><sub><b>William Templier</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=templierw" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/magittan"><img src="https://avatars0.githubusercontent.com/u/14024202?v=4?s=100" width="100px;" alt="William Zheng"/><br /><sub><b>William Zheng</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=magittan" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=magittan" title="Tests">⚠️</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/wirrywoo"><img src="https://avatars.githubusercontent.com/u/148647848?v=4?s=100?s=100" width="100px;" alt="Wilson Cheung"/><br /><sub><b>Wilson Cheung</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=wirrywoo" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=wirrywoo" title="Documentation">📖</a></td>
-      <td align="center" valign="top" width="11.11%"><a href="https://github.com/wpdonders"><img src="https://avatars.githubusercontent.com/u/1868824?v=4?s=100?s=100" width="100px;" alt="Wouter Donders"/><br /><sub><b>Wouter Donders</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=wpdonders" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=wpdonders" title="Tests">⚠️</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="11.11%"><a href="https://github.com/wpdonders"><img src="https://avatars.githubusercontent.com/u/1868824?v=4?s=100?s=100" width="100px;" alt="Wouter Donders"/><br /><sub><b>Wouter Donders</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=wpdonders" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=wpdonders" title="Tests">⚠️</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/XinyuWuu"><img src="https://avatars.githubusercontent.com/u/57612792?v=4?s=100" width="100px;" alt="Xinyu Wu"/><br /><sub><b>Xinyu Wu</b></sub></a><br /><a href="https://github.com/sktime/sktime/issues?q=author%3AXinyuWuu" title="Bug reports">🐛</a> <a href="https://github.com/sktime/sktime/commits?author=XinyuWuu" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=XinyuWuu" title="Tests">⚠️</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/yairbeer"><img src="https://avatars.githubusercontent.com/yairbeer?s=100" width="100px;" alt="Yair Beer"/><br /><sub><b>Yair Beer</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=yairbeer" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://www.linkedin.com/in/yann-hallouard/"><img src="https://avatars.githubusercontent.com/YHallouard?s=100" width="100px;" alt="Yann Hallouard"/><br /><sub><b>Yann Hallouard</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=YHallouard" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=YHallouard" title="Tests">⚠️</a></td>
@@ -379,9 +380,9 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/yashlamba"><img src="https://avatars.githubusercontent.com/u/44164398?v=4?s=100" width="100px;" alt="Yash Lamba"/><br /><sub><b>Yash Lamba</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=yashlamba" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/xuyxu"><img src="https://avatars2.githubusercontent.com/u/22359569?v=4?s=100" width="100px;" alt="Yi-Xuan Xu"/><br /><sub><b>Yi-Xuan Xu</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=xuyxu" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=xuyxu" title="Tests">⚠️</a> <a href="#maintenance-xuyxu" title="Maintenance">🚧</a> <a href="https://github.com/sktime/sktime/commits?author=xuyxu" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/y-mx"><img src="https://avatars.githubusercontent.com/y-mx?s=100" width="100px;" alt="Yimeng Xie"/><br /><sub><b>Yimeng Xie</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=y-mx" title="Code">💻</a></td>
-      <td align="center" valign="top" width="11.11%"><a href="https://github.com/sz85512678"><img src="https://avatars.githubusercontent.com/sz85512678?s=100" width="100px;" alt="Zhen Shao"/><br /><sub><b>Zhen Shao</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=sz85512678" title="Code">💻</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="11.11%"><a href="https://github.com/sz85512678"><img src="https://avatars.githubusercontent.com/sz85512678?s=100" width="100px;" alt="Zhen Shao"/><br /><sub><b>Zhen Shao</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=sz85512678" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/ZiyaoWei"><img src="https://avatars.githubusercontent.com/u/940823?v=4?s=100" width="100px;" alt="Ziyao Wei"/><br /><sub><b>Ziyao Wei</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=ZiyaoWei" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/aa25desh"><img src="https://avatars1.githubusercontent.com/u/29518290?v=4?s=100" width="100px;" alt="aa25desh"/><br /><sub><b>aa25desh</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=aa25desh" title="Code">💻</a> <a href="https://github.com/sktime/sktime/issues?q=author%3Aaa25desh" title="Bug reports">🐛</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/abandus"><img src="https://avatars2.githubusercontent.com/u/46486474?v=4?s=100" width="100px;" alt="abandus"/><br /><sub><b>abandus</b></sub></a><br /><a href="#ideas-abandus" title="Ideas, Planning, & Feedback">🤔</a> <a href="https://github.com/sktime/sktime/commits?author=abandus" title="Code">💻</a></td>
@@ -390,9 +391,9 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/big-o"><img src="https://avatars1.githubusercontent.com/u/1134151?v=4?s=100" width="100px;" alt="big-o"/><br /><sub><b>big-o</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=big-o" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=big-o" title="Tests">⚠️</a> <a href="#design-big-o" title="Design">🎨</a> <a href="#ideas-big-o" title="Ideas, Planning, & Feedback">🤔</a> <a href="https://github.com/sktime/sktime/pulls?q=is%3Apr+reviewed-by%3Abig-o" title="Reviewed Pull Requests">👀</a> <a href="#tutorial-big-o" title="Tutorials">✅</a> <a href="#mentoring-big-o" title="Mentoring">🧑‍🏫</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/bobbys-dev"><img src="https://avatars.githubusercontent.com/bobbys-dev?s=100" width="100px;" alt="bobbys"/><br /><sub><b>bobbys</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=bobbys-dev" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/brettkoonce"><img src="https://avatars2.githubusercontent.com/u/11281814?v=4?s=100" width="100px;" alt="brett koonce"/><br /><sub><b>brett koonce</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=brettkoonce" title="Documentation">📖</a></td>
-      <td align="center" valign="top" width="11.11%"><a href="https://github.com/btrtts"><img src="https://avatars3.githubusercontent.com/u/66252156?v=4?s=100" width="100px;" alt="btrtts"/><br /><sub><b>btrtts</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=btrtts" title="Documentation">📖</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="11.11%"><a href="https://github.com/btrtts"><img src="https://avatars3.githubusercontent.com/u/66252156?v=4?s=100" width="100px;" alt="btrtts"/><br /><sub><b>btrtts</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=btrtts" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/chizzi25"><img src="https://avatars3.githubusercontent.com/u/67911243?v=4?s=100" width="100px;" alt="chizzi25"/><br /><sub><b>chizzi25</b></sub></a><br /><a href="#blog-chizzi25" title="Blogposts">📝</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/chrisholder"><img src="https://avatars.githubusercontent.com/u/4674372?v=4?s=100" width="100px;" alt="chrisholder"/><br /><sub><b>chrisholder</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=chrisholder" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=chrisholder" title="Tests">⚠️</a> <a href="https://github.com/sktime/sktime/commits?author=chrisholder" title="Documentation">📖</a> <a href="#design-chrisholder" title="Design">🎨</a> <a href="#example-chrisholder" title="Examples">💡</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/morestart"><img src="https://avatars.githubusercontent.com/u/35556811?s=100" width="100px;" alt="ctl"/><br /><sub><b>ctl</b></sub></a><br /><a href="https://github.com/sktime/sktime/issues?q=author%3Amorestart" title="Bug reports">🐛</a></td>
@@ -401,9 +402,9 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/hiqbal2"><img src="https://avatars3.githubusercontent.com/u/10302415?v=4?s=100" width="100px;" alt="hiqbal2"/><br /><sub><b>hiqbal2</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=hiqbal2" title="Documentation">📖</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/jesellier"><img src="https://avatars0.githubusercontent.com/u/51952076?v=4?s=100" width="100px;" alt="jesellier"/><br /><sub><b>jesellier</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=jesellier" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/jschemm"><img src="https://avatars.githubusercontent.com/u/81151346?v=4?s=100" width="100px;" alt="jschemm"/><br /><sub><b>jschemm</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=jschemm" title="Code">💻</a></td>
-      <td align="center" valign="top" width="11.11%"><a href="https://github.com/kkoziara"><img src="https://avatars1.githubusercontent.com/u/4346849?v=4?s=100" width="100px;" alt="kkoziara"/><br /><sub><b>kkoziara</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=kkoziara" title="Code">💻</a> <a href="https://github.com/sktime/sktime/issues?q=author%3Akkoziara" title="Bug reports">🐛</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="11.11%"><a href="https://github.com/kkoziara"><img src="https://avatars1.githubusercontent.com/u/4346849?v=4?s=100" width="100px;" alt="kkoziara"/><br /><sub><b>kkoziara</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=kkoziara" title="Code">💻</a> <a href="https://github.com/sktime/sktime/issues?q=author%3Akkoziara" title="Bug reports">🐛</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/matteogales"><img src="https://avatars0.githubusercontent.com/u/9269326?v=4?s=100" width="100px;" alt="matteogales"/><br /><sub><b>matteogales</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=matteogales" title="Code">💻</a> <a href="#design-matteogales" title="Design">🎨</a> <a href="#ideas-matteogales" title="Ideas, Planning, & Feedback">🤔</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/oleskiewicz"><img src="https://avatars1.githubusercontent.com/u/5682158?v=4?s=100" width="100px;" alt="oleskiewicz"/><br /><sub><b>oleskiewicz</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=oleskiewicz" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=oleskiewicz" title="Documentation">📖</a> <a href="https://github.com/sktime/sktime/commits?author=oleskiewicz" title="Tests">⚠️</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/pabworks"><img src="https://avatars.githubusercontent.com/u/32725127?v=4?s=100" width="100px;" alt="pabworks"/><br /><sub><b>pabworks</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=pabworks" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=pabworks" title="Tests">⚠️</a></td>
@@ -412,9 +413,9 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/simone-pignotti"><img src="https://avatars1.githubusercontent.com/u/44410066?v=4?s=100" width="100px;" alt="simone-pignotti"/><br /><sub><b>simone-pignotti</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=simone-pignotti" title="Code">💻</a> <a href="https://github.com/sktime/sktime/issues?q=author%3Asimone-pignotti" title="Bug reports">🐛</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/sophijka"><img src="https://avatars2.githubusercontent.com/u/47450591?v=4?s=100" width="100px;" alt="sophijka"/><br /><sub><b>sophijka</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=sophijka" title="Documentation">📖</a> <a href="#maintenance-sophijka" title="Maintenance">🚧</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/sri1419"><img src="https://avatars2.githubusercontent.com/u/65078278?v=4?s=100" width="100px;" alt="sri1419"/><br /><sub><b>sri1419</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=sri1419" title="Code">💻</a></td>
-      <td align="center" valign="top" width="11.11%"><a href="https://github.com/tensorflow-as-tf"><img src="https://avatars.githubusercontent.com/u/51345718?v=4?s=100" width="100px;" alt="tensorflow-as-tf"/><br /><sub><b>tensorflow-as-tf</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=tensorflow-as-tf" title="Code">💻</a></td>
     </tr>
     <tr>
+      <td align="center" valign="top" width="11.11%"><a href="https://github.com/tensorflow-as-tf"><img src="https://avatars.githubusercontent.com/u/51345718?v=4?s=100" width="100px;" alt="tensorflow-as-tf"/><br /><sub><b>tensorflow-as-tf</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=tensorflow-as-tf" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/vedazeren"><img src="https://avatars3.githubusercontent.com/u/63582874?v=4?s=100" width="100px;" alt="vedazeren"/><br /><sub><b>vedazeren</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=vedazeren" title="Code">💻</a> <a href="https://github.com/sktime/sktime/commits?author=vedazeren" title="Tests">⚠️</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/vincent-nich12"><img src="https://avatars3.githubusercontent.com/u/36476633?v=4?s=100" width="100px;" alt="vincent-nich12"/><br /><sub><b>vincent-nich12</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=vincent-nich12" title="Code">💻</a></td>
       <td align="center" valign="top" width="11.11%"><a href="https://github.com/vollmersj"><img src="https://avatars2.githubusercontent.com/u/12613127?v=4?s=100" width="100px;" alt="vollmersj"/><br /><sub><b>vollmersj</b></sub></a><br /><a href="https://github.com/sktime/sktime/commits?author=vollmersj" title="Documentation">📖</a></td>
diff --git a/docs/source/about/team.rst b/docs/source/about/team.rst
index 72837ff85..6e1041372 100644
--- a/docs/source/about/team.rst
+++ b/docs/source/about/team.rst
@@ -25,7 +25,7 @@ Community Council
      - :user:`onyekaugochukwu`
 
 For further information on past and future council elections,
-refer to the `elections repository <www.github.com/sktime/elections>`_.
+refer to the `elections repository <https://www.github.com/sktime/elections>`_.
 
 Community Council Observers
 ---------------------------
diff --git a/pyproject.toml b/pyproject.toml
index e4c63d135..1888e514c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -124,7 +124,7 @@ all_extras_pandas2 = [
   'autots<0.7,>=0.6.1; python_version < "3.13"',
   'cloudpickle; python_version < "3.13"',
   'dash!=2.9.0; python_version < "3.13"',
-  "dask<2024.12.2; extra == 'dataframe'",
+  "dask<2025.1.1; extra == 'dataframe'",
   'dtaidistance<2.4; python_version < "3.13"',
   'dtw-python; python_version < "3.13"',
   'esig==0.9.7; python_version < "3.10"',
diff --git a/sktime/classification/sklearn/_rotation_forest.py b/sktime/classification/sklearn/_rotation_forest.py
index 645e590d7..338100a19 100644
--- a/sktime/classification/sklearn/_rotation_forest.py
+++ b/sktime/classification/sklearn/_rotation_forest.py
@@ -10,18 +10,17 @@ __all__ = ["RotationForest"]
 import time
 
 import numpy as np
-import pandas as pd
-from sklearn.base import BaseEstimator
+from sklearn.base import BaseEstimator, ClassifierMixin
 from sklearn.decomposition import PCA
 from sklearn.tree import DecisionTreeClassifier
 from sklearn.utils import check_random_state
+from sklearn.utils.validation import check_is_fitted
 
 from sktime.base._base import _clone_estimator
-from sktime.exceptions import NotFittedError
 from sktime.utils.validation import check_n_jobs
 
 
-class RotationForest(BaseEstimator):
+class RotationForest(BaseEstimator, ClassifierMixin):
     """A rotation forest (RotF) vector classifier.
 
     Implementation of the Rotation Forest classifier described in Rodriguez et al
@@ -160,18 +159,14 @@ class RotationForest(BaseEstimator):
         """
         from joblib import Parallel, delayed
 
-        if isinstance(X, np.ndarray) and len(X.shape) == 3 and X.shape[1] == 1:
-            X = np.reshape(X, (X.shape[0], -1))
-        elif isinstance(X, pd.DataFrame) and len(X.shape) == 2:
-            X = X.to_numpy()
-        elif not isinstance(X, np.ndarray) or len(X.shape) > 2:
-            raise ValueError(
-                "RotationForest is not a time series classifier. "
-                "A valid sklearn input such as a 2d numpy array is required."
-                "Sparse input formats are currently not supported."
-            )
-        X, y = self._validate_data(X=X, y=y, ensure_min_samples=2)
-
+        X, y = self._validate_data(
+            X,
+            y,
+            dtype=[np.float32, np.float64],
+            ensure_2d=True,
+            allow_nd=True,
+            force_all_finite=True,
+        )
         self._n_jobs = check_n_jobs(self.n_jobs)
 
         self.n_instances_, self.n_atts_ = X.shape
@@ -269,6 +264,7 @@ class RotationForest(BaseEstimator):
         y : array-like, shape = [n_instances]
             Predicted class labels.
         """
+        check_is_fitted(self)
         rng = check_random_state(self.random_state)
         return np.array(
             [
@@ -292,27 +288,20 @@ class RotationForest(BaseEstimator):
         """
         from joblib import Parallel, delayed
 
-        if not self._is_fitted:
-            raise NotFittedError(
-                f"This instance of {self.__class__.__name__} has not "
-                f"been fitted yet; please call `fit` first."
-            )
+        check_is_fitted(self)
 
         # treat case of single class seen in fit
         if self.n_classes_ == 1:
             return np.repeat([[1]], X.shape[0], axis=0)
 
-        if isinstance(X, np.ndarray) and len(X.shape) == 3 and X.shape[1] == 1:
-            X = np.reshape(X, (X.shape[0], -1))
-        elif isinstance(X, pd.DataFrame) and len(X.shape) == 2:
-            X = X.to_numpy()
-        elif not isinstance(X, np.ndarray) or len(X.shape) > 2:
-            raise ValueError(
-                "RotationForest is not a time series classifier. "
-                "A valid sklearn input such as a 2d numpy array is required."
-                "Sparse input formats are currently not supported."
-            )
-        X = self._validate_data(X=X, reset=False)
+        X = self._validate_data(
+            X,
+            dtype=[np.float32, np.float64],
+            reset=False,
+            ensure_2d=True,
+            allow_nd=True,
+            force_all_finite=True,
+        )
 
         # replace missing values with 0 and remove useless attributes
         X = X[:, self._useful_atts]
@@ -338,21 +327,15 @@ class RotationForest(BaseEstimator):
     def _get_train_probs(self, X, y):
         from joblib import Parallel, delayed
 
-        if not self._is_fitted:
-            raise NotFittedError(
-                f"This instance of {self.__class__.__name__} has not "
-                f"been fitted yet; please call `fit` first."
-            )
-        if isinstance(X, np.ndarray) and len(X.shape) == 3 and X.shape[1] == 1:
-            X = np.reshape(X, (X.shape[0], -1))
-        elif isinstance(X, pd.DataFrame) and len(X.shape) == 2:
-            X = X.to_numpy()
-        elif not isinstance(X, np.ndarray) or len(X.shape) > 2:
-            raise ValueError(
-                "RotationForest is not a time series classifier. "
-                "A valid sklearn input such as a 2d numpy array is required."
-                "Sparse input formats are currently not supported."
-            )
+        check_is_fitted(self)
+        X = self._validate_data(
+            X,
+            dtype=[np.float32, np.float64],
+            reset=False,
+            ensure_2d=True,
+            allow_nd=True,
+            force_all_finite=True,
+        )
         X = self._validate_data(X=X, reset=False)
 
         # handle the single-class-label case
diff --git a/sktime/datasets/_single_problem_loaders.py b/sktime/datasets/_single_problem_loaders.py
index 35741c506..9835ce1ca 100644
--- a/sktime/datasets/_single_problem_loaders.py
+++ b/sktime/datasets/_single_problem_loaders.py
@@ -87,24 +87,31 @@ def load_UCR_UEA_dataset(
     split : None or str{"train", "test"}, optional (default=None)
         Whether to load the train or test partition of the problem. By default it
         loads both into a single dataset, otherwise it looks only for files of the
-        format <name>_TRAIN.ts or <name>_TEST.ts.
+        format ``<name>_TRAIN.ts`` or ``<name>_TEST.ts``.
+
     return_X_y : bool, optional (default=False)
         it returns two objects, if False, it appends the class labels to the dataframe.
+
     return_type: valid Panel mtype str or None, optional (default=None="nested_univ")
         Memory data format specification to return X in, None = "nested_univ" type.
         str can be any supported sktime Panel mtype,
-            for list of mtypes, see datatypes.MTYPE_REGISTER
-            for specifications, see examples/AA_datatypes_and_datasets.ipynb
+
+        * for list of mtypes, see ``datatypes.MTYPE_REGISTER``
+        * for specifications, see ``examples/AA_datatypes_and_datasets.ipynb``
+
         commonly used specifications:
-            "nested_univ: nested pd.DataFrame, pd.Series in cells
-            "numpy3D"/"numpy3d"/"np3D": 3D np.ndarray (instance, variable, time index)
-            "numpy2d"/"np2d"/"numpyflat": 2D np.ndarray (instance, time index)
-            "pd-multiindex": pd.DataFrame with 2-level (instance, time) MultiIndex
+
+        * "numpy3D"/"numpy3d"/"np3D": 3D np.ndarray (instance, variable, time index)
+        * "numpy2d"/"np2d"/"numpyflat": 2D np.ndarray (instance, time index)
+        * "pd-multiindex": pd.DataFrame with 2-level (instance, time) MultiIndex
+        * "nested_univ: nested pd.DataFrame, pd.Series in cells
+
         Exception is raised if the data cannot be stored in the requested type.
+
     extract_path : str, optional (default=None)
         the path to look for the data. If no path is provided, the function
         looks in ``sktime/datasets/data/``. If a path is given, it can be absolute,
-        e.g. C:/Temp or relative, e.g. Temp or ./Temp.
+        e.g. ``C:/Temp`` or relative, e.g. ``Temp`` or ``./Temp``.
 
     Returns
     -------
diff --git a/sktime/forecasting/auto_reg.py b/sktime/forecasting/auto_reg.py
index 5f209c6b8..fb59e8417 100644
--- a/sktime/forecasting/auto_reg.py
+++ b/sktime/forecasting/auto_reg.py
@@ -18,8 +18,6 @@ class AutoREG(_StatsModelsAdapter):
 
     Parameters
     ----------
-    endog : array_like
-        A 1-d endogenous response variable. The dependent variable.
     lags : {None, int, list[int]}
         The number of lags to include in the model if an integer or the
         list of lag indices to include.  For example, [1, 4] will only
@@ -37,10 +35,6 @@ class AutoREG(_StatsModelsAdapter):
         Flag indicating whether to include seasonal dummies in the model. If
         seasonal is True and trend includes 'c', then the first period
         is excluded from the seasonal terms.
-    exog : array_like, optional
-        Exogenous variables to include in the model. Must have the same number
-        of observations as endog and should be aligned so that endog[i] is
-        regressed on exog[i].
     hold_back : {None, int}
         Initial observations to exclude from the estimation sample.  If None,
         then hold_back is equal to the maximum lag in the model.  Set to a
@@ -51,7 +45,7 @@ class AutoREG(_StatsModelsAdapter):
         the model.
     period : {None, int}
         The period of the data. Only used if seasonal is True. This parameter
-        can be omitted if using a pandas object for endog that contains a
+        can be omitted if using a pandas object  that contains a
         recognized frequency.
     missing : str
         Available options are 'none', 'drop', and 'raise'. If 'none', no nan
diff --git a/sktime/forecasting/compose/_reduce.py b/sktime/forecasting/compose/_reduce.py
index 3505ab2a2..29f4d2736 100644
--- a/sktime/forecasting/compose/_reduce.py
+++ b/sktime/forecasting/compose/_reduce.py
@@ -2374,8 +2374,12 @@ class RecursiveReductionForecaster(BaseForecaster, _ReducerMixin):
         # lagger_y_to_X_ will lag y to obtain the sklearn X
         lags = self._lags
         lagger_y_to_X = Lag(lags=lags, index_out="extend")
-        if impute_method is not None:
-            lagger_y_to_X = lagger_y_to_X * Imputer(method=impute_method)
+
+        if isinstance(self.impute_method, Imputer):
+            lagger_y_to_X = lagger_y_to_X * impute_method
+        else:
+            if self.impute_method is not None:
+                lagger_y_to_X = lagger_y_to_X * Imputer(method=impute_method)
         self.lagger_y_to_X_ = lagger_y_to_X
 
         Xt = lagger_y_to_X.fit_transform(y)
@@ -2482,8 +2486,12 @@ class RecursiveReductionForecaster(BaseForecaster, _ReducerMixin):
             Xt = lagger_y_to_X.transform(y_plus_preds)
 
             lag_plus = Lag(lags=1, index_out="extend")
-            if self.impute_method is not None:
-                lag_plus = lag_plus * Imputer(method=self.impute_method)
+
+            if isinstance(self.impute_method, Imputer):
+                lag_plus = lag_plus * self.impute_method
+            else:
+                if self.impute_method is not None:
+                    lag_plus = lag_plus * Imputer(method=self.impute_method)
 
             Xtt = lag_plus.fit_transform(Xt)
             y_plus_one = lag_plus.fit_transform(y_plus_preds)
@@ -2534,8 +2542,12 @@ class RecursiveReductionForecaster(BaseForecaster, _ReducerMixin):
         Xt = lagger_y_to_X.transform(y)
 
         lag_plus = Lag(lags=1, index_out="extend")
-        if self.impute_method is not None:
-            lag_plus = lag_plus * Imputer(method=self.impute_method)
+
+        if isinstance(self.impute_method, Imputer):
+            lag_plus = lag_plus * self.impute_method
+        else:
+            if self.impute_method is not None:
+                lag_plus = lag_plus * Imputer(method=self.impute_method)
 
         Xtt = lag_plus.fit_transform(Xt)
 
@@ -2582,7 +2594,14 @@ class RecursiveReductionForecaster(BaseForecaster, _ReducerMixin):
         """
         from sklearn.linear_model import LinearRegression
 
+        from sktime.forecasting.compose._reduce import DirectReductionForecaster
+        from sktime.transformations.series.impute import Imputer
+
         est = LinearRegression()
+        forecaster_imputer = Imputer(
+            method="forecaster", forecaster=DirectReductionForecaster(estimator=est)
+        )
+
         params1 = {
             "estimator": est,
             "window_length": 3,
@@ -2593,8 +2612,20 @@ class RecursiveReductionForecaster(BaseForecaster, _ReducerMixin):
             "window_length": 4,
             "pooling": "local",
         }
+        params3 = {
+            "estimator": est,
+            "window_length": 4,
+            "pooling": "local",
+            "impute_method": forecaster_imputer,  # test imputation with forecaster
+        }
+        params4 = {
+            "estimator": est,
+            "window_length": 4,
+            "pooling": "global",
+            "impute_method": forecaster_imputer,
+        }
 
-        return [params1, params2]
+        return [params1, params2, params3, params4]
 
 
 class YfromX(BaseForecaster, _ReducerMixin):
diff --git a/sktime/forecasting/hf_transformers_forecaster.py b/sktime/forecasting/hf_transformers_forecaster.py
index 4e7384aa7..5efb77935 100644
--- a/sktime/forecasting/hf_transformers_forecaster.py
+++ b/sktime/forecasting/hf_transformers_forecaster.py
@@ -36,6 +36,7 @@ class HFTransformersForecaster(BaseForecaster):
     fit_strategy : str, default="minimal"
         Strategy to use for fitting (fine-tuning) the model. This can be one of
         the following:
+
         - "minimal": Fine-tunes only a small subset of the model parameters,
           allowing for quick adaptation with limited computational resources.
         - "full": Fine-tunes all model parameters, which may result in better
@@ -45,6 +46,7 @@ class HFTransformersForecaster(BaseForecaster):
           Note: If the 'peft' package is not available, a `ModuleNotFoundError` will
           be raised, indicating that the 'peft' package is required. Please install
           it using `pip install peft` to use this fit strategy.
+
     validation_split : float, default=0.2
         Fraction of the data to use for validation
     config : dict, default={}
diff --git a/sktime/forecasting/pytorchforecasting.py b/sktime/forecasting/pytorchforecasting.py
index 03360dd94..a12b9fc79 100644
--- a/sktime/forecasting/pytorchforecasting.py
+++ b/sktime/forecasting/pytorchforecasting.py
@@ -53,7 +53,7 @@ class PytorchForecastingTFT(_PytorchForecastingAdapter):
     >>> max_prediction_length = 5
     >>> fh = ForecastingHorizon(range(1, max_prediction_length + 1), is_relative=True)
     >>> # split X, y data for train and test
-    >>> x = data["c0", "c1"]
+    >>> x = data[["c0", "c1"]]
     >>> y = data["c2"].to_frame()
     >>> X_train, X_test, y_train, y_test = train_test_split(
     ...     x, y, test_size=0.2, train_size=0.8, shuffle=False
diff --git a/sktime/forecasting/ttm.py b/sktime/forecasting/ttm.py
index f6a391f76..ea9fc5711 100644
--- a/sktime/forecasting/ttm.py
+++ b/sktime/forecasting/ttm.py
@@ -31,41 +31,101 @@ class TinyTimeMixerForecaster(_BaseGlobalForecaster):
     open-sourced by IBM Research. With less than 1 Million parameters, TTM introduces
     the notion of the first-ever "tiny" pre-trained models for Time-Series Forecasting.
 
+    **Zero-shot and Fine-tuning**:
+
+    This model is designed as a versatile foundation model, capable of both
+    zero-shot forecasting and fine-tuning on custom datasets.
+    The choice between zero-shot and fine-tuning is managed internally and
+    is determined by the final configuration used to initialize the model.
+
+    **Initialization Process**:
+
+    1. **Model Path**: The ``model_path`` parameter points to a local folder or
+       huggingface repo that contains both *configuration files*
+       and *pretrained weights*.
+
+    2. **Default Configuration**: The model loads its default configuration from the
+       *configuration files*.
+
+    3. **Custom Configuration**: Users can provide a custom configuration via the
+       ``config`` parameter during model initialization.
+
+    4. **Configuration Override**: If custom configuration is provided,
+       it overrides the default configuration.
+
+    5. **Forecasting Horizon**: If the forecasting horizon (``fh``) specified during
+       ``fit`` exceeds the default ``config.prediction_length``,
+       the configuration is updated to reflect ``max(fh)``.
+
+    6. **Model Architecture**: The final configuration is used to construct the
+       *model architecture*.
+
+    7. **Pretrained Weights**: *pretrained weights* are loded from the ``model_path``,
+       these weights are then aligned and loaded into the *model architechture*.
+
+    8. **Weight Alignment**: However sometimes, *pretrained weights* do not align with
+       the *model architechture*, because the config was changed which created a
+       *model architechture* of different size than the default one.
+       This causes some of the weights in *model architechture* to be reinitialized
+       randomly instead of using the pre-trained weights.
+
+    **Forecasting Modes**:
+
+    - **Zero-shot Forecasting**: When all the *pre-trained weights* are correctly
+      aligned with the *model architechture*, fine-tuing part is bypassed and
+      the model preforms zero-short forecasting.
+
+    - **Fine-tuning**: When not all the *pre-trained weights* are correctly aligned
+      with the *model architechture*, rather some weights are re-initialized,
+      these re-initialized weights are fine-tuned on the provided data.
+
     Parameters
     ----------
     model_path : str, default="ibm/TTM"
         Path to the Huggingface model to use for forecasting.
         This can be either:
+
         - The name of a Huggingface repository (e.g., "ibm/TTM")
+
         - A local path to a folder containing model files in a format supported
           by transformers. In this case, ensure that the directory contains all
           necessary files (e.g., configuration, tokenizer, and model weights).
+
     revision: str, default="main"
         Revision of the model to use:
+
         - "main": For loading model with context_length of 512
           and prediction_length of 96.
+
         - "1024_96_v1": For loading model with context_length of 1024
           and prediction_length of 96.
+
     validation_split : float, default=0.2
         Fraction of the data to use for validation
+
     config : dict, default={}
         Configuration to use for the model. See the ``transformers``
         documentation for details.
+
     training_args : dict, default={}
         Training arguments to use for the model. See ``transformers.TrainingArguments``
         for details.
         Note that the ``output_dir`` argument is required.
+
     compute_metrics : list, default=None
         List of metrics to compute during training. See ``transformers.Trainer``
         for details.
+
     callbacks : list, default=[]
         List of callbacks to use during training. See ``transformers.Trainer``
+
     broadcasting : bool, default=False
         if True, multiindex data input will be broadcasted to single series.
         For each single series, one copy of this forecaster will try to
         fit and predict on it. The broadcasting is happening inside automatically,
         from the outerside api perspective, the input and output are the same,
         only one multiindex output from ``predict``.
+
     use_source_package : bool, default=False
         If True, the model and configuration will be loaded directly from the source
         package ``tsfm_public.models.tinytimemixer``. This is useful if you
@@ -80,9 +140,9 @@ class TinyTimeMixerForecaster(_BaseGlobalForecaster):
     ----------
     .. [1] https://github.com/ibm-granite/granite-tsfm/tree/main/tsfm_public/models/tinytimemixer
     .. [2] Ekambaram, V., Jati, A., Dayama, P., Mukherjee, S.,
-    Nguyen, N.H., Gifford, W.M., Reddy, C. and Kalagnanam, J., 2024.
-    Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced
-    Zero/Few-Shot Forecasting of Multivariate Time Series. CoRR.
+           Nguyen, N.H., Gifford, W.M., Reddy, C. and Kalagnanam, J., 2024.
+           Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced
+           Zero/Few-Shot Forecasting of Multivariate Time Series. CoRR.
     .. [3] https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_tutorial.ipynb
     .. [4] https://github.com/ibm-granite/granite-tsfm/tree/ttm
 
@@ -92,6 +152,7 @@ class TinyTimeMixerForecaster(_BaseGlobalForecaster):
     >>> from sktime.datasets import load_airline
     >>> y = load_airline()
     >>> forecaster = TinyTimeMixerForecaster() # doctest: +SKIP
+    >>> # performs zero-shot forecasting, as default config (unchanged) is used
     >>> forecaster.fit(y, fh=[1, 2, 3]) # doctest: +SKIP
     >>> y_pred = forecaster.predict() # doctest: +SKIP
 
@@ -118,7 +179,7 @@ class TinyTimeMixerForecaster(_BaseGlobalForecaster):
     ...     },
     ... ) # doctest: +SKIP
     >>>
-    >>> # train and predict
+    >>> # model is fine-tuned, as a config different from default is provided.
     >>> forecaster.fit(y, fh=[1, 2, 3]) # doctest: +SKIP
     >>> y_pred = forecaster.predict() # doctest: +SKIP
     """
@@ -304,18 +365,27 @@ class TinyTimeMixerForecaster(_BaseGlobalForecaster):
                 _model = getattr(_model, attr_name)
             _model.weight.requires_grad = True
 
-        y_train, y_test = temporal_train_test_split(y, test_size=self.validation_split)
+        if self.validation_split is not None:
+            y_train, y_eval = temporal_train_test_split(
+                y, test_size=self.validation_split
+            )
+        else:
+            y_train = y
+            y_eval = None
 
         train = PyTorchDataset(
             y=y_train,
             context_length=config.context_length,
             prediction_length=config.prediction_length,
         )
-        test = PyTorchDataset(
-            y=y_test,
-            context_length=config.context_length,
-            prediction_length=config.prediction_length,
-        )
+
+        eval = None
+        if self.validation_split is not None:
+            eval = PyTorchDataset(
+                y=y_eval,
+                context_length=config.context_length,
+                prediction_length=config.prediction_length,
+            )
 
         # Get Training Configuration
         training_args = TrainingArguments(**self._training_args)
@@ -325,7 +395,7 @@ class TinyTimeMixerForecaster(_BaseGlobalForecaster):
             model=self.model,
             args=training_args,
             train_dataset=train,
-            eval_dataset=test,
+            eval_dataset=eval,
             compute_metrics=self.compute_metrics,
             callbacks=self.callbacks,
         )
diff --git a/sktime/performance_metrics/detection/__init__.py b/sktime/performance_metrics/detection/__init__.py
index b8e80b14c..0361676e5 100644
--- a/sktime/performance_metrics/detection/__init__.py
+++ b/sktime/performance_metrics/detection/__init__.py
@@ -3,5 +3,6 @@
 from sktime.performance_metrics.detection._chamfer import DirectedChamfer
 from sktime.performance_metrics.detection._count import DetectionCount
 from sktime.performance_metrics.detection._hausdorff import DirectedHausdorff
+from sktime.performance_metrics.detection._randindex import RandIndex
 
-__all__ = ["DirectedChamfer", "DirectedHausdorff", "DetectionCount"]
+__all__ = ["DirectedChamfer", "DirectedHausdorff", "DetectionCount", "RandIndex"]
diff --git a/sktime/performance_metrics/detection/_randindex.py b/sktime/performance_metrics/detection/_randindex.py
new file mode 100644
index 000000000..d3c4cb209
--- /dev/null
+++ b/sktime/performance_metrics/detection/_randindex.py
@@ -0,0 +1,307 @@
+import pandas as pd
+
+from sktime.performance_metrics.detection._base import BaseDetectionMetric
+
+
+class RandIndex(BaseDetectionMetric):
+    r"""Segmentation Rand Index metric.
+
+    The Segmentation Rand Index (SRI) for comparing two segmentations
+    is the integrated version of the Rand Index
+    for clustering (RI). It measures the similarity between two segmentations of the
+    same time series, where each segmentation is a sequence of non-overlapping segments.
+
+    It takes values between 0 and 1, where 1 indicates identical segmentations.
+
+    The segmentation Rand Index (SRI) is obtained from the clustering Rand Index (RI),
+    in qualitative terms, by:
+
+    - Interpreting segments as clusters along an index or time axis.
+
+    - Counting pairwise “agreements” (same vs different) along that axis.
+
+    A mathematical definition follows.
+
+    For clarity, we define the clustering Rand Index (RI) first:
+
+    The Rand Index (RI) between two clusterings of n elements is defined as:
+
+    .. math:: RI = (a + b) / (a + b + c + d),
+
+    where:
+
+    - a = #pairs in the same cluster in both segmentations
+
+    - b = #pairs in different clusters in both segmentations
+
+    - c = #pairs in the same cluster in the first segmentation but different in the second
+
+    - d = #pairs in different clusters in the first segmentation but same in the second
+
+    The segmentation Rand Index (SRI) is defined as follows:
+
+    For a time series starting at time index :math:`S \in \mathbb{R}`
+    and ending at time index :math:`T \in \mathbb{R}`, let
+
+    :math:`a: [S, T] \rightarrow \mathbb{N}` be the first segmentation,
+    and :math:`b: [S, T] \rightarrow \mathbb{N}` be the second segmentation.
+
+    Let :math:`\mathbb{I}(s_1, s_2, t_1, t_2)` be the indicator function that is 1
+    iff at least one of the following holds, and 0 otherwise:
+
+    - :math:`s_1 = s_2` and :math:`t_1 = t_2`
+    - :math:`s_1 \neq s_2` and :math:`t_1 \neq t_2`
+
+    Then the SRI is defined as:
+
+    .. math:: SRI = \frac{1}{T-S} \int_S^T \int_S^T \mathbb{I}(a(s), a(t), b(s), b(t)) ds dt
+
+    By default, if ``X`` is provided, this metric computes distances in loc-based units
+    by looking up the corresponding labels in ``X.index``.
+    Otherwise, or if ``use_loc=False``, it uses iloc-based indexing.
+
+    If an existing "label" column is present in y_true/y_pred, those labels are used
+    directly for matching; otherwise, segments are considered to be numbered
+    consecutively, starting from 0.
+    """  # noqa: E501
+
+    _tags = {
+        "object_type": ["metric_detection", "metric"],
+        "scitype:y": "segments",
+        "requires_X": False,
+        "requires_y_true": True,
+        "lower_is_better": False,  # Higher Rand Index is better
+    }
+
+    def __init__(self, use_loc=True):
+        """
+        Parameters
+        ----------
+        use_loc : bool, optional (default=True)
+
+            - If True (and X is provided), segment lengths/overlaps are computed as the
+              difference of X.index[end_iloc] - X.index[start_iloc].
+
+            - If False, or X=None, uses iloc-based distances (end_iloc - start_iloc) as before.
+        """  # noqa: D205, E501
+        self.use_loc = use_loc
+        super().__init__()
+
+    def _evaluate(self, y_true, y_pred, X=None):
+        """
+        Calculate Rand Index between true and predicted segments.
+
+        Parameters
+        ----------
+        y_true : pd.DataFrame
+            Ground truth segments with 'start', 'end', or an 'ilocs' column (interval or int),
+            plus an optional 'label' column.
+
+        y_pred : pd.DataFrame
+            Predicted segments with 'start', 'end', or an 'ilocs' column (interval or int),
+            plus an optional 'label' column.
+
+        X : pd.DataFrame, optional (default=None)
+            If provided (and use_loc=True), loc-based distances are used.
+
+        Returns
+        -------
+        float
+            Rand Index score between 0.0 and 1.0.
+        """  # noqa: E501
+        # 1) Validate and extract segments (still iloc-based).
+        y_true_segments = self._extract_segments(y_true, "y_true")
+        y_pred_segments = self._extract_segments(y_pred, "y_pred")
+
+        # 2) Assign unique cluster IDs to each segment (does not overwrite 'label').
+        y_true_segments = self._assign_unique_ids(y_true_segments, prefix="true")
+        y_pred_segments = self._assign_unique_ids(y_pred_segments, prefix="pred")
+
+        # 3) Determine total_length in loc or iloc units
+        if X is not None and self.use_loc:
+            # Loc-based => difference of X.index[-1] - X.index[0], if more than one row
+            if len(X) > 1:
+                total_length = float(X.index[-1] - X.index[0])
+            else:
+                total_length = len(X)
+        else:
+            # Iloc-based => max 'end' among true/pred
+            max_end_true = max((seg["end"] for seg in y_true_segments), default=0)
+            max_end_pred = max((seg["end"] for seg in y_pred_segments), default=0)
+            total_length = max(max_end_true, max_end_pred)
+
+        # 4) If there's <2 points in total, the Rand Index is trivially 1.
+        if total_length < 2:
+            return 1.0
+
+        # 5) Compute same-cluster pairs in y_true, y_pred
+        same_cluster_true = sum(
+            self._pairs_count(self._compute_length(seg["start"], seg["end"], X))
+            for seg in y_true_segments
+        )
+        same_cluster_pred = sum(
+            self._pairs_count(self._compute_length(seg["start"], seg["end"], X))
+            for seg in y_pred_segments
+        )
+
+        # 6) same-cluster pairs that both segmentations agree on
+        same_cluster_both = self._compute_same_cluster_both(
+            y_true_segments, y_pred_segments, X
+        )
+
+        # 7) total pairs
+        total_pairs = self._pairs_count(total_length)
+
+        # 8) Rand Index = (a + d) / total_pairs, with
+        #    a = same_cluster_both
+        #    d = total_pairs - same_cluster_true - same_cluster_pred + a
+        agreements = same_cluster_both + (
+            total_pairs - same_cluster_true - same_cluster_pred + same_cluster_both
+        )
+        rand_index = agreements / total_pairs
+        return rand_index
+
+    def _compute_length(self, start_iloc, end_iloc, X):
+        """Compute the length of a segment in either loc-based or iloc-based units."""
+        if end_iloc <= start_iloc:
+            return 0
+        length_iloc = end_iloc - start_iloc
+        if X is not None and self.use_loc:
+            # clamp to avoid out-of-bounds
+            start_iloc = max(0, min(start_iloc, len(X) - 1))
+            end_iloc = max(0, min(end_iloc, len(X) - 1))
+            return float(X.index[end_iloc] - X.index[start_iloc])
+        else:
+            return float(length_iloc)
+
+    def _compute_same_cluster_both(self, y_true_segments, y_pred_segments, X=None):
+        """Compute # of same-cluster pairs (a) in the overlap of segments with the same label."""  # noqa: E501
+        a = 0
+        y_true_sorted = sorted(y_true_segments, key=lambda x: x["start"])
+        y_pred_sorted = sorted(y_pred_segments, key=lambda x: x["start"])
+
+        i, j = 0, 0
+        while i < len(y_true_sorted) and j < len(y_pred_sorted):
+            t_seg = y_true_sorted[i]
+            p_seg = y_pred_sorted[j]
+
+            overlap_start = max(t_seg["start"], p_seg["start"])
+            overlap_end = min(t_seg["end"], p_seg["end"])
+
+            if overlap_start < overlap_end:
+                overlap_length = self._compute_length(overlap_start, overlap_end, X)
+                # Compare existing 'label' columns
+                if t_seg["label"] == p_seg["label"]:
+                    a += self._pairs_count(overlap_length)
+
+            if t_seg["end"] <= p_seg["end"]:
+                i += 1
+            else:
+                j += 1
+
+        return a
+
+    def _pairs_count(self, length):
+        """
+        Number of unique pairs among 'length' items.
+
+        For discrete or continuous:
+
+        - If length < 2, returns 0.
+
+        - Otherwise, we round length to integer n and compute (n*(n-1))//2
+        """  # noqa: D401
+        if length < 2:
+            return 0
+        n = int(round(length))
+        return (n * (n - 1)) // 2 if n >= 2 else 0
+
+    def _extract_segments(self, y, var_name):
+        """
+        Extract segments from the DataFrame.
+
+        1) 'start'/'end' => direct use
+
+        2) A single int column 'ilocs' => interpret each row as [i, i+1)
+
+        3) A single interval column 'ilocs' => read left/right from each Interval
+
+        If a 'label' column is present, that is used directly for identifying clusters.
+        Otherwise, the row index i is used as a fallback label.
+        """
+        segments = []
+
+        # Case 1
+        if {"start", "end"}.issubset(y.columns):
+            for i, row in y.iterrows():
+                seg_start = row["start"]
+                seg_end = row["end"]
+                seg_label = row["label"] if "label" in y.columns else i
+                segments.append(
+                    {"start": seg_start, "end": seg_end, "label": seg_label}
+                )
+            return segments
+
+        # Case 2/3: user-provided 'ilocs'
+        if "ilocs" in y.columns:
+            col_dtype = y["ilocs"].dtype
+            if pd.api.types.is_interval_dtype(col_dtype):
+                for i, row in y.iterrows():
+                    interval = row["ilocs"]
+                    seg_start, seg_end = interval.left, interval.right
+                    seg_label = row["label"] if "label" in y.columns else i
+                    segments.append(
+                        {"start": seg_start, "end": seg_end, "label": seg_label}
+                    )
+            else:
+                # integer => [iloc_val, iloc_val+1)
+                for i, row in y.iterrows():
+                    iloc_val = row["ilocs"]
+                    seg_label = row["label"] if "label" in y.columns else i
+                    segments.append(
+                        {"start": iloc_val, "end": iloc_val + 1, "label": seg_label}
+                    )
+            return segments
+
+        raise ValueError(
+            f"Expected columns ['start','end'] or 'ilocs' in {var_name}, "
+            f"but found columns {list(y.columns)}."
+        )
+
+    def _assign_unique_ids(self, segments, prefix):
+        """
+        Assign a unique 'cluster_id' to each segment (for internal reference),
+        but do not overwrite the existing 'label' if it is present.
+
+        Parameters
+        ----------
+        segments : list of dict
+            Each dict has 'start', 'end', and 'label'.
+
+        prefix : str
+            E.g. "true" or "pred", to differentiate ground truth vs predicted.
+
+        Returns
+        -------
+        segments : list of dict
+            Same as input, but each dict gains a 'cluster_id' key.
+        """  # noqa: D205
+        for idx, seg in enumerate(segments):
+            # We do NOT overwrite seg['label'] -- we only create a separate ID if needed
+            seg["cluster_id"] = f"{prefix}_{seg['label']}_{idx}"
+        return segments
+
+    @classmethod
+    def get_test_params(cls, parameter_set="default"):
+        """
+        Return testing parameter settings for the estimator.
+
+        Creates two configurations:
+
+        1) default RandIndex (use_loc=True)
+
+        2) RandIndex(use_loc=False)
+        """
+        param1 = {}  # default => use_loc=True
+        param2 = {"use_loc": False}
+        return [param1, param2]
diff --git a/sktime/performance_metrics/forecasting/_classes.py b/sktime/performance_metrics/forecasting/_classes.py
index c5986cb0f..b1ef7c090 100644
--- a/sktime/performance_metrics/forecasting/_classes.py
+++ b/sktime/performance_metrics/forecasting/_classes.py
@@ -124,6 +124,14 @@ class BaseForecastingErrorMetric(BaseMetric):
         * If 'uniform_average_time', metric is applied to all data,
           ignoring level index.
         * If 'raw_values', does not average errors across levels, hierarchy is retained.
+
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
     """
 
     _tags = {
@@ -135,16 +143,18 @@ class BaseForecastingErrorMetric(BaseMetric):
         "lower_is_better": True,
         # "y_inner_mtype": ["pd.DataFrame", "pd-multiindex", "pd_multiindex_hier"]
         "inner_implements_multilevel": False,
-        "reserved_params": ["multioutput", "multilevel"],
+        "reserved_params": ["multioutput", "multilevel", "by_index"],
     }
 
     def __init__(
         self,
         multioutput="uniform_average",
         multilevel="uniform_average",
+        by_index=False,
     ):
         self.multioutput = multioutput
         self.multilevel = multilevel
+        self.by_index = by_index
 
         if not hasattr(self, "name") or self.name is None:
             self.name = type(self).__name__
@@ -230,6 +240,8 @@ class BaseForecastingErrorMetric(BaseMetric):
               of shape ``(n_levels, y_true.columns)`` if ``multioutput="raw_values"``.
               metric is applied per level, row averaging (yes/no) as in ``multioutput``.
         """  # noqa: E501
+        if self.by_index:
+            return self.evaluate_by_index(y_true, y_pred, **kwargs)
         return self.evaluate(y_true, y_pred, **kwargs)
 
     def _apply_sample_weight_to_kwargs(self, y_true, y_pred, **kwargs):
@@ -884,13 +896,16 @@ class _DynamicForecastingErrorMetric(BaseForecastingErrorMetricFunc):
         multioutput="uniform_average",
         multilevel="uniform_average",
         lower_is_better=True,
+        by_index=False,
     ):
         self.multioutput = multioutput
         self.multilevel = multilevel
         self.func = func
         self.name = name
         self.lower_is_better = lower_is_better
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput, multilevel=multilevel, by_index=by_index
+        )
 
         self.set_tags(**{"lower_is_better": lower_is_better})
 
@@ -1060,6 +1075,14 @@ class MeanAbsoluteScaledError(_ScaledMetricTags, BaseForecastingErrorMetricFunc)
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MedianAbsoluteScaledError
@@ -1108,9 +1131,12 @@ class MeanAbsoluteScaledError(_ScaledMetricTags, BaseForecastingErrorMetricFunc)
         multioutput="uniform_average",
         multilevel="uniform_average",
         sp=1,
+        by_index=False,
     ):
         self.sp = sp
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput, multilevel=multilevel, by_index=by_index
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -1181,6 +1207,14 @@ class MedianAbsoluteScaledError(_ScaledMetricTags, BaseForecastingErrorMetricFun
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsoluteScaledError
@@ -1229,9 +1263,14 @@ class MedianAbsoluteScaledError(_ScaledMetricTags, BaseForecastingErrorMetricFun
         multioutput="uniform_average",
         multilevel="uniform_average",
         sp=1,
+        by_index=False,
     ):
         self.sp = sp
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -1303,6 +1342,14 @@ class MeanSquaredScaledError(_ScaledMetricTags, BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsoluteScaledError
@@ -1349,10 +1396,15 @@ class MeanSquaredScaledError(_ScaledMetricTags, BaseForecastingErrorMetricFunc):
         multilevel="uniform_average",
         sp=1,
         square_root=False,
+        by_index=False,
     ):
         self.sp = sp
         self.square_root = square_root
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -1424,6 +1476,14 @@ class MedianSquaredScaledError(_ScaledMetricTags, BaseForecastingErrorMetricFunc
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsoluteScaledError
@@ -1470,10 +1530,15 @@ class MedianSquaredScaledError(_ScaledMetricTags, BaseForecastingErrorMetricFunc
         multilevel="uniform_average",
         sp=1,
         square_root=False,
+        by_index=False,
     ):
         self.sp = sp
         self.square_root = square_root
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -1545,6 +1610,14 @@ class MeanAbsoluteError(BaseForecastingErrorMetric):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MedianAbsoluteError
@@ -1661,6 +1734,14 @@ class MedianAbsoluteError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsoluteError
@@ -1801,6 +1882,14 @@ class MeanSquaredError(BaseForecastingErrorMetric):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsoluteError
@@ -1847,9 +1936,14 @@ class MeanSquaredError(BaseForecastingErrorMetric):
         multioutput="uniform_average",
         multilevel="uniform_average",
         square_root=False,
+        by_index=False,
     ):
         self.square_root = square_root
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     def _evaluate(self, y_true, y_pred, **kwargs):
         """Evaluate the desired metric on given inputs.
@@ -2009,6 +2103,13 @@ class MeanSquaredErrorPercentage(BaseForecastingErrorMetricFunc):
         If 'uniform_average' (default), errors are mean-averaged across levels.
         If 'uniform_average_time', metric is applied to all data, ignoring level index.
         If 'raw_values', does not average errors across levels, hierarchy is retained.
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
     """
 
     def __init__(
@@ -2016,11 +2117,16 @@ class MeanSquaredErrorPercentage(BaseForecastingErrorMetricFunc):
         multioutput="uniform_average",
         multilevel="uniform_average",
         square_root=False,
+        by_index=False,
     ):
         self.square_root = square_root
         self.multioutput = multioutput
 
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     def _evaluate(self, y_true, y_pred, **kwargs):
         r"""
@@ -2139,22 +2245,48 @@ class MeanSquaredErrorPercentage(BaseForecastingErrorMetricFunc):
 
 
 class MedianSquaredError(BaseForecastingErrorMetricFunc):
-    """Median squared error (MdSE) or root median squared error (RMdSE).
+    r"""Median squared error (MdSE) or root median squared error (RMdSE).
 
     If ``square_root`` is False then calculates MdSE and if ``square_root`` is True
     then RMdSE is calculated. Both MdSE and RMdSE return non-negative floating
     point. The best value is 0.0.
 
+    The Median Squared Error (MdSE) is calculated as:
+
+    .. math::
+        \\text{MdSE} = \text{median}\left((y_{i} - \\widehat{y}_{i})^2\right)_{i=1}^{n}
+
+    where:
+    - \\( y_i \\) are the true values,
+    - \\( \\hat{y}_i \\) are the predicted values,
+    - \\( n \\) is the number of observations.
+
+    If ``square_root`` is set to True,
+    the Root Median Squared Error Percentage (RMdSE) is computed:
+
+    .. math::
+        \\text{RMdSE} = \\sqrt{ \\text{MdSE} }
+
     Like MSE, MdSE is measured in squared units of the input data. RMdSE is
-    on the same scale as the input data like RMSE. Because MdSE and RMdSE
-    square the forecast error rather than taking the absolute value, they
-    penalize large errors more than MAE or MdAE.
+    on the same scale as the input data, like RMSE.
 
     Taking the median instead of the mean of the squared errors makes
     this metric more robust to error outliers relative to a meean based metric
     since the median tends to be a more robust measure of central tendency in
     the presence of outliers.
 
+    Because the median commutes with monotonic transformations, MdSE and RMdSE
+    do not penalize large errors more than the MdAE.
+
+    ``evaluate_by_index`` returns, at a time index :math:`t_i`:
+
+    * if ``square_root`` is False, the squared error at that time index,
+        :math:`(y_i - \widehat{y}_i)^2`,
+    * if ``square_root`` is True, the absolute error at that time index,
+      the absolute error at that time index, :math:`|y_i - \widehat{y}_i|`,
+
+    for all time indices :math:`t_1, \dots, t_n` in the input.
+
     Parameters
     ----------
     square_root : bool, default = False
@@ -2180,6 +2312,14 @@ class MedianSquaredError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsoluteError
@@ -2230,9 +2370,61 @@ class MedianSquaredError(BaseForecastingErrorMetricFunc):
         multioutput="uniform_average",
         multilevel="uniform_average",
         square_root=False,
+        by_index=False,
     ):
         self.square_root = square_root
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
+
+    def _evaluate_by_index(self, y_true, y_pred, **kwargs):
+        """Return the metric evaluated at each time point.
+
+        private _evaluate_by_index containing core logic, called from evaluate_by_index
+
+        Parameters
+        ----------
+        y_true : time series in sktime compatible pandas based data container format
+            Ground truth (correct) target values
+            y can be in one of the following formats:
+            Series scitype: pd.DataFrame
+            Panel scitype: pd.DataFrame with 2-level row MultiIndex
+            Hierarchical scitype: pd.DataFrame with 3 or more level row MultiIndex
+        y_pred : time series in sktime compatible data container format
+            Forecasted values to evaluate
+            must be of same format as y_true, same indices and columns if indexed
+
+        Returns
+        -------
+        loss : pd.Series or pd.DataFrame
+            Calculated metric, by time point.
+            pd.Series if self.multioutput="uniform_average" or array-like
+                index is equal to index of y_true
+                entry at index i is metric at time i, averaged over variables
+            pd.DataFrame if self.multioutput="raw_values"
+                index and columns equal to those of y_true
+                i,j-th entry is metric at time i, at variable j
+        """
+        multioutput = self.multioutput
+
+        raw_values = (y_true - y_pred) ** 2
+
+        if self.square_root:
+            raw_values = np.sqrt(raw_values)
+
+        raw_values = self._get_weighted_df(raw_values, **kwargs)
+
+        if isinstance(multioutput, str):
+            if multioutput == "raw_values":
+                return raw_values
+
+            if multioutput == "uniform_average":
+                return raw_values.median(axis=1)
+
+        # else, we expect multioutput to be array-like
+        return raw_values.dot(multioutput)
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -2291,6 +2483,14 @@ class GeometricMeanAbsoluteError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     mean_absolute_error
@@ -2377,6 +2577,14 @@ class GeometricMeanSquaredError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     mean_absolute_error
@@ -2441,9 +2649,12 @@ class GeometricMeanSquaredError(BaseForecastingErrorMetricFunc):
         multioutput="uniform_average",
         multilevel="uniform_average",
         square_root=False,
+        by_index=False,
     ):
         self.square_root = square_root
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput, multilevel=multilevel, by_index=by_index
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -2531,6 +2742,14 @@ class MeanAbsolutePercentageError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MedianAbsolutePercentageError
@@ -2581,9 +2800,14 @@ class MeanAbsolutePercentageError(BaseForecastingErrorMetricFunc):
         multioutput="uniform_average",
         multilevel="uniform_average",
         symmetric=False,
+        by_index=False,
     ):
         self.symmetric = symmetric
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     def _evaluate_by_index(self, y_true, y_pred, **kwargs):
         """Return the metric evaluated at each time point.
@@ -2716,6 +2940,14 @@ class MedianAbsolutePercentageError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsolutePercentageError
@@ -2766,9 +2998,14 @@ class MedianAbsolutePercentageError(BaseForecastingErrorMetricFunc):
         multioutput="uniform_average",
         multilevel="uniform_average",
         symmetric=False,
+        by_index=False,
     ):
         self.symmetric = symmetric
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -2839,6 +3076,14 @@ class MeanSquaredPercentageError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsolutePercentageError
@@ -2893,10 +3138,15 @@ class MeanSquaredPercentageError(BaseForecastingErrorMetricFunc):
         multilevel="uniform_average",
         symmetric=False,
         square_root=False,
+        by_index=False,
     ):
         self.symmetric = symmetric
         self.square_root = square_root
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -2971,6 +3221,14 @@ class MedianSquaredPercentageError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanAbsolutePercentageError
@@ -3025,10 +3283,15 @@ class MedianSquaredPercentageError(BaseForecastingErrorMetricFunc):
         multilevel="uniform_average",
         symmetric=False,
         square_root=False,
+        by_index=False,
     ):
         self.symmetric = symmetric
         self.square_root = square_root
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -3086,6 +3349,14 @@ class MeanRelativeAbsoluteError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MedianRelativeAbsoluteError
@@ -3161,6 +3432,14 @@ class MedianRelativeAbsoluteError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanRelativeAbsoluteError
@@ -3237,6 +3516,14 @@ class GeometricMeanRelativeAbsoluteError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanRelativeAbsoluteError
@@ -3321,6 +3608,14 @@ class GeometricMeanRelativeSquaredError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     MeanRelativeAbsoluteError
@@ -3369,9 +3664,14 @@ class GeometricMeanRelativeSquaredError(BaseForecastingErrorMetricFunc):
         multioutput="uniform_average",
         multilevel="uniform_average",
         square_root=False,
+        by_index=False,
     ):
         self.square_root = square_root
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -3460,6 +3760,14 @@ class MeanAsymmetricError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     mean_linex_error
@@ -3519,6 +3827,7 @@ class MeanAsymmetricError(BaseForecastingErrorMetricFunc):
         right_error_function="absolute",
         left_error_penalty=1.0,
         right_error_penalty=1.0,
+        by_index=False,
     ):
         self.asymmetric_threshold = asymmetric_threshold
         self.left_error_function = left_error_function
@@ -3526,7 +3835,11 @@ class MeanAsymmetricError(BaseForecastingErrorMetricFunc):
         self.left_error_penalty = left_error_penalty
         self.right_error_penalty = right_error_penalty
 
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -3607,6 +3920,14 @@ class MeanLinexError(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     See Also
     --------
     mean_asymmetric_error
@@ -3663,10 +3984,15 @@ class MeanLinexError(BaseForecastingErrorMetricFunc):
         b=1.0,
         multioutput="uniform_average",
         multilevel="uniform_average",
+        by_index=False,
     ):
         self.a = a
         self.b = b
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
@@ -3742,6 +4068,14 @@ class RelativeLoss(BaseForecastingErrorMetricFunc):
         * If ``'raw_values'``,
           does not average errors across levels, hierarchy is retained.
 
+    by_index : bool, default=False
+        Determines averaging over time points in direct call to metric object.
+
+        * If False, direct call to the metric object averages over time points,
+          equivalent to a call of the``evaluate`` method.
+        * If True, direct call to the metric object evaluates the metric at each
+          time point, equivalent to a call of the ``evaluate_by_index`` method.
+
     References
     ----------
     Hyndman, R. J and Koehler, A. B. (2006). "Another look at measures of
@@ -3788,9 +4122,14 @@ class RelativeLoss(BaseForecastingErrorMetricFunc):
         multioutput="uniform_average",
         multilevel="uniform_average",
         relative_loss_function=mean_absolute_error,
+        by_index=False,
     ):
         self.relative_loss_function = relative_loss_function
-        super().__init__(multioutput=multioutput, multilevel=multilevel)
+        super().__init__(
+            multioutput=multioutput,
+            multilevel=multilevel,
+            by_index=by_index,
+        )
 
     @classmethod
     def get_test_params(cls, parameter_set="default"):
diff --git a/sktime/performance_metrics/forecasting/_functions.py b/sktime/performance_metrics/forecasting/_functions.py
index 28fa97357..b109c833d 100644
--- a/sktime/performance_metrics/forecasting/_functions.py
+++ b/sktime/performance_metrics/forecasting/_functions.py
@@ -1122,7 +1122,7 @@ def median_squared_error(
     penalize large errors more than MAE or MdAE.
 
     Taking the median instead of the mean of the squared errors makes
-    this metric more robust to error outliers relative to a meean based metric
+    this metric more robust to error outliers relative to a mean based metric
     since the median tends to be a more robust measure of central tendency in
     the presence of outliers.
 
diff --git a/sktime/split/sameloc.py b/sktime/split/sameloc.py
index 395533f85..38fd312ec 100644
--- a/sktime/split/sameloc.py
+++ b/sktime/split/sameloc.py
@@ -23,6 +23,18 @@ class SameLocSplitter(BaseSplitter):
     Splits ``y`` in ``split`` and ``split_loc`` such that ``loc`` indices of splits
     are identical to loc indices of ``cv`` applied to ``y_template``.
 
+    This splitter is useful when you need to replicate train-test splits across multiple
+    time series with consistent loc-based indexing.
+
+    Mathematically, let :math:`y_{template}` represent the reference time series,
+    corresponding to the input ``y_template``,
+    and let :math:`I_{train, 1}, I_{test, 1}, \ldots, I_{train, K}, I_{test, K}`
+    be the loc-based train-test splits generated by ``cv`` on ``y_template``.
+
+    The `SameLocSplitter` returns the corresponding positional indices in ``y``,
+    such that the loc-based splits are identical to the splits generated by ``cv``,
+    namely, :math:`I_{train, 1}, I_{test, 1}, \ldots, I_{train, K}, I_{test, K}`.
+
     Parameters
     ----------
     cv : BaseSplitter


Test patch:
diff --git a/sktime/classification/sklearn/tests/test_rotation_forest.py b/sktime/classification/sklearn/tests/test_rotation_forest.py
index 9186e71e6..1ba351de6 100644
--- a/sktime/classification/sklearn/tests/test_rotation_forest.py
+++ b/sktime/classification/sklearn/tests/test_rotation_forest.py
@@ -2,12 +2,23 @@
 
 import numpy as np
 import pytest
+from sklearn.utils.estimator_checks import parametrize_with_checks
 
 from sktime.classification.sklearn import RotationForest
 from sktime.datasets import load_unit_test
 from sktime.tests.test_switch import run_test_for_class
 
 
+@pytest.mark.skipif(
+    not run_test_for_class(RotationForest),
+    reason="run test if softdeps are present and incrementally (if requested)",
+)
+@parametrize_with_checks([RotationForest()])
+def test_sklearn_compatible_estimator(estimator, check):
+    """Run sklearn estimator compatibility checks."""
+    check(estimator)
+
+
 @pytest.mark.skipif(
     not run_test_for_class(RotationForest),
     reason="run test only if softdeps are present and incrementally (if requested)",
diff --git a/sktime/performance_metrics/forecasting/tests/test_all_metrics_forecasting.py b/sktime/performance_metrics/forecasting/tests/test_all_metrics_forecasting.py
index 821ee4d41..f8b77ec9e 100644
--- a/sktime/performance_metrics/forecasting/tests/test_all_metrics_forecasting.py
+++ b/sktime/performance_metrics/forecasting/tests/test_all_metrics_forecasting.py
@@ -134,6 +134,21 @@ class TestAllForecastingPtMetrics(ForecastingMetricPtFixtureGenerator, QuickTest
 
         assert (res.index == y_true.index).all()
 
+        metric = metric.clone()
+
+        metric.set_params(by_index=True)
+
+        res_call = metric(
+            y_true=y_true,
+            y_pred=y_pred,
+            y_pred_benchmark=y_pred,
+            y_train=y_true,
+        )
+        if isinstance(multioutput, str) and multioutput == "raw_values":
+            pd.testing.assert_frame_equal(res, res_call)
+        else:
+            pd.testing.assert_series_equal(res, res_call)
+
     def test_uniform_average_time(self, estimator_instance):
         """Tests that uniform_average_time indeed ignores index."""
         metric_obj = estimator_instance.set_params(multilevel="uniform_average_time")