Repository: sktime/sktime

Issue:
[DOC] BaseForecaster score function uses MAPE instead of sMAPE as metric
#### Describe the issue linked to the documentation

The [score function](https://github.com/sktime/sktime/blob/ed4efe9e47fa998ad6b21d63b0ac58f932ab1b3f/sktime/forecasting/base/_base.py#L1234C1-L1234C1) of the BaseForecaster suggests the metric that is used for evaluation is sMAPE:

```
Returns

        -------

        score : float
            sMAPE loss of self.predict(fh, X) with respect to y_test.
```
            
The code calls the [mean_absolute_percentage_error function](https://github.com/sktime/sktime/blob/ed4efe9e47fa998ad6b21d63b0ac58f932ab1b3f/sktime/performance_metrics/forecasting/_functions.py#L1448) without any further parameters. 

```     
from sktime.performance_metrics.forecasting import (
            mean_absolute_percentage_error,
        )

        return mean_absolute_percentage_error(y, self.predict(fh, X))
``` 

But the default values of the [mean_absolute_percentage_error function ](https://github.com/sktime/sktime/blob/ed4efe9e47fa998ad6b21d63b0ac58f932ab1b3f/sktime/performance_metrics/forecasting/_functions.py#L1448) are: 

```
def mean_absolute_percentage_error(
    y_true,
    y_pred,
    horizon_weight=None,
    multioutput="uniform_average",
    symmetric=False,
    **kwargs,
):
```

and the doc states:

```
... 
    symmetric : bool, default=False
        Calculates symmetric version of metric if True.
...
```

Thus I conclude that instead of the sMAPE, MAPE is returned.

#### Suggest a potential alternative/fix

Either adjust the documentation such that it states that the MAPE metric is used or adjust the implementation by adding the symmetric parameter. I did not investigate if there are other places with the documentation error. ;) Many thanks!! 

[DOC] BaseForecaster score function uses MAPE instead of sMAPE as metric
#### Describe the issue linked to the documentation

The [score function](https://github.com/sktime/sktime/blob/ed4efe9e47fa998ad6b21d63b0ac58f932ab1b3f/sktime/forecasting/base/_base.py#L1234C1-L1234C1) of the BaseForecaster suggests the metric that is used for evaluation is sMAPE:

```
Returns

        -------

        score : float
            sMAPE loss of self.predict(fh, X) with respect to y_test.
```
            
The code calls the [mean_absolute_percentage_error function](https://github.com/sktime/sktime/blob/ed4efe9e47fa998ad6b21d63b0ac58f932ab1b3f/sktime/performance_metrics/forecasting/_functions.py#L1448) without any further parameters. 

```     
from sktime.performance_metrics.forecasting import (
            mean_absolute_percentage_error,
        )

        return mean_absolute_percentage_error(y, self.predict(fh, X))
``` 

But the default values of the [mean_absolute_percentage_error function ](https://github.com/sktime/sktime/blob/ed4efe9e47fa998ad6b21d63b0ac58f932ab1b3f/sktime/performance_metrics/forecasting/_functions.py#L1448) are: 

```
def mean_absolute_percentage_error(
    y_true,
    y_pred,
    horizon_weight=None,
    multioutput="uniform_average",
    symmetric=False,
    **kwargs,
):
```

and the doc states:

```
... 
    symmetric : bool, default=False
        Calculates symmetric version of metric if True.
...
```

Thus I conclude that instead of the sMAPE, MAPE is returned.

#### Suggest a potential alternative/fix

Either adjust the documentation such that it states that the MAPE metric is used or adjust the implementation by adding the symmetric parameter. I did not investigate if there are other places with the documentation error. ;) Many thanks!! 



Discussion:
Good spot! I thik the default for MAPE (non-symmetric) was changed a while ago, which affected the `score` function.

In order to not break functionality downstream, it's better to change the docstring than the code logic.

Would you like to make a PR to change the docstring? Would be much appreciated!
Good spot! I thik the default for MAPE (non-symmetric) was changed a while ago, which affected the `score` function.

In order to not break functionality downstream, it's better to change the docstring than the code logic.

Would you like to make a PR to change the docstring? Would be much appreciated!

Resolving patch:
diff --git a/sktime/forecasting/base/_base.py b/sktime/forecasting/base/_base.py
--- a/sktime/forecasting/base/_base.py
+++ b/sktime/forecasting/base/_base.py
@@ -1232,7 +1232,7 @@ def predict_residuals(self, y=None, X=None):
         return y_res
 
     def score(self, y, X=None, fh=None):
-        """Scores forecast against ground truth, using MAPE.
+        """Scores forecast against ground truth, using MAPE (non-symmetric).
 
         Parameters
         ----------
@@ -1252,7 +1252,7 @@ def score(self, y, X=None, fh=None):
         Returns
         -------
         score : float
-            sMAPE loss of self.predict(fh, X) with respect to y_test.
+            MAPE loss of self.predict(fh, X) with respect to y_test.
 
         See Also
         --------
@@ -1260,12 +1260,12 @@ def score(self, y, X=None, fh=None):
         """
         # no input checks needed here, they will be performed
         # in predict and loss function
-        # symmetric=True is default for mean_absolute_percentage_error
         from sktime.performance_metrics.forecasting import (
             mean_absolute_percentage_error,
         )
 
-        return mean_absolute_percentage_error(y, self.predict(fh, X))
+        # specify non-symmetric explicitly as it changed in the past
+        return mean_absolute_percentage_error(y, self.predict(fh, X), symmetric=False)
 
     def get_fitted_params(self, deep=True):
         """Get fitted parameters.


Test patch: