Repository: sktime/sktime

Issue:
[ENH] naming of symmetricMAPE vs adjustedMAPE - clarifying docstring
_Originally posted by @mtorabirad in https://github.com/sktime/sktime/issues/1986#issuecomment-1470320909_

A minor point but shouldn't we call symmetricMAPE adjustedMAPE. After all, sMAPE is not really symmetric and the original developer of the Metric, Armstrong, never called it symmetricMAPE. https://www.youtube.com/watch?v=_uaJWg5eNpE&ab_channel=MLBoost


            


Discussion:
thanks for the comment, @mtorabirad!
(kindly note that we rarely look in closed PR for comments and suggestions for improvement, I found this by chance only)

I suppose you have a point - how would you call the metrics and/or clarify the docstring?
(PR with suggestions are also appreciated)

The main problem with renaming is, users might be (or, in fact, are quite certainly) importing the metrics in production systems, so it would need deprecation if we were to rename any python function or class that is public.

I think we can approach it by making clarifications in the docstring first.

I've also noticed some degree of incompleteness and inaccuracy in the metrics docstrings, here's the umbrella issue:
https://github.com/sktime/sktime/issues/4303
If you (or someone else) want to take care of MAPE metrics, that would be great! The issue contains an example.


It appears that the issue is just with the naming convention. I found a [blog](https://robjhyndman.com/hyndsight/smape/) that briefly discusses the literature on sMAPE. It seems that there is a disagreement on whether we should call it sMAPE or aMAPE, but to be fair both just refer to the same metrics. In my opinion, I agree with you that improving the clarifications in the docstrings that symmetricMAPE and adjustedMAPE refer to the same equation would be a way forward. I would be happy to improve the docstring for MAPE metrics (mean and median).
ok, that would be great!

If there is controversy or inconsistency on naming, you could do it as in the `CRPS` metric and just list the different names in use.

(btw, I notice that th `CRPS` is lacking references, that would also be a good thing to add if you are working on metrics docstrings anyway)

Resolving patch:
diff --git a/sktime/performance_metrics/forecasting/_classes.py b/sktime/performance_metrics/forecasting/_classes.py
--- a/sktime/performance_metrics/forecasting/_classes.py
+++ b/sktime/performance_metrics/forecasting/_classes.py
@@ -1677,30 +1677,57 @@ def get_test_params(cls, parameter_set="default"):
 
 
 class MeanAbsolutePercentageError(BaseForecastingErrorMetricFunc):
-    """Mean absolute percentage error (MAPE) or symmetric version.
+    r"""Mean absolute percentage error (MAPE) or symmetric version.
 
-    If `symmetric` is False then calculates MAPE and if `symmetric` is True
-    then calculates symmetric mean absolute percentage error (sMAPE). Both
-    MAPE and sMAPE output is non-negative floating point. The best value is 0.0.
+    For a univariate, non-hierarchical sample
+    of true values :math:`y_1, \dots, y_n` and
+    predicted values :math:`\widehat{y}_1, \dots, \widehat{y}_n`,
+    at time indices :math:`t_1, \dots, t_n`,
+    `evaluate` or call returns the Mean Absolute Percentage Error,
+    :math:`\frac{1}{n} \sum_{i=1}^n \left|\frac{y_i-\widehat{y}_i}{y_i} \right|`.
+    (the time indices are not used)
+
+    if `symmetric` is True then calculates
+    symmetric mean absolute percentage error (sMAPE), defined as
+    :math:`\frac{2}{n} \sum_{i=1}^n \frac{|y_i - \widehat{y}_i|}
+    {|y_i| + |\widehat{y}_i|}`.
+
+    Both MAPE and sMAPE output non-negative floating point which is in fractional units
+    rather than percentage. The best value is 0.0.
 
     sMAPE is measured in percentage error relative to the test data. Because it
     takes the absolute value rather than square the percentage forecast
     error, it penalizes large errors less than MSPE, RMSPE, MdSPE or RMdSPE.
 
-    There is no limit on how large the error can be, particulalrly when `y_true`
+    MAPE has no limit on how large the error can be, particulalrly when `y_true`
     values are close to zero. In such cases the function returns a large value
-    instead of `inf`.
+    instead of `inf`. While sMAPE is bounded at 2.
+
+    `multioutput` and `multilevel` control averaging across variables and
+    hierarchy indices, see below.
+
+    `evaluate_by_index` returns, at a time index :math:`t_i`,
+    the abolute percentage error at that time index,
+    :math:`\left| \frac{y_i - \widehat{y}_i}{y_i} \right|`,
+    or :math:`\frac{2|y_i - \widehat{y}_i|}{|y_i| + |\widehat{y}_i|}`,
+    the symmetric version, if `symmetric` is True, for all time indices
+    :math:`t_1, \dots, t_n` in the input.
 
     Parameters
     ----------
     symmetric : bool, default = False
         Whether to calculate the symmetric version of the percentage metric
-    multioutput : {'raw_values', 'uniform_average'}  or array-like of shape \
-            (n_outputs,), default='uniform_average'
+    multioutput : str or 1D array-like (n_outputs,), default='uniform_average'
+        if str, must be one of {'raw_values', 'uniform_average'}
         Defines how to aggregate metric for multivariate (multioutput) data.
         If array-like, values used as weights to average the errors.
         If 'raw_values', returns a full set of errors in case of multioutput input.
         If 'uniform_average', errors of all outputs are averaged with uniform weight.
+    multilevel : {'raw_values', 'uniform_average', 'uniform_average_time'}
+        Defines how to aggregate metric for hierarchical data (with levels).
+        If 'uniform_average' (default), errors are mean-averaged across levels.
+        If 'uniform_average_time', errors are mean-averaged across rows.
+        If 'raw_values', does not average errors across levels, hierarchy is retained.
 
     See Also
     --------
@@ -1716,8 +1743,7 @@ class MeanAbsolutePercentageError(BaseForecastingErrorMetricFunc):
     Examples
     --------
     >>> import numpy as np
-    >>> from sktime.performance_metrics.forecasting import \
-    MeanAbsolutePercentageError
+    >>> from sktime.performance_metrics.forecasting import MeanAbsolutePercentageError
     >>> y_true = np.array([3, -0.5, 2, 7, 2])
     >>> y_pred = np.array([2.5, 0.0, 2, 8, 1.25])
     >>> mape = MeanAbsolutePercentageError(symmetric=False)
@@ -1781,11 +1807,21 @@ def get_test_params(cls, parameter_set="default"):
 
 
 class MedianAbsolutePercentageError(BaseForecastingErrorMetricFunc):
-    """Median absolute percentage error (MdAPE) or symmetric version.
+    r"""Median absolute percentage error (MdAPE) or symmetric version.
+
+    For a univariate, non-hierarchical sample of true values :math:`y_1, \dots, y_n`
+    and predicted values :math:`\widehat{y}_1, \dots, \widehat{y}_n`,
+    at time indices :math:`t_1, \dots, t_n`,
+    `evaluate` or call returns the Median Absolute Percentage Error,
+    :math:`median(\left|\frac{y_i - \widehat{y}_i}{y_i} \right|)`.
+    (the time indices are not used)
+
+    if `symmetric` is True then calculates
+    symmetric Median Absolute Percentage Error (sMdAPE), defined as
+    :math:`median(\frac{2|y_i-\widehat{y}_i|}{|y_i|+|\widehat{y}_i|})`.
 
-    If `symmetric` is False then calculates MdAPE and if `symmetric` is True
-    then calculates symmetric median absolute percentage error (sMdAPE). Both
-    MdAPE and sMdAPE output is non-negative floating point. The best value is 0.0.
+    Both MdAPE and sMdAPE output non-negative floating point which is in fractional
+    units rather than percentage. The best value is 0.0.
 
     MdAPE and sMdAPE are measured in percentage error relative to the test data.
     Because it takes the absolute value rather than square the percentage forecast
@@ -1795,20 +1831,28 @@ class MedianAbsolutePercentageError(BaseForecastingErrorMetricFunc):
     makes this metric more robust to error outliers since the median tends
     to be a more robust measure of central tendency in the presence of outliers.
 
-    There is no limit on how large the error can be, particulalrly when `y_true`
+    MAPE has no limit on how large the error can be, particulalrly when `y_true`
     values are close to zero. In such cases the function returns a large value
-    instead of `inf`.
+    instead of `inf`. While sMAPE is bounded at 2.
+
+    `multioutput` and `multilevel` control averaging across variables and
+    hierarchy indices, see below.
 
     Parameters
     ----------
     symmetric : bool, default = False
         Whether to calculate the symmetric version of the percentage metric
-    multioutput : {'raw_values', 'uniform_average'}  or array-like of shape \
-            (n_outputs,), default='uniform_average'
+    multioutput : str or 1D array-like (n_outputs,), default='uniform_average'
+        if str, must be one of {'raw_values', 'uniform_average'}
         Defines how to aggregate metric for multivariate (multioutput) data.
         If array-like, values used as weights to average the errors.
         If 'raw_values', returns a full set of errors in case of multioutput input.
         If 'uniform_average', errors of all outputs are averaged with uniform weight.
+    multilevel : {'raw_values', 'uniform_average', 'uniform_average_time'}
+        Defines how to aggregate metric for hierarchical data (with levels).
+        If 'uniform_average' (default), errors are mean-averaged across levels.
+        If 'uniform_average_time', errors are mean-averaged across rows.
+        If 'raw_values', does not average errors across levels, hierarchy is retained.
 
     See Also
     --------
@@ -1824,8 +1868,7 @@ class MedianAbsolutePercentageError(BaseForecastingErrorMetricFunc):
     Examples
     --------
     >>> import numpy as np
-    >>> from sktime.performance_metrics.forecasting import \
-    MedianAbsolutePercentageError
+    >>> from sktime.performance_metrics.forecasting import MedianAbsolutePercentageError
     >>> y_true = np.array([3, -0.5, 2, 7, 2])
     >>> y_pred = np.array([2.5, 0.0, 2, 8, 1.25])
     >>> mdape = MedianAbsolutePercentageError(symmetric=False)
@@ -1852,7 +1895,7 @@ class MedianAbsolutePercentageError(BaseForecastingErrorMetricFunc):
     >>> smdape = MedianAbsolutePercentageError(multioutput=[0.3, 0.7], symmetric=True)
     >>> smdape(y_true, y_pred)
     0.5066666666666666
-    """
+    """  # noqa: E501
 
     func = median_absolute_percentage_error
 


Test patch: