Repository: pytorch/ignite

Issue:
More metrics for regression tasks and forecasting
Following the discussion in #305 
We propose to implement the following list of  metrics

D1 = magnitude of error
- Mean Error
- Mean Normalized Bias
- Fractional Bias

D2 = absolute error
- Median Absolute Error
- Mean Absolute Relative Error / Mean Absolute Percentage Error
- Median Absolute Percentage Error
- Median Relative Absolute Error
- Geometric Mean Absolute Error
- Geometric Mean Relative Absolute Error
- Wave Hedges Distance
- Fractional absolute error
- Canberra Metric

based on the paper: 
https://arxiv.org/pdf/1809.03006.pdf
See appendix 2, page 30.

What do you think ?

Implementation to put into [contrib](https://github.com/pytorch/ignite/tree/master/ignite/contrib/metrics)

Discussion:
anmolsjoshi: cc @zasdfgbnm 
anmolsjoshi: cc @vfdev-5 

I have completed implementations of the metrics that require the history of training, all the tests, and I added tests for _base.py for code coverage.

Let me know if you have any comments.
vfdev-5: @anmolsjoshi thank you for working on this PR !
Could you please search how `np.median` is implemented and can we insprire of this in our code ?
anmolsjoshi: @vfdev-5 I looked into [numpy median](https://github.com/numpy/numpy/blob/v1.13.0/numpy/lib/function_base.py#L4113-L4117) implementation. 

NumPy implementation is closer to what is the known definition of median, for odd number of elements - it's the middle element of the sorted array, for even number of elements - its the average of the two middle elements of the sorted array. 

PyTorch does not intend to change there implementation, see [here](https://github.com/pytorch/pytorch/issues/1837#issuecomment-314992655).

I ran some code locally, here are the results.

```python
import numpy as np
import torch

a = np.random.rand(4)
a.sort()
print (a)
# [0.15983365 0.20260213 0.40420191 0.43118176]
print (np.median(a))
# 0.3034020154974049
print (torch.from_numpy(a).median())
# tensor(0.2026, dtype=torch.float64)

a = np.random.rand(5)
a.sort()
print (a)
# [0.14675148 0.18874507 0.64389564 0.72931892 0.75430595]
print (np.median(a))
# 0.6438956443116628
print (torch.from_numpy(a).median())
# tensor(0.6439, dtype=torch.float64)
```

Instead of calculating torch.median, we can use the code below:

```python
def median(tensor):
    tensor = tensor.sort()[0]
    length = tensor.shape[0]

    if length % 2 == 0:
        szh = length // 2
        kth = [szh - 1, szh]
    else:
        kth = [(length - 1) // 2]
    return tensor[kth].mean()

a = np.random.rand(4)
print (median(torch.from_numpy(a)).item())
# 0.6643668870940789
print (np.median(a))
# 0.6643668870940789

a = np.random.rand(5)

print (median(torch.from_numpy(a)).item())
# 0.42556139026154693
print (np.median(a))
# 0.42556139026154693
```

If we want to be more in line with PyTorch, we can choose to not incorporate this, but if we want to be closer to the definition of median it might be useful to implement the above code. Let me know what you think. 
vfdev-5: @anmolsjoshi thanks for digging that ! So both actually works (evidently on the whole data history). Maybe we would like also to know whether there exists a running median implementation ...
However if we should store the complete historym yes we could use such implementation you provided.
anmolsjoshi: @vfdev-5 thanks! Looking into a 'running median' implementation, I only methods that retained the history of the entire stream of data. let me know if you find something different! I'll wait on what you decide. 

As for GMRAE, we can try a running average for `y`, this will prevent us from keeping the entire history of the training data. 
anmolsjoshi: cc @vfdev-5 
vfdev-5: > As for GMRAE, we can try a running average for y, this will prevent us from keeping the entire history of the training data.

yes, I agree, we can deal with it as for an average mean.
vfdev-5: @anmolsjoshi please review the updates.
anmolsjoshi: > It remains to reimplement GMRAE.

@vfdev-5 does that mean this PR is complete after GMRAE is converted to running mean method?

vfdev-5: @anmolsjoshi I would say yes, but we need to review.
anmolsjoshi: @vfdev-5 I have updated the GMRAE implementation and tests.

The metric now keeps a running total of y and number of examples, thus for each batch a mean is calculated and a GMRAE is calculated and summed to a running total. At compute, the total GMRAE is divided by the number of examples, and it's exponential is calculated.

please review
anmolsjoshi: @vfdev-5 changes to docs and tests LGTM!
vfdev-5: @jasonkriss please review and merge if OK

Patch:
diff --git a/docs/source/contrib/metrics.rst b/docs/source/contrib/metrics.rst
index f92093a7..74396e6c 100644
--- a/docs/source/contrib/metrics.rst
+++ b/docs/source/contrib/metrics.rst
@@ -19,7 +19,50 @@ metrics useful for regression tasks. Definitions of metrics are based on `Botchk
 .. _`Botchkarev 2018`:
         https://arxiv.org/ftp/arxiv/papers/1809/1809.03006.pdf
 
+Complete list of metrics:
 
-.. automodule:: ignite.contrib.metrics.regression
-   :members:
-   :imported-members:
+    - :class:`~ignite.contrib.metrics.regression.CanberraMetric`
+    - :class:`~ignite.contrib.metrics.regression.FractionalAbsoluteError`
+    - :class:`~ignite.contrib.metrics.regression.FractionalBias`
+    - :class:`~ignite.contrib.metrics.regression.GeometricMeanAbsoluteError`
+    - :class:`~ignite.contrib.metrics.regression.GeometricMeanRelativeAbsoluteError`
+    - :class:`~ignite.contrib.metrics.regression.ManhattanDistance`
+    - :class:`~ignite.contrib.metrics.regression.MaximumAbsoluteError`
+    - :class:`~ignite.contrib.metrics.regression.MeanAbsoluteRelativeError`
+    - :class:`~ignite.contrib.metrics.regression.MeanError`
+    - :class:`~ignite.contrib.metrics.regression.MeanNormalizedBias`
+    - :class:`~ignite.contrib.metrics.regression.MedianAbsoluteError`
+    - :class:`~ignite.contrib.metrics.regression.MedianAbsolutePercentageError`
+    - :class:`~ignite.contrib.metrics.regression.MedianRelativeAbsoluteError`
+    - :class:`~ignite.contrib.metrics.regression.WaveHedgesDistance`
+
+
+.. currentmodule:: ignite.contrib.metrics.regression
+
+.. autoclass:: CanberraMetric
+
+.. autoclass:: FractionalAbsoluteError
+
+.. autoclass:: FractionalBias
+
+.. autoclass:: GeometricMeanAbsoluteError
+
+.. autoclass:: GeometricMeanRelativeAbsoluteError
+
+.. autoclass:: ManhattanDistance
+
+.. autoclass:: MaximumAbsoluteError
+
+.. autoclass:: MeanAbsoluteRelativeError
+
+.. autoclass:: MeanError
+
+.. autoclass:: MeanNormalizedBias
+
+.. autoclass:: MedianAbsoluteError
+
+.. autoclass:: MedianAbsolutePercentageError
+
+.. autoclass:: MedianRelativeAbsoluteError
+
+.. autoclass:: WaveHedgesDistance
diff --git a/ignite/contrib/metrics/regression/__init__.py b/ignite/contrib/metrics/regression/__init__.py
index 1843b096..d486f7ca 100644
--- a/ignite/contrib/metrics/regression/__init__.py
+++ b/ignite/contrib/metrics/regression/__init__.py
@@ -8,3 +8,7 @@ from ignite.contrib.metrics.regression.canberra_metric import CanberraMetric
 from ignite.contrib.metrics.regression.fractional_absolute_error import FractionalAbsoluteError
 from ignite.contrib.metrics.regression.wave_hedges_distance import WaveHedgesDistance
 from ignite.contrib.metrics.regression.geometric_mean_absolute_error import GeometricMeanAbsoluteError
+from ignite.contrib.metrics.regression.median_absolute_error import MedianAbsoluteError
+from ignite.contrib.metrics.regression.median_relative_absolute_error import MedianRelativeAbsoluteError
+from ignite.contrib.metrics.regression.median_absolute_percentage_error import MedianAbsolutePercentageError
+from ignite.contrib.metrics.regression.geometric_mean_relative_absolute_error import GeometricMeanRelativeAbsoluteError
diff --git a/ignite/contrib/metrics/regression/_base.py b/ignite/contrib/metrics/regression/_base.py
index 6686c87e..aa178e31 100644
--- a/ignite/contrib/metrics/regression/_base.py
+++ b/ignite/contrib/metrics/regression/_base.py
@@ -1,12 +1,16 @@
 from abc import abstractmethod
 
-from ignite.metrics import Metric
+import warnings
+
+import torch
+
+from ignite.metrics import Metric, EpochMetric
 
 
 class _BaseRegression(Metric):
     # Base class for all regression metrics
     # `update` method check the shapes and call internal overloaded
-    # method `_update`
+    # method `_update`.
 
     def update(self, output):
         y_pred, y = output
@@ -33,3 +37,32 @@ class _BaseRegression(Metric):
     @abstractmethod
     def _update(self, output):
         pass
+
+
+class _BaseRegressionEpoch(_BaseRegression, EpochMetric):
+    # Base class for all median-based regression metrics
+    # `update` method check the shapes and call internal overloaded method `_update`.
+    # Class internally stores complete history of predictions and targets of type float32.
+
+    def __init__(self, compute_fn, output_transform=lambda x: x):
+        EpochMetric.__init__(self, compute_fn=compute_fn, output_transform=output_transform)
+
+    def reset(self):
+        self._predictions = torch.tensor([], dtype=torch.float32)
+        self._targets = torch.tensor([], dtype=torch.float32)
+
+    def _update(self, output):
+        y_pred, y = output
+        y_pred = y_pred.type_as(self._predictions)
+        y = y.type_as(self._targets)
+
+        self._predictions = torch.cat([self._predictions, y_pred], dim=0)
+        self._targets = torch.cat([self._targets, y], dim=0)
+
+        # Check once the signature and execution of compute_fn
+        if self._predictions.shape == y_pred.shape:
+            try:
+                self.compute_fn(self._predictions, self._targets)
+            except Exception as e:
+                warnings.warn("Probably, there can be a problem with `compute_fn`:\n {}".format(e),
+                              RuntimeWarning)
diff --git a/ignite/contrib/metrics/regression/geometric_mean_relative_absolute_error.py b/ignite/contrib/metrics/regression/geometric_mean_relative_absolute_error.py
new file mode 100644
index 00000000..93940dd2
--- /dev/null
+++ b/ignite/contrib/metrics/regression/geometric_mean_relative_absolute_error.py
@@ -0,0 +1,43 @@
+from __future__ import division
+
+import torch
+
+from ignite.contrib.metrics.regression._base import _BaseRegression
+
+
+class GeometricMeanRelativeAbsoluteError(_BaseRegression):
+    r"""
+    Calculates the Geometric Mean Relative Absolute Error:
+
+    :math:`\text{GMRAE} = \exp(\frac{1}{n}\sum_{j=1}^n \ln\frac{|A_j - P_j|}{|A_j - \bar{A}|})`
+
+    where :math:`A_j` is the ground truth and :math:`P_j` is the predicted value.
+
+    More details can be found in `Botchkarev 2018`__.
+
+    - `update` must receive output of the form `(y_pred, y)`.
+    - `y` and `y_pred` must be of same shape `(N, )` or `(N, 1)`.
+
+
+    __ https://arxiv.org/abs/1809.03006
+
+    """
+    def reset(self):
+        self._sum_y = 0.0
+        self._num_examples = 0
+        self._sum_of_errors = 0.0
+
+    def _update(self, output):
+        y_pred, y = output
+        self._sum_y += y.sum()
+        self._num_examples += y.shape[0]
+        y_mean = self._sum_y / self._num_examples
+        numerator = torch.abs(y.view_as(y_pred) - y_pred)
+        denominator = torch.abs(y.view_as(y_pred) - y_mean)
+        self._sum_of_errors += torch.log(numerator / denominator).sum()
+
+    def compute(self):
+        if self._num_examples == 0:
+            raise NotComputableError('GeometricMeanRelativeAbsoluteError must have at least '
+                                     'one example before it can be computed.')
+        return torch.exp(torch.mean(self._sum_of_errors / self._num_examples)).item()
diff --git a/ignite/contrib/metrics/regression/median_absolute_error.py b/ignite/contrib/metrics/regression/median_absolute_error.py
new file mode 100644
index 00000000..f35f0b3d
--- /dev/null
+++ b/ignite/contrib/metrics/regression/median_absolute_error.py
@@ -0,0 +1,34 @@
+import torch
+
+from ignite.contrib.metrics.regression._base import _BaseRegressionEpoch
+
+
+def median_absolute_error_compute_fn(y_pred, y):
+    e = torch.abs(y.view_as(y_pred) - y_pred)
+    return torch.median(e).item()
+
+
+class MedianAbsoluteError(_BaseRegressionEpoch):
+    r"""
+    Calculates the Median Absolute Error:
+
+    :math:`\text{MdAE} = \text{MD}_{j=1,n} \left( |A_j - P_j| \right)`,
+
+    where :math:`A_j` is the ground truth and :math:`P_j` is the predicted value.
+
+    More details can be found in `Botchkarev 2018`__.
+
+    - `update` must receive output of the form `(y_pred, y)`.
+    - `y` and `y_pred` must be of same shape `(N, )` or `(N, 1)` and of type `float32`.
+
+    .. warning::
+
+        Current implementation stores all input data (output and target) in as tensors before computing a metric.
+        This can potentially lead to a memory error if the input data is larger than available RAM.
+
+
+    __ https://arxiv.org/abs/1809.03006
+
+    """
+    def __init__(self, output_transform=lambda x: x):
+        super(MedianAbsoluteError, self).__init__(median_absolute_error_compute_fn, output_transform)
diff --git a/ignite/contrib/metrics/regression/median_absolute_percentage_error.py b/ignite/contrib/metrics/regression/median_absolute_percentage_error.py
new file mode 100644
index 00000000..d3b9741c
--- /dev/null
+++ b/ignite/contrib/metrics/regression/median_absolute_percentage_error.py
@@ -0,0 +1,37 @@
+from __future__ import division
+
+import torch
+
+from ignite.contrib.metrics.regression._base import _BaseRegressionEpoch
+
+
+def median_absolute_percentage_error_compute_fn(y_pred, y):
+    e = torch.abs(y.view_as(y_pred) - y_pred) / torch.abs(y.view_as(y_pred))
+    return 100.0 * torch.median(e).item()
+
+
+class MedianAbsolutePercentageError(_BaseRegressionEpoch):
+    r"""
+    Calculates the Median Absolute Percentage Error:
+
+    :math:`\text{MdAPE} = 100 \cdot \text{MD}_{j=1,n} \left( \frac{|A_j - P_j|}{|A_j|} \right)`,
+
+    where :math:`A_j` is the ground truth and :math:`P_j` is the predicted value.
+
+    More details can be found in `Botchkarev 2018`__.
+
+    - `update` must receive output of the form `(y_pred, y)`.
+    - `y` and `y_pred` must be of same shape `(N, )` or `(N, 1)` and of type `float32`.
+
+    .. warning::
+
+        Current implementation stores all input data (output and target) in as tensors before computing a metric.
+        This can potentially lead to a memory error if the input data is larger than available RAM.
+
+
+    __ https://arxiv.org/abs/1809.03006
+
+    """
+    def __init__(self, output_transform=lambda x: x):
+        super(MedianAbsolutePercentageError, self).__init__(median_absolute_percentage_error_compute_fn,
+                                                            output_transform)
diff --git a/ignite/contrib/metrics/regression/median_relative_absolute_error.py b/ignite/contrib/metrics/regression/median_relative_absolute_error.py
new file mode 100644
index 00000000..32a18d53
--- /dev/null
+++ b/ignite/contrib/metrics/regression/median_relative_absolute_error.py
@@ -0,0 +1,37 @@
+from __future__ import division
+
+import torch
+
+from ignite.contrib.metrics.regression._base import _BaseRegressionEpoch
+
+
+def median_relative_absolute_error_compute_fn(y_pred, y):
+    e = torch.abs(y.view_as(y_pred) - y_pred) / torch.abs(y.view_as(y_pred) - torch.mean(y))
+    return torch.median(e).item()
+
+
+class MedianRelativeAbsoluteError(_BaseRegressionEpoch):
+    r"""
+    Calculates the Median Relative Absolute Error:
+
+    :math:`\text{MdRAE} = \text{MD}_{j=1,n} \left( \frac{|A_j - P_j|}{|A_j - \bar{A}|} \right)`,
+
+    where :math:`A_j` is the ground truth and :math:`P_j` is the predicted value.
+
+    More details can be found in `Botchkarev 2018`__.
+
+    - `update` must receive output of the form `(y_pred, y)`.
+    - `y` and `y_pred` must be of same shape `(N, )` or `(N, 1)` and of type `float32`.
+
+    .. warning::
+
+        Current implementation stores all input data (output and target) in as tensors before computing a metric.
+        This can potentially lead to a memory error if the input data is larger than available RAM.
+
+
+    __ https://arxiv.org/abs/1809.03006
+
+    """
+    def __init__(self, output_transform=lambda x: x):
+        super(MedianRelativeAbsoluteError, self).__init__(median_relative_absolute_error_compute_fn,
+                                                          output_transform)


Test patch:
diff --git a/tests/ignite/contrib/metrics/regression/test__base.py b/tests/ignite/contrib/metrics/regression/test__base.py
new file mode 100644
index 00000000..5a65fbca
--- /dev/null
+++ b/tests/ignite/contrib/metrics/regression/test__base.py
@@ -0,0 +1,103 @@
+import torch
+import numpy as np
+import pytest
+from ignite.contrib.metrics.regression._base import _BaseRegression, _BaseRegressionEpoch
+
+
+def test_base_regression_shapes():
+
+    class L1(_BaseRegression):
+        def reset(self):
+            self._sum_of_errors = 0.0
+
+        def _update(self, output):
+            y_pred, y = output
+            errors = torch.abs(y.view_as(y_pred) - y_pred)
+            self._sum_of_errors += torch.sum(errors).item()
+
+        def compute(self):
+            return self._sum_of_errors
+
+    m = L1()
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4, 1)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1),
+                  torch.rand(4, 1, 2)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4,)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4,),
+                  torch.rand(4, 1, 2)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 3),
+                  torch.rand(4, 1)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1),
+                  torch.rand(4, 3)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 7),
+                  torch.rand(4,)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4,),
+                  torch.rand(4, 7)))
+
+
+def test_base_regression_epoch_shapes():
+
+    def compute_fn(y_pred, y):
+        return 0.0
+
+    class ZeroEpoch(_BaseRegressionEpoch):
+        def __init__(self, output_transform=lambda x: x):
+            super(ZeroEpoch, self).__init__(compute_fn, output_transform)
+
+    m = ZeroEpoch()
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4, 1)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1),
+                  torch.rand(4, 1, 2)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4,)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4,),
+                  torch.rand(4, 1, 2)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 3),
+                  torch.rand(4, 1)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1),
+                  torch.rand(4, 3)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 7),
+                  torch.rand(4,)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4,),
+                  torch.rand(4, 7)))
+
+
+def test_base_regression_compute_fn():
+    # Wrong compute function
+    with pytest.raises(TypeError):
+        _BaseRegressionEpoch(12345)
diff --git a/tests/ignite/contrib/metrics/regression/test_geometric_mean_relative_absolute_error.py b/tests/ignite/contrib/metrics/regression/test_geometric_mean_relative_absolute_error.py
new file mode 100644
index 00000000..032cfd64
--- /dev/null
+++ b/tests/ignite/contrib/metrics/regression/test_geometric_mean_relative_absolute_error.py
@@ -0,0 +1,119 @@
+import torch
+import numpy as np
+import pytest
+from ignite.engine import Engine
+from ignite.contrib.metrics.regression import GeometricMeanRelativeAbsoluteError
+
+
+def test_wrong_input_shapes():
+    m = GeometricMeanRelativeAbsoluteError()
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4, 1)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1),
+                  torch.rand(4, 1, 2)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4,)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4,),
+                  torch.rand(4, 1, 2)))
+
+
+def test_geometric_mean_relative_absolute_error():
+    size = 51
+    np_y_pred = np.random.rand(size,)
+    np_y = np.random.rand(size,)
+    np_gmrae = np.exp(np.log(np.abs(np_y - np_y_pred) / np.abs(np_y - np_y.mean())).mean())
+
+    m = GeometricMeanRelativeAbsoluteError()
+    y_pred = torch.from_numpy(np_y_pred)
+    y = torch.from_numpy(np_y)
+
+    m.reset()
+    m.update((y_pred, y))
+
+    assert np_gmrae == pytest.approx(m.compute())
+
+
+def test_geometric_mean_relative_absolute_error_2():
+
+    np.random.seed(1)
+    size = 105
+    np_y_pred = np.random.rand(size, 1)
+    np_y = np.random.rand(size, 1)
+    np.random.shuffle(np_y)
+
+    np_y_sum = 0
+    num_examples = 0
+    num_sum_of_errors = 0
+    np_gmrae = 0
+
+    m = GeometricMeanRelativeAbsoluteError()
+    y_pred = torch.from_numpy(np_y_pred)
+    y = torch.from_numpy(np_y)
+
+    m.reset()
+    n_iters = 15
+    batch_size = size // n_iters
+    for i in range(n_iters + 1):
+        idx = i * batch_size
+        np_y_i = np_y[idx: idx + batch_size]
+        np_y_pred_i = np_y_pred[idx: idx + batch_size]
+
+        np_y_sum += np_y_i.sum()
+        num_examples += np_y_i.shape[0]
+        np_mean = np_y_sum / num_examples
+
+        np_gmrae += np.log(np.abs(np_y_i - np_y_pred_i) / np.abs(np_y_i - np_mean)).sum()
+        m.update((y_pred[idx: idx + batch_size], y[idx: idx + batch_size]))
+
+    assert np.exp(np_gmrae / num_examples) == pytest.approx(m.compute())
+
+
+def test_integration_geometric_mean_relative_absolute_error_with_output_transform():
+
+    np.random.seed(1)
+    size = 105
+    np_y_pred = np.random.rand(size, 1)
+    np_y = np.random.rand(size, 1)
+    np.random.shuffle(np_y)
+
+    np_y_sum = 0
+    num_examples = 0
+    num_sum_of_errors = 0
+    np_gmrae = 0
+
+    n_iters = 15
+    batch_size = size // n_iters
+    for i in range(n_iters + 1):
+        idx = i * batch_size
+        np_y_i = np_y[idx: idx + batch_size]
+        np_y_pred_i = np_y_pred[idx: idx + batch_size]
+
+        np_y_sum += np_y_i.sum()
+        num_examples += np_y_i.shape[0]
+        np_mean = np_y_sum / num_examples
+
+        np_gmrae += np.log(np.abs(np_y_i - np_y_pred_i) / np.abs(np_y_i - np_mean)).sum()
+
+    def update_fn(engine, batch):
+        idx = (engine.state.iteration - 1) * batch_size
+        y_true_batch = np_y[idx:idx + batch_size]
+        y_pred_batch = np_y_pred[idx:idx + batch_size]
+        return idx, torch.from_numpy(y_pred_batch), torch.from_numpy(y_true_batch)
+
+    engine = Engine(update_fn)
+
+    m = GeometricMeanRelativeAbsoluteError(output_transform=lambda x: (x[1], x[2]))
+    m.attach(engine, 'geometric_mean_relative_absolute_error')
+
+    data = list(range(size // batch_size))
+    gmrae = engine.run(data, max_epochs=1).metrics['geometric_mean_relative_absolute_error']
+
+    assert np.exp(np_gmrae / num_examples) == pytest.approx(m.compute())
diff --git a/tests/ignite/contrib/metrics/regression/test_median_absolute_error.py b/tests/ignite/contrib/metrics/regression/test_median_absolute_error.py
new file mode 100644
index 00000000..1712adf3
--- /dev/null
+++ b/tests/ignite/contrib/metrics/regression/test_median_absolute_error.py
@@ -0,0 +1,98 @@
+import torch
+import numpy as np
+import pytest
+from ignite.engine import Engine
+from ignite.contrib.metrics.regression import MedianAbsoluteError
+
+
+def test_wrong_input_shapes():
+    m = MedianAbsoluteError()
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4, 1)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1),
+                  torch.rand(4, 1, 2)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4,)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4,),
+                  torch.rand(4, 1, 2)))
+
+
+def test_median_absolute_error():
+
+    # See https://github.com/torch/torch7/pull/182
+    # For even number of elements, PyTorch returns middle element
+    # NumPy returns average of middle elements
+    # Size of dataset will be odd for these tests
+
+    size = 51
+    np_y_pred = np.random.rand(size,)
+    np_y = np.random.rand(size,)
+    np_median_absolute_error = np.median(np.abs(np_y - np_y_pred))
+
+    m = MedianAbsoluteError()
+    y_pred = torch.from_numpy(np_y_pred)
+    y = torch.from_numpy(np_y)
+
+    m.reset()
+    m.update((y_pred, y))
+
+    assert np_median_absolute_error == pytest.approx(m.compute())
+
+
+def test_median_absolute_error_2():
+
+    np.random.seed(1)
+    size = 105
+    np_y_pred = np.random.rand(size, 1)
+    np_y = np.random.rand(size, 1)
+    np.random.shuffle(np_y)
+    np_median_absolute_error = np.median(np.abs(np_y - np_y_pred))
+
+    m = MedianAbsoluteError()
+    y_pred = torch.from_numpy(np_y_pred)
+    y = torch.from_numpy(np_y)
+
+    m.reset()
+    batch_size = 16
+    n_iters = size // batch_size + 1
+    for i in range(n_iters):
+        idx = i * batch_size
+        m.update((y_pred[idx: idx + batch_size], y[idx: idx + batch_size]))
+
+    assert np_median_absolute_error == pytest.approx(m.compute())
+
+
+def test_integration_median_absolute_error_with_output_transform():
+
+    np.random.seed(1)
+    size = 105
+    np_y_pred = np.random.rand(size, 1)
+    np_y = np.random.rand(size, 1)
+    np.random.shuffle(np_y)
+    np_median_absolute_error = np.median(np.abs(np_y - np_y_pred))
+
+    batch_size = 15
+
+    def update_fn(engine, batch):
+        idx = (engine.state.iteration - 1) * batch_size
+        y_true_batch = np_y[idx:idx + batch_size]
+        y_pred_batch = np_y_pred[idx:idx + batch_size]
+        return idx, torch.from_numpy(y_pred_batch), torch.from_numpy(y_true_batch)
+
+    engine = Engine(update_fn)
+
+    m = MedianAbsoluteError(output_transform=lambda x: (x[1], x[2]))
+    m.attach(engine, 'median_absolute_error')
+
+    data = list(range(size // batch_size))
+    median_absolute_error = engine.run(data, max_epochs=1).metrics['median_absolute_error']
+
+    assert np_median_absolute_error == pytest.approx(median_absolute_error)
diff --git a/tests/ignite/contrib/metrics/regression/test_median_absolute_percentage_error.py b/tests/ignite/contrib/metrics/regression/test_median_absolute_percentage_error.py
new file mode 100644
index 00000000..ab40726d
--- /dev/null
+++ b/tests/ignite/contrib/metrics/regression/test_median_absolute_percentage_error.py
@@ -0,0 +1,98 @@
+import torch
+import numpy as np
+import pytest
+from ignite.engine import Engine
+from ignite.contrib.metrics.regression import MedianAbsolutePercentageError
+
+
+def test_wrong_input_shapes():
+    m = MedianAbsolutePercentageError()
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4, 1)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1),
+                  torch.rand(4, 1, 2)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4,)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4,),
+                  torch.rand(4, 1, 2)))
+
+
+def test_median_absolute_percentage_error():
+
+    # See https://github.com/torch/torch7/pull/182
+    # For even number of elements, PyTorch returns middle element
+    # NumPy returns average of middle elements
+    # Size of dataset will be odd for these tests
+
+    size = 51
+    np_y_pred = np.random.rand(size,)
+    np_y = np.random.rand(size,)
+    np_median_absolute_percentage_error = 100.0 * np.median(np.abs(np_y - np_y_pred) / np.abs(np_y))
+
+    m = MedianAbsolutePercentageError()
+    y_pred = torch.from_numpy(np_y_pred)
+    y = torch.from_numpy(np_y)
+
+    m.reset()
+    m.update((y_pred, y))
+
+    assert np_median_absolute_percentage_error == pytest.approx(m.compute())
+
+
+def test_median_absolute_percentage_error_2():
+
+    np.random.seed(1)
+    size = 105
+    np_y_pred = np.random.rand(size, 1)
+    np_y = np.random.rand(size, 1)
+    np.random.shuffle(np_y)
+    np_median_absolute_percentage_error = 100.0 * np.median(np.abs(np_y - np_y_pred) / np.abs(np_y))
+
+    m = MedianAbsolutePercentageError()
+    y_pred = torch.from_numpy(np_y_pred)
+    y = torch.from_numpy(np_y)
+
+    m.reset()
+    batch_size = 16
+    n_iters = size // batch_size + 1
+    for i in range(n_iters):
+        idx = i * batch_size
+        m.update((y_pred[idx: idx + batch_size], y[idx: idx + batch_size]))
+
+    assert np_median_absolute_percentage_error == pytest.approx(m.compute())
+
+
+def test_integration_median_absolute_percentage_error_with_output_transform():
+
+    np.random.seed(1)
+    size = 105
+    np_y_pred = np.random.rand(size, 1)
+    np_y = np.random.rand(size, 1)
+    np.random.shuffle(np_y)
+    np_median_absolute_percentage_error = 100.0 * np.median(np.abs(np_y - np_y_pred) / np.abs(np_y))
+
+    batch_size = 15
+
+    def update_fn(engine, batch):
+        idx = (engine.state.iteration - 1) * batch_size
+        y_true_batch = np_y[idx:idx + batch_size]
+        y_pred_batch = np_y_pred[idx:idx + batch_size]
+        return idx, torch.from_numpy(y_pred_batch), torch.from_numpy(y_true_batch)
+
+    engine = Engine(update_fn)
+
+    m = MedianAbsolutePercentageError(output_transform=lambda x: (x[1], x[2]))
+    m.attach(engine, 'median_absolute_percentage_error')
+
+    data = list(range(size // batch_size))
+    median_absolute_percentage_error = engine.run(data, max_epochs=1).metrics['median_absolute_percentage_error']
+
+    assert np_median_absolute_percentage_error == pytest.approx(median_absolute_percentage_error)
diff --git a/tests/ignite/contrib/metrics/regression/test_median_relative_absolute_error.py b/tests/ignite/contrib/metrics/regression/test_median_relative_absolute_error.py
new file mode 100644
index 00000000..05399934
--- /dev/null
+++ b/tests/ignite/contrib/metrics/regression/test_median_relative_absolute_error.py
@@ -0,0 +1,98 @@
+import torch
+import numpy as np
+import pytest
+from ignite.engine import Engine
+from ignite.contrib.metrics.regression import MedianRelativeAbsoluteError
+
+
+def test_wrong_input_shapes():
+    m = MedianRelativeAbsoluteError()
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4, 1)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1),
+                  torch.rand(4, 1, 2)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4, 1, 2),
+                  torch.rand(4,)))
+
+    with pytest.raises(ValueError):
+        m.update((torch.rand(4,),
+                  torch.rand(4, 1, 2)))
+
+
+def test_median_relative_absolute_error():
+
+    # See https://github.com/torch/torch7/pull/182
+    # For even number of elements, PyTorch returns middle element
+    # NumPy returns average of middle elements
+    # Size of dataset will be odd for these tests
+
+    size = 51
+    np_y_pred = np.random.rand(size,)
+    np_y = np.random.rand(size,)
+    np_median_absolute_relative_error = np.median(np.abs(np_y - np_y_pred) / np.abs(np_y - np_y.mean()))
+
+    m = MedianRelativeAbsoluteError()
+    y_pred = torch.from_numpy(np_y_pred)
+    y = torch.from_numpy(np_y)
+
+    m.reset()
+    m.update((y_pred, y))
+
+    assert np_median_absolute_relative_error == pytest.approx(m.compute())
+
+
+def test_median_relative_absolute_error_2():
+
+    np.random.seed(1)
+    size = 105
+    np_y_pred = np.random.rand(size, 1)
+    np_y = np.random.rand(size, 1)
+    np.random.shuffle(np_y)
+    np_median_absolute_relative_error = np.median(np.abs(np_y - np_y_pred) / np.abs(np_y - np_y.mean()))
+
+    m = MedianRelativeAbsoluteError()
+    y_pred = torch.from_numpy(np_y_pred)
+    y = torch.from_numpy(np_y)
+
+    m.reset()
+    batch_size = 16
+    n_iters = size // batch_size + 1
+    for i in range(n_iters + 1):
+        idx = i * batch_size
+        m.update((y_pred[idx: idx + batch_size], y[idx: idx + batch_size]))
+
+    assert np_median_absolute_relative_error == pytest.approx(m.compute())
+
+
+def test_integration_median_relative_absolute_error_with_output_transform():
+
+    np.random.seed(1)
+    size = 105
+    np_y_pred = np.random.rand(size, 1)
+    np_y = np.random.rand(size, 1)
+    np.random.shuffle(np_y)
+    np_median_absolute_relative_error = np.median(np.abs(np_y - np_y_pred) / np.abs(np_y - np_y.mean()))
+
+    batch_size = 15
+
+    def update_fn(engine, batch):
+        idx = (engine.state.iteration - 1) * batch_size
+        y_true_batch = np_y[idx:idx + batch_size]
+        y_pred_batch = np_y_pred[idx:idx + batch_size]
+        return idx, torch.from_numpy(y_pred_batch), torch.from_numpy(y_true_batch)
+
+    engine = Engine(update_fn)
+
+    m = MedianRelativeAbsoluteError(output_transform=lambda x: (x[1], x[2]))
+    m.attach(engine, 'median_absolute_relative_error')
+
+    data = list(range(size // batch_size))
+    median_absolute_relative_error = engine.run(data, max_epochs=1).metrics['median_absolute_relative_error']
+
+    assert np_median_absolute_relative_error == pytest.approx(median_absolute_relative_error)