Repository: pytorch/ignite

Issue:
[Feature Request] More general metrics
I find the metrics to be a bit limited as one might want to pass additional options (even tensors) to the loss.
For instance in recurrent models with different sequence lengths, one would use a mask to avoid counting errors on padded time steps.
The mask is necessary in the loss to know which outputs to use in the final averaging/ loss.

Discussion:

Patch:
diff --git a/ignite/metrics/loss.py b/ignite/metrics/loss.py
index 0e9bf9e0..a8ddac9f 100644
--- a/ignite/metrics/loss.py
+++ b/ignite/metrics/loss.py
@@ -8,9 +8,21 @@ class Loss(Metric):
     """
     Calculates the average loss according to the passed loss_fn.
 
-    - `loss_fn` must return the average loss over all observations in the batch.
-    - `update` must receive output of the form `(y_pred, y)`.
+    Args:
+        loss_fn (callable): a callable taking a prediction tensor, a target
+            tensor, optionally other arguments, and returns the average loss
+            over all observations in the batch.
+        output_transform (callable): a callable that is used to transform the
+            :class:`ignite.engine.Engine`'s `process_function`'s output into the
+            form expected by the metric.
+            This can be useful if, for example, you have a multi-output model and
+            you want to compute the metric with respect to one of the outputs.
+            The output is is expected to be a tuple (prediction, target) or
+            (prediction, target, kwargs) where kwargs is a dictionary of extra
+            keywords arguments.
+
     """
+
     def __init__(self, loss_fn, output_transform=lambda x: x):
         super(Loss, self).__init__(output_transform)
         self._loss_fn = loss_fn
@@ -20,8 +32,12 @@ class Loss(Metric):
         self._num_examples = 0
 
     def update(self, output):
-        y_pred, y = output
-        average_loss = self._loss_fn(y_pred, y)
+        if len(output) == 2:
+            y_pred, y = output
+            kwargs = {}
+        else:
+            y_pred, y, kwargs = output
+        average_loss = self._loss_fn(y_pred, y, **kwargs)
         assert len(average_loss.shape) == 0, '`loss_fn` did not return the average loss'
         self._sum += average_loss.item() * y.shape[0]
         self._num_examples += y.shape[0]


Test patch:
diff --git a/tests/ignite/metrics/test_loss.py b/tests/ignite/metrics/test_loss.py
index cc494d75..b5e7c59b 100644
--- a/tests/ignite/metrics/test_loss.py
+++ b/tests/ignite/metrics/test_loss.py
@@ -50,6 +50,15 @@ def test_non_averaging_loss():
         loss.update((y_pred, y))
 
 
+def test_kwargs_loss():
+    loss = Loss(nll_loss)
+
+    y_pred = torch.Tensor([[0.1, 0.4, 0.5], [0.1, 0.7, 0.2]]).log()
+    y = torch.LongTensor([2, 2])
+    loss.update((y_pred, y, {"weight": torch.Tensor([0, 0, 0])}))
+    assert_almost_equal(loss.compute(), 0)
+
+
 def test_reset():
     loss = Loss(nll_loss)