Repository: Project-MONAI/MONAI

Issue:
Issues trying to mask DiceCELoss
**Goal**
I am trying to segment 2 objects from (3D) images with one network. However, sometimes the masks are missing for one of the object. In that case I want to essentially ignore the loss for that object. 

**Method**
In order to do this I modify the 'outputs' and 'masks' to be equal for these predictions, resulting in 0 loss over it, and add a correction factor for loss averaging, so. that loss will not be too optimistic from the perfect predictions. 

Possibly related. Before using this 'trick' for the loss value calculation I tried to work with 'reduction='mean'' loss function output instead. However, DiceCELoss would throw an error because of a mismatch in the shape of the Dice and CE parts when summing. 

**Expected behavior**

Note: Should be fully reproducible with the code below with the prints containing the necessary information.

Let's take a situation where all masks for 1 object are missing in a batch. Now, if I would split the batch at the channel dimension for both objects and calculate loss, the one with present masks should yield, say 'x', and the one with missing masks should yield 0, after applying above mentioned trick. If the trick should be working properly, one should also get a value of 'x' for loss calculated over the batch, without splitting the batch first.

I find that this is the case when I use DiceLoss. It is also the case when I use a basic binary cross entropy calculation (apart for a small offset to ensure no ln(0) happens). When using DiceCELoss this is not the case, however. I am not sure why this is. The DiceCELoss summing error is also included at the end.

```python
from monai.losses import DiceLoss, DiceCELoss
import torch
import numpy as np

def BinaryCrossEntropy(y_true, y_pred):
    y_true = np.array(y_true)
    y_pred = np.array(y_pred)
    term_0 = (1-y_true) * np.log(1-y_pred + 1e-7)
    term_1 = y_true * np.log(y_pred + 1e-7)
    return -np.mean(term_0+term_1)

def loss_split_and_combined(preds, masks, loss_function, trick_multiplier = None):
    object1_preds = preds[:,0:1,...]
    object2_preds = preds[:,1:2,...]
    
    object1_masks = masks[:,0:1,...]
    object2_masks = masks[:,1:2,...]

    if trick_multiplier is None:
        print("Plain: ")
        print("Object 1, Object 2, Combined")
        print(loss_function(object1_preds, object1_masks), loss_function(object2_preds, object2_masks), loss_function(preds, masks))
    else:
        print("After trick: ")
        print("Object 1, Object 2, Combined")
        print(loss_function(object1_preds, object1_masks), loss_function(object2_preds, object2_masks), loss_function(preds, masks) * trick_multiplier)

preds = torch.rand(4,2,160,160,16)
masks = torch.randint(high = 2, size = (4,2,160,160,16))
masks[:,1:2,...] = 0

# This is not the normal trick calculation, but good for an example if all object 2 masks in the batch are missing
object1_preds = preds[:,0:1,...]
object2_masks = masks[:,1:2,...]
trick_preds = torch.zeros(preds.shape)
trick_preds[:,0:1,...] = object1_preds
trick_preds[:,1:2,...] = object2_masks
trick_multiplier = trick_preds.numel()/object2_masks.numel()

print("\nDICE LOSS")
print("-" * 40)
loss_split_and_combined(preds, masks, DiceLoss(sigmoid = False))
loss_split_and_combined(trick_preds, masks, DiceLoss(sigmoid = False), trick_multiplier = trick_multiplier)
print("\nBINARY CROSS ENTROPY LOSS")
print("-" * 40)
loss_split_and_combined(preds, masks, BinaryCrossEntropy)
loss_split_and_combined(trick_preds, masks, BinaryCrossEntropy, trick_multiplier=trick_multiplier)
print("\nDICE CE LOSS")
print("-" * 40)
loss_split_and_combined(preds, masks, DiceCELoss(sigmoid = False))
loss_split_and_combined(trick_preds, masks, DiceCELoss(sigmoid = False), trick_multiplier = trick_multiplier)

DiceCELoss(sigmoid = True, reduction = 'none')(preds, masks)
```

**Additional context**
I am of course also open to other solutions to obtain my original goal.

Edit: Code didn't run as is after all, sorry. Added 
```
object1_preds = preds[:,0:1,...]
object2_masks = masks[:,1:2,...]
```
before creating trick_preds


Discussion:
Hi @demcbs , as for the error of `DiceCELoss(sigmoid = True, reduction = 'none')(preds, masks)`, it is reasonable since we cannot use reduction = "none" (as you can see in the [docstrings](https://github.com/Project-MONAI/MONAI/blob/4b66746764e200a9a933d2198a7aee13e53b1206/monai/losses/dice.py#L652)).
Hi @yiheng-wang-nv  Thanks for the reply. I guess that it is reasonable. However, when I used a batch size of two it did not throw an error with reduction = 'none', so if one can consider it an easy mistake to make it might be better to catch it as an input error. 
Hi @demcbs , thanks for your suggestion, and I will add the input type check.

Resolving patch:
diff --git a/monai/losses/dice.py b/monai/losses/dice.py
--- a/monai/losses/dice.py
+++ b/monai/losses/dice.py
@@ -21,7 +21,7 @@
 from monai.losses.focal_loss import FocalLoss
 from monai.losses.spatial_mask import MaskedLoss
 from monai.networks import one_hot
-from monai.utils import LossReduction, Weight, look_up_option
+from monai.utils import DiceCEReduction, LossReduction, Weight, look_up_option
 
 
 class DiceLoss(_Loss):
@@ -671,6 +671,7 @@ def __init__(
 
         """
         super().__init__()
+        reduction = look_up_option(reduction, DiceCEReduction).value
         self.dice = DiceLoss(
             include_background=include_background,
             to_onehot_y=to_onehot_y,
diff --git a/monai/utils/__init__.py b/monai/utils/__init__.py
--- a/monai/utils/__init__.py
+++ b/monai/utils/__init__.py
@@ -19,6 +19,7 @@
     BlendMode,
     ChannelMatching,
     CommonKeys,
+    DiceCEReduction,
     ForwardMode,
     GridSampleMode,
     GridSamplePadMode,
diff --git a/monai/utils/enums.py b/monai/utils/enums.py
--- a/monai/utils/enums.py
+++ b/monai/utils/enums.py
@@ -25,6 +25,7 @@
     "Average",
     "MetricReduction",
     "LossReduction",
+    "DiceCEReduction",
     "Weight",
     "ChannelMatching",
     "SkipMode",
@@ -166,6 +167,16 @@ class LossReduction(Enum):
     SUM = "sum"
 
 
+class DiceCEReduction(Enum):
+    """
+    See also:
+        - :py:class:`monai.losses.dice.DiceCELoss`
+    """
+
+    MEAN = "mean"
+    SUM = "sum"
+
+
 class Weight(Enum):
     """
     See also: :py:class:`monai.losses.dice.GeneralizedDiceLoss`


Test patch:
diff --git a/tests/test_dice_ce_loss.py b/tests/test_dice_ce_loss.py
--- a/tests/test_dice_ce_loss.py
+++ b/tests/test_dice_ce_loss.py
@@ -80,6 +80,11 @@ def test_ill_shape(self):
         with self.assertRaisesRegex(ValueError, ""):
             loss(torch.ones((1, 2, 3)), torch.ones((1, 1, 2, 3)))
 
+    def test_ill_reduction(self):
+        with self.assertRaisesRegex(ValueError, ""):
+            loss = DiceCELoss(reduction="none")
+            loss(torch.ones((1, 2, 3)), torch.ones((1, 1, 2, 3)))
+
     @SkipIfBeforePyTorchVersion((1, 7, 0))
     def test_script(self):
         loss = DiceCELoss()