Original: T HE HIDDEN LABEL - MARGINAL BIASES OF SEGMENTATION LOSSES
Revised: T HE HIDDEN LABEL - MARGINAL BIASES OF SEGMENTATION LOSSES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Most segmentation losses are arguably variants of the Cross-Entropy (CE) or Dice losses.
Revised: Most segmentation losses are arguably variants of the Cross-Entropy (CE) or Dice losses.

Original: In the abundant segmentation literature, there is no clear consensus as to which of these losses is a better choice, with varying performances for each across different benchmarks and applications.
Revised: In the abundant segmentation literature, there is no clear consensus as to which of these losses is a better choice, with varying performances for each across different benchmarks and applications.

Original: In this work, we develop a theoretical analysis that links these two types of losses, exposing their advantages and weaknesses.
Revised: In this work, we develop a theoretical analysis that links these two types of losses, exposing their advantages and weaknesses.

Original: First, we provide a constrained-optimization perspective showing that CE and Dice share a much deeper connection than previously thought: They both decompose into label-marginal penalties and closely related ground-truth matching penalties.
Revised: First, we provide a constrained-optimization perspective showing that CE and Dice share a much deeper connection than previously thought: They both decompose into label-marginal penalties and closely related ground-truth matching penalties.

Original: Then, we provide bound relationships and an informationtheoretic analysis, which uncover hidden label-marginal biases: Dice has an intrinsic bias towards speciﬁc extremely imbalanced solutions, whereas CE implicitly encourages the ground-truth region proportions.
Revised: Then, we provide bound relationships and an informationtheoretic analysis, which uncover hidden label-marginal biases: Dice has an intrinsic bias towards speciﬁc extremely imbalanced solutions, whereas CE implicitly encourages the ground-truth region proportions.

Original: Our theoretical results explain the wide experimental evidence in the medical-imaging literature, whereby Dice losses bring improvements for imbalanced segmentation.
Revised: Our theoretical results explain the wide experimental evidence in the medical-imaging literature, whereby Dice losses bring improvements for imbalanced segmentation.

Original: It also explains why
Revised: It also explains why

Original: CE dominates natural-image problems with diverse class proportions, in which case Dice might have difﬁculty adapting to different label-marginal distributions.
Revised: CE dominates natural-image problems with diverse class proportions, in which case Dice might have difﬁculty adapting to different label-marginal distributions.

Original: Based on our theoretical analysis, we propose a principled and simple solution, which enables to control explicitly the label-marginal bias.
Revised: Based on our theoretical analysis, we propose a principled and simple solution, which enables to control explicitly the label-marginal bias.

Original: Our loss integrates CE with explicit L 1 regularization, which encourages label marginals to match target class proportions, thereby mitigating class imbalance but without losing generality.
Revised: Our loss integrates CE with explicit L 1 regularization, which encourages label marginals to match target class proportions, thereby mitigating class imbalance but without losing generality.

Original: Comprehensive experiments and ablation studies over different losses and applications validate our theoretical analysis, as well as the effectiveness of our explicit label-marginal regularizers.
Revised: Comprehensive experiments and ablation studies over different losses and applications validate our theoretical analysis, as well as the effectiveness of our explicit label-marginal regularizers.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Semantic segmentation is one of the most investigated problems in computer vision, and has been impacting a breadth of applications, from natural-scene understanding (Cordts et al., 2016; Kirillov et al., 2019) to medical image analysis (Litjens et al., 2017; Dolz et al., 2018).
Revised: Semantic segmentation is one of the most investigated problems in computer vision, and has been impacting a breadth of applications, from natural-scene understanding (Cordts et al., 2016; Kirillov et al., 2019) to medical image analysis (Litjens et al., 2017; Dolz et al., 2018).

Original: In the recent years, deep learning methods have dominated the ﬁeld, as a result of the great capacity of Convolutional
Revised: In the recent years, deep learning methods have dominated the ﬁeld, as a result of the great capacity of Convolutional

Original: Neural Networks (CNN) (He et al., 2016) to automatically learn representations from large-scale data sets (Long et al., 2015; Ronneberger et al., 2015; Zhao et al., 2017; Chen et al., 2018; Yuan et al., 2020).
Revised: Neural Networks (CNN) (He et al., 2016) to automatically learn representations from large-scale data sets (Long et al., 2015; Ronneberger et al., 2015; Zhao et al., 2017; Chen et al., 2018; Yuan et al., 2020).

Original: Semantic segmentation is often stated as a pixel-wise classiﬁcation task, following the optimization of a loss function expressed with summations over the ground-truth regions, as in the standard Cross-Entropy (CE) loss.
Revised: Semantic segmentation is often stated as a pixel-wise classiﬁcation task, following the optimization of a loss function expressed with summations over the ground-truth regions, as in the standard Cross-Entropy (CE) loss.

Original: A relevant aspect of segmentation problems is class imbalance, i.e. , unequal proportions of the segmentation regions, which may cause large-region terms in the objective to completely dominate small-region ones.
Revised: A relevant aspect of segmentation problems is class imbalance, i.e. , unequal proportions of the segmentation regions, which may cause large-region terms in the objective to completely dominate small-region ones.

Original: A representative example is the popular
Revised: A representative example is the popular

Original: Cityscapes dataset (Cordts et al., 2016), where the average proportions of some classes, such as motorcycle or bicycle , are below 1% .
Revised: Cityscapes dataset (Cordts et al., 2016), where the average proportions of some classes, such as motorcycle or bicycle , are below 1% .

Original: In these scenarios, besides speciﬁcally designed CNN architectures or training schemes (Tao et al., 2020; Bao et al., 2021), the loss function to be minimized during learning plays a critical role, and has triggered a large body of research works in the last years (Lin et al., 2017;
Revised: In these scenarios, besides speciﬁcally designed CNN architectures or training schemes (Tao et al., 2020; Bao et al., 2021), the loss function to be minimized during learning plays a critical role, and has triggered a large body of research works in the last years (Lin et al., 2017;

Original: Wong et al., 2018; Kervadec et al., 2021b; Milletari et al., 2016; Sudre et al., 2017;
Revised: Wong et al., 2018; Kervadec et al., 2021b; Milletari et al., 2016; Sudre et al., 2017;

Original: Kervadec et al., 2021a).
Revised: Kervadec et al., 2021a).

Original: There exists a great diversity of loss functions for image segmentation, which can be categorized into two main families, and are arguably variants of CE, the Dice loss (Ma et al., 2021; Yeung et al., 2021), or combinations of both (Wong et al., 2018; Taghanaki et al., 2019).
Revised: There exists a great diversity of loss functions for image segmentation, which can be categorized into two main families, and are arguably variants of CE, the Dice loss (Ma et al., 2021; Yeung et al., 2021), or combinations of both (Wong et al., 2018; Taghanaki et al., 2019).

Original: The ﬁrst family is mo-
Revised: The ﬁrst family is mo-

Original: [Figure]
Revised: [Figure]

Original: m o t
Revised: m o t

Original: o r c
Revised: o r c

Original: y
Revised: y

Original: c l e b u s r i d e r tivated by distribution measures, i.e. , CE and its variants, and is directly adapted from classiﬁcation tasks.
Revised: c l e b u s r i d e r tivated by distribution measures, i.e. , CE and its variants, and is directly adapted from classiﬁcation tasks.

Original: To deal with class imbalance, various extensions of CE have been investigated, such as increasing the relative weights for minority classes (Ronneberger et al., 2015), or modifying the loss so as to account for performance indicators during training, as in the popular Focal loss (Lin et al., 2017) or TopK loss (Wu et al., 2016).
Revised: To deal with class imbalance, various extensions of CE have been investigated, such as increasing the relative weights for minority classes (Ronneberger et al., 2015), or modifying the loss so as to account for performance indicators during training, as in the popular Focal loss (Lin et al., 2017) or TopK loss (Wu et al., 2016).

Original: The second main family of losses is inspired by geometrical metrics.
Revised: The second main family of losses is inspired by geometrical metrics.

Original: In this category, the most popular losses are linear Dice (Milletari et al., 2016) and its extensions, such as the logarithmic (Wong et al., 2018) or generalized (Sudre et al., 2017)
Revised: In this category, the most popular losses are linear Dice (Milletari et al., 2016) and its extensions, such as the logarithmic (Wong et al., 2018) or generalized (Sudre et al., 2017)

Original: Dice loss.
Revised: Dice loss.

Original: Borrowing the idea of the weighted CE, the latter introduces class weights to increase the contributions of the minority classes.
Revised: Borrowing the idea of the weighted CE, the latter introduces class weights to increase the contributions of the minority classes.

Original: These loss functions are motivated by the geometric Dice coefﬁcient, which measures the overlap between the ground-truth and predicted segmentation regions.
Revised: These loss functions are motivated by the geometric Dice coefﬁcient, which measures the overlap between the ground-truth and predicted segmentation regions.

Original: In the literature, to our knowledge, there is no clear consensus as to which category of losses is better, with the performances of each varying across data sets and applications.
Revised: In the literature, to our knowledge, there is no clear consensus as to which category of losses is better, with the performances of each varying across data sets and applications.

Original: It has been empirically argued that the Dice loss and its variants are more appropriate for extreme class imbalance, and such empirical observations are the main motivation behind the wide use and popularity of Dice in medical-imaging applications (Milletari et al., 2016; Jha et al., 2020).
Revised: It has been empirically argued that the Dice loss and its variants are more appropriate for extreme class imbalance, and such empirical observations are the main motivation behind the wide use and popularity of Dice in medical-imaging applications (Milletari et al., 2016; Jha et al., 2020).

Original: CE, however, dominates most recent models in the context of natural images (Chen et al., 2018;
Revised: CE, however, dominates most recent models in the context of natural images (Chen et al., 2018;

Original: Zhao et al., 2017; 2018; Yuan et al., 2020).
Revised: Zhao et al., 2017; 2018; Yuan et al., 2020).

Original: Therefore, beyond experimental evidence, there is a need for a theoretical analysis that clariﬁes which segmentation loss to adopt for a given task, a decision that affects performance signiﬁcantly.
Revised: Therefore, beyond experimental evidence, there is a need for a theoretical analysis that clariﬁes which segmentation loss to adopt for a given task, a decision that affects performance signiﬁcantly.

Original: On the surface, these two categories of losses ( i.e. , geometry-based vs. distribution-based) seem unrelated.
Revised: On the surface, these two categories of losses ( i.e. , geometry-based vs. distribution-based) seem unrelated.

Original: Furthermore, an important body of work in the literature suggests that CE and Dice are complementary, which has motivated composite losses integrating both, e.g. , (Wong et al., 2018;
Revised: Furthermore, an important body of work in the literature suggests that CE and Dice are complementary, which has motivated composite losses integrating both, e.g. , (Wong et al., 2018;

Original: Taghanaki et al., 2019).
Revised: Taghanaki et al., 2019).

Original: Such composite losses perform very competitively in extremely imbalanced segmentation, as shown by (Wong et al., 2018; Taghanaki et al., 2019), among several other recent works (Isensee et al., 2018; Ma et al., 2021).
Revised: Such composite losses perform very competitively in extremely imbalanced segmentation, as shown by (Wong et al., 2018; Taghanaki et al., 2019), among several other recent works (Isensee et al., 2018; Ma et al., 2021).

Original: In this paper, we provide a constrained-optimization perspective showing that, in fact, CE and Dice share a much deeper connection than previously thought: They both decompose into label-marginal penalties and closely related ground-truth matching penalties.
Revised: In this paper, we provide a constrained-optimization perspective showing that, in fact, CE and Dice share a much deeper connection than previously thought: They both decompose into label-marginal penalties and closely related ground-truth matching penalties.

Original: Our theoretical analysis highlights encoded hidden label-marginal biases in Dice and CE, and shows that the main difference between the two types of losses lies essentially in those label-marginal biases: Dice has an intrinsic bias preferring very small regions, while CE implicitly encourages the right (ground-truth) region proportions.
Revised: Our theoretical analysis highlights encoded hidden label-marginal biases in Dice and CE, and shows that the main difference between the two types of losses lies essentially in those label-marginal biases: Dice has an intrinsic bias preferring very small regions, while CE implicitly encourages the right (ground-truth) region proportions.

Original: Our results explain the wide experimental evidence in the medical-imaging literature, whereby using or adding Dice losses brings improvements for imbalanced segmentation with extremely small regions.
Revised: Our results explain the wide experimental evidence in the medical-imaging literature, whereby using or adding Dice losses brings improvements for imbalanced segmentation with extremely small regions.

Original: It also explains why CE dominates natural-image problems with diverse class proportions, in which case Dice might have difﬁculty adapting to different label-marginal distributions (see Table
Revised: It also explains why CE dominates natural-image problems with diverse class proportions, in which case Dice might have difﬁculty adapting to different label-marginal distributions (see Table

Original: 3 and the examples depicted in Fig. 1).
Revised: 3 and the examples depicted in Fig. 1).

Original: Based on our theoretical analysis, we propose principled and simple loss functions, which enable to control explicitly the label-marginal bias.
Revised: Based on our theoretical analysis, we propose principled and simple loss functions, which enable to control explicitly the label-marginal bias.

Original: Our solution integrates the beneﬁts of both categories of losses, mitigating class imbalance but without losing generality, as shown in the examples in Fig.
Revised: Our solution integrates the beneﬁts of both categories of losses, mitigating class imbalance but without losing generality, as shown in the examples in Fig.

Original: Our contributions are summarized as follows:
Revised: Our contributions are summarized as follows:

Original: • Showing through an explicit bound relationship (Proposition 1) that the Dice loss has a hidden label-marginal bias towards speciﬁc extremely imbalanced solutions, preferring small structures, while losing the ﬂexibility to deal effectively with arbitrary class proportions.
Revised: • Showing through an explicit bound relationship (Proposition 1) that the Dice loss has a hidden label-marginal bias towards speciﬁc extremely imbalanced solutions, preferring small structures, while losing the ﬂexibility to deal effectively with arbitrary class proportions.

Original: • Providing an information-theoretic perspective of CE, via Monte-Carlo approximation of the entropy of the learned features (Proposition 2).
Revised: • Providing an information-theoretic perspective of CE, via Monte-Carlo approximation of the entropy of the learned features (Proposition 2).

Original: This highlights a hidden label-marginal bias of CE, which encourages the proportions of the predicted segmentation regions to match the ground-truth proportions.
Revised: This highlights a hidden label-marginal bias of CE, which encourages the proportions of the predicted segmentation regions to match the ground-truth proportions.

Original: • Introducing new loss functions to control label-marginal biases: Our losses integrate CE with explicit regularization terms based on L 1 or the KL divergence, which encourage label marginals to match target class proportions.
Revised: • Introducing new loss functions to control label-marginal biases: Our losses integrate CE with explicit regularization terms based on L 1 or the KL divergence, which encourage label marginals to match target class proportions.

Original: • Comprehensive experiments and ablation studies over different losses and applications, including natural and medical-imaging data, validate our theoretical analysis, as well as the effectiveness of our explicit label-marginal regularizers.
Revised: • Comprehensive experiments and ablation studies over different losses and applications, including natural and medical-imaging data, validate our theoretical analysis, as well as the effectiveness of our explicit label-marginal regularizers.

Original:  F ORMULATION
Revised:  F ORMULATION

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: Semantic segmentation is often stated as a pixel-wise classiﬁcation task, following the optimization of a loss function for training a deep network.
Revised: Semantic segmentation is often stated as a pixel-wise classiﬁcation task, following the optimization of a loss function for training a deep network.

Original: In Table
Revised: In Table

Original: [Table]
Revised: [Table]

Original: we present the notations, formulations and approximations used in our subsequent discussions.
Revised: we present the notations, formulations and approximations used in our subsequent discussions.

Original: Besides the basic notations of the task (such as networks predictions), we explicitly include the loss functions, label-marginal regularizers and information-theoretic quantities that will be discussed in the following sections.
Revised: Besides the basic notations of the task (such as networks predictions), we explicitly include the loss functions, label-marginal regularizers and information-theoretic quantities that will be discussed in the following sections.

Original: We note that, to facilitate the reading of our analysis, we write the CE and Dice losses in a non-standard way using summations over the ground-truth segmentation regions, rather than as functions of the labels.
Revised: We note that, to facilitate the reading of our analysis, we write the CE and Dice losses in a non-standard way using summations over the ground-truth segmentation regions, rather than as functions of the labels.

Original: Also, while we provide the CE loss for all segmentation regions, we give Dice for a single region.
Revised: Also, while we provide the CE loss for all segmentation regions, we give Dice for a single region.

Original: This is to accommodate two variants of the Dice loss in the literature: in the binary case, Dice is typically used for the foreground region only (Milletari et al., 2016); in the multi-region case, it is commonly used over all the regions (Wong et al., 2018).
Revised: This is to accommodate two variants of the Dice loss in the literature: in the binary case, Dice is typically used for the foreground region only (Milletari et al., 2016); in the multi-region case, it is commonly used over all the regions (Wong et al., 2018).

Original: Finally, to simplify notation, we give all the loss functions for a single training image, without summations over all training samples (as this does not lead to any ambiguity, neither does it alter the analysis hereafter).
Revised: Finally, to simplify notation, we give all the loss functions for a single training image, without summations over all training samples (as this does not lead to any ambiguity, neither does it alter the analysis hereafter).

Original: In the training iterations, we use the mean values across all the training samples via standard mini-batch optimization.
Revised: In the training iterations, we use the mean values across all the training samples via standard mini-batch optimization.

Original: 2. D EFINITION OF LABEL - MARGINAL BIASES AND PENALTY FUNCTIONS
Revised: 2. D EFINITION OF LABEL - MARGINAL BIASES AND PENALTY FUNCTIONS

Original: In the following, we analyse the label-marginal biases inherent to CE and Dice losses, and show that the main difference between the two types of losses lies essentially in those label-marginal biases.
Revised: In the following, we analyse the label-marginal biases inherent to CE and Dice losses, and show that the main difference between the two types of losses lies essentially in those label-marginal biases.

Original: To do so, we provide a constrained-optimization perspective of the losses: We deﬁne a label-marginal bias as a soft penalty function for the hard equality constraint p = t , where t is a given (ﬁxed) target distribution.
Revised: To do so, we provide a constrained-optimization perspective of the losses: We deﬁne a label-marginal bias as a soft penalty function for the hard equality constraint p = t , where t is a given (ﬁxed) target distribution.

Original: Such a penalty encourages the predicted label-marginal p to match a given target distribution t .
Revised: Such a penalty encourages the predicted label-marginal p to match a given target distribution t .

Original: In the general context of constrained optimization, penalty functions are widely used (Bertsekas, 1995).
Revised: In the general context of constrained optimization, penalty functions are widely used (Bertsekas, 1995).

Original: In general, penalty methods replace equality constraints of the form p = t by adding a term g ( p ) to the main objective being minimized.
Revised: In general, penalty methods replace equality constraints of the form p = t by adding a term g ( p ) to the main objective being minimized.

Original: Such a penalty function g increases when p deviates from target t .
Revised: Such a penalty function g increases when p deviates from target t .

Original: By deﬁnition, for the constraint p = t , with the domain of p being probability simplex ∆ K , a penalty g ( p ) is a continuous and differentiable function, which reaches its global minimum when the constraint is satisﬁed, i.e. , it veriﬁes: g ( t )
Revised: By deﬁnition, for the constraint p = t , with the domain of p being probability simplex ∆ K , a penalty g ( p ) is a continuous and differentiable function, which reaches its global minimum when the constraint is satisﬁed, i.e. , it veriﬁes: g ( t )

Original: ≤ g ( p ) ∀
Revised: ≤ g ( p ) ∀

Original: p ∈ ∆ K .
Revised: p ∈ ∆ K .

Original: 2. T HE LINK BETWEEN C ROSS E NTROPY AND D ICE
Revised: 2. T HE LINK BETWEEN C ROSS E NTROPY AND D ICE

Original: To ease the discussion in what follows, we will start by analyzing the link between CE and the logarithmic Dice, along with the label-marginal bias of the latter (Proposition 1).
Revised: To ease the discussion in what follows, we will start by analyzing the link between CE and the logarithmic Dice, along with the label-marginal bias of the latter (Proposition 1).

Original: Then, we discuss a bounding relationship between the different Dice variants.
Revised: Then, we discuss a bounding relationship between the different Dice variants.

Original: Finally, we will provide an informationtheoretic analysis, which highlights the hidden label-marginal bias of CE (Proposition 2).
Revised: Finally, we will provide an informationtheoretic analysis, which highlights the hidden label-marginal bias of CE (Proposition 2).

Original: Let us consider the logarithmic Dice loss in the multi-class case.
Revised: Let us consider the logarithmic Dice loss in the multi-class case.

Original: This loss decomposes (up to a constant) into two terms, a ground-truth matching term and a label-marginal bias:
Revised: This loss decomposes (up to a constant) into two terms, a ground-truth matching term and a label-marginal bias:

Original: [Equation]
Revised: [Equation]

Original: where c = stands for equality up to an additive and/or non-negative multiplicative constant.
Revised: where c = stands for equality up to an additive and/or non-negative multiplicative constant.

Original: The ground-truth matching term in Eq.
Revised: The ground-truth matching term in Eq.

Original: (1) is a lower bound on the cross-entropy loss (CE) due to Jensen’s inequality and the convexity of function − log( x ) : DF ≤ CE.
Revised: (1) is a lower bound on the cross-entropy loss (CE) due to Jensen’s inequality and the convexity of function − log( x ) : DF ≤ CE.

Original: Therefore, minimizing CE could be viewed as a proxy for minimizing term DF that appears in the logarithmic
Revised: Therefore, minimizing CE could be viewed as a proxy for minimizing term DF that appears in the logarithmic

Original: Dice.
Revised: Dice.

Original: In fact, from a constrained-optimization perspective, DF and CE are very closely related and could be viewed as two different penalty functions enforcing the same equality constraints: p ik = 1 , ∀
Revised: In fact, from a constrained-optimization perspective, DF and CE are very closely related and could be viewed as two different penalty functions enforcing the same equality constraints: p ik = 1 , ∀

Original: i ∈
Revised: i ∈

Original: Ω k , ∀ k .
Revised: Ω k , ∀ k .

Original: Both DF and CE are monotonically decreasing functions of each softmax and reach their global minimum when these equality constraints are satisﬁed.
Revised: Both DF and CE are monotonically decreasing functions of each softmax and reach their global minimum when these equality constraints are satisﬁed.

Original: Therefore, they encourage softmax predictions p ik for each region Ω k to reach their target ground-truth values of 1 .
Revised: Therefore, they encourage softmax predictions p ik for each region Ω k to reach their target ground-truth values of 1 .

Original: Of course, this does not mean that penalties CE and DF yield exactly the same results.
Revised: Of course, this does not mean that penalties CE and DF yield exactly the same results.

Original: The difference in the results that they may yield is due to the optimization technique ( e.g. , different gradient dynamics in the standard training of deep networks as the penalty functions have different forms).
Revised: The difference in the results that they may yield is due to the optimization technique ( e.g. , different gradient dynamics in the standard training of deep networks as the penalty functions have different forms).

Original: 2. T HE HIDDEN LABEL - MARGINAL BIAS OF D ICE
Revised: 2. T HE HIDDEN LABEL - MARGINAL BIAS OF D ICE

Original: The following proposition highlights how the label-marginal term DB in Eq.
Revised: The following proposition highlights how the label-marginal term DB in Eq.

Original: (1) encourages speciﬁc extremely imbalanced solutions.
Revised: (1) encourages speciﬁc extremely imbalanced solutions.

Original: Proposition 1.
Revised: Proposition 1.

Original: Let t = (cid:0) ˆ t j (cid:1) 1 ≤ j ≤ K ∈ { 0 , 1 } K denote the simplex vertex verifying:
Revised: Let t = (cid:0) ˆ t j (cid:1) 1 ≤ j ≤ K ∈ { 0 , 1 } K denote the simplex vertex verifying:

Original: ˆ t j = 1 when ˆ y j = max 1 ≤ k ≤ K ˆ y k and ˆ t j = 0 otherwise.
Revised: ˆ t j = 1 when ˆ y j = max 1 ≤ k ≤ K ˆ y k and ˆ t j = 0 otherwise.

Original: For variables p
Revised: For variables p

Original: =
Revised: =

Original: (ˆ p k ) 1 ≤ k ≤ K and ﬁxed distribution
Revised: (ˆ p k ) 1 ≤ k ≤ K and ﬁxed distribution

Original: y =
Revised: y =

Original: (ˆ y k ) 1 ≤ k ≤ K , the label-marginal term in Eq.
Revised: (ˆ y k ) 1 ≤ k ≤ K , the label-marginal term in Eq.

Original: (1) reaches its minimum over the simplex at t :
Revised: (1) reaches its minimum over the simplex at t :

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The details of the proof are deferred to Appendix A.1.
Revised: The details of the proof are deferred to Appendix A.1.

Original: The main technical ingredient is based on Jensen’s inequality and the concavity of penalty DB with respect to simplex variables p .
Revised: The main technical ingredient is based on Jensen’s inequality and the concavity of penalty DB with respect to simplex variables p .

Original: Inequality (2) means that the label-marginal term in Dice in Eq.
Revised: Inequality (2) means that the label-marginal term in Dice in Eq.

Original: (1) is a penalty function for constraint p = t , where t is the simplex vertex given in Proposition 1.
Revised: (1) is a penalty function for constraint p = t , where t is the simplex vertex given in Proposition 1.

Original: Therefore, it encourages extremely imbalanced segmentations, where a speciﬁc region includes all the pixels and the remaining regions are empty.
Revised: Therefore, it encourages extremely imbalanced segmentations, where a speciﬁc region includes all the pixels and the remaining regions are empty.

Original: All in all, the logarithmic Dice loss integrates a hidden label-marginal prior preferring extremely imbalanced segmentations, which is optimized jointly with a ground-truth matching term similar to CE.
Revised: All in all, the logarithmic Dice loss integrates a hidden label-marginal prior preferring extremely imbalanced segmentations, which is optimized jointly with a ground-truth matching term similar to CE.

Original: It is worth noting that, in the two-class (binary) segmentation case, Dice might be used for the foreground region only, as in the popular work in (Milletari et al., 2016), for instance.
Revised: It is worth noting that, in the two-class (binary) segmentation case, Dice might be used for the foreground region only, as in the popular work in (Milletari et al., 2016), for instance.

Original: Similarly to the multi-class case discussed above, a single Dice term also decomposes into a ground-truth matching term and label-marginal penalty, with the latter encouraging extremely imbalanced binary segmentations.
Revised: Similarly to the multi-class case discussed above, a single Dice term also decomposes into a ground-truth matching term and label-marginal penalty, with the latter encouraging extremely imbalanced binary segmentations.

Original: We provide more details for this case in Appendix B.
Revised: We provide more details for this case in Appendix B.

Original: 2. O N THE LINK BETWEEN THE DIFFERENT VARIANTS OF D ICE
Revised: 2. O N THE LINK BETWEEN THE DIFFERENT VARIANTS OF D ICE

Original: The label-marginal analysis we discussed above is based on the standard logarithmic Dice loss.
Revised: The label-marginal analysis we discussed above is based on the standard logarithmic Dice loss.

Original: Here, we argue that both logarithmic and linear Dice are very closely related and, hence, the linear
Revised: Here, we argue that both logarithmic and linear Dice are very closely related and, hence, the linear

Original: Dice also hides a class-imbalance bias.
Revised: Dice also hides a class-imbalance bias.

Original: In fact, from a constrained-optimization perspective, the two losses could be viewed as different penalty functions for imposing constraints: Dice k = 1 ∀ k .
Revised: In fact, from a constrained-optimization perspective, the two losses could be viewed as different penalty functions for imposing constraints: Dice k = 1 ∀ k .

Original: Both functions – log( x ) and (1 − x ) are monotonically decreasing in [0 , 1] and achieve their minimum in [0 , 1] at x = 1 .
Revised: Both functions – log( x ) and (1 − x ) are monotonically decreasing in [0 , 1] and achieve their minimum in [0 , 1] at x = 1 .

Original: Furthermore, the logarithmic Dice is an upper bound on the linear one.
Revised: Furthermore, the logarithmic Dice is an upper bound on the linear one.

Original: This follows directly from: − log ( t ) ≥ 1 − t ∀ t > 0 .
Revised: This follows directly from: − log ( t ) ≥ 1 − t ∀ t > 0 .

Original: Of course, this does not mean that optimizing these two variants leads to exactly the same results.
Revised: Of course, this does not mean that optimizing these two variants leads to exactly the same results.

Original: The differences in their results might be due to optimization ( i.e. , different gradient dynamics stemming from logarithmic and linear penalties).
Revised: The differences in their results might be due to optimization ( i.e. , different gradient dynamics stemming from logarithmic and linear penalties).

Original: 2. T HE HIDDEN LABEL - MARGINAL BIAS OF CE
Revised: 2. T HE HIDDEN LABEL - MARGINAL BIAS OF CE

Original: In the following, we give an information-theoretic perspective of CE, via a generative view of network predictions and a Monte-Carlo approximation of the entropy of the learned features given the labels.
Revised: In the following, we give an information-theoretic perspective of CE, via a generative view of network predictions and a Monte-Carlo approximation of the entropy of the learned features given the labels.

Original: This highlights a hidden label-marginal bias of CE, which encourages the proportions of the predicted segmentation regions to match the ground-truth proportions.
Revised: This highlights a hidden label-marginal bias of CE, which encourages the proportions of the predicted segmentation regions to match the ground-truth proportions.

Original: Proposition 2.
Revised: Proposition 2.

Original: Let F and K denote the random variables associated with the learned features and the labels, respectively, and H ( F|K ) the conditional entropy of learned features given the labels, estimated via Monte-Carlo :
Revised: Let F and K denote the random variables associated with the learned features and the labels, respectively, and H ( F|K ) the conditional entropy of learned features given the labels, estimated via Monte-Carlo :

Original: [Equation]
Revised: [Equation]

Original: where H ( F|K = k ) is the empirical estimate of the conditional entropy of features given a speciﬁc class k (expression in Table 1) and P ( f θi | k ) denotes the probability of the learned features given class k .
Revised: where H ( F|K = k ) is the empirical estimate of the conditional entropy of features given a speciﬁc class k (expression in Table 1) and P ( f θi | k ) denotes the probability of the learned features given class k .

Original: We have the following generative view of CE :
Revised: We have the following generative view of CE :

Original: [Equation]
Revised: [Equation]

Original: The detailed proof is deferred to Appendix A.2.
Revised: The detailed proof is deferred to Appendix A.2.

Original: The approximation of H ( F|K = k ) in the second line of Eq.
Revised: The approximation of H ( F|K = k ) in the second line of Eq.

Original: (3) is based on the well-known Monte-Carlo estimation (Kearns et al., 1997; Tang et al., 2019).
Revised: (3) is based on the well-known Monte-Carlo estimation (Kearns et al., 1997; Tang et al., 2019).

Original: Then the relationship in Eq.
Revised: Then the relationship in Eq.

Original: (4) follows from Eq.
Revised: (4) follows from Eq.

Original: (3), after some manipulations, using Bayes rule P ( f θi | k ) ∝ p
Revised: (3), after some manipulations, using Bayes rule P ( f θi | k ) ∝ p

Original: ik ˆ p k
Revised: ik ˆ p k

Original: and (cid:80) i ∈
Revised: and (cid:80) i ∈

Original: Ω k
Revised: Ω k

Original: log(ˆ p k ) =
Revised: log(ˆ p k ) =

Original: | Ω k | log(ˆ p k ) .
Revised: | Ω k | log(ˆ p k ) .

Original: This information-theoretic view of CE shows that the latter has an implicit (hidden) label-marginal bias towards the ground-truth region proportions (the KL term).
Revised: This information-theoretic view of CE shows that the latter has an implicit (hidden) label-marginal bias towards the ground-truth region proportions (the KL term).

Original: This bias competes with the entropy term, which encourages low uncertainty (variations) within each ground-truth segmentation region Ω k .
Revised: This bias competes with the entropy term, which encourages low uncertainty (variations) within each ground-truth segmentation region Ω k .

Original: The entropy term could be viewed as a ground-truth matching term: it reaches its global minima when the feature embedding is constant within each region.
Revised: The entropy term could be viewed as a ground-truth matching term: it reaches its global minima when the feature embedding is constant within each region.

Original: If used alone, the entropy term may lead to trivial imbalanced solutions.
Revised: If used alone, the entropy term may lead to trivial imbalanced solutions.

Original: The label-marginal KL term avoids such trivial solutions by matching the ground-truth class proportions.
Revised: The label-marginal KL term avoids such trivial solutions by matching the ground-truth class proportions.

Original: Note that there is no mechanism in CE to control the relative contributions of those two competing terms as they are implicit in CE.
Revised: Note that there is no mechanism in CE to control the relative contributions of those two competing terms as they are implicit in CE.

Original: 2. O UR SOLUTION
Revised: 2. O UR SOLUTION

Original: Our analysis shows that Dice, CE and their combinations, e.g. , CE − log( Dice ) , are closely related and enforce two types of competing constraints : ground-truth matching and label-marginal constraints.
Revised: Our analysis shows that Dice, CE and their combinations, e.g. , CE − log( Dice ) , are closely related and enforce two types of competing constraints : ground-truth matching and label-marginal constraints.

Original: However, there is no clear consensus in the literature as to which loss is better, with the performances of each varying across data sets and applications.
Revised: However, there is no clear consensus in the literature as to which loss is better, with the performances of each varying across data sets and applications.

Original: This variability in performances could be explained by two fundamental factors:
Revised: This variability in performances could be explained by two fundamental factors:

Original: • The difference in the label-marginal prior .
Revised: • The difference in the label-marginal prior .

Original: The label-marginal priors are different as Dice has an intrinsic bias preferring very small regions, while CE encourages the right (ground-truth) region proportions.
Revised: The label-marginal priors are different as Dice has an intrinsic bias preferring very small regions, while CE encourages the right (ground-truth) region proportions.

Original: This might explain the wide experimental evidence in the medical imaging literature, where using or adding Dice losses brings improvements for imbalanced segmentation with extremely small regions.
Revised: This might explain the wide experimental evidence in the medical imaging literature, where using or adding Dice losses brings improvements for imbalanced segmentation with extremely small regions.

Original: Weighting the contribution of the bias term .
Revised: Weighting the contribution of the bias term .

Original: Our analysis suggests that CE should be preferred over Dice in all cases and applications (both balanced/imbalanced segmentation, or segmentation problems with high variability in region proportions) as it promotes the right label-marginal distribution.
Revised: Our analysis suggests that CE should be preferred over Dice in all cases and applications (both balanced/imbalanced segmentation, or segmentation problems with high variability in region proportions) as it promotes the right label-marginal distribution.

Original: While this seems to be widely the case in natural image segmentation, where Dice is uncommon, the extensive experimental evidence in the medical-image segmentation literature suggests otherwise, especially in extremely imbalanced problems.
Revised: While this seems to be widely the case in natural image segmentation, where Dice is uncommon, the extensive experimental evidence in the medical-image segmentation literature suggests otherwise, especially in extremely imbalanced problems.

Original: We argue that this is due to the relative contribution of the label-marginal term in the overall objective.
Revised: We argue that this is due to the relative contribution of the label-marginal term in the overall objective.

Original: Controlling such label-marginal contribution is very important in imbalanced problems.
Revised: Controlling such label-marginal contribution is very important in imbalanced problems.

Original: In particular, it mitigates the difﬁculty that the ground-truth matching terms differ by several orders of magnitude across regions, as in CE, which causes large-region terms to completely dominate small-region ones.
Revised: In particular, it mitigates the difﬁculty that the ground-truth matching terms differ by several orders of magnitude across regions, as in CE, which causes large-region terms to completely dominate small-region ones.

Original: This analysis also resonates with the fact that combo losses such as CE - λ log (Dice) perform very competitively in imbalanced segmentation, as shown by (Wong et al., 2018; Taghanaki et al., 2019), among several other recent works.
Revised: This analysis also resonates with the fact that combo losses such as CE - λ log (Dice) perform very competitively in imbalanced segmentation, as shown by (Wong et al., 2018; Taghanaki et al., 2019), among several other recent works.

Original: In this case, controlling the relative contribution of each of these terms indirectly controls the weight of the label-marginal bias.
Revised: In this case, controlling the relative contribution of each of these terms indirectly controls the weight of the label-marginal bias.

Original: Note that such control is not possible when using CE alone or Dice alone, as the label-marginal biases in these losses are hidden (implicit) 
Revised: Note that such control is not possible when using CE alone or Dice alone, as the label-marginal biases in these losses are hidden (implicit) =

Original: We propose a principled and simple solution, which enables to control explicitly the labelmarginal bias, via regularization losses that encourage the correct class proportions and are used in conjunction with CE :
Revised: We propose a principled and simple solution, which enables to control explicitly the labelmarginal bias, via regularization losses that encourage the correct class proportions and are used in conjunction with CE :

Original: [Equation]
Revised: [Equation]

Original: Our label-marginal regularizers increase the contribution of the minority classes in imbalanced problems, but, unlike Dice, do not lose adaptability to problems with various class proportions.
Revised: Our label-marginal regularizers increase the contribution of the minority classes in imbalanced problems, but, unlike Dice, do not lose adaptability to problems with various class proportions.

Original: Our extensive experiments and ablation studies over different losses and applications demonstrate the effectiveness of our explicit label-marginal regularizers.
Revised: Our extensive experiments and ablation studies over different losses and applications demonstrate the effectiveness of our explicit label-marginal regularizers.

Original: We investigate different forms of regularization, including the L 1 norm, i.e. , R ( y ; p ) =
Revised: We investigate different forms of regularization, including the L 1 norm, i.e. , R ( y ; p ) =

Original: L 1 ( y , p ) , and the KL divergence, i.e. , R ( y ; p ) =
Revised: L 1 ( y , p ) , and the KL divergence, i.e. , R ( y ; p ) =

Original: D KL ( y || p ) ; see Table 1 for the expressions of D KL and L 1 .
Revised: D KL ( y || p ) ; see Table 1 for the expressions of D KL and L 1 .

Original: In Fig.
Revised: In Fig.

Original: 2, we depict our different regularizers as functions of the label-marginal distribution for a binary-segmentation case, with the foreground-region proportion set to 0 .
Revised: 2, we depict our different regularizers as functions of the label-marginal distribution for a binary-segmentation case, with the foreground-region proportion set to 0 .

Original: 1 , along with the bias terms in Dice.
Revised: 1 , along with the bias terms in Dice.

Original: While our D KL and L 1 regularizers may deliver comparable performances (see the experimental section), L 1 might be a better option for extremely imbalanced segmentations, due to its gradient properties and stability at the vicinity of 0 , i.e. , when the label-marginal probability ˆ pis close to 0 .
Revised: While our D KL and L 1 regularizers may deliver comparable performances (see the experimental section), L 1 might be a better option for extremely imbalanced segmentations, due to its gradient properties and stability at the vicinity of 0 , i.e. , when the label-marginal probability ˆ p 1 is close to 0 .

Original: Notice that, at the vicinity of zero, both ﬁrst and second derivatives of the regularizer are unbounded for D KL , but bounded and constant for L 1 .
Revised: Notice that, at the vicinity of zero, both ﬁrst and second derivatives of the regularizer are unbounded for D KL , but bounded and constant for L 1 .

Original: Our experiments on imbalanced medical image segmentation conﬁrm the effectiveness ofthe
Revised: Our experiments on imbalanced medical image segmentation conﬁrm the effectiveness of

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Datasets.
Revised: Datasets.

Original: We report results on both medical and natural image segmentation benchmarks, i.e. ,
Revised: We report results on both medical and natural image segmentation benchmarks, i.e. ,

Original: Retinal Lesions (Wei et al., 2020) and Cityscapes (Cordts et al., 2016).
Revised: Retinal Lesions (Wei et al., 2020) and Cityscapes (Cordts et al., 2016).

Original: Retinal Lesions is a large collection of color fundus images, as shown in Fig.
Revised: Retinal Lesions is a large collection of color fundus images, as shown in Fig.

Original: 3, with pixel-level lesion annotations.
Revised: 3, with pixel-level lesion annotations.

Original: In our experiments, we employ the public version of Retinal Lesions 1 , containing 1 , 593 samples, and conduct our experiments in the binary setting to segment the lesion region versus background.
Revised: In our experiments, we employ the public version of Retinal Lesions 1 , containing 1 , 593 samples, and conduct our experiments in the binary setting to segment the lesion region versus background.

Original: The dataset is randomly divided into training ( 70% ), validation ( 10% ) and test ( 20% ) set, whose images are resized to 512 × 512 .
Revised: The dataset is randomly divided into training ( 70% ), validation ( 10% ) and test ( 20% ) set, whose images are resized to 512 × 512 .

Original: We tune the hyper-parameters of each model on the validation set, and report the results on the test set.
Revised: We tune the hyper-parameters of each model on the validation set, and report the results on the test set.

Original: The natural scene dataset, Cityscapes , is a large-scale dataset with high quality pixel-level annotations of 5 , 000 images across 19 categories, containing both stuff and objects with high variation in the class proportions distribution.
Revised: The natural scene dataset, Cityscapes , is a large-scale dataset with high quality pixel-level annotations of 5 , 000 images across 19 categories, containing both stuff and objects with high variation in the class proportions distribution.

Original: We use the ofﬁcial data split, which contains 2 , 975 samples for training, 500 samples for validation and 1 , 512 samples for testing.
Revised: We use the ofﬁcial data split, which contains 2 , 975 samples for training, 500 samples for validation and 1 , 512 samples for testing.

Original: All the input images are resized to 512 × 1024 for training, and 1024 × 2048 for testing, without the strategy of multi-scale or sliding window in inference.
Revised: All the input images are resized to 512 × 1024 for training, and 1024 × 2048 for testing, without the strategy of multi-scale or sliding window in inference.

Original: Fig.
Revised: Fig.

Original: C.1 in Appendix C shows the distribution of region proportions for Retinal Lesions and the 19 classes in Cityscapes, highlighting the challenging class imbalance and high diversity on target sizes.
Revised: C.1 in Appendix C shows the distribution of region proportions for Retinal Lesions and the 19 classes in Cityscapes, highlighting the challenging class imbalance and high diversity on target sizes.

Original: Backbones.
Revised: Backbones.

Original: It is worth stressing that the theoretical discussion in this paper is model-agnostic.
Revised: It is worth stressing that the theoretical discussion in this paper is model-agnostic.

Original: Thus, in our experiments we employ  Res50-FPN (Lin et al., 2017) and  Res50-Unet (Ronneberger et al., 2015) on Retinal Lesions, while we show results with Res50-FPN and Res101-FPN on Cityscapes, whose implementations are publicly available 2 .
Revised: Thus, in our experiments we employ Res34-FPN, Res50-FPN (Lin et al., 2017) and Res34-Unet, Res50-Unet (Ronneberger et al., 2015) on Retinal Lesions, while we show results with Res50-FPN and Res101-FPN on Cityscapes, whose implementations are publicly available 2 .

Original: Training.
Revised: Training.

Original: The parameters of the encoder are initialized with the pre-trained weights on Imagenet, while those in the decoder head are randomly initialized.
Revised: The parameters of the encoder are initialized with the pre-trained weights on Imagenet, while those in the decoder head are randomly initialized.

Original: On Retinal Lesions, we train the model during 60 epochs, with a batch size of 8 , via the Adam optimizer.
Revised: On Retinal Lesions, we train the model during 60 epochs, with a batch size of 8 , via the Adam optimizer.

Original: The initial learning rate is set to 1 e- 4 , and halved if the loss on the validation set does not decrease within 5 epochs.
Revised: The initial learning rate is set to 1 e- 4 , and halved if the loss on the validation set does not decrease within 5 epochs.

Original: On Citycapes, the main setting is adopted from the state-of-the-art library 3 .
Revised: On Citycapes, the main setting is adopted from the state-of-the-art library 3 .

Original: Speciﬁcally, the SGD optimizer is used for training, with an initial learning rate set to 0 .
Revised: Speciﬁcally, the SGD optimizer is used for training, with an initial learning rate set to 0 .

Original: 01 and a batch size of 8 .
Revised: 01 and a batch size of 8 .

Original: During the 100 training epochs, we use an iteration-wise polynomial strategy to linearly scale the learning rate down to a minimum of 1 e- 4 .
Revised: During the 100 training epochs, we use an iteration-wise polynomial strategy to linearly scale the learning rate down to a minimum of 1 e- 4 .

Original: For all the models, we tune the hyper-parameters on the validation and report the best results on the corresponding test set.
Revised: For all the models, we tune the hyper-parameters on the validation and report the best results on the corresponding test set.

Original: Conventional data augmentations, such as random cropping, mirror ﬂipping and changes in brightness, are employed on both datasets.
Revised: Conventional data augmentations, such as random cropping, mirror ﬂipping and changes in brightness, are employed on both datasets.

Original: Losses.
Revised: Losses.

Original: We evaluate the proposed loss function in Eq.
Revised: We evaluate the proposed loss function in Eq.

Original: (5) with two regularizing terms, i.e. , D KL and L 1 .
Revised: (5) with two regularizing terms, i.e. , D KL and L 1 .

Original: In our implementation, we use a modiﬁed soft-max function with a temperature parameter when computing the predicted region proportion, as this enables a better estimate of the actual region proportion (refer to Appendix D for details).
Revised: In our implementation, we use a modiﬁed soft-max function with a temperature parameter when computing the predicted region proportion, as this enables a better estimate of the actual region proportion (refer to Appendix D for details).

Original: We compare our results with the following baselines:
Revised: We compare our results with the following baselines:

Original: CE, Focal loss (Lin et al., 2017), Dice related losses, i.e. , linear/logarithmic/generalized Dice, and the composite loss combining CE and logarithmic Dice.
Revised: CE, Focal loss (Lin et al., 2017), Dice related losses, i.e. , linear/logarithmic/generalized Dice, and the composite loss combining CE and logarithmic Dice.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: On Retinal Lesions, we use two standard metrics in medical image segmentation, i.e. , Dice Similarity Coefﬁcient (DSC) and modiﬁed Hausdorff Distance (HD-95).
Revised: On Retinal Lesions, we use two standard metrics in medical image segmentation, i.e. , Dice Similarity Coefﬁcient (DSC) and modiﬁed Hausdorff Distance (HD-95).

Original: Particularly,
Revised: Particularly,

Original: HD-95 represents the 95th percentile of the symmetric Hausdorff Distance (HD) between the binary objects in two images.
Revised: HD-95 represents the 95th percentile of the symmetric Hausdorff Distance (HD) between the binary objects in two images.

Original: On Cityscapes, we resort to the standard Intersection-Over-Union (IoU) score, which is widely employed in natural image segmentation.
Revised: On Cityscapes, we resort to the standard Intersection-Over-Union (IoU) score, which is widely employed in natural image segmentation.

Original: Results on Retinal Lesions.
Revised: Results on Retinal Lesions.

Original: Table 2 reports the quantitative comparison between the proposed loss, with the two different regularizers, and baselines on the Retinal Lesions dataset.
Revised: Table 2 reports the quantitative comparison between the proposed loss, with the two different regularizers, and baselines on the Retinal Lesions dataset.

Original: Regarless of the backbone network, the proposed CE + L 1 loss consistently outperforms all the others with a DSC score of 54 .
Revised: Regarless of the backbone network, the proposed CE + L 1 loss consistently outperforms all the others with a DSC score of 54 .

Original: 5% for Res50-FPN and 54 .
Revised: 5% for Res50-FPN and 54 .

Original: 3% for Res50-Unet.
Revised: 3% for Res50-Unet.

Original: Compared to CE, our best model, i.e. ,
Revised: Compared to CE, our best model, i.e. ,

Original: L 1 as the label-marginal regularizer, brings nearly 3 . 0% improvement in terms of DSC and between 5 .
Revised: L 1 as the label-marginal regularizer, brings nearly 3 . 0% improvement in terms of DSC and between 5 .

Original: 6% and 8 .
Revised: 6% and 8 .

Original: 6% in terms of HD-95, depending on the backbone.
Revised: 6% in terms of HD-95, depending on the backbone.

Original: Even though these differences are reduced with respect to the best baseline, this gap is still signiﬁcant, particularly in terms of HD-95.
Revised: Even though these differences are reduced with respect to the best baseline, this gap is still signiﬁcant, particularly in terms of HD-95.

Original: It is noteworthy to mention that while DSC is more sensitive to the internal ﬁlling of the target region, the HD-95 is more sensitive to the segmentation boundary.
Revised: It is noteworthy to mention that while DSC is more sensitive to the internal ﬁlling of the target region, the HD-95 is more sensitive to the segmentation boundary.

Original: Thus, the proposed loss is more effective to predict better boundaries than existing losses.
Revised: Thus, the proposed loss is more effective to predict better boundaries than existing losses.

Original: Furthermore, as shown in Fig.
Revised: Furthermore, as shown in Fig.

Original: C.1 in Appendix C,
Revised: C.1 in Appendix C,

Original: [Table]
Revised: [Table]

Original: [Figure]

Original: [Table]

Original: [Table]

Original: [Table] [Table] [Table] the average region proportion is considerably low, i.e. , 3 . 8% , and varies signiﬁcantly (the standard deviation is 6 .
Revised: the average region proportion is considerably low, i.e. , 3 . 8% , and varies signiﬁcantly (the standard deviation is 6 .

Original: 6% with a maximum of 57 .
Revised: 6% with a maximum of 57 .

Original: Under this highly heterogeneous scenario in terms of class region proportions, the proposed losses present a more stable performance across different runs and backbones, which is reﬂected in their lower variances.
Revised: Under this highly heterogeneous scenario in terms of class region proportions, the proposed losses present a more stable performance across different runs and backbones, which is reﬂected in their lower variances.

Original: This can be explained by their better adaptability to different target sizes and better gradient dynamics at the vicinity of label marginal
Revised: This can be explained by their better adaptability to different target sizes and better gradient dynamics at the vicinity of label marginal

Original: ˆ p k
Revised: ˆ p k

Original: =
Revised: =

Original: (as shown in Fig. 2).
Revised: (as shown in Fig. 2).

Original: Another interesting ﬁnding is that, by combining CE with the bias term of Dice (CE + DB 1 ), we obtain results close to combo loss CE − log( Dice 1 )
Revised: Another interesting ﬁnding is that, by combining CE with the bias term of Dice (CE + DB 1 ), we obtain results close to combo loss CE − log( Dice 1 )

Original: (Wong et al., 2018).
Revised: (Wong et al., 2018).

Original: This validates our theoretical insight that the fundamental difference between CE and Dice lies in their distinct hidden biases.
Revised: This validates our theoretical insight that the fundamental difference between CE and Dice lies in their distinct hidden biases.

Original: On the trade-off weightλ in Eq.
Revised: On the balancing weight in the composite losses.

Original: In this section we evaluate the impact of  λ  in Eq.
Revised: In this section we evaluate the impact of the balancing weight λ of the proposed loss in Eq.

Original: (5), which balances the effects of the regularization term.
Revised: (5), which balances the effects of the regularization term, as well as the balancing weight in CE − log( Dice 1 ) .

Original: In our experiments, we empirically found that the best λ values  are : 1 . 0 for CE + L 1 , 0 . 1 for CE + D KL and 0 .
Revised: In our experiments, we empirically found that the best λ values fordifferentpenaltyterms are : 1 . 0 for CE + L 1 , 0 . 1 for CE + D KL , 0 .

Original: 01 for CE + DB 1 .
Revised: 01 for CE + DB 1 .

Original: These values are consistent with the given arguments in Sec. 2.6, which emphasize that L 1 has a better gradient evolution for large learning rates and relatively high weighting, especially at the beginning of training.
Revised: These values are consistent with the given arguments in Sec. 2.6, which emphasize that L 1 has a better gradient evolution for large learning rates and relatively high weighting, especially at the beginning of training.

Revised: Regarding the widely used combo loss of CE − log( Dice 1 ) , setting its balancing weight to 1 . 0 yielded consistent performance across the datasets and networks we used.

Original: Details of this experimental study can be found in Appendix E.  Note that we use the best empirical values of λ found on Retinal Lesions for the experiments on Cityscapes.
Revised: Details of this experimental study can be found in Appendix E. Therefore, with the same hyperparameter budget, CE + L 1 performed better than all the other related composite loss functions. Note that we use the best empirical values of λ found on Retinal Lesions for the experiments on Cityscapes.

Original: [Figure]
Revised: [Figure]

Original: indicate the corresponding obtained DSC score.

Original: Qualitative results on the Retinal Lesions dataset.
Revised: Qualitative results on the Retinal Lesions dataset.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 3, we can observe that the model trained with Dice variants ( third and fourth column ) tends to undersegment large and medium target regions, while ignoring several small lesions and generating false positives.
Revised: 3, we can observe that the model trained with Dice variants ( third and fourth column ) tends to undersegment large and medium target regions, while ignoring several small lesions and generating false positives.

Original: In contrast, the proposed solution enables a better trade-off between ﬁnding small regions, reducing the number of false positives and matching the size of the larger targets.
Revised: In contrast, the proposed solution enables a better trade-off between ﬁnding small regions, reducing the number of false positives and matching the size of the larger targets.

Original: Furthermore, one can notice that our loss yields better consistencies with the target region proportions than CE. 
Revised: Furthermore, one can notice that our loss yields better consistencies with the target region proportions than CE. [Table]

Original: Results on Cityscapes.
Revised: Results on Cityscapes.

Original: Table 3 reports the comparative per-class IoU and mean IoU (mIoU) on the validation set of Cityscapes with two network architectures.
Revised: Table 3 reports the comparative per-class IoU and mean IoU (mIoU) on the validation set of Cityscapes with two network architectures.

Original: First, we can observe that in this[Table] multi-class dataset, regardless of the network, the proposed learning objectives outperform all the evaluated losses in terms of mIoU. Then, by investigating the relationship between mRegProp (second row in Table 3) and the corresponding segmentation performance across small region classes, we can observe that Dice-related losses have a hidden label-marginal bias towards extremely imbalanced solutions, preferring small structures.
Revised: First, we can observe that in this multi-class dataset, regardless of the network, the proposed learning objectives outperform all the evaluated losses in terms of mIoU. Then, by investigating the relationship between mRegProp (second row in Table 3) and the corresponding segmentation performance across small region classes, we can observe that Dice-related losses have a hidden label-marginal bias towards extremely imbalanced solutions, preferring small structures.

Original: In particular, the linear Dice often obtains the highest mIoU for the smallest structures.
Revised: In particular, the linear Dice often obtains the highest mIoU for the smallest structures.

Original: This bias comes at the cost of less ﬂexibility when dealing with arbitrary class proportions, which is reﬂected in its poor average mIoU ( right column ).
Revised: This bias comes at the cost of less ﬂexibility when dealing with arbitrary class proportions, which is reﬂected in its poor average mIoU ( right column ).

Original: [Table]
Revised: [Table]

Original: Quantitative evaluation on the Cityscapes test set is reported in Table 4.
Revised: Quantitative evaluation on the Cityscapes test set is reported in Table 4.

Original: The observations in this table are consistent with the results on the validation set.
Revised: The observations in this table are consistent with the results on the validation set.

Original: Similarly to the validation set, our method achieves better results on both architectures.
Revised: Similarly to the validation set, our method achieves better results on both architectures.

Original: For the per-class scores on the test set, please refer to Appendix F.
Revised: For the per-class scores on the test set, please refer to Appendix F.

Original: It is noteworthy to highlight that both linear Dice and logarithmic Dice perform relatively poorly on Cityscapes, on average, empirically showing why Dice is rarely adopted in natural-image segmentation tasks.
Revised: It is noteworthy to highlight that both linear Dice and logarithmic Dice perform relatively poorly on Cityscapes, on average, empirically showing why Dice is rarely adopted in natural-image segmentation tasks.

Original: As we mentioned earlier, this might be due to its inherent label-marginal bias, which is inappropriate for segmenting regions with arbitrary class proportions.
Revised: As we mentioned earlier, this might be due to its inherent label-marginal bias, which is inappropriate for segmenting regions with arbitrary class proportions.

Original: Furthermore, our simple solution yielded an improvement over CE by nearly 1.5%, regardless of the backbone.
Revised: Furthermore, our simple solution yielded an improvement over CE by nearly 1.5%, regardless of the backbone.

Original: These empirical observations suggest that the proposed formulation in Eq.
Revised: These empirical observations suggest that the proposed formulation in Eq.

Original: (5) results in better region-proportion guidance and training stability than existing segmentation losses.
Revised: (5) results in better region-proportion guidance and training stability than existing segmentation losses.

Original: Finally, we can observe that employing L 1 as regularizer consistently results in better performance than using the D KL term.
Revised: Finally, we can observe that employing L 1 as regularizer consistently results in better performance than using the D KL term.

Original: Hence, L 1 might be a better option in extremely imbalanced segmentations, due to its gradient properties and stability when the label-marginal probability is close to 0.
Revised: Hence, L 1 might be a better option in extremely imbalanced segmentations, due to its gradient properties and stability when the label-marginal probability is close to 0.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We provided a detailed theoretical analysis of the two most popular semantic segmentation losses, i.e. , Cross-entropy and Dice, which revealed non-obvious bounding relationships and hidden labelmarginal biases, suggesting that CE is a better option in general.
Revised: We provided a detailed theoretical analysis of the two most popular semantic segmentation losses, i.e. , Cross-entropy and Dice, which revealed non-obvious bounding relationships and hidden labelmarginal biases, suggesting that CE is a better option in general.

Original: Then, we showed how both loss functions could be written within a common formulation, containing a ground-truth matching term and a label-marginal bias.
Revised: Then, we showed how both loss functions could be written within a common formulation, containing a ground-truth matching term and a label-marginal bias.

Original: The implicit bias in Dice prefers small regions, improving its performance in highly imbalanced conditions, as in medical-imaging applications.
Revised: The implicit bias in Dice prefers small regions, improving its performance in highly imbalanced conditions, as in medical-imaging applications.

Original: The bias hidden in CE encourages the ground-truth region proportion, which makes it a generally better option in complex scenarios with diverse class proportions.
Revised: The bias hidden in CE encourages the ground-truth region proportion, which makes it a generally better option in complex scenarios with diverse class proportions.

Original: Furthermore, we proposed a principled solution, which enables to control the label-marginal bias via L 1 and KL regularizers that encourage the target class proportions, while improving training stability.
Revised: Furthermore, we proposed a principled solution, which enables to control the label-marginal bias via L 1 and KL regularizers that encourage the target class proportions, while improving training stability.

Original: Our ﬂexible formulation enables the minority classes to have better inﬂuence on training, without losing adaptability to medium-to-large regions.
Revised: Our ﬂexible formulation enables the minority classes to have better inﬂuence on training, without losing adaptability to medium-to-large regions.

Original: Extensive experiments on a natural and medical imaging datasets validate the theoretical analysis in this paper, as well as the effectiveness of the presented solution.
Revised: Extensive experiments on a natural and medical imaging datasets validate the theoretical analysis in this paper, as well as the effectiveness of the presented solution.

Original: A P ROOFS A. P ROPOSITION
Revised: A P ROOFS A. P ROPOSITION

Original: Proof.
Revised: Proof.

Original: Let us write the label-marginal bias term in the logarithmic Dice as a vector-valued function of probability simplex vector p :
Revised: Let us write the label-marginal bias term in the logarithmic Dice as a vector-valued function of probability simplex vector p :

Original: [Equation]
Revised: [Equation]

Original: where symbol (cid:62) denotes transpose and 1 K is the K -dimensional vector of ones.
Revised: where symbol (cid:62) denotes transpose and 1 K is the K -dimensional vector of ones.

Original: Function g is concave because its Hessian is a negative semi-deﬁnite matrix:
Revised: Function g is concave because its Hessian is a negative semi-deﬁnite matrix:

Original: The Hessian of g is a diagonal matrix whose diagonal elements are given by − 1ˆ p 2 k
Revised: The Hessian of g is a diagonal matrix whose diagonal elements are given by − 1ˆ p 2 k

Original: and, hence, are all non-positive.
Revised: and, hence, are all non-positive.

Original: Therefore, using
Revised: Therefore, using

Original: Jensen’s inequality and the fact that p is within the simplex, we have the following lower bound on penalty g in Eq.
Revised: Jensen’s inequality and the fact that p is within the simplex, we have the following lower bound on penalty g in Eq.

Original: [Equation]
Revised: [Equation]

Original: where e k ∈ { 0 , 1 } K denote the k -th vertex of the simplex: the k -th component of e k is equal to 1 while the other components are all equal to 0 .
Revised: where e k ∈ { 0 , 1 } K denote the k -th vertex of the simplex: the k -th component of e k is equal to 1 while the other components are all equal to 0 .

Original: Now, recall the deﬁnition of simplex vector t = (cid:0)
Revised: Now, recall the deﬁnition of simplex vector t = (cid:0)

Original: ˆ t j (cid:1) 1 ≤ j ≤ K : ˆ t j = 1 when ˆ y j = max 1 ≤ k ≤ K ˆ y k and ˆ t j = 0 otherwise.
Revised: ˆ t j (cid:1) 1 ≤ j ≤ K : ˆ t j = 1 when ˆ y j = max 1 ≤ k ≤ K ˆ y k and ˆ t j = 0 otherwise.

Original: Given this deﬁnition, one could easily verify the following fact:
Revised: Given this deﬁnition, one could easily verify the following fact:

Original: [Equation]
Revised: [Equation]

Original: To see this, let j denotes the integer verifying ˆ y j = max 1 ≤ k ≤ K ˆ y k and let k (cid:54) =
Revised: To see this, let j denotes the integer verifying ˆ y j = max 1 ≤ k ≤ K ˆ y k and let k (cid:54) =

Original: j .
Revised: j .

Original: Then, we have:
Revised: Then, we have:

Original: [Equation]
Revised: [Equation]

Original: This is due to the fact that function log (cid:0) 1 + 1 x (cid:1) is monotonically decreasing in [0 , 1] and ˆ y k ≤ ˆ y j .
Revised: This is due to the fact that function log (cid:0) 1 + 1 x (cid:1) is monotonically decreasing in [0 , 1] and ˆ y k ≤ ˆ y j .

Original: Now, combining inequalities (7) and (8), and using the fact that (cid:80)
Revised: Now, combining inequalities (7) and (8), and using the fact that (cid:80)

Original: Kk
Revised: Kk

Original: =

Revised: =

Original: ˆ p k
Revised: ˆ p k

Original: =1, we obtain:
Revised: =1, we obtain:

Original: [Equation]
Revised: [Equation]

Original: A. P ROPOSITION
Revised: A. P ROPOSITION

Original: Proof.
Revised: Proof.

Original: Considering a generative view of the prediction model, random variable F associated with the learned features is continuous, while the random variable describing the labels, i.e. , K , takes its possible values in a ﬁnite set { 1 , . .
Revised: Considering a generative view of the prediction model, random variable F associated with the learned features is continuous, while the random variable describing the labels, i.e. , K , takes its possible values in a ﬁnite set { 1 , . .

Original: . , K } .
Revised: . , K } .

Original: Then, the marginal distribution of the labels could be empirically estimated by the GT proportion of each segmentation region (as listed in Table 1) (Kim et al., 2005) :
Revised: Then, the marginal distribution of the labels could be empirically estimated by the GT proportion of each segmentation region (as listed in Table 1) (Kim et al., 2005) :

Original: [Equation]
Revised: [Equation]

Original: Also, we express the conditional entropy of the learned features as follows :
Revised: Also, we express the conditional entropy of the learned features as follows :

Original: [Equation]
Revised: [Equation]

Original: with each H ( F|K = k ) given by :
Revised: with each H ( F|K = k ) given by :

Original: [Equation]
Revised: [Equation]

Original: Hereafter, for notation simplicity, we omit K and use H ( F| k ) instead of H ( F|K = k ) .
Revised: Hereafter, for notation simplicity, we omit K and use H ( F| k ) instead of H ( F|K = k ) .

Original: Also, we use P ( f θ | k ) instead of P ( f θ |K = k ) .
Revised: Also, we use P ( f θ | k ) instead of P ( f θ |K = k ) .

Original: To estimate the conditional entropy in Eq.
Revised: To estimate the conditional entropy in Eq.

Original: (13), let us refer to the following well known Monte-Carlo estimation (Kearns et al., 1997; Tang et al., 2019) :
Revised: (13), let us refer to the following well known Monte-Carlo estimation (Kearns et al., 1997; Tang et al., 2019) :

Original: Monte-Carlo estimation.
Revised: Monte-Carlo estimation.

Original: For any discrete set of points S ⊂ Ω , any function g and any feature embedding f , we have :
Revised: For any discrete set of points S ⊂ Ω , any function g and any feature embedding f , we have :

Original: [Equation]
Revised: [Equation]

Original: where f i denotes a feature vector at point i , and P ( f | S ) stands for the density of { f i , i ∈ S } .
Revised: where f i denotes a feature vector at point i , and P ( f | S ) stands for the density of { f i , i ∈ S } .

Original: Therefore, applying Montre-Carlo to H ( F| k ) in Eq.
Revised: Therefore, applying Montre-Carlo to H ( F| k ) in Eq.

Original: (13), we can re-write Eq.
Revised: (13), we can re-write Eq.

Original: (12) as follows :
Revised: (12) as follows :

Original: [Equation]
Revised: [Equation]

Original: Furthermore, using Bayes rule P ( f θi | k ) ∝ p ik ˆ p k , in addition to the fact that (cid:80) i ∈
Revised: Furthermore, using Bayes rule P ( f θi | k ) ∝ p ik ˆ p k , in addition to the fact that (cid:80) i ∈

Original: Ω k
Revised: Ω k

Original: log(ˆ p k ) =
Revised: log(ˆ p k ) =

Original: | Ω k | log(ˆ p k ) , we obtain :
Revised: | Ω k | log(ˆ p k ) , we obtain :

Original: [Equation]
Revised: [Equation]

Original: Finally, due to the deﬁnition of the label-marginal KL divergence, we have :
Revised: Finally, due to the deﬁnition of the label-marginal KL divergence, we have :

Original: [Equation]
Revised: [Equation]

Original: This yields :
Revised: This yields :

Original: [Equation]
Revised: [Equation]

Original: In summary, we give an information-theoretic prospective of CE.
Revised: In summary, we give an information-theoretic prospective of CE.

Original: The entropy term can be considered as a ground-truth matching term, while the label-marginal KL term avoids trivial solutions and encourages the proportions of the predicted segmentation regions to match the ground-truth proportions.
Revised: The entropy term can be considered as a ground-truth matching term, while the label-marginal KL term avoids trivial solutions and encourages the proportions of the predicted segmentation regions to match the ground-truth proportions.

Original: B T HE BINARY SEGMENTATION CASE
Revised: B T HE BINARY SEGMENTATION CASE

Original: In the two-class (binary) segmentation case, Dice might be used for the foreground region only (Milletari et al., 2016).
Revised: In the two-class (binary) segmentation case, Dice might be used for the foreground region only (Milletari et al., 2016).

Original: Similarly to the multi-class case discussed in the paper, a single Dice term also decomposes into a ground-truth matching term and label-marginal penalty, with the latter encouraging extremely imbalanced binary segmentations.
Revised: Similarly to the multi-class case discussed in the paper, a single Dice term also decomposes into a ground-truth matching term and label-marginal penalty, with the latter encouraging extremely imbalanced binary segmentations.

Original: For this speciﬁc case, the logarithmic Dice and CE could be written as summations over the foreground and background segmentation regions:
Revised: For this speciﬁc case, the logarithmic Dice and CE could be written as summations over the foreground and background segmentation regions:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: In Eq.
Revised: In Eq.

Original: (19), the term DB 1 can be expressed, up to an additive constant, as a function of the marginal probability of the foreground class ( k = 1 ) as follows:
Revised: (19), the term DB 1 can be expressed, up to an additive constant, as a function of the marginal probability of the foreground class ( k = 1 ) as follows:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Clearly, the marginal probability ˆ p 1 measures the predicted proportion of pixels within the foreground region.
Revised: Clearly, the marginal probability ˆ p 1 measures the predicted proportion of pixels within the foreground region.

Original: This term reaches its minimum when the foreground region is empty ( p i 1 = 0 ∀ i ).
Revised: This term reaches its minimum when the foreground region is empty ( p i 1 = 0 ∀ i ).

Original: Therefore, since log is monotonically increasing, minimizing term DB 1 in Eq.
Revised: Therefore, since log is monotonically increasing, minimizing term DB 1 in Eq.

Original: (19) introduces a bias preferring small foreground structures.
Revised: (19) introduces a bias preferring small foreground structures.

Original: Note that this label-marginal term in the logarithmic
Revised: Note that this label-marginal term in the logarithmic

Original: Dice loss is important to avoid trivial solutions: when using the foreground-matching term alone, the model may assign all the pixels in the image to the foreground region.
Revised: Dice loss is important to avoid trivial solutions: when using the foreground-matching term alone, the model may assign all the pixels in the image to the foreground region.

Original: The foreground-matching terms, CE 1 and DF 1 , are closely related, with the former being and upper bound on the latter, due to Jensen’s inequality: DF 1 ≤ CE 1 .
Revised: The foreground-matching terms, CE 1 and DF 1 , are closely related, with the former being and upper bound on the latter, due to Jensen’s inequality: DF 1 ≤ CE 1 .

Original: Both foreground-matching terms are monotonically decreasing functions of each softmax and reach their global minimum when all the softmax predictions in the ground-truth foreground are equal to 1 ( i.e. , reach their target).
Revised: Both foreground-matching terms are monotonically decreasing functions of each softmax and reach their global minimum when all the softmax predictions in the ground-truth foreground are equal to 1 ( i.e. , reach their target).

Original: Hence, the matching terms in Dice and CE can be viewed as two different penalty functions for imposing the same equality constraints, p i 1 = 1 , ∀ i ∈ Ω 1 , thereby encouraging the predicted foreground to include the ground-truth foreground.
Revised: Hence, the matching terms in Dice and CE can be viewed as two different penalty functions for imposing the same equality constraints, p i 1 = 1 , ∀ i ∈ Ω 1 , thereby encouraging the predicted foreground to include the ground-truth foreground.

Original: C D ISTRIBUTION OF REGION PROPORTIONS ACROSS CLASSES
Revised: C D ISTRIBUTION OF REGION PROPORTIONS ACROSS CLASSES

Original: [Figure]
Revised: [Figure]

Original: In Fig.
Revised: In Fig.

Original: C.1, we present the distribution of region portions for each class in Cityscapes and Retinal
Revised: C.1, we present the distribution of region portions for each class in Cityscapes and Retinal

Original: Lesions.
Revised: Lesions.

Original: Cityscapes has highly variable region sizes (ranging from an average of 1 . 0% for rider and motorbike to an average of 32 .
Revised: Cityscapes has highly variable region sizes (ranging from an average of 1 . 0% for rider and motorbike to an average of 32 .

Original: 6% for road).
Revised: 6% for road).

Original: In Retinal Lesions, although the average region proportion is 3 . 8% , there are some examples with large region proportions (above 50 . 0% ), and the standard deviation is 6 .
Revised: In Retinal Lesions, although the average region proportion is 3 . 8% , there are some examples with large region proportions (above 50 . 0% ), and the standard deviation is 6 .

Original: Thus, these plots present the signiﬁcant heterogeneity on different classes found across these two data sets.
Revised: Thus, these plots present the signiﬁcant heterogeneity on different classes found across these two data sets.

Original: D T HE TEMPERATURE SCALING
Revised: D T HE TEMPERATURE SCALING

Original: In our implementation, we employ a modiﬁed soft-max function with a temperature scaling parameter when computing the predicted region proportion ˆ p k (also referred to as the predicted label- marginal probability, as in Table 1) :
Revised: In our implementation, we employ a modiﬁed soft-max function with a temperature scaling parameter when computing the predicted region proportion ˆ p k (also referred to as the predicted label- marginal probability, as in Table 1) :

Original: [Equation]
Revised: [Equation]

Original: where z =
Revised: where z =

Original: ( z i ) 1 ≤ i ≤ K is the input vector of the soft-max function, and τ > 0 acts as the temperature hyper-parameter.
Revised: ( z i ) 1 ≤ i ≤ K is the input vector of the soft-max function, and τ > 0 acts as the temperature hyper-parameter.

Original: High values of τ > 0 yield high conﬁdence of the soft-max prediction, as shown in Fig.
Revised: High values of τ > 0 yield high conﬁdence of the soft-max prediction, as shown in Fig.

Original: D.1: They push the softmax vector towards the vertices of the simplex, with prediction values approaching either 0 or 1 .
Revised: D.1: They push the softmax vector towards the vertices of the simplex, with prediction values approaching either 0 or 1 .

Original: As a result, this enables a better estimate of the actual region proportion (or relative size).
Revised: As a result, this enables a better estimate of the actual region proportion (or relative size).

Original: Throughout all our experiments, we set τ to 10 .
Revised: Throughout all our experiments, we set τ to 10 .

Original: E S TUDY OF THE TRADE - OFF WEIGHT IN THE PROPOSED LOSS
Revised: E S TUDY OF THE TRADE - OFF WEIGHT IN THE COMPOSITE LOSSES

Revised: [Figure]

Original: The proposed method isbased on a  composite loss.
Revised: proposed method is a type of composite loss.

Original: Thus, the trade-off hyper-parameter λ could inﬂuence the ﬁnalperformance.
Revised: Thus, the trade-off hyper-parameter λ could inﬂuence the ﬁnal

Original: Fig.
Revised: Fig.

Original: E.1 shows the performancesof the different composite losses examined inthe paper, with different trade-off hyper-parameter values.
Revised: E. performances the different composite losses examined in different trade- hyper-parameter values we the perfor

Original: While we plot the performances on the test set of Retinal Lesions, the behaviour on the validation set is similar, with the best trade-off parameters reached at the same value for both the validation and test sets. From the curves, we can notice thatCE+ L 1 is more stable due to its gradient properties and stability at the vicinity of 0 (see Fig.
Revised: From the curves, we can notice that L 1 is a better choice than KL divergence for the proposed method because of its better stability. This may relate to its gradient properties and stability at the vicinity of 0 (see Fig.

Original: 2 in the main document).
Revised: 2 in the main document).

Original: Thus, we can use a relatively larger weighting values for CE+ L 1 (best performance achieved when λ = 1 . 0 ).
Revised: Thus, we can use a relatively larger weighting values for CE+ L 1 (best performance achieved when λ = 1 . 0 ).

Original: Observe that CE+ D KL leads to comparable results at its best when λ = 0 . 1 , but it seems more sensitive to the trade-off hyperparameter.
Revised: Observe that CE+ D KL leads to comparable results at its best when λ = 0 . 1 , but it seems more sensitive to the trade-off hyperparameter.

Original: For the loss integrating CE with the Dice bias term, i.e. , CE+DB 1 , we demonstrate that it can achieve performances similar to CE- log( Dice ) , but it drops signiﬁcantly with high weighting values ( λ ≥ 0 . 1 ).
Revised: For the loss integrating CE with the Dice bias term, i.e. , CE+DB 1 , we demonstrate that it can achieve performances similar to CE- log( Dice ) , but it drops signiﬁcantly with high weighting values ( λ ≥ 0 . 1 ).

Original: This might be due to its gradient characteristics.
Revised: This might be due to its gradient characteristics.

Revised: Comparing to the widely suggested composite loss of CE and Dice, our method deliver better performance with the same hyper-parmeter budget.

Original: F P ER - CLASS RESULTS ON C ITYSCAPES TEST SET
Revised: F P ER - CLASS RESULTS ON C ITYSCAPES TEST SET

Original: [Table]
Revised: [Table]

Original: In Table F.1, we report the detailed per-class results on the Cityscapes test set.
Revised: In Table F. [Table] we report the detailed per-class results on the Cityscapes test set.

Original: The proposed methods outperform related losses on both network settings.
Revised: The proposed methods outperform related losses on both network settings.

Original: In term of mIoU, we achieve 68 .
Revised: In term of mIoU, we achieve 68 .

Original: 35% and 70 .
Revised: 35% and 70 .

Original: 11% on Res50-FPN and Res101-FPN, respectively.
Revised: 11% on Res50-FPN and Res101-FPN, respectively.

Original: Compared to the baseline CE, our method yields improvement for most of the minority classes, like fence, train and motorbike, while it shows better adaptability for diverse classes than the Dice loss.
Revised: Compared to the baseline CE, our method yields improvement for most of the minority classes, like fence, train and motorbike, while it shows better adaptability for diverse classes than the Dice loss [Table] [Figure]

Revised: G M ORE QUALITATIVE RESULTS ON THE R ETINAL L ESIONS D ATASET

Revised: Fig.

Revised: F.1 gives more examples from Retinal Lesions Dataset.

Revised: It is shown that our method is able to adapt to the highly variant segmenting regions from small to large, while Dice loss obviously degrades in the case of large regions like the example in the third row.