“In this paper, we propose a new loss function that acts as a more effective alternative to previous approaches for dealing with class imbalance. The loss function is a dynamically scaled cross entropy loss, where the scaling factor decays to zero as confidence in the correct class increases, see Fig. 1. Intuitively, this scaling factor can automatically down-weight the contribution of easy examples during training and rapidly focus the model on hard examples. Experiments show that our proposed Focal Loss enables us to train a high-accuracy, one-stage detector that significantly outperforms the alternatives of training with the sampling heuristics or hard example mining, the previous state-of-the-art techniques for training one-stage detectors. Finally, we note that the exact form of the focal loss is not crucial, and we show other instantiations can achieve similar results.

Fig. 1. -
We propose a novel loss we term the Focal Loss that adds a factor (1 - p_\mathrm {t})^\gamma(1-pt)γ to the standard cross entropy criterion. Setting \gamma >0γ>0 reduces the relative loss for well-classified examples (p_\mathrm {t}>.5pt>.5), putting more focus on hard, misclassified examples. As our experiments will demonstrate, the proposed focal loss enables training highly accurate dense object detectors in the presence of vast numbers of easy background examples.
Fig. 1.
We propose a novel loss we term the Focal Loss that adds a factor (1−pt)γ to the standard cross entropy criterion. Setting γ>0 reduces the relative loss for well-classified examples (pt>.5), putting more focus on hard, misclassified examples. As our experiments will demonstrate, the proposed focal loss enables training highly accurate dense object detectors in the presence of vast numbers of easy background examples.

Class Imbalance. Both classic one-stage object detection methods, like boosted detectors [19], [25] and DPMs [20], and more recent methods, like SSD [9], face a large class imbalance during training. These detectors evaluate 104-105 candidate locations per image but only a few locations contain objects. This imbalance causes two problems: (1) training is inefficient as most locations are easy negatives that contribute no useful learning signal; (2) en masse, the easy negatives can overwhelm training and lead to degenerate models. A common solution is to perform some form of hard negative mining [9], [16], [17], [19], [20] that samples hard examples during training or more complex sampling/reweighing schemes [34]. In contrast, we show that our proposed focal loss naturally handles the class imbalance faced by a one-stage detector and allows us to efficiently train on all examples without sampling and without easy negatives overwhelming the loss and computed gradients.

Robust Estimation. There has been much interest in designing robust loss functions (e.g., Huber loss [35]) that reduce the contribution of outliers by down-weighting the loss of examples with large errors (hard examples). In contrast, rather than addressing outliers, our focal loss is designed to address class imbalance by down-weighting inliers (easy examples) such that their contribution to the total loss is small even if their number is large. In other words, the focal loss performs the opposite role of a robust loss: it focuses training on a sparse set of hard examples.

SECTION 3Focal Loss
The Focal Loss is designed to address the one-stage object detection scenario in which there is an extreme imbalance between foreground and background classes during training (e.g., 1:1000). We introduce the focal loss starting from the cross entropy (CE) loss for binary classification1:
CE(p,y)={−log§−log(1−p)if y=1otherwise.(1)
View SourceRight-click on figure for MathML and additional features.In the above y∈{±1} specifies the ground-truth class and p∈[0,1] is the model’s estimated probability for the class with label y=1. For notational convenience, we define pt:
pt={p1−pif y=1otherwise,(2)
View SourceRight-click on figure for MathML and additional features.and rewrite CE(p,y)=CE(pt)=−log(pt).

The CE loss can be seen as the blue (top) curve in Fig. 1. One notable property of this loss, which can be easily seen in its plot, is that even examples that are easily classified (pt≫.5) incur a loss with non-trivial magnitude. When summed over a large number of easy examples, these small loss values can overwhelm the rare class.

3.1 Balanced Cross Entropy
A common method for addressing class imbalance is to introduce a weighting factor α∈[0,1] for class 1 and 1−α for class −1. In practice α may be set by inverse class frequency or treated as a hyperparameter to set by cross validation. For notational convenience, we define αt analogously to how we defined pt. We write the α-balanced CE loss as:
CE(pt)=−αtlog(pt).(3)
View SourceRight-click on figure for MathML and additional features.This loss is a simple extension to CE that we consider as an experimental baseline for our proposed focal loss.

3.2 Focal Loss Definition
As our experiments will show, the large class imbalance encountered during training of dense detectors overwhelms the cross entropy loss. Easily classified negatives comprise the majority of the loss and dominate the gradient. While α balances the importance of positive/negative examples, it does not differentiate between easy/hard examples. Instead, we propose to reshape the loss function to down-weight easy examples and thus focus training on hard negatives.

More formally, we propose to add a modulating factor (1−pt)γ to the cross entropy loss, with tunable focusing parameter γ≥0. We define the focal loss as:
FL(pt)=−(1−pt)γlog(pt).(4)
View SourceRight-click on figure for MathML and additional features.The focal loss is visualized for several values of γ∈[0,5] in Fig. 1. We note two properties of the focal loss. (1) When an example is misclassified and pt is small, the modulating factor is near 1 and the loss is unaffected. As pt→1, the factor goes to 0 and the loss for well-classified examples is down-weighted. (2) The focusing parameter γ smoothly adjusts the rate at which easy examples are down-weighted. When γ=0, FL is equivalent to CE, and as γ is increased the effect of the modulating factor is likewise increased (we found γ=2 to work best in our experiments).

Intuitively, the modulating factor reduces the loss contribution from easy examples and extends the range in which an example receives low loss. For instance, with γ=2, an example classified with pt=0.9 would have 100× lower loss compared with CE and with pt≈0.968 it would have 1000× lower loss. This in turn increases the importance of correcting misclassified examples (whose loss is scaled down by at most 4× for pt≤.5 and γ=2).

In practice we use an α-balanced variant of the focal loss:
FL(pt)=−αt(1−pt)γlog(pt).(5)
View SourceRight-click on figure for MathML and additional features.We adopt this form in our experiments as it yields slightly improved accuracy over the non-α-balanced form. Finally, we note that the implementation of the loss layer combines the sigmoid operation for computing p with the loss computation, resulting in greater numerical stability.

While in our main experimental results we use the focal loss definition above, its precise form is not crucial. In Section 3.5 we consider other instantiations of the focal loss and demonstrate that these can be equally effective.

3.3 Class Imbalance and Model Initialization
Binary classification models are by default initialized to have equal probability of outputting either y=−1 or 1. Under such an initialization, in the presence of class imbalance, the loss due to the frequent class can dominate total loss and cause instability in early training. To counter this, we introduce the concept of a ‘prior’ for the value of p estimated by the model for the rare class (foreground) at the start of training. We denote the prior by π and set it so that the model’s estimated p for examples of the rare class is low, e.g., 0.01. We note that this is a change in model initialization (see Section 4.1) and not of the loss function. We found this to improve training stability for both the cross entropy and focal loss in the case of heavy class imbalance.”
From the above paper, summarize and explain why focal loss can tackle the imabalanced dataset problem with figures and equation shown above. Try use your own words, do not directly use the above sentences.