# Balanced Loss Function for Accurate Surface Defect Segmentation
## 2 Method
### Loss Imbalance
#### 2.2.2 Easy-Hard Example Imbalance  
Easy-hard example imbalance refers to the situation that easy examples or hard examples have a dominantly large loss value compared to other examples. In the context of segmentation, an example refers to one of the point predictions of an image. Easy/difficult examples are points that have small/large point classification errors, which produce small/large loss values. The difficulty of examples varies with parameter updating, so this concept is restricted to a certain state of the model. When easy-hard example imbalance occurs, the drop of loss does not reflect an equivalent improvement on segmentation accuracy.  
The formation and the harm of easy example imbalance and hard example imbalance are different. Figure 1 is a visualization of different situations that occurred at a certain time point in the training of U-Net by cross-entropy loss on the KolektorSDD2 dataset.  
In most learning tasks, the majority are easy examples. Even if easy examples have a small loss value, the sum can still outweigh other examples due to its large quantity. For example, the second row of Figure 1 displays that when easy examples are the majority and very few hard examples exist, most of the loss value and the gradient is attributed to the easy examples. Other examples near the \"critical\" point have a minor influence. When easy examples altogether have a major contribution to the overall loss, the training is largely influenced by those easy examples on which the model has already been making correct predictions. Penalizing predictions on easy examples have zero improvement on segmentation accuracy.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Defect Name** & **Pixel Ratio** & **Type Ratio** \\ \hline \multirow{4}{*}{Severstal [18]} & class 1 & 49.32\% & 12.64\% \\ \cline{2-4}  & class 2 & 51.77\% & 3.481\% \\ \cline{2-4}  & class 3 & 53.04\% & 72.58\% \\ \cline{2-4}  & class 4 & 50.62\% & 11.29\% \\ \hline KolektorSDD2 [17] & defect & 2.697\% & 10.55\% \\ \hline \multirow{4}{*}{Aluminum Alloy} & corrosion spot & 3.010\% & 42.5\% \\ \cline{2-4}  & corrosion product & 15.73\% & 37.0\% \\ \cline{2-4}  & corrosion pit & 8.103\% & 5.0\% \\ \cline{2-4}  & chemical denudation & 89.24\% & 7.5\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: Label imbalance in surface defect datasets. Pixel ratio is the mean ratio of defective pixels in the images. Type ratio is the ratio of the images in the dataset containing the defect.  
On another hand, the hard example imbalance occurs on a few examples with a large loss value. The third row of Figure 1 demonstrates the consequence of having a small amount of hard examples that produce very high loss. Hard examples take up more than half of the loss value and the gradient. Although fitting hard examples have potential long-term benefit, there is a risk of over-fitting on noisy data. For example, the commonly used focal loss suppresses the influence of easy examples and amplifies the loss of hard examples. When the hard examples consists of noisy data, focal loss will lead to an instable training process. In addition, over-optimization on a small portion of examples provides little improvement on overall segmentation accuracy.  
Intuitively, easy-hard example imbalance can be tackled by limiting the loss value of easy/hard examples within a reasonable range. Our solution is the truncated cross-entropy loss, which is detailed in Section 2.3.2.  
Figure 1: The point-wise loss histogram, loss cumulative histogram, and gradient cumulative histogram in three conditions: balance, easy example imbalance and hard example imbalance. The black line is the “critical” point of the prediction above which the prediction is wrong and or vice versa.