# Balanced Loss Function for Accurate Surface Defect Segmentation
## 3 Experiments
### Quantitative Evaluations
#### 3.2.2. Severstal  
We report the segmentation performance of different networks trained by the balanced loss and other counterparts in Figure 9. Most of our findings in KolektorSDD2 are still true in Severstal. The balanced loss outperforms other functions significantly. However, Dice loss or focal loss only has marginally advantageous CE and WCE. What makes this difference from the results on KolektorSDD2 is that Severstal has more salient (about 50% of the image area on average) and more contiguous defects. The training is less influenced by hard examples and the weaker label imbalance, which reduce the advantage of Dice loss and focal loss. For the same reason, skip-connection equipped networks (U-Net and DeepLabV3) have less advantage over others. Because the defects are salient, the model needs large context information to make point predictions. Figure 10 shows segmentation predictions on test examples of Severstal. Figure 8: KolektorSDD2 test examples semantic segmentation result of U-Net trained with the balanced loss. Figure 9: Mean IoU comparison of the balanced loss function and other loss functions on various networks trained on Severstal. To analyze the impact of label imbalance on the model, the class-wise IoU of trained U-Net is presented in Table 3. Classes with more instances, such as defect type class 3 that appears on 72.58% images, tend to have better segmentation accuracy, while the accuracy on less represented classes, e.g., class 2 of 3.481% images, is worse. However, the balanced loss achieves significantly higher performance on class 2, and it displays the effectiveness of the balanced loss on solving label imbalance problems. In addition to class 2, others classes show minor improvement from using focal loss or Dice loss. The result suggests that using focal loss or Dice loss for the segmentation of a salient surface defect is not very effective. The sensitivity of the hyper-parameters of the balanced loss function is reported in Table 4. The \(\beta\) factor of dynamical class weighting varies in \(\{0.0,0.9,0.99,0.999\}\). When \(\beta=0\), the weight is always 1.0 (see Equation (1)), and \(M_{class}\) is ineffective. In this experiment, the interval \([\epsilon_{inf},\epsilon_{sub}]\) of the truncated cross-entropy loss (Equation (5)) always has 0.5 as the center. Only one variable \(\Delta\epsilon\in(0,0.5]\) will decide the interval by letting \(\epsilon_{inf}=0.5-\Delta\epsilon\) and \(\epsilon_{sup}=0.5+\Delta\epsilon\). When \(\Delta\epsilon=0.5\), the interval is \([0,1]\), and the truncated cross-entropy loss degenerates to the normal cross-entropy loss. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Truncated CE \(\Delta\epsilon\)**} & \multicolumn{4}{c}{\(\beta\) **Class Weighting**} \\ \cline{2-5}  & **0.0** & **0.9** & **0.99** & **0.999** \\ \hline
0.1 & 0.73 & 0.73 & 0.82 & 0.79 \\
0.2 & 0.70 & 0.70 & 0.87 & 0.86 \\
0.3 & 0.70 & 0.69 & 0.86 & 0.83 \\
0.4 & 0.69 & 0.70 & 0.78 & 0.79 \\
0.5 & 0.60 & 0.62 & 0.65 & 0.65 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Mean IoU of U-Net trained on Steveral with varying truncated cross-entropy loss thresholds \(\Delta\epsilon\) and class weighting factor \(\beta\). Figure 10: Several test examples semantic segmentation result of U-Net trained with the balanced loss. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Cls1** & **Cls2** & **Cls3** & **Cls4** & **mIOU** \\ \hline CE & 0.72 & 0.42 & 0.76 & 0.69 & 0.65 \\ WCE & 0.60 & 0.44 & 0.75 & 0.59 & 0.59 \\ Focal \(\beta=2\) & 0.68 & 0.61 & 0.82 & 0.69 & 0.70 \\ Focal \(\beta=4\) & 0.78 & 0.63 & 0.80 & 0.79 & 0.75 \\ Dice & 0.67 & 0.59 & 0.82 & 0.67 & 0.69 \\ \hline Balanced Loss & 0.82 & 0.76 & 0.95 & 0.89 & 0.87 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Class-wise IoU and mIoU of U-Net trained on Steveral with different loss functions. The result shows that the optimal choice of \(\beta\) is 0.99. For \(\beta=0.9\), the performance approximately equals that of \(\beta=0\).