# Automated 3D Image Processing System for Inspection of Residential Wall Spalls
## 2 Materials and Methods
### Randla-Spall
#### 2.3.4 Losses  
The total loss of the RandLA-Spall model contains the following three components: the weighted cross-entropy loss function \(L_{WCE}\), the Lovasz-softmax loss function \(L_{LZS}\), and the consistency constraint loss function \(L_{cc}\).  
To address the class imbalance in the dataset, a weighted cross-entropy loss function and a Lovasz-softmax loss function are employed [38]. The weighted cross-entropy loss function assigns different weights based on the proportion of each class, thereby improving its segmentation accuracy. The formula for the weighted cross-entropy loss function can be expressed as follows (16):  
\[L_{WCE}=-\frac{1}{N}{\sum_{i=1}^{N}{\sum_{j=1}^{C}{w_{j}y_{ij}log(y_{ij})}}} \tag{16}\]  
where N is the number of input points in a single batch, \(y_{ij}\) is the probability of indicating that the true category of the ith point is j, \(y_{ij}\) is the probability of the ith point belonging to category j as predicted by the model, \(w_{j}\) is the weight of the jth category, and C is the number of categories.  
While the effectiveness of the weighted cross-entropy loss function has been proven in semantic segmentation, there remains room for improvement in the predictions. Such improvements are particularly crucial for objects that occupy a small proportion of the space, as RandLA-Spall often struggles to accurately segment these using the weighted cross-entropy loss function. Although the spall is clearly evident in the input point cloud data, it typically represents a very small proportion. To address this issue, the Lovasz-softmax loss function was implemented in RandLA-Spall. Introduced by Berman in 2018 [38], this loss function is designed to optimize the mIoU and has proven its effectiveness on smaller objects. The functional formula for the Lovasz-softmax loss function is expressed as (17):  
\[L_{LZS}=\frac{1}{|C|}{\sum_{j=1}^{N}{\overline{\Delta J_{j}}(m(j))}} \tag{17}\]  
where \(\overline{\Delta J_{j}}\) is [PERSON]'s extended \(\Delta J_{j}\) approximating the Jaccard index j of the class, and \(\overline{\Delta J_{j}}(m(j))\) denotes the Jaccard coefficient \(m(j)\).  
Furthermore, to measure and constrain the discrepancies between the predicted probabilities output by the dual-branch module, the Jensen-Shannon (JS) divergence is employed to compute a consistency constraint loss. This approach aims to effectively increase the training samples and enhance the model's robustness while preventing inconsistent feature representations learned by the two branches of the model, potentially leading to a degradation in model performance. The consistency constraint loss function formula can be expressed as (18):  
\[\begin{array}{c}L_{cc}=\frac{1}{2N}\sum_{i}^{N}JS(P_{A}\|P_{B})\\ =\frac{1}{4N}\sum_{j}^{N}\sum_{j}^{C}KL\Big{(}P_{Aij}\Big{\|}\frac{P_{Aij}+P_{Bij }}{2}\Big{)}+KL\Big{(}P_{Bij}\Big{\|}\frac{P_{Aij}+P_{Bij}}{2}\Big{)}\\ =\frac{1}{2N}\sum_{i}^{N}\sum_{j}^{C}P_{Aij}log\frac{P_{Aij}}{P_{Aij}+P_{Bij}} +P_{Bij}log\frac{P_{Bij}}{P_{Aij}+P_{Bij}}\end{array} \tag{18}\]  
where KL denotes the Kullback-Leibler divergence. \(P_{Aij}\), \(P_{Bij}\) represent the probabilities that the input data point \(p_{i}\) belongs to class J, predicted by the original and enhanced modules, respectively.  
Therefore, the consistency constraint loss, weighted cross-entropy, and Lovasz-softmax loss function are combined to achieve better model segmentation performance. The model loss function formula can be expressed as (19):  
\[L_{sum}=L_{WCE}+L_{LZS}+L_{cc} \tag{19}\]