## 4 Evaluation Metrics  
Understanding common evaluation schemes and the metrics to measure performance is necessary for developing and applying deep learning methods. Although the overall accuracy (OA) is a significant index on a minimum unit basis (e.g., pixel, point, voxel, and mesh), it is not a well-defined metric to evaluate model predictions. Accuracy assigns equal cost to false positive and false negative cases, which, in reality, is rare. For example, engineering is known for high safety requirements where false negative predictions can have a crucial impact, while false positives may be neglected. Binary test results can be best visualized in a confusion matrix, as shown in Figure 9. Within the confusion matrix, the capital letters TP/FP refer to the number of positive predictions that are true/false-classified, and similarly, FN/TN refer to the number of negative predictions that are false/true-classified. The sum, \(N\), of all four cells is equal to the total number of classified points. The capital letters PP/PN correspond to the horizontal sum of positive/negative predictions, and vice versa RP/RN correspond to the vertical sum of positive/negative real units. By convention, the lowercase letters, \(tp\), \(fp\), \(fn\), \(tn\) and \(rp\), \(rn\) and \(pp\), \(pn\), refer to the joint and marginal probabilities, and the four contingency cells and the two pairs of marginal probabilities (\(rp+rn=1\) and \(pp+pn=1\)) each must sum to 1 [112]. Figure 7c shows an example of a confusion matrix for multi-class classification. In the multi-class case, FP, FN, and TN cannot be obtained directly from the confusion matrix as in the binary case; instead, they must be determined individually for each class according to the summation scheme in Figure 10.  
Common statistical performance measures can be calculated from the evaluation of a binary classification problem and the per-class discretization. Precision (also known as confidence) denotes the proportion of predicted positive cases that are correctly real positives. Recall (also known as sensitivity) aims to identify all real positive (often called relevant) cases. The F1-score denotes the weighted average (harmonic mean) of precision and recall and is a measure of a test's fidelity, which is usually more useful than accuracy since it requires both precision and recall to have a higher value at the same time for the f1-score to rise, especially if dealing with uneven class distributions [113; 114]. The formulas  
Figure 10: Multi-class confusion matrix summation scheme to determine the following: true positive (TP), false positive (FP), false negative (FN), and true negative (TN). An example is shown for the class _wall_. Note: other references may use a different convention for axes. to calculate precision, recall, and the F1-Score are reported in Equations (1), (2), and (3), respectively:  
\[\text{Precision}=\frac{\text{TP}}{\text{(TP+FP)}}=\frac{\text{TP}}{\text{PP}} \tag{1}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{(TP+FN)}}=\frac{\text{TP}}{\text{RP}} \tag{2}\]  
\[\text{F1-Score}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}} \tag{3}\]  
Based on the confusion matrix kernel, different evaluation metrics were proposed to test the performance of various point cloud understanding tasks, not only in the binary scenario but also in multi-class cases. We assume a set of \(K+1\) classes, where \(p_{ij}\) denotes the smallest instance (e.g., pixel, voxel, mesh, and point) of class \(i\) implied to belong to class \(j\) (\(i=j\) represents _true_ and \(i\
eq j\) represents _false_ classifications). To generalize the implementation, matrix indexing is applied so that \(p_{ii}\) and \(p_{jj}\) represent _true positives_ and _true negatives_, while \(p_{ij}\) and \(p_{ji}\) represent _false positives_ and _false negatives_, respectively [26]. The formulas to calculate the overall accuracy (OA) and the mean class accuracy (mAcc) are reported in Equations (4) and (5), respectively.