## 3 Public Datasets for Scene Understanding Methods
### Why Class Imbalance Is a Problem for Deep Learning  
The availability of datasets that are designed for semantic segmentation tasks, applicable in the construction industry, and acquired from real-world scenes are limited and often characterized by a lack of object class balance. To achieve optimal performance and avoid overfitting on individual classes, deep learning models must be trained on training sets that are as balanced as possible. The evaluation of the well-adopted S3 DIS dataset [31] in Figure 5 shows that, in the real-world example, wall instances are heavily over-represented in common indoor scenes. In the case of educational and office areas, a predominant amount of site-specific furniture can be found. In the S3 DIS example, chairs dominate the object classes. A deeper examination of the per-point level in Figure 6 reveals that chair instances often appear in the dataset but occupy only a small amount of data points. The situation is opposite for both the structural elements ceiling and floor, as these elements are single connected large segments with a wide surface area. Comparing the number of scan points by their semantic labels shows that only three classes (_wall, ceiling,_ and _floor_) occupy about 63% of the training data (85% of the building component objects), while beam and column, but also window and door classes, are marginalized. If a class imbalance exists within training data, learners typically over-classify the majority group due to its increased prior probability. As a result, the instances belonging to the minority group are misclassified more often than those belonging to the majority group [90]. Dataset imbalances of this form are commonly referred to [91] as follows:  
_Intrinsic imbalance_, which is caused by the naturally occurring frequency of data, e.g., rooms usually have four walls but only one door and measurable damage in sensor data streams only occur in very rare catastrophic events. _Extrinsic imbalance_, which is caused by external factors, e.g., ceilings did not get scanned due to camera viewing angles, and most buildings' architecture is biased by a country's certain style. The mechanism behind the poor network adoption for subordinate classes can be explained by [PERSON] et al. [92]. Within the training process of feedforward networks, they could show that the backpropagation (BP) gradient vector lengths are proportional to the sizes of the training sets. In standard BP, the sum of all gradient vectors corresponds to the direction to follow to change perceptron weights. In imbalanced scenarios, themajority class gradients dominate the net gradient vector, which can result in a moving direction pointing upwards for minority classes. Consequently, standard BP will make major improvements in reducing the net error in the first steps and will likely get stuck in a slow mode of error reduction. Besides poor convergence behavior, class imbalance can lead to further problems. In the case of strong object variation, it can happen that the model does not see enough data on the underrepresented class to classify this class correctly, even if the dataset seems extensive and detailed. For performance evaluation, some metrics, such as accuracy, might mislead the analyst with high scores that incorrectly indicate good performance [90]. Finally, class imbalance makes it difficult to create appropriate validation sets if the sample pool to choose from is small. Nevertheless, creating well-balanced datasets is not just the challenge of balancing the numbers. [PERSON] [93] concluded, through an experiment with an artificial dataset, that a system's sensitivity to training data imbalance increases with the degree of complexity of the system and that non-complex linear separable domains do not appear sensitive to any amount of imbalance.