## 3 Public Datasets for Scene Understanding Methods  
Efficient training in machine learning and the applicability of deep neural networks heavily depends on the available training data. While the performance of most shallow machine learning algorithms converges at some point, deep learning can increase performance relative to the amount of training data [32]. Creating huge datasets for the individual is not always feasible. Most protagonists in the machine learning community, therefore, necessarily rely on the use of freely available data. However, famous achievements demonstrate that the need for public datasets and the need for people to work together does not have to be a burden but offers great opportunities for accelerated progress. The MNIST dataset, which features 70,000 gray-scale images of handwritten digits, was first published in 1999 [33] under a free-to-use, copy, share, and adapt license (Creative Commons license: CC BY-SA 3.0) and is still widely used as a benchmark for evaluating the performance of convolutional neural network (CNN) models [34]. The ImageNet database, which was introduced in 2012 ([PERSON] et al., 2015), has been instrumental in driving the development of deep learning algorithms in the community to groundbreaking innovation [35]. Three-dimensional point cloud semantic segmentation (PCSS) is a fast-growing field of research, and the amount and variety of open-source datasets to train, validate, and compare new model architectures are almost exclusively credited to the car industry and autonomous driving [36; 37; 38; 39; 40]. Three-dimensional datasets designed for computer vision scene understanding applications in the construction industry are rare and deviate in pure volume from their automotive counterparts. A literature review was conducted to summarize existing civil engineering-related open-source datasets and make them more visible for future research. The review was performed with a combination of search items including \"3D\", \"dataset\", \"point cloud\", \"scene understanding\", \"semantic segmentation\", and \"instance segmentation\", which revealed a range of different datasets. In total, 52 datasets were found that met the authors' criteria. To the best of the authors' knowledge, this list is the most complete database of construction-related 3D datasets for computer vision and scene understanding. A tree structure is presented in Figure 2 to visualize the extent of combinations in the wider scope of 3D scene understanding. The tree structure shows how certain sensors, data types, and data formats are commonly paired for given use cases in the literature. The main scene understanding tasks are arranged as use cases on top of the tree. The layers represent certain objectives, attributes, and methods that we extracted from the reviewed datasets. The connecting edges show the configurations found within the papers in the following Table 1. This table includes a comprehensive compilation of a wide range of freely available datasets that are relevant for different 3D point cloud scene understanding tasks in the scope of the construction industry. We categorized the available data by its binary data type of acquisition [_real-word; synthetic_], the content of the dataset [_scene type_], utilized hardware [_sensors_], the dataset file format {_representation_}, the intended use case {_application task_}, and the available point cloud features {_features_}. In addition, the number of used semantic annotation classes is given. The presented datasets consider different scene types and were recorded with different sensors to capture 3D spaces. A quantitative evaluation of the look-up table for scene types and sensors is given in Figure 3. Sub-figure (a) shows the occurrence frequency of each scene type in our database. Sub-figure (b) breaks down which sensors were used to capture the scene types. As shown, some included datasets feature dynamic urban scenes from autonomous driving (D). We considered these datasets for two reasons.