## 1 Introduction
Multiple surveillance tasks, especially site monitoring, are performed already with the help of image-based computer vision. Adding 3D data has the potential to significantly increase information density and the level of automation. Below, we list practical examples of applications where learning-based point cloud segmentation can be beneficial to quality, efficiency, and safety in construction. Shape and position control in concrete casting and precast installation requires finding accurate object/segment boundaries [7]. Volume calculations in the construction and demolition (C&D) phases can be automated from laser scanner, UAV, or satellite syn  
Figure 1: Concept of point cloud segmentation with learned building classes. **Right**: raw point cloud with color information. **Left**: semantically enriched point cloud with predicted class labels output by a neural network. All points with the same predicted class are colored in the same color. thetic aperture radar (SAR) footage with the help of semantic segmentation [8; 9]. Reverse engineering digital twins of existing buildings (as built and scan to BIM) requires assigning element classes to scan data to reconstruct element geometries [10; 11]. Site progress monitoring can be automated through 3D component recognition and comparison with a planned database [12; 13]. In addition to many surveillance applications, semantic segmentation of real-time sensor data as a subfield of computer vision is one crucial component for guiding (partially) autonomous robots. We expect robots to take over even more construction tasks in the future, but sites are constantly changing environments and scattered with moving obstacles. Thus, real-time scene understanding will be mandatory. Price drops for sensor hardware have ultimately made it economically feasible to install the necessary hardware. However, software development is needed to utilize the information, as the following examples of non-learning approaches illustrate. [PERSON] et al. [14] used colored point cloud segmentation to estimate the position of precast concrete rebar elements. With appropriate training data and a learning-based approach, this can be applied to all types of construction equipment that robots interact with. Automated guided vehicles (AGVs) require vehicle guidance and collision avoidance. This can be performed with cameras, but imaging sensors rely on good lighting conditions. This is not always the case on construction sites, and light detection and ranging (LiDAR) can be a solution for this [15]. Finally, semantic-enhanced sensor data can dramatically improve the overall safety of construction sites where humans and machines work together. [PERSON] and [PERSON] [16] used ray tracing to calculate the blind spots of construction equipment, but the algorithm cannot distinguish between multiple obstacles. Machine safety decision making can be improved with learning-based point cloud understanding. The scope of this paper is to focus on learning-driven PCSS with deep neural networks, which can learn deep geometric connections of complex structures by transforming the feature space into a high-dimensional representation problem. The early adopters, PointNet [17] and PointNet++ [18], have proven that deep learning techniques are fundamental for many computer vision tasks in the sparse 3D point cloud domain. Multiple strategies were published in the following years, transferring approaches from image-based computer vision and natural language processing to point cloud processing (convolution-, recurrent-, graph-, and transformer-based methods). Within the civil engineering domain, only a few papers covered the use of deep learning to perform PCSS. Some remarkable publications automated as-built BIM generation [19; 20], building reconstruction [21; 22], and bridge part semantic segmentation [23; 24]. Nevertheless, up to now, there is no settled trend for the civil engineering branch, and most approaches only feature outdated PointNet network architectures.