## 4 Data Analysis and Evaluation by AI  
Many of the measurement systems presented here generate large amounts of data, such as point clouds in clearance measurement and images from measurement cameras. These need to be evaluated quickly and comprehensively. Automated analysis of the inspection data can reduce the time required and the use of human resources. Machine learning algorithms can be applied to reduce the time and cost of detecting defects or anomalies. In recent years, the evaluation and analysis techniques have also proven to be effective in the field of artificial intelligence (AI) in connection with the rail industry. AI is defined as a computerized system that is capable of performing physical tasks and cognitive functions, solving various problems or making decisions without explicit human instruction [88]. Ref. [89] shows that AI research in rail infrastructure is mainly concerned with the track system (77%); civil structures (14%), substructures (5%) and catenary system (4%) receive less attention. However, papers from the areas of railway signaling technology, rolling stock, and operations were excluded in this research. AI can be used to solve various clearly defined tasks. Classification and segmentation involve the task of distinguishing between different defects or objects, with segmentation assigning the optical data of the point cloud or image to an object class. Even major rail operators have already investigated the benefits of AI in their sector. For example, SIGNON ([[https://signon-group.com/](https://signon-group.com/)]([https://signon-group.com/](https://signon-group.com/)) (accessed on 12 September 2024)), a fully owned subsidiary of Deutsche Bahn AG, has developed a neural network for recognizing infrastructure elements based on image data. Objects such as boundary signs,  
Figure 11: UAV with detected insulators on the overhead line. The green boxes show the detected insulators. Source: DLR/[83]. catenary masts, signals, and signs are recognized. Using data from Deutsche Bahn AG's LIMEZ III measurement train in 2020, Fraunhofer IPM has implemented a demonstrator that uses AI to classify or segment object types in images and then project them back into the point cloud (see Figure 12). In this way, a segmented point cloud is generated in which each point is given an object label. Reliable results were achieved for manually matched point clouds. For an automated back projection, more parameters of camera calibration and orientation need to be known. Network Rail ([[https://www.networkrail.co.uk/](https://www.networkrail.co.uk/)]([https://www.networkrail.co.uk/](https://www.networkrail.co.uk/)) (accessed on 12 September 2024)), owner of the property assets of the former British Rail, uses AI and video images to find and then remove forgotten scrap at the side of the track. They are also working with the University of Sheffield to use a camera and AI to identify contaminants such as leaves (generally referred to as 'black layer') on the rails and freeze them with dry ice particles in a supersonic air stream to clean the railhead [90; 91]. In the following section, we only deal with AI analyses of data from optical sensor technology. A survey on using AI in the entire railroad sector can be found here [89; 92; 93]. There are several defects that affect railway networks. Defects of the rail area include defects of the rail surface, fasteners, ballast, and sleepers. In order to detect defects on the track surface, ref. [94] has merged two Deep Learning (DL) models with each other. Features of the two models are combined to achieve a higher accuracy. Using classical image processing such as contrast equalization, the images are pre-processed and achieve a classification success of 97.10%. To detect so-called \"squats\", which can lead to rail fractures, on the rail surface, ref. [95] uses a deep convolutional neural network (DCNN), an N-step ahead prediction model for defect severity and crack growth analysis, and a Bayesian inference model for failure-probability estimation.