## 4 Synthetic Data Generation  
### Overview  
The dataset comprising images of actual ballast particles has demonstrated its effectiveness in building and training deep learning models for 2D ballast particle segmentation in field images. Nevertheless, the dataset has the following limitations:* **Limited accessibility. ** Acquiring a high-quality ballast dataset for training robust deep learning models requires access to dedicated in-service railroad sites, high-end equipment, and coordination with specialized departments with domain knowledge. * **Limited scalability. ** The processes of gathering, processing, and annotating field data are time-consuming and labor-intensive. * **Limited flexibility. ** Extracting specific forms of data, such as individual ballast particles within a 3D reconstructed point cloud, can be extremely challenging and may be unattainable from images of actual ballast in real-world scenarios. To overcome these limitations, a synthetic ballast dataset was established with support from Blender 3D modeling software. This supplemental ballast dataset is accessible, scalable, flexible, and capable of augmenting the existing actual ballast image datasets with high-quality synthetic ballast images to enhance the performance of the deep learning models in the task of ballast particle segmentation from field-collected images. The synthetic ballast dataset is structured around \"scenes\" as its primary unit of analysis. Each scene represents an individual sample of ballast aggregate, encompassing all necessary information to generate comprehensive synthetic ballast data and annotations. This methodology obviates the need for the labor-intensive manual labeling process by automating it. The synthetic dataset consists of 120 generated scenes, each containing hundreds of ballast particles. These particles display a range of sizes, shapes, and textures, adhering to an assigned gradation distribution. Each scene is composed of three principal elements: ballast data, ballast labels, and statistical information that characterizes the ballast data (as illustrated in Figure 1). * **Ballast data. ** The ballast dataset provides high-resolution (2500 pixel \(\times\) 2500 pixel) RGB images captured from a top-down perspective under various lighting conditions (Figure 1a-c). It also contains a 3D heightmap (Figure 1d) corresponding to the base plane, a 3D textured point cloud (Figure 1f) encompassing all the ballast particles within the scene, and individual, complete meshes for each ballast particle. Figure 1d visualizes the generated 3D heightmap, where the height of each pixel to the base plane is encoded with a colormap ranging from dark blue to red, where red pixels indicate the highest points in the physical world. The diversity of the synthetic data allows for the emulation of a wide range of commonly used ballast imaging devices, including but not limited to line scan cameras, area scan cameras, and 3D laser scanners. * **Ballast labels. ** Within a generated scene, each synthetic ballast particle is assigned a unique particle ID ranging from 1 to the number of particles in the scene. The label of a synthetic 2D ballast image (2D particle masks) paired with the ballast data is a map from each pixel in the 2D ballast image to its corresponding ballast particle's unique ID. Figure 1e visualizes the 2D particle mask for a top-view RGB image, where the red polygons are the outlines of each ballast particle, and the color of each pixel inside the red polygons encodes its unique ballast particle ID. Similarly, the label of a generated 3D ballast point cloud is a map from each 3D point in the point cloud to its corresponding ballast particle's unique ID. Figure 1g visualizes the label of a 3D point cloud by rendering each point in the point cloud with a unique color encoding its unique ballast ID, indicating that points with the same color belong to the same ballast particle. The gray cuboids in Figure 1g show the 3D bounding box of each ballast particle.