# Searching for the Ideal Recipe for Preparing Synthetic Data in the Multi-Object Detection Problem
## 2 Materials and Methods
### Synthetic Data Generation  
In the process of generating synthetic data, a primary consideration was given to ensuring diversity in the images. The Unreal Engine 5 was chosen as a guarantee of high data quality. Unreal Engine provides unparalleled graphics quality, advanced physics, and world-building utilities, thereby expediting work processes and elevating the overall quality of generated datasets. Unreal Engine 5 offers sophisticated tools for managing physics, creating realistic human simulations, and controlling object behavior within a scene. It empowers developers to achieve industry-leading graphics quality. The 3D models of classes listed in Table 2 were created in the Blender environment with various materials (Figure 3). To retrieve detections of specific generated objects, the Unreal GT library [36] was employed. This library enables the direct extraction of detections from crowd simulations (Figure 4d), a tool built for Unreal Engine, during the synthetic data generation process. As mentioned above, there are three common ways of generating synthetic images for object detection. The Img method consists of adding background-less images to the background image. Its main feature is, generating hundreds of images in a short time. The problem lies in varying the pose of an object, which is crucial while learning more complex shapes. Obj method uses the possibilities given by modern 3D graphic engines and is an extension of an Img method, but instead of using flat images of objects to be learned, it uses 3D models rendered on a random background. The main advantage of such an approach is \begin{table}
\begin{tabular}{l c c} \hline \hline
**Class Name** & **Total/Train/Val/Test** & **COCO** \\ \hline Bicycle & 3236/2268/651/317 & Yes \\ Trolley & 3249/2262/651/336 & No \\ Wheelchair & 3236/2270/654/312 & No \\ Box & 3277/2280/627/370 & No \\ Suitcase & 3281/2290/674/317 & Yes \\ Bag & 3251/2270/671/310 & Mix \\ \hline \hline \end{tabular}
\end{table} 
Table 2: A summary of the amount of data for individual classes used in research (Total), with division into training (Train), validation (Val), and test (Test) sets. Additionally, information about the class’s availability in the COCO benchmark is included. Figure 3: View of 3D models used in synthetic data with sample materials. that an object is being seen from multiple views, which provides more information during the learning process. This is the reason why a decision was made to skip the simple Img approach and use Obj instead. The Sim method is based on a dedicated simulation, and its main power is allowing us to recreate the destination environment, which is the reason why it is being so often used in unusual problems. The main disadvantage of Sim is the difficulty in varying the environment, which is in many cases very time-consuming. To overcome this problem, the 3 rd method (Square) was developed, which is a mixture of Obj and Sim methods. The last mentioned method is the mix of images from Obj, Square, and Sim approaches, which provide a rich and diverse dataset. A detailed summary and description of general strategies of data generation are given below:  
* Realistic city crowd simulation (Sim: Figure 3(a))--Features a realistic crowd simulation in an urban setting with moving people, vehicles, weather conditions, and proper lighting effects ensured by various sources of light positions. The scene includes 3D buildings and additional elements like paper, waste bins, letter boxes, and trees. Large objects on sidewalks are registered and marked, and cameras move around objects with randomized positions. Such an approach results in a scene with numerous objects blocking each other, situated at a relatively large distance from the camera. * Static square with changing background (Square: Figure 3(b))--Takes place in a static square with a changing background, where a plane contains randomly placed objects such as benches, lampposts, trees, garbage bins, and static cars, along with pedestrians.