## 2 Materials and Methods
### Synthetic Data Generation  
The synthetic dataset was built to emulate the environment and technical specifications of the real world dataset. Figure 2 summarizes the synthetic data creation process. Blender 2.80 was used for the creation of the different scenes, which were rendered using the Cycles rendering engine. All scenes were built by hand to represent real construction sites, depicting a varied set of scenes. Using a physically-based renderer allows for more realistic images than game engines, but also increases render time. For the comparison of real and synthetic data, the two real scenes were recreated (see Figure 1c,d). Overall, 8 different scenes with a total of 3835 frames and 32 tracked subjects were created for this work. Table 1 lists all created scenes, with sample frames shown in Figure 3. Similar to the real world dataset, the synthetic dataset incorporates different lighting and weather conditions, ground surface types, and surrounding clutter. As for the real dataset, this increases the variation of the data samples as well as modeling the most common conditions and construction site types. Figure 1: Comparing the different environmental scenarios of the real world dataset in (**a**,**b**) to their 3D generated counterparts in (**c**,**d**), respectively. Ground truth labels are marked as green rectangles. The lighting conditions include sunny, cloudy, and rainy. The incidence angle of the sun varies to simulate the different lighting conditions at different times of the day, including long shadows in the morning and evening. Overex emulates the effect present in the real world dataset (compare Figure 1a,c). The virtual workers wear different safety vests (in either green, yellow, or orange) and different helmets (in either white or yellow) to ensure variability in the dataset. To ensure that the pose and movement of the virtual workers are realistic, motion capture data from the Carnegie Mellon University Motion Capture Database was used. The motion capture data was rigged onto the construction worker model to create realistic animations. Virtual workers conduct multiple different actions in the created scenes, including walking, picking up objects, interacting with each other, or repairing or using equipment. Several other moving objects are added, such as cars, forklifts, and construction material. Dangerous situations are also included, where construction material is lifted above workers' heads (see Figure 3a,d). Workers may also be partly occluded by objects. Ground truth labels were extracted automatically from the 3D data. For this, a spherical hull was placed, which encompasses the head and part of the shoulders of a virtual construction worker. \begin{table}
\begin{tabular}{c l l l l} \hline \hline
**Scene** & **Description** & **Weather** & **Workers** & **Frames** \\ \hline
1 & \
\begin{tabular}{l} Recreation of real scene 1 \\ \end{tabular} &
\begin{tabular}{l} Sunny, \\ overexposed \\ \end{tabular} & 4 & 250 \\ \hline
2 & \
\begin{tabular}{l} Recreation of real scene 2 \\ \end{tabular} &
\begin{tabular}{l} Sunny \\ \end{tabular} & 4 & 250 \\ \hline
3 &
\begin{tabular}{l} Street, workers walk in a straight line, a car passes \\ \end{tabular} & Cloudy & 4 & 500 \\ \hline
5 &
\begin{tabular}{l} Construction site, one worker working on a forklift, \\ two workers walk past, a car drives past, one \\ worker walks in circles \\ \end{tabular} & Sunny & 4 & 600 \\ \hline
6 &
\begin{tabular}{l} Construction site, two workers working on a forklift, one worker working on a car, one worker \\ picking up items and walking around \\ \end{tabular} & Cloudy & 4 & 661 \\ \hline
7 &
\begin{tabular}{l} Construction site, one worker directs a driving \\ forklift, two workers sit down to chat, one worker \\ saws \\ \end{tabular} & Sunny, late & 4 & 662 \\ \hline
8 &
\begin{tabular}{l} Same as scene 7, but with a different ground type \\ and lighting \\ \end{tabular} & Sunny, slightly overexposed & 4 & 662 \\ \hline \hline \end{tabular}
\end{table}