# Investigating a Toolchain from Trajectory Recording to Resimulation
## 2 Material and Methods
### Recordings and Trajectory Extraction  
To record the movement of the traffic participants in the scenery, a full-frame Sony Alpha 7R (Sony Corporation, Bangkok, Thailand) camera with a fixed focal length lens (Sony FE35F1.8) was mounted on the roof of Fraunhofer IPM. From the rooftop, it was not possible to film the scenery in a top view; therefore, a laser scan of the T-junction was made. We filmed the different scenarios with a frame rate of 100 fps and an image size of 1920 \(\times\) 1080 pixels to detect the trajectories. With the help of a laser scan and the homography, the images were projected onto the street plane. In order to overlay the camera recordings with a 2D top-down image of the laser scan recordings, the video output has to be distorted. With OpenCV, this can be reached by computing the homography matrix for an image and a planar surface or a reference image of the scene in orthographic projection [30]. Points that display the same real-world position can be marked with their respective pixel coordinates and will feed the computation of the matrix. At least four fixed points are needed to successfully compute the matrix, but more points increase the accuracy of the final image projection. Therefore, 55 points are used for the transformation. For the object detection, we chose a Regions with Convolutional Neural Networks features (R-CNN) object detector based on the MobileNetV2 [31] CNN, which is pre-trained on a 1000-class ImageNet dataset [32]. MobileNetV2 is a very efficient and versatile network architecture and can easily be fine-tuned to a specific use case. Since our model targets a very specific scenario with a fixed camera perspective and only one type of car to detect, it was fine-tuned using additional training data for cars and pedestrians obtained from video images. The system needs to be calibrated once to adjust for boundary conditions, such as cars driving into the street scene. As only the center point of the detected objects is used to extrapolate the trajectories, a car that is not fully visible yet will output a shifted center point, which would be interpreted as a change in velocity. To correct the shifted data points, we use the static character of the system. First, we simply cut out the first few points where the center point of a viewed car is not actually  
Figure 2: The three scenarios depicted on the T-junction. A car interferes with another road user, namely (1, red) a breaking car in front, (2, blue) a car in the LTAP maneuver, and (3, green) a crossing pedestrian. The camera position is also presented.  
visible. Afterward, we can calculate how far the center point is off from the ground truth by sampling video data. Since the car length and camera angle do not change, the error is the same in every video. We can simply calculate a common correction factor and adjust the data points.  
After the object detection, trajectories for every car and pedestrian within the scenario can be extracted by tracking the objects in the projected street plane. Several objects, e.g., trees and street signs, within the scene can disturb this tracking. To adjust for these influences, we used the median between the detected road user center points of ten consecutive frames, so we ended up with 10 points per second. The final resulting trajectories are used to derive the speeds of the investigated traffic participants.