Calibration of a Vehicle Camera System with Divergent Fields-of-view in an Urban Environment

Multi camera systems are used in modern cars for environment perception. To combine information extracted from different cameras, their relative pose has to be known. Therefore, a multi camera system can be calibrated. In this contribution, a calibration approach using the structure-from-motion (SfM) method is described. From images of urban structures, 3d points are calculated using SfM. These 3d points are used as reference to estimate the pose of the vehicle cameras. Test images are taken in a car park with a DSLR camera, together with two action cameras mounted in a test vehicle recording the car park. More than 26,000 3d points representing the car park can be obtained. With these points, the poses of the vehicle cameras are estimated successfully. The distance between the vehicle cameras shows the highest deviation from laser reference measurements for images showing a low point density area of the car park. 1 Single and system camera calibration The road the automotive industry is driving down currently is characterized by autonomously driving cars. To allow advanced driver assistance systems to control a car reliably, they need information about the car’s environment. To capture this information, cars can be equipped with several cameras observing the environment. As the relative position of environment objects to the car is important, the positions and orientations (pose) of the vehicle cameras relative to the car have to be known. To determine the poses of the vehicle cameras, a system calibration can be performed. To keep costs low, the number of cameras is kept small, too. 1 Technische Universität München, Photogrammetry & Remote Sensing, Arcisstraße 21, D-80333 München, E-mail: [alexander.hanel, stilla]@tum.de Fig. 1: Top-down view on a car, whose cameras with divergent fields-ofview (in red) can be orientated using 3d points (blue) generated for the calibration environment. 37. Wissenschaftlich-Technische Jahrestagung der DGPF in Würzburg – Publikationen der DGPF, Band 26, 2017 161 So, their fields-of-view are typically non-overlapping, making stereo camera calibration with the same planar calibration pattern seen in all cameras impossible. Due to their wide field-of-view, so called “action cameras” can cover a huge part of the car’s environment despite their small number. Furthermore, a removable mount available for such cameras on the car windows with a suction pad can avoid in Germany problems with the car registration caused by additional “built-in” installations in the car. Therefore, it has to be considered, that the wide field-of-view might cause large image distortions requiring reliable single camera calibration. In addition, the mounting on the windshield might not be rigid over time caused by mechanical movements during a car drive. To check the poses again and again, the system calibration has to be repeated from time to time. To perform the calibration, a suitable calibration environment has to be selected. This environment has to be large enough to contain cars. Therefore, calibration laboratories are typically not feasible. Urban environments, as for example car parks, are large enough. As it might not be allowed by the owner of the car park and to keep costs and time effort low, the calibration procedure should work without manually placed reference marks. In addition, the number of placed reference marks is typically limited due to the available space. The number of reference marks extracted automatically using image feature descriptors can be higher. In contrast, it is not possible to provide 3d reference information for automatically extracted reference marks in advance. All three mentioned tasks, single camera calibration, system calibration and generation of 3d reference information can be performed with the structure-from-motion method (Fig. 1). Therefore, in this contribution, a structure-from-motion software toolbox will be used to complete these tasks.