# Calibration of Camera and Flash LiDAR System with a Triangular Pyramid Target
###### Abstract  
Calibration between multiple sensors is a fundamental procedure for data fusion. To address the problems of large errors and tedious operation, we present a novel method to conduct the calibration between light detection and ranging (LiDAR) and camera. We invent a calibration target, which is an arbitrary triangular pyramid with three chessboard patterns on its three planes. The target contains both 3D information and 2D information, which can be utilized to obtain intrinsic parameters of the camera and extrinsic parameters of the system. In the proposed method, the world coordinate system is established through the triangular pyramid. We extract the equations of triangular pyramid planes to find the relative transformation between two sensors. One capture of camera and LiDAR is sufficient for calibration, and errors are reduced by minimizing the distance between points and planes. Furthermore, the accuracy can be increased by more captures. We carried out experiments on simulated data with varying degrees of noise and numbers of frames. Finally, the calibration results were verified by real data through incremental validation and analyzing the root mean square error (RMSE), demonstrating that our calibration method is robust and provides state-of-the-art performance. calibration; camera; LiDAR; point cloud 2021  
115822020103990/app1102058212 December 2020  
6 January 2021  
Published: 8 January 2021  
**Published: 8 January 2021**  
complexity, while challenging the reliability of the moving parts under long-term use. A solid-state LiDAR is usually a radar with no moving parts at all. It mainly includes micro electro mechanical systems (MEMS) LiDAR, optical phased array (OPA) LiDAR, and flash LiDAR. MEMS LiDAR do not completely cancel the mechanical moving structure [10]. The mechanical structure is integrated into a small silicon-based chip by MEMS technology, inside which there is a rotatable MEMS micro-mirror, through which the transmitting angle of a single transmitter is changed. OPA LiDAR consists of an array of light-emitting elements where the phase can be manipulated [11]. On top of that, the entire system is fabricated on a single chip using a silicon photonics platform. Flash LiDAR illuminates the entire scene in a single shot and detects the light reflected from the scene with an array of detectors [12]. As there are no moving parts, solid-state LiDAR systems can be built with enhanced reliability. Camera and LiDAR system calibration consists of intrinsic calibration and extrinsic calibration. The intrinsic parameters of the camera are related to the characteristics of the camera itself, including focal length and size of pixels. In most literature, it is accepted that the intrinsic parameters of the camera are calibrated in advance. The intrinsic parameters of LiDAR sensors are calibrated in the factory. The extrinsic calibration is to find the rotation and translation relationship between the camera and LiDAR. In this paper, we present a novel method to calibrate the intrinsic parameters of the camera and extrinsic parameters of the system. Our work has four characteristics differentiating itself from prior research:  
* The traditional calibration method has limited accuracy because of low vertical resolution of 3D mechanical scanning LiDAR. For instance, Velodyne-64 LiDAR can only measure 64 channels vertically, and generate sparse point clouds of the environment. Our method deals with dense point cloud, thus the result has high accuracy. We first apply flash LiDAR to acquire dense point clouds both in the horizontal direction and vertical direction to improve calibration accuracy. * We design a novel calibration target made up of an arbitrary triangular pyramid, with three chessboards on it. The sizes of the triangular pyramid are unknown, and it can be unsymmetrical, hence the manufacturing of the pyramid has no effect on the accuracy. Our method only employs planes of triangular pyramid from camera images and LiDAR point clouds in calibration.