# A Method for the Calibration of a LiDAR and Fisheye Camera System
## 1 Introduction  
In recent decades, LiDAR sensors have improved their performance, becoming lighter and more economical, and providing accurate distance measurements. While cameras were the most common sensors in the past for autonomous navigation or other applications in robotics, the use of LiDAR sensors has increased in recent years, and the combination of both kinds of sensors provides complementary information that has the potential to address autonomous navigation more safely and efficiently. LiDAR sensors provide accurate distance measurements and are invariant to the lighting conditions of the scene since the scenario is illuminated by the laser sensor itself. However, laser sensors typically have limited resolution, mainly along the vertical axis. Meanwhile, cameras are economical and present low weight, and provide a large amount of information about the environment. Together, these systems enable a more complete and accurate representation of the surrounding environment, which is essential to ensure safety and efficiency in autonomous navigation. While cameras are sensitive to the lighting conditions or seasonal changes, LiDAR sensors are sensitive to the presence of dust or rain, which can affect the quality of the data. The combination of LiDAR and camera data may improve the understanding of the environment in various applications, including the semantic segmentation of point clouds and images, thus facilitating the identification and classification of objects and surfaces in three-dimensional scenes. In addition, object detection may benefit from the fusion of three-dimensional and visual information, enabling more accurate localization and recognition of objects in real time and in challenging operations [1; 2]. Some approaches developed in recent years for point cloud semantic segmentation [3] and object recognition [4] demonstrate that the successful design of methods that fuse information correctly provides clear improvements over single-sensor-based methods. This paper addresses the estimation of the translation and rotation between a fisheye camera and a LiDAR sensor. The precise calibration of the transformation between the reference systems associated with each sensor is key to ensure that the data collected by both sensors are effectively fused, enabling accurate real-time interpretation of the environment. The field of view of pinhole cameras is limited and typically below 90\({}^{\circ}\), often between 40\({}^{\circ}\) and 75\({}^{\circ}\), so the use of fisheye cameras, whose field of view could be up to 180\({}^{\circ}\) or more, is becoming more common. Fisheye cameras have an expanded field of view, which permits a more complete but distorted observation of the environment. However, the calibration of fisheye cameras is more complex than that of pinhole cameras since the fisheye lens introduces non-linear distortions in the image. Pinhole cameras are the most widely used; thus, the methods for the calibration of LiDAR systems and this type of camera are abundant in the literature [5]. However, this article focuses on the use of a fisheye camera, which provides a much wider field of view than a standard camera, enabling a more complete observation of the environment. As described below, the calibration methods in the literature are not applicable to the proposed problem since most of them are designed for the calibration of pinhole cameras. Those methods, specifically designed for fisheye cameras, are not suitable for the calibration of a LiDAR-fisheye system since they require the transformation of the image to a pinhole model, which is not necessary in the proposed method. The main contribution of this paper is the development of a method for the calibration of a LiDAR and fisheye camera system to obtain the translation and rotation parameters between the reference systems of both sensors. The algorithm is based on the use of a simple calibration pattern, which enables relating the 3D position of the pattern directly between the camera and LiDAR systems.