# A Method for the Calibration of a LiDAR and Fisheye Camera System
## 2 Materials and Methods
### Three-Dimensional LiDAR Coordinates  
The LiDAR itself provides the three-dimensional coordinates of the points in space, but it is not known exactly which points belong to the calibration rectangle. The edges and corners of the rectangle have to be extracted from the LiDAR point cloud. The algorithm proposed to extract the edges and corners of the calibration rectangle based on the LiDAR point cloud follows the structure of Figure 1.  
The first step is to segment which points in the cloud correspond to the reference rectangle. The calibration rectangle is considered to produce a set of \(N\) points reflected in the LiDAR \({}^{L}P=\{^{L}\vec{p}_{1},{}^{L}\vec{p}_{2},\ldots,{}^{L}\vec{p}_{N}\}\). To this end, by visual inspection of a range image, such as the one in Figure 2, any point in the \({}^{L}\vec{p}_{i}\) plane is manually selected. By using a \(kd-tree\) of the point cloud, points neighboring \({}^{L}\vec{p}_{i}\) that are within a radius \(r\) are found and added to the set of points \({}^{L}P\). The process is repeated for each added point \({}^{L}\vec{p}_{j}\) of \({}^{L}P\) until no neighboring points \({}^{L}\vec{p}_{i}\) are found within \(r\). The process is described in detail in Algorithm 1. For this method to converge, it is necessary that the rectangle is not in contact with any other surface, so it has to be completely isolated since no features are considered to differentiate points on the calibration rectangle from other surrounding objects. The estimation of the equation of the plane defined by the set of points of the set \({}^{L}P\) is carried out in order to continue the process. By using a RANSAC algorithm [12], an equation of the plane is obtained, in which the points \({}^{L}P\) are projected and from which its normal vector is extracted. Afterwards, by obtaining the normal vector of the plane, the reference system of the plane is changed to work on the set of projected points \({}^{L}P^{\prime}\) in 2D  
Figure 1: Block diagram with the steps to obtain the 3D LiDAR coordinates of the corners of the rectangular polygon. Figure 2: Equirectangular projection of a point cloud with two reference rectangles with a reflectivity filter applied. The black area denotes the absence of points. The color map ranges from red for closer points to blue for more distant points. coordinates, removing the coordinate that would correspond to the depth, as shown in Figure 3.  
```
0:\({}^{L}P\)
1:\({}^{L}P_{i}\) = FindRadiusPoints(\(kdtree,r,{}^{L}P\))
2:while size of \({}^{L}P_{i}\) is not 0 do
3:for\({}^{L}p_{i}\) in \({}^{L}p_{i}\)do
4:\({}^{L}P_{j}\) = FindRadiusPoints(\(kdtree,r,{}^{L}p_{i}\))
5:\({}^{L}P_{i}^{\prime}\gets{}^{L}P_{i}^{\prime}\cup{}^{L}P_{j}\)
6:endfor
7: DeleteRepeatedPoints(\({}^{L}P_{i}^{\prime}\))
8:\({}^{L}P_{i}\gets{}^{L}P_{i}^{\prime}\)
9:\({}^{L}P\gets{}^{L}P\cup{}^{L}P_{i}\)
10:endwhile
```  
**Algorithm 1** Algorithm to obtain the LiDAR points belonging to the rectangle. Finally, the minimum area rectangle that surrounds this set of points \({}^{L}P^{\prime}\) is calculated. For this purpose, \({}^{L}P^{\prime}\) is repeatedly rotated by an angle \(\alpha\in[0,\pi/2]\), obtaining the maximum and minimum coordinates of each axis that would determine a rectangular polygon containing the rotated set \({}^{L}P^{\prime}\). The rectangle of minimum area will have an orientation \(\rho=-\alpha_{Aretmin}\), where \(\alpha_{Aretmin}\) corresponds to the rotation of \({}^{L}P^{\prime}\) in which the rectangular polygon that bounds the points minimizes its area, this being area calculated from the maximum and minimum coordinates in x and y. Lastly, keeping the orientation of the bounding rectangle of minimum area and maintaining its center fixed, the width and height dimensions of the rectangle are increased, if necessary, by making them equal to those of the actual reference pattern.