# Detection of Parking Slots Based on Mask R-CNN
## 3 Generation of Around-View Images  
The around-view parking-assist system provides drivers with real-time information on the surrounding environment of the vehicle in an imaging manner. It usually consists of multiple fisheye cameras installed around the vehicle, each camera facing a different direction, thus ensuring that each fisheye camera and the images taken by two adjacent fisheye cameras have overlapping areas.  
Four fisheye cameras are mounted on the experimental vehicle, each of which has a horizontal field of view of 180 degrees. We use four fisheye images and camera parameters are input to synthesize a 360-degree panorama around the vehicle image, as shown in Figure 1b.  
Figure 1: Generation of around-view monitoring (AVM) image. (**a**) Experimental vehicle; (**b**) AVM image.  
The generation of the AVM image is the process of mapping from the bird's-eye view coordinate system to the original fisheye image. The key step is to generate the mapping table \(T_{B\to F}\), then to take a pixel point F out of the original fisheye image and fill it into a corresponding position B in the bird's-eye view image based on OpenCL [42] quickly and efficiently. As shown in Figure 2, the mapping process and related coordinate transformation of the bird's-eye view image are explained.  
We divide the bird's-eye view into four areas and each area corresponds to a mapping table \(T_{B\to F}\), as shown in Figure 2. In the traversal of bird's-eye view image, the coordinates of each pixel are mapped to the world coordinate system by a transformation matrix \(P_{B\to W}\), then the world coordinate system is projected to the undistorted image coordinate system by a homography matrix \(P_{W\to U}\). Finally, the lookup table \(T_{U\to F}\) maps a point on the undistorted image to a position on the original input fisheye image.  
The following describes how to obtain the mapping table \(T_{B\to F}\): we create a bird's-eye view image with a width of \(u\) pixels and a height of \(v\) pixels, which is used to display the ground range with the width of \(W\) mm and the length of \(H\) mm in the world coordinate system, as shown in Figure 3. Therefore, we get the transformation matrix \(P_{B\to W}\):  
\[P_{B\to W}=\left[\begin{array}{ccc}u/W&0&v/H\\ 0&u/2&v/2\\ 0&0&1\end{array}\right] \tag{1}\]  
The mapping table \(T_{U\to F}\) can be determined based on obtaining the distortion coefficients and the parameters of the fisheye camera by [PERSON]'s calibration method. Accordingly, we correct the original fisheye images to undistorted images based on \(T_{U\to F}\). To obtain the homography matrix \(P_{W\to U}\), we arrange a checkerboard around the experimental vehicle to obtain the feature points \(X_{u}^{i}\) in the world coordinate system. Next, we manually select the corresponding feature points \(X_{w}^{i}\) in the undistorted image. The number of feature points \(i\) cannot be less than four, as shown in Figure 4. It is known that \(X_{u}^{i}=P_{W\to U}X_{w}^{i}\), therefore, \(P_{W\to U}\) can be estimated based on the least-squares method.  
Figure 2: Reverse mapping process of bird’s-eye image.