# 3-D People Counting with a Stereo Camera on GPU Embedded Board
## 2 Proposed Method
### Camera Calibration  
The image acquired from the camera is affected by the intrinsic camera parameters and the presence of distortion. The camera's intrinsic parameters are the focal length and principal point. Generally, in order to perform image processing in a stereo camera system, it is necessary to correct the distortion by using the intrinsic camera parameters. In addition, the horizontal axis of the left and right images should be made the same as the extrinsic camera parameters. To obtain the depth map, stereo matching should be applied. If the horizontal axis of the left and right images are not matched, a disparity map of good quality cannot be obtained. The extrinsic camera parameter is the relationship between the camera coordinates and the world coordinates as a rotation and translation matrix. The process of acquiring camera parameters is called calibration. The calibration proceeds through Equation (1), which describes the pinhole camera model [20]:  
\[s\left[ \begin{array}{c}x\\ y\\ 1\end{array} \right]=\left[ \begin{array}{ccc}f_{x}&0&u_{0}\\ 0&f_{y}&v_{0}\\ 0&0&1\end{array} \right]\left[ \begin{array}{ccc}r1&r2&r3&t1\\ r4&r5&r6&t2\\ r7&r8&r9&t3\end{array} \right]\left[ \begin{array}{c}X\\ Y\\ Z\\ 1\end{array} \right] \tag{1}\]  
where \(f_{x}\) and \(f_{y}\) denote the camera focal distances with respect to the \(x\)-axis and \(y\)-axis, respectively, and \(u_{0}\) and \(v_{0}\) denote the principal points of the camera with respect to the \(x\)-axis and the \(y\)-axis, respectively. \(r\) and \(t\) denote the elements of the rotation matrix and the translation matrix, respectively, and are the external parameters of the camera. \(X,Y,\) and \(Z\) denote points in the world coordinates, and \(x\) and \(y\) denote points in the image coordinates. In Equation (1), the pair of coordinates from the image coordinates and the corresponding points of the world coordinates make it possible to acquire the intrinsic camera parameters. To obtain the  
Figure 3: Stereo camera configuration of the proposed method. position of the point for each coordinate system, we used a uniform chess board as shown in Figure 4. The corner point of a chess board is easy to extract because its characteristics are clear. The coordinates of the corner points extracted from the image are image coordinates, and the distance between the actual corner points of the chessboard is the world coordinate. Substituting these pairs of coordinates into Equation (1), an intrinsic parameter can be obtained. We obtain an undistorted image pair by correcting the distorted image using the intrinsic parameters [21]. Stereo matching is a method of obtaining a disparity map by calculating the distances of corresponding points in left and right images. In stereo matching, if the external factors are unified except for the distance between the cameras, that is, if the two images are perfectly horizontal, the problem of finding the corresponding point becomes one-dimensional, such that the corresponding point can be found easier. However, even if two cameras are installed in parallel, there is a slight physical difference. Therefore, it makes the two images parallel via software [22]. Figure 5 shows the result of rectifying the stereo image. Figure 5 shows the left image and right image before. The positions of the corresponding points on the horizontally drawn orange line were different. Figure 5 shows the left and right images after stereo rectification and it can be clearly seen that the positions of corresponding points on the orange line were the same. After stereo rectification, the counting line was drawn. When a person crosses the counting line, the number in the crossing direction is counted. Mono cameras only have two-dimensional image coordinates. Therefore, whether or not a person actually crosses the counting line spatially cannot be judged. This results in the case where a person who has been judged to have passed though the counting line has not actually passed. This reduces the accuracy of the counting.