# Fast Object Detection in Light Field Imaging by Integrating Deep Learning with Defocusing
## 3 Experimental Results
### Real Word Experiment  
The proposed method is also used in real scene to further examine the validity in object detection. The light field data is collected by a commercial light field camera named Lytro Illum [8], and the data is decoded by the procedure given by [PERSON](Lytro, Inc., California, CA, USA) [8]. Before the processing, it is necessary to correct the color of the image by gamma correction.  
Figure 7 presents the process of the experiment. First of all, the 4D light field was taken by Lytro Illum, and the raw data is shown in Figure 7a. 4D light field data can be presented by a 4D vector contains (_u,v,s,t_) after decoded the raw data. Three different kinds of images are produced from the 4D light field data, including the 13 \(\times\) 13 multi-view images, as shown in Figure 7b, the all-in-focus image, as shown in Figure 6c, and the focal stacks as shown in Figure 7d. The CNN is then used to detect the objects on the all-in-focus image. As shown in Figure 7c, four objects are detected. The focal stacks are then trimmed by the bounding box of the detected objects, and are used to estimate the depths of the objects based on defocus response. When considering that the disparity range between two adjacent views is from \(-1\) pixel to 1 pixel [8], subpixel shift should be performed if more than two layers of the focal stacks are required. Based the theory of Discrete Fourier Transform (DFT), continuous movement of images can be implemented. In the present study, 0.1 pixel shift is implemented between two adjacent views. Hence, 21 layers of focal stacks are used to analyze the defocus response of the detected objects.  
The defocus response is analyzed by the gradient of the focal stacks by SSML operator, as shown in Figure 7e. It is clearly seen from the results that the focused layer has very sharp response can easily and robustly be detected from the series of focal stacks, as shown in Figure 7d. The detected layers are marked with colored rectangles. Then, by taking the estimated depth by defocus response as input, multi-baseline stereo matching is performed on each bounding box, and the results are shown in Figure 7f. It is shown from the disparity maps that the cup and pot have very large percentage of mismatched pixels since they have low texture and are semitransparent. In such a scenario, the estimated depth maps are completely useless since it is almost impossible for a computer to judge which part of the depth map is correct or not. However, for the defocus response, the gradient of high frequency region like edges of the object can be much bigger than the low frequency region like low texture region. Hence, the defocus response can still be very sharp at the focused layer, which makes the defocus response based depth estimation method, although being relatively rough, is still very robust even for those objects which has low texture and are semitransparent.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Object**} & \multirow{2}{*}{**Views**} & \multirow{2}{*}{**Resolution**} & \multicolumn{2}{c}{**Percentage of Inaccurate Matching**} \\ \cline{3-5}  & & & **Binocular Stereo** & **Multi-View Stereo** \\ \hline Mona & 9 \(\times\) 9 & 93 \(\times\) 453 & 3\% & 1\% \\ Ball & 9 \(\times\) 9 & 190 \(\times\) 181 & 57\% & 52\% \\ Flowerpot & 9 \(\times\) 9 & 113 \(\times\) 95 & 39\% & 5\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: A summary of the stereo matching results of the identified objects.