# Fast Object Detection in Light Field Imaging by Integrating Deep Learning with Defocusing
## 3 Experimental Results
### Benchmark Dataset Experiment  
The proposed method is tested on a light field dataset, named Mona [10], in mainly in four steps, in accordance with the process given in Figure 1. In the first step, three different kinds of images are produced from the light field dataset, including multi-view images, as shown in Figure 5(a), an all-in-focus image as shown in Figure 5(b), and 41 layers of focal stacks in different depths as shown in Figure 5(c). In the second step, CNN is used to perform the object detection on the all-in-focus image. A total of three objects are detected, including Mona, a ball, and a flowerpot. The identified regions as highlighted by the bounding box in Figure 5(b) are then used to sample the 21 focal, as shown in Figure 5(c). The depths of the identified objects are determined by analyzing the defocus of the corresponding focal stacks. Figure 5(d) shows the defocus response of the three objects based on sum squared modified Laplacian (SSML) operator. It is clearly illustrated from the results that the defocus response has very sharp response at the focused layer, which can be used to determine the depths of the corresponding objects with high robustness. The depths of Mona, ball, and flowerpot are identified to be \(-0.45\), \(0.25\), and \(0.4\), respectively, according to the defocus response. It is emphasized that, only a single value was assigned for each object in the present study. Although focal stacks can achieve much better resolution by more comprehensive defocusing analysis, the process would take long time for the computation and would be not able to perform in real time. In the current version, the proposed method can achieve around 10 frame rate on NVIDIA GeForce 1080P GPU card. The main time consumption is in the object detection process. In case of the scenarios which require depth estimation per pixel, multi-baseline stereo matching can be performed. One of the problems anticipated in the present study is that dense depth estimation inevitably cost expense computation in stereo matching. As a result, although currently available methods can achieve very high matching accurate in light field image comparing with conventional stereo matching, it would take relatively long time for the depth estimation, normally in several or even dozens of seconds [12; 13; 14], which hinders the light field imaging to be used in many applications. For the proposed method, since initial depths are already estimated by defocus response for every detected object in the third step, the efficiency of the stereo matching can be dramatically improved by shrinking the searching space of the disparity in the fourth step. Figure 6d shows the depths of the objects evaluated by multi-baseline phase shifting stereo matching based on light field. It can be inferred from the results that the depths estimated by the stereo matching generally match well with that are obtained by the defocusing. The results are also compared with conventional binocular stereo  
Figure 6: Result of Mona synthetic 4D light. (**a**) The input 4D light field. (**b**) Identified objects in all focus image. (**c**) Focal stacks of object regions. (**d**) Depth estimation by defocus analysis of the down sampled focal stacks. (**e**) Depth estimation of objects by stereo matching. matching method [4]. A summary of the comparison is shown in Table 1. It is inferred from the results that the multi-baseline phase shifting stereo matching has much better performance than conventional stereo matching especially at the occlusion regions, such as the boundaries of flowerpot and the ball. However, the multi-view stereo matching inevitably requires much more time for the computation than binocular stereo matching. Currently, the proposed method can achieve around 5 frame rate on NVIDIA GeForce 1080P GPU card. On the other hand, it is also shown from the results that both of the methods have worse performance in matching the ball due to the high spot and low texture.