# Fast Object Detection in Light Field Imaging by Integrating Deep Learning with Defocusing
## 2 Methodology
### Object Detection and Defocus Response  
In the present study, a state-of-the-art deep learning method, named you only look once (YOLO), is used to detect the object on the all-in-focus image as well as the focal stacks [19]. Previous approaches, like region proposal convolutional neural network (R-CNN) [20; 21], use region proposal method to choose candidate boxes in image, and take CNN to extract features to carry out the classification. While YOLO considers the object detection as a single regression problem and uses single CNN to deal with the regional proposal, classification, and bounding box locating as an end-to-end process, which makes it much computational efficient than other methods. The system of the YOLO is performed from three aspects [19]. Firstly, an image is input and is resized into 448 \(\times\) 448. The resized image is divided into S*S grid and each grid is responsible for predicting whether the centers of the detected objects were located in this region. Secondly, each grid containing the center of the detected object is used as initial condition to predict the bounding box for the corresponding object. Each bounding box contains five parameters, including x, y, w, h, and corresponding confidence. \((\mathrm{x},\mathrm{y})\) are the shift of the center of bounding box from the corresponding grid, and the \((\mathrm{w},\mathrm{h})\) are the dimension of the bounding box. The confidence means the possibility that there is a detected object and how precisely the box is located. Thirdly, the object contained in each box also needs to be classified by the confidence of every class \(\mathrm{Pr}(\mathrm{Class_{i}}\,|\,\mathrm{Object})\). Hence, in the currently study, the CNN contains 26 layers, including 24 convolutional layers and two fully connected layers, as shown in Figure 4. The convolutional layers are made of two different kernels including 3 \(\times\) 3 and 1 \(\times\) 1 kernels. Besides, two boxes with different initial values are used to get a more precise box for complex scenario. The first FCN is a 4096 vector and the output is 7 \(\times\) 7 \(\times\) 30. Here 7 \(\times\) 7 are the number of the grids. 30 corresponds to 20 classes and 10 parameters for the two boxes. The CNN is trained in two steps. In the first step, the first 20 convolutional layers, the average pooling layer and the fully connected layer are pre-trained by Image-Net 1000-class datasets [22] to get a robust and general feature extractor. In the second step, the final two layers are deleted and other four convolutional layers and two fully connected layers are trained together with pre-trained 20 convolutional layers by using the PASCAL [23]. More details of the CNN can be referred to [19]. The depths of the detected objects are then estimated by the defocus response of the focal stacks at the bounding box. As shown in Figure 5, the bounding boxes are used to sub-sample the series of the focal stacks that were generated by digital refocusing, as presented in Section 2.2. Then, the sub-sampled regions are analyzed by defocus to find the clearest layer. For defocus analyzing, the gradient stacks can be computed via convolution. Different convolution kernels would result in different defocus response. It is stated in previous work [19] that the sum modified Laplacian (SML) operator has better performance than traditional operators, such as Sobel, Laplacian. This paper modifies the SML and present the sum squared modified Laplacian (SSML) operator to get sharper response at the focused layer, given by Equation (7):  
\[L_{p}=\left[ \begin{array}{ccc}-1&2&1\end{array} \right] \tag{7}\]  
Then, the defocus response _D(d)_ is computed by:  
\[D(d)=\frac{1}{M\cdot N}{\sum_{s}{\sum_{t}{({l_{d}(s,t)L_{p}})^{2}}}}+{({l_{d}(s,t){L_{p}}^{T}})}^{2} \tag{8}\]  
where \({l_{d}}\) denotes the \(d\)-th layer of focal stacks. High response \(D(d)\) means that the corresponding layer is in focus.