# Feature Detection of Focused Plenoptic Camera Based on Central Projection Stereo Focal Stack
## 1 Introduction  
Plenoptic (or light field) cameras have become increasingly developed and widespread in recent years. In contrast with traditional pinhole cameras, they capture the light distribution both in spatial and angular dimension by the help of the inserted micro-lens array (MLA). According the MLA placement, plenoptic cameras can be classified into unfocused [1] and focused plenoptic cameras [2]. In an unfocused plenoptic camera, the main lens focuses the subject into the MLA (like Lytro). Furthermore, based on the position of MLA, focused plenoptic cameras can be classified into two types with the MLA behind (Keplerian) or in front of (Galilean) the focused main lens image. As a special case of focused plenoptic cameras, multi-focus plenoptic cameras use multiple types of micro-lenses (like Raytrix) to further extend the depth-of-field. Due to the better ability to restore depth information compared with unfocused plenoptic camera, the focused plenoptic camera is gaining more attention in fields like structure from motion (SFM) and visual odometry. Robust and accurate feature detection methods are the foundation of many SFM algorithms [3; 4; 5]. At present, many works have been conducted with regard to feature extraction of the focused plenoptic camera. [PERSON] et al. [6] propose a checkerboard line feature detector performed on the raw image, and [PERSON] et al. [7] present a checkerboard corner detector. [PERSON] et al. [8] introduce an adaptive checkerboard corner detector, which can select the sharpest corners automatically. However, the methods mentioned [6; 7; 8] are designed for specific feature pattern, which is hardto use in SFM. [PERSON] et al. [9] use the scale-invariant feature transform (SIFT) detector [10] to extract features and calculate depth on the raw image. However, the small size of micro-image makes the results less robust and accurate. [PERSON] et al. [11] propose a LIFF feature detector for the unfocused plenoptic camera based on focal stack. However, the locations of detected features are restrict to the central sub-aperture image other than the whole raw image. [PERSON] et al. [12] carry out SURF feature detector directly on total focus image. However, the computation of dense depth image and total focus image is very time-consuming. Although it is easier to extract features from the total focus image, the traditional refocusing algorithm of focused plenoptic camera has some limitations. The traditional refocusing algorithm is based on the concept of virtual depth [2], which is widely used in the calculation of depth maps and total focus image [9; 13; 14; 15; 16; 17]. However, the theoretical imaging range of the refocused image based on traditional method [2] is limited by the sensor size and the traditional focal stack do not conform to the central projection in theory. Besides, most methods [9; 13; 14; 15; 16; 17] ignore the difference between the center of micro-lens and micro-image when calculating depth maps and total focus images, which will result in the deviation of calculated results from the theoretical model. In order to extract features more accurately and efficiently, we perform an innovative feature extraction on the central projection stereo focal stack (CPSFS). Specifically, we present a refocusing model that conforms to the central projection with regard to the center of main lens. The calculated refocused image is strictly consistent with the theoretical model. Then, we propose the feature detection algorithm performed on CPSFS based on the stereo focal stack proposed by [PERSON] et al. [17]. We analyze the relationship between the layer selection of CPSFS and the detection results through simulated experiments. Furthermore, the proposed feature detection method is tested on both simulated and real data. The experiment results indicate that the proposed detection method has good performance in speed, accuracy and noise condition.