# All-in-Focus Three-Dimensional Reconstruction Based on Edge Matching for Artificial Compound Eye
## 1 Introduction  
Artificial compound eye (ACE) is a type of camera with a miniature volume and large FOV. Different kinds of artificial compound eye systems have been proposed [1; 2; 3]. The ACE-captured image is an array of sub-images with slightly different viewpoints. All the sub-images can be reconstructed into one large FOV image, but depth information is required for the all-in-focus image reconstruction. As one of the most famous and typical ACE systems, the eCley (Electronic Cluster Eye [1]) (shown in Figure 1) uses a lens array of \(17\times 13\) channels, and adjacent channels have an inherent offset. The height of its optical module, about two times shorter than a comparable single-aperture camera with the same resolution, is only 1.4 mm. [PERSON] et al. present a stitching method that merges all the sub-images and reconstructs an image with a resolution of \(700\times 550\) pixels [4]. This method does not consider the effect of parallax, and the images are merged at a specified distance. Consequently, the object located at another distance will be blurred, as shown in the left image of Figure 2. The reconstructed image with all \(17\times 13\) channels without depth information is shown in Figure 1(a), and our reconstructed image with central \(13\times 13\) channels is shown in Figure 1(b). To address this problem, the object distance (depth) information is essential. Stereo matching, one of the fundamental tasks in computer vision, can estimate the depth of objects from at least two cameras with overlapped FOV [5]. Many stereo matching methods have been proposed in recent years (BP [6], SGBM [7], cost-volume [8], Lac-GwcNet [9], PCW-Net [10], IGEV-Stereo [11], etc.) and have achieved state-of-the-art results. Although the adjacent images of ACE have overlapped FOV, traditional stereo matching methods cannot obtain satisfied depth results directly. As shown in Figure 2(e),f, the depth map of the traditional method is noisy and inconsistent. This is mainly because of the following:1. The low-resolution and small FOV. In this paper, the rectified sub-image has only \(101\times 101\) pixels, and the FOV of the sub-image is about \(9.18^{\circ}\), which will lead to a large textureless area in the sub-image. 2. The large FOV offset. For the eCley, the inherent offset between adjacent sub-images is about 34 pixels, and the overlap area between adjacent images is less than 70%. As shown in Figure 4, Figure 4a gives the target sub-image located in the center and its four neighboring sub-images placed at the corresponding location. Figure 4b-d show the overlap between the target sub-image and each adjacent sub-image, where the green area indicates the non-overlapped area. This will result in much less contextual information being used. 3. The photo-consistency may not hold for all pixels. Due to the low resolution and noise, the boundary of objects will be blurred, and the point-wise depth map will be very noisy. To obtained accurate depth for the ACE, an edge matching and propagation (EdMP) method is proposed. The method takes inspiration from local feature matching and propagation methods [12]. Since there is less texture in the sub-image, the edge feature is used to indicate the salient information. To accurately match the edge features between adjacent images, the edge matching cost is computed by combining a modified census transform [13] and shape context method [14]. Additionally, to eliminate the no-match problem caused by the inherent offset, the edge matching results between a sub-image and its four adjacent sub-images are fused to obtain the complete and robust sparse edge depth. The edge depth is then propagated to the whole sub-image to obtain the dense  
Figure 1: The eCley [1]. (**a**) The imaging principle. (**b**) The eCley sensor, the red box is the lens array. Figure 2: The reconstructed image. (**a**) The reconstructed image without depth information. (**b**) The reconstructed image with our method. depth map.