# All-in-Focus Three-Dimensional Reconstruction Based on Edge Matching for Artificial Compound Eye
## 4 Edge Matching and Propagation Method
### Edge Depth Estimation
#### 4.2.2 Edge Feature Matching  
In the eCley, the focal length is \(f=0.778\) mm, the pixel size is \(3.2\)\(\upmu\)m, the rectified image is upsampled by a factor of 2 from the original image, and there are \(111\) pixels between centers of the rectified adjacent sub-images, so \(\Delta x\) of the rectified sub-image is \(1.6\)\(\upmu\)m. According to the relationship between disparity and distance of the stereo image in Equation (1),  
\[Z=\frac{B\cdot f}{d\cdot\Delta x} \tag{1}\]  
where \(Z\) is the distance of the object, and \(d\) is the estimated disparity (or object depth). Therefore, when \(d=1\), then \(Z=172\) mm; when \(d=10\), then \(Z=17.2\) mm; and when \(d=15\), then \(Z=11.47\) mm. The distance of the object is only \(1.1\) cm when the disparity reaches \(15\), so, in the edge matching, the disparity is assumed to be below \(15\).  
Although eCley images can extract a large number of Canny edge features, the edge features only respond to the location where the image color changes suddenly, and edge matching is prone to mismatching if only the color information is used. To obtain robust edge depths, two different types of operators (improved Census operator [13] and SC (Shape Context) operator [14]) are fused for edge feature matching.  
Supposed for the sub-image \(I\), \(I(p)\) indicates the intensity of pixel \(p\). The improved Census operator of \(p\) is given in Equation (2),  
\[T(p)=\bigotimes_{q\in W_{p}^{\prime}}\xi(I(p),I(q)),\]  
\[\xi(I(p),I(q))=\left\{\begin{array}{cc}1,&b+\alpha<c\\ 0,&b-\alpha\leq c\leq b+\alpha\\ -1,&b-\alpha>c\end{array}\right. \tag{2}\]  
where \(\bigotimes\) denotes concatenation, \(W_{p}^{\prime}\) is the local window centered in pixel \(p\), and \(\alpha\) denotes noise tolerance. Due to the low resolution of the sub-image, the system noise may lead to fluctuations in the edge transition region. Hence, the noise tolerance is introduced to improve the matching stability.  
The SC operator is widely used in object matching with similar shapes such as digit recognition. Therefore, the SC operator introduces the geometry similarity for edge matching. For pixel \(p\), the SC operator obtains the edge content information from statistical histogram \(h_{p}(k)\) of window \(W_{p}\) centered in \(p\) using Equation (3),  
\[h_{p}(k)=\#\{q\
eq p:(q-p)\in bin(k)\} \tag{3}\]  
where \(bin(\cdot)\) uses the log-polar coordinate system and uniformly partitions the coordinate system into K regions. \(bin(k)\) is the \(k_{th}\) part. \(q\) is the edge point in \(W_{p}\), and \(h_{p}(k)\) indicates the number of \(q\) in \(bin(k)\).  
The Census and SC operators are used to match the edge point. Suppose \(I\) is the target sub-image, \(I_{i}\in N_{I}\) is the adjacent sub-image, and \(N_{I}\) denotes the four neighboring sub-images of \(I\). For sub-image pair \(I\) and \(I_{i}\), the matching cost of edge point \(I(p)\) to have a disparity of \(d\) is  
\[C^{i}(p,d)=(1-\beta)\cdot\mathrm{C}_{s}(p,p^{\prime}(d))+\beta\cdot\mathrm{C} _{c}(p,p^{\prime}(d)) \tag{4}\]  
where \(\mathrm{C}_{s}(p,p^{\prime}(d))\) and \(\mathrm{C}_{c}(p,p^{\prime}(d))\) are the cost of SC and Census operators (given in Equation (5)), respectively. \(p^{\prime}(d)\) indicates the edge point with disparity \(\mathrm{d}\) in the matching sub-image \(I_{i}\).  
\[\begin{array}{rl}\mathrm{C}_{c}(p,p^{\prime}(d))=&\quad Hamm(T(p),T_{i}(p^{ \prime}(d)))\\ \mathrm{C}_{s}(p,p^{\prime}(d))=&\frac{1}{2}\sum_{k=1}^{K}\frac{[h_{p}(k)-h_{ p^{\prime}(d)}(k)]^{2}}{h_{p}(k)+h_{p^{\prime}(d)}(k)}\end{array} \tag{5}\]where \(Hamm(\cdot)\) denotes Hamming distance. If no matching point exists in the matching sub-image with disparity d for pixel \(p\), \(C^{i}(p,d)=\infty\), then the winner-takes-all (WTA) method is used to select the edge depth map \(D^{i}(p)\).