# Iterative Refinement of Uniformly Focused Image Set for Accurate Depth from Focus
## 2 Proposed Depth from Focus
### Initial Depth Estimation  
In order to find best-focused frame of each pixel, we calculate neighborhood patch dissimilarity. Small window size causes noisy depth due to the lack of enough observation for focus estimation. On the other hand, a bigger patch loses details in depth. In our work, we use adaptive patch size at each iterative estimation. In addition, we use a circular window to minimize the artifact of estimated depth. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Focus Measure Operators** & **Abbr.** & **Year** & **Depth from Focus** & **Abbr.** & **Year** \\ \hline Gaussian derivative [29] & **GDER** & 2000 & DFF from mobile phone [18] & **DFM** & 2015 \\ \hline Variance of laplacian [30] & **LAPV** & 2000 & Variational DFF [1] & **VAR** & 2015 \\ \hline Image contrast [31] & **CONT** & 2001 & Noise robust RDF [19] & **RDF** & 2017 \\ \hline Wavelet sum [32] & **WAVS** & 2003 & Composite Focus Measure [38] & **CFM** & 2017 \\ \hline Variance of Wavelet [32] & **WAVV** & 2003 & PAD method of multipliers [39] & **PAD** & 2018 \\ \hline DCT Energy measure [33] & **DCTE** & 2006 & Deep Depth from Focus [22] & **DDFFNet** & 2018 \\ \hline Wavelet ratio [34] & **WAVR** & 2006 & Fast and noise robust RDF [20] & **F-RDF** & 2019 \\ \hline DCT Energy ratio [35] & **DCTR** & 2009 & & & \\ \hline Diagonal Laplacian [36] & **LAPD** & 2009 & & & \\ \hline Steerable filters-based [37] & **SFIL** & 2009 & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Focus measure operators and depth from focus (DFF) methods with abbreviation and year of publication. Figure 1: Pipeline of our proposed depth from focus method. A circular window creates less perceivable noise. Figure 2 shows the difference of 3D reconstruction result between square and circle window. Using patch dissimilarity to find the best-focused frame, we chose a reference frame arbitrarily among focal-stack. Then, we calculate the normalized average absolute difference of patches between the reference frame and all other frames at each pixel. The best-focused frame becomes a local maximum in the difference. The best-focused frame detection task for each pixel becomes a simple local maximum detection task. However, if the best-focused frame is identical or very close to the reference frame, the best-focused frame cannot be detected easily. To avoid such a singular case, we blur the reference frame before the difference computation. The blurring amount of the reference frame is selected not to have a similar blur amount in all other focused image frames (Figure 3a). We use Gaussian blurring assuming that it simulates optical blur well in out-focus images. Normalized average absolute difference between the reference frame and \(k\)th frame is calculated on both spatial and frequency domain information of the given patch using Equation (1) for spatial domain and Equation (2) for frequency domain:  
\[p_{k}^{x,y}=\sum_{i=x-w}^{x+w}\sum_{j=i-w}^{y+w}(\left|R_{i,j}-F_{i,j}^{k} \right|*mask_{i,j}), \tag{1}\]  
\[\bar{p}_{k}^{x,y}=\sum_{i=x-w}^{x+w}\sum_{j=i-w}^{y+w}(\left|\bar{R}_{i,j}-\bar {F}_{i,j}^{k}\right|), \tag{2}\]  
Figure 3: **(a)** Adding blur for reference frame (yellow: reference frame, blue: other frames); **(b)** difference between reference frame and other frames (curve form); **(c)** comparison of obtained 3D point clouds with and without curve fitting. Figure 2: From left to right: all-in-focused image, depth map, and 3D reconstruction results using circle and square windows (highlighted part zoomed-in). Three-dimensional reconstruction result shows that circle window produces less artifacts. where \(x\) and \(y\) are pixel indexes, \(w\) is window size, \(R\) is reference frame in spatial domain, \(\bar{R}\) is reference frame (neglect edge effects using mask) in frequency domain, \(k\) is index of frames (\(1\leqslant k\leqslant n\), \(n\) is number of frames in focus-stack), \(F\) is \(k\)th frame in spatial domain, \(\bar{F}\) is \(k\)th frame in frequency domain, and _mask_ is binary circle mask (to use circle window).