# Iterative Refinement of Uniformly Focused Image Set for Accurate Depth from Focus
## 1 Introduction  
Estimating three dimensional shape of a scene from color image is a challenging task [1]. Without any prior knowledge on the scene, it is an ill-posed problem to recover three dimensional shape of objects using single color camera. Many researchers have proposed diverse approaches for 3D depth acquisition. For example, in Reference [2, 3], the authors estimate depth information based on a coded aperture framework. However, it requires hardware modification which cannot be applied to off-the-shelf camera, such as smart phone camera. Depth from focus (DFF), or shape from focus (SFF), is a technique for depth estimation from a set of image frames having continuously changing focus amount that are taken at the same location and viewpoint. By identifying best focused frame of each pixel, depth of each pixel can be estimated. In DFF, three factors determine the quality of reconstructed depth: lens aperture, focal length, and focusing distance. While many computer vision techniques assume that given images are obtained with pin-hole camera, DFF assumes that a real aperture camera is used. Real aperture cameras have relatively short depth of field resulting in images focused only on a small distance range of a scene. For instance, [PERSON] and [PERSON] [4] use Laplacian of Gaussian pyramids in order to find best focused frame at each pixel. There are lots of traditional Depth from focus methods [5, 6, 7, 8, 9]. [PERSON] et al. [6] propose new SFF algorithm based on Markov Random Fields, and [PERSON] et al. [7] use SUSAN operator. [PERSON] et al. [8] use energy of high-frequency components in S transform and [PERSON] et al. [9] propose to use 3D anisotropic nonlinear diffusion filtering for accurate SFF. Recently, depth from focus (DFF) method has been improved in various ways producing more accurate depth map andhigh quality all-focused image. [PERSON] and [PERSON] [10] use a neural network in order to extract 3D shape of objects based on DFF. In Reference [11], they use neural networks to learn the shape of focused image surface (FIS). In Reference [12; 13], the authors use dynamic programming optimization technique to get depth (3D shape) of the scene from set of focused images. Their method works significantly faster than previous FIS algorithms. In Reference [14], the authors propose local search algorithm for SFF problem to reduce computational complexity. In Reference [15], the authors propose focus on measurement method based on an optical transfer function, which is implemented in Fourier domain for 3D shape recovery. [PERSON] et al. [16] employ the entropy of high frequency bands as the amount of blur combining texture information for improved performance. [PERSON] et al. [17] propose a semi global DFF approach that enforces the adaptive smooth constraint on reconstructed depth of the scene. [PERSON] et al. [1] propose variational approach using an efficient non-convex minimization scheme to produce depth map. [PERSON] et al. [18] formulate uncalibrated DFF problem and propose a new focal-stack aligning algorithm to estimate depth of given scene using hand held cameras. In Reference [19], the authors propose a new focus measurement that is robust to noise with higher accuracy in focus measurement. They present Ring Difference filter (RDF) by inserting a gap and looking at the pixels that are located farther away from the point of interest (POI). They extended their work [20] by proposing RDF-based cost aggregation that utilizes both local and non-local characteristics by inheriting the structure of RDF. Recently, [PERSON] et al. [21] proposed depth recovery framework including depth reconstruction and refinement process. They use non-local matting Laplacian prior and variance based confidence level computation. It is able to produce depth map robust to texture-less regions and give more clean edges. [PERSON] et al. [22] propose auto-encoder-style network to predict depth from focal stack. In order to train their network, they create 12-Scene Benchmark dataset.