# Iterative Refinement of Uniformly Focused Image Set for Accurate Depth from Focus
## 2 Proposed Depth from Focus
### Textureless Region and Post Processing  
Estimating depth on the textureless region is one of the challenging problems in DFF. Most existing DFF methods fail in textureless regions because of the lack of enough information to observe. Figure 8 describes two different parts of the input scene: textured and textureless regions. The image shows that textured surface has more information to observe, and the textureless surface has a lack of enough information to chose the best-focused frame. In Figure 9, the depth estimation result has a bunch of erroneous points on textureless regions. To fix these erroneous points, we have to use depth information from its neighbors. In order to identify textureless region, we investigate all-in-focused image (generated using depth map) and find inaccurate (noisy) regions from depth map, as well. We create two binary masks:  
* Textureless Region Mask (TRM): textureless regions of all-in-focused image, and
* Inaccurate Depth Mask (IDM): erroneous depth regions on depth map. Figure 8: Choosing peak-point(best-focused frame) of normalized absolute difference between the reference frame and all other frames from textured and textureless parts of the scene. Figure 6: Input image, point cloud(without post-processing) and zoomed-in version of point cloud result of each iterations. Iteration 1-green, 2-yellow, 3-blue, 4-purple. A—includes all iterations’ result, B—includes 3 iterations’ result, C—includes 2 iterations’ result, D—includes last iterations’ result. First iteration’s (green) point cloud cannot provide detailed depth, next iterations’ result shows more precise depth. Rectangle color of each point cloud result means that iterations are visible in that figure. Full view of depth map and point cloud shown in Figure 1.  
Figure 7: Iterative depth refinement creating uniformly focused image set. _TRM mask_. Textureless regions have lower variance in color. In order to create _TRM_ mask, we scan a window over all-in-focused image calculating color variance within the window. If the variance is lower than a threshold, we mark the point as textureless. Threshold is the squared value of the window size (\(th=w^{2}\), \(w\) is window size). Threshold value is chosen empirically as the value that produces a better result than other variations when pixel values are in between 0 and 255. Figure 9D describes _TRM_ mask (black-color: textureless region). _IDM mask_. Inaccurate depth points which are from textureless part have higher variance while accurate depth points have lower variance. Based on this assumption we create _IDM_ mask. Generating _IDM_ mask process is similar to _TRM_. If the variance is higher than a threshold, we mark that point as an inaccurate depth point. Figure 9E describes _IDM_ mask (black-color: inaccurate depth). _IDM_ sometimes finds inaccurate regions even at the correct depth around edges of objects. We overlap both masks to figure out textureless regions. We indicate _erroneous depth from textureless_ regions as (_EDfT_). Finally, hole-filling is performed with _EDfT_ regions. We replace _EDfT_ points by its neighbors of high-score-confidence. Depth confidence gets higher if the peak point in the sharpness estimation is more greater than its neighbors in the focus measure process. Based on this idea, we created a score-confidence map in the depth estimation process. After fixing depth values of textureless region, we perform post-processing for getting cleaner depth around the object boundary using a filter called guided filter [24]. The guided filter investigates intensity changes in an all-in-focus image (Figure 10b). Finally, we scale estimated depth values by scaling factor s to convert frame index to physically meaningful depth value. Median and Gaussian filters are applied, in turn, to eliminate any remaining noisy value. After obtaining depth information of the scene, we can get point clouds of the scene, as shown in Figure 1 (point clouds).