# Depth Image Completion through Iterative Low-Pass Filtering
## 2 Approach
### Iterative Low-Pass Filter (ILF)  
After the extraction of boundaries and surfaces, the Iterative Low-Pass Filter (ILF) was applied to complete the depth matrices. In an ILF, depth matrices undergo the following process until the expanded depth matrix reaches the stopping criteria:  
1. Multiplication with a low-pass filter in the Fourier domain, where the cut-off frequency changes from low to high in different iterations. 2. Updating of the expanded depth matrix with a raw depth matrix. After several iterations, the low-frequency term reaches a steady state, and the incoming high frequency modifies the fine shape and detail of the surfaces. The expanded depth matrix is then cropped in the shape of the operation block, and the input depth image is updated in the same position. Consequently, the output depth image contains the updated depth matrix. The initial spatial frequency of the low-pass filter starts from the baseband of the image, corresponding to the reciprocal of the most extended length of the target surface. Since the operation block can be seen as a 2D point distribution, we used Principal Component Analysis (PCA) to analyze the shortest length of the operation block. The shortest length was applied for the initial radius of the low-pass filter for the raw depth matrix. For instance, if the random shape of the operation block represents 10 pixels in the shortest length from the PCA, the starting spatial frequency will be 10 pixel units. Frequencies lower than the initial frequency contribute less to the shape of the surfaces. Thus, the information within these frequencies reaches a steady state in several epochs of the first low-pass filter. By applying the initial spatial frequency and dynamically adjusting the starting spatial frequency from zero to the initial frequency, we ensure a gradual restoration of information and accelerate the ILF process. Following this, we iteratively expanded the low-pass filter's radius to obtain the surface's depth information in the shape of the operation block, moving from a low spatial frequency to a high spatial frequency. This step strengthens the included information from the previous iteration and minimizes interference from upcoming spatial frequencies. The depth information is updated iteratively to recover the defective area. As the depth information expands after each process with a low-pass filter, it becomes uniform due to the filter's smoothing effect. This uniformity makes the depth information inside the defective area similar to the depth information outside it. To address the compromise introduced by the low-pass filter, we reintegrated it with the raw depth matrix in each iteration. After several iterations, the depth information reaches a steady state, resulting in the recovery of depth information. For noise reduction, the ILF process starts from a low spatial frequency to a high spatial frequency, which contains most of the noise. The ILF effectively reduces the high spatial frequency component of noise while preserving most of the main signal, as the high-frequency noise does not contribute to the update of the initial expanded depth matrix. Here, we established a stop criterion to enhance the optimized performance and quality of the recovered depths. As each epoch of the Iterative Low-Pass Filter (ILF) yields an expanded depth matrix, it becomes essential to evaluate the termination point of the ILF process. To determine the optimal iteration steps, we analyzed the alternation between standard deviations in each epoch and the number of iterations. The stop criterion was designed to satisfy two conditions: the deviation between the results of consecutive iterations was sufficiently small, and it reached the maximum iteration limit.