## 2 The Algorithm
### Expansion
#### 2.2.2 Optimization  
As the initial position and normal are not accurate, iterative optimization based on photo-consistency is conducted to get the final accurate position and normal. Yet before the iterative optimization, a reference image and some auxiliary images need to be selected for each expanding point. For image selection, we mainly follow the methods in [14]. What we emphasize here is that only the best reference image is selected, which poses a view direction (i.e., one leading from the point to the optical center) as close as possible to the normal of the point and at a relatively shorter view distance as well. This is because only such an image can provide a relatively high resolution and relatively complete information of the region around the expanding point. After the selection of a reference image \(R\), several auxiliary images around it are selected, which provide view direction changes usually from 15 to 40 degrees. The former is selected to guarantee the highest possible reconstruction accuracy of the position of the expanding point on its tangent plane, whereas the later mainly serve to determine its position along its normal. After each iteration of optimization, auxiliary images with low similarity will be replaced and never be selected again if the reference image remains the same. After every four iterations, the position and normal of the point may increase in accuracy, and a new reference image with a view angle \(\theta_{72}\) (the angle between the normal of the point and the vector from the point to the center of the camera) smaller than the previous view angle \(\theta_{71}\) by an amount larger than \(th_{p}\), as indicated by Expression (5), will be considered to replace the previous one. The threshold \(th_{p}\) is set to prevent the frequent replacement of reference images. As the experiment shows, if the view angle is  
Figure 2: Illustration of the principle of homogeneous direct spatial expansion: (**a**) Direct expansion of seed points in 3D space along tangent planes; (**b**) The initial positions of points for the first three iterations of expansion. Black, green and yellow points stand for the first, second and third iteration, respectively. For convenience, these points are drawn on the same plane with equal separation; actually, after optimization, they would sit on different planes with slightly different separations. not large (\(\theta_{r2}<30\)), a small change of view angle (\(\theta_{r1}-\theta_{r2}<10\)) will produce little difference, even if the reference image is changed. However, a very small change of view angle (\(\theta_{r1}-\theta_{r2}>5\)) may result in significant improvement if the view angle is relatively large (\(\theta_{r2}>30\)). Once the reference image has been replaced, auxiliary images are reselected with all previously-replaced images reactivated. \[\theta_{r1}-\theta_{r2}\geq th_{p},\;th_{p}\;=\left\{ \begin{array}{cc}10& \theta_{r2}&<30\\ 5&\theta_{r2}&>30\end{array} \right. \tag{5}\]  
In Expression (5), \(\theta_{r1}\) and \(\theta_{r2}\) are the view angles in degree. After the selection of reference and auxiliary images, the iterative optimization based on photo-consistency can be conducted following [14]. When the final position and normal of an expanding point become accurate, the two \(m\)\(\times\)\(m\) windows centered at its projections, one in the reference image and one in the auxiliary image, would have a similar intensity distribution to Equation (6)  
\[I_{R}(s+i,t+j)=c_{K}\times I_{K}((s+i,t+j)^{K}) \tag{6}\]  
where \(I_{R}\) and \(I_{K}\) are the intensities in the reference image \(R\) and auxiliary image \(K\), respectively. (\(s\), \(t\)) is the pixel coordinate of the expanding point in the reference image. (\(s+i\), \(t+j\)) and (\(s+i\), \(t+j\))\({}^{K}\) are the (\(i\), \(j\))th pixel coordinates in the reference image and auxiliary image \(K\). \(i\), \(j=(m-1)/2\), \(\ldots\), \((m+1)/2\), \(c_{K}\) is a constant. \(K=1\), \(\ldots\), \(n\), with \(n\) being the number of auxiliary images.