# A Photometric Stereo Using Re-Projected Images for Active Stereo Vision System
## 3 Our Approach
### Re-Projection Method  
To use the photometric stereo measurement method, three or more images from a single camera with different light sources are needed. For the re-projection method, such images can be obtained by re-projecting the images obtained at various viewpoints in the stereo vision measurement process. If a reference view image (Figure 5b) and an image from a different view (Figure 5c) are obtained at two different viewpoints (as shown in Figure 5), a new image (Figure 5d) can be generated by re-projecting an image from view 2 to reference view 1. The projector in view 2 works as an external light source. As a result, the re-projected image (Figure 5d) can be assumed to be an image taken at view 1, but with a light source from view 2.  
Figure 4: (**a**) 3D scanning configuration at various viewpoints; (**b**) 3D scan data from position #1; (**c**) 3D scan data from position #2; (**d**) 3D scan data from position #3; (**e**) Registration result of the 3D scan data (**b–d**).  
Figure 5: (**a**) Image re-projection in two different positions; (**b**) Reference view image taken from left camera at position #1; (**c**) Other view image taken from left camera at position #2; (**d**) Re-projected image from the other view to the reference view which shows data acquired from position #2 in the perspective of camera #1.  
In the image generation process, the 3D scan data (\(X\)) obtained at another viewpoint in the local coordinate system is converted into 3D scan data (\(X^{\prime}\)) in the world coordinate system using the transformation matrix (P).  
\[X^{\prime}\big{(}x^{\prime},\,y^{\prime},\,z^{\prime}\big{)}=\mathrm{P}X \tag{1}\]  
The converted 3D scan data are applied to the function (\(g\)), including the radial and tangent distortion of the lens in accordance with the camera model, and are re-projected onto the image plane at the reference point.  
\[u^{\prime} =f\frac{x^{\prime}}{z^{\prime}} \tag{2}\] \[v^{\prime} =f\frac{y^{\prime}}{z^{\prime}}\] (3) \[(k_{1},k_{2}) =g\big{(}u^{\prime},v^{\prime}\big{)} \tag{4}\]  
Since the re-projected coordinates (\(k_{1},k_{2}\)) do not fit the camera pixel coordinates, they are converted to integer coordinates through bilinear interpolation. Because the corresponding points of the 3D scan data (\(X\)) obtained at the other viewpoints and the other viewpoint image pixels (\(u,v\)) can be known in the 3D scan data measurement process of the stereo vision, the corresponding points can be found between the other viewpoint image (\(u,v\)) and the reference viewpoint image pixel (\(u^{\prime},v^{\prime}\)). Thereafter, the intensity values (\(I_{1},\,\,I_{2},\,\,I_{3},\,\,I_{4}\)) of the image corresponding to the corresponding point are bilinearly interpolated to obtain a re-projected image as shown in Figure 6. Through this process, images obtained from various positions can be re-projected to the reference coordinate system to obtain images for use in photometric stereo.  
\[I_{p} =(1-t)\cdot\,\,I_{1}+t\cdot\,I_{3} \tag{5}\] \[I_{q} =(1-t)\cdot\,I_{2}+t\cdot\,I_{4}\] (6) \[I_{c} =(1-t)\cdot\,I_{p}+t\cdot\,I_{q} \tag{7}\]