PATENT ABSTRACT
A system is described for providing a three-dimensional representation of a scene from a single image. The system includes a reflector having a plurality of reflective surfaces for providing an interior reflective area defining a substantially quadrilateral cross section, wherein the reflector reflective surfaces are configured to provide nine views of an image. An imager is included for converting the nine-view image into digital data. Computer systems and computer program products for converting the data into three-dimensional representations of the scene are described.

PATENT DESCRIPTION
[0001]    This application claims the benefit of U.S. Provisional Patent Application Ser. No. 61/178,776, filed May 15, 2009, the entirety of the disclosure of which is incorporated herein by reference. 
     
    
     TECHNICAL FIELD 
       [0002]    The present invention relates to the art of three dimensional imaging. More particularly, the invention relates to devices and methods for three-dimensional imaging, capable of generating stereoscopic images and image-plus-depth utilizing a single imager and image. 
       COPYRIGHT 
       [0003]    A portion of the disclosure of this document contains materials subject to a claim of copyright protection. The copyright owner has no objection to the reproduction by anyone of the patent document or the patent disclosure as it appears in the U.S. Patent and Trademark Office patent files or records, but reserves all other rights with respect to the work. 
       BACKGROUND OF THE INVENTION 
       [0004]    Conventional stereo imaging systems require multiple imagers such as cameras, to obtain images of the same scene from different angles. The cameras are separated by a distance, similar to human eyes. A device such as a computer then calculates depths of objects in the scene by comparing images shot by the multiple cameras. This is typically done by shifting one image on top of the other one to identify matching points. The shifted amount is called the disparity. The disparity at which objects in the images best match is used by the computer to calculate their depths. 
         [0005]    Prior art multi-view imaging systems use only one camera to calculate the object depth. In most cases, such a system uses specially designed mirrored surfaces to create virtual cameras. With the views captured by the real camera and the virtual cameras the computer can use the same scheme as in classic computer vision to calculate the depth of an object. 
         [0006]    One prior art multi-view imaging system (Yuuki Uranishi, Mika Naganawa, Yoshihiro Yasumuro, Masataka Imura, Yoshitsugu Manabe, Kunihiro Chihara: Three-Dimensional Measurement System Using a Cylindrical Mirror,  SCIA  2005: 399-408) uses a cylindrical mirror (CM) to create virtual cameras. The CM is a hollow tube or chamber providing mirrored surfaces on the interior. The camera, equipped with a fish eye lens, captures the scene through the mirror. A CM can create infinitely many symmetric virtual cameras, one for each radial line, if the real camera lies on the center line of the CM, for each point in the captured image (image inside the center circle), correspondence can be found on some radial lines of the image. Another prior art system (U.S. Pat. No. 7,420,750) provides a cylindrical mirror device wherein a front end and rear end of the CM can have different dimensions. 
         [0007]    The advantage of such a cylindrical mirror device is that the user can always find corresponding points on the same diameter line of the image. This is because each radial slice of the captured image has its own virtual camera. However, this property requires that the optical axis pass through a center axis of the mirror and further that the optical axis be parallel to every mirror surface tangent plane. Such devices are difficult to calibrate, and generate heavily blurred images. A point on the object corresponds to a very large area in the reflection if that point is close to the center of the mirror. This is because the distance between the object and the virtual camera is much longer than the distance between the object and the real camera, but the focusing distances of the real and virtual cameras are still the same. The blurring of the images makes the work of identifying the corresponding point for a point on the object very difficult. 
         [0008]    Accordingly, a need is identified for an improved devices and method for multi-view imaging systems. The multi-view imaging systems set forth in the present disclosure provides a plurality of corresponding images from a single camera image, without the blurring of images noted in prior art systems. Still further, the present disclosure provides methods for deriving stereoscopic images and image-plus-depth utilizing a single imager and image. The described imaging system finds use in a variety of devices and applications, including without limitation (1) providing three-dimensional contents to three-dimensional photo frames, three-dimensional personal computer displays and three-dimensional television displays; (2) specialized lenses for document cameras and endoscopes so these devices can generate stereoscopic images and image-plus-depth; (3) three-dimensional Web cameras for personal computers and three-dimensional cameras for three-dimensional photo frames and mobile devices (such as intelligent cell-phones); (4) three-dimensional representations of the mouth and eyes of a patient. 
       SUMMARY OF THE INVENTION 
       [0009]    To solve the aforementioned and other problems, there are provided herein novel multi-view imaging systems. In accordance with a first aspect of the invention, a system is described for providing a three-dimensional representation of a scene from a single image. The system includes a reflector for providing an interior reflective area defining a substantially quadrilateral cross section, wherein the reflector reflective surfaces are configured to provide nine views of an image. In particular embodiments, the reflector may define a square or rectangle in side view, or may define an isosceles trapezoid in side view. An imager may be provided to convert the nine-view image from the reflector into digital data. The data may be rendered into stereoscopic images or image-plus-depth renderings. 
         [0010]    In another aspect there is provided a software for rendering a nine view image provided by the system described above into a stereoscopic image or an image-plus-depth rendering, including a first component for identifying a camera location relative to a scene of which a nine view image is to be taken, a second component for identifying a selected point in a central view of the nine view image and for identifying points corresponding to the selected point in the remaining eight views, and a third component for identifying a depth of the selected point or points in the central view. A fourth software component combines the corresponding points data and the depth data to provide a three-dimensional image. The second and third components may be the same, and/or may identify depth and corresponding points concurrently. 
         [0011]    In yet another aspect, there is provided a computing system for rendering a nine view image into a stereoscopic image or an image-plus-depth rendering. The computing system includes a camera for translating an image into a digital form, and a reflector as described above. There is also provided a computing device or processor for receiving data from the camera and converting those data as described above to provide a three-dimensional image from a single image obtained by the camera. 
         [0012]    These and other embodiments, aspects, advantages, and features will be set forth in the description which follows, and in part will become apparent to those of ordinary skill in the art by reference to the following description of the invention and referenced drawings or by practice of the invention. The aspects, advantages, and features of the invention are realized and attained by means of the instrumentalities, procedures, and combinations particularly pointed out in the appended claims. Unless otherwise indicated, any patent and non-patent references discussed herein are incorporated in their entirety into the present disclosure specifically by reference. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWING 
         [0013]    The accompanying drawings, incorporated herein and forming a part of the specification, illustrate several aspects of the present invention and together with the description serve to explain certain principles of the invention. In the drawings: 
           [0014]      FIG. 1  shows a square-tube mirror based imaging system according to the present disclosure; 
           [0015]      FIG. 2  shows the square-tube mirror (STM) of  FIG. 1 ; 
           [0016]      FIG. 3  graphically depicts a nine-view STM image; 
           [0017]      FIG. 4  shows a focused STM image; 
           [0018]      FIG. 5  shows a defocused STM image; 
           [0019]      FIG. 6  graphically depicts the corners of the central, left, and right views of  FIG. 3 ; 
           [0020]      FIG. 7  shows the image of  FIG. 6  after rotation, clipping, and translation; 
           [0021]      FIG. 8  depicts a point (P) in a virtual central view of an STM image, and the corresponding point (P′) in the virtual right view; 
           [0022]      FIG. 9  schematically shows a top view of an STM with d&gt;l; 
           [0023]      FIG. 10  schematically shows an STM wherein the field of view of a camera covers the STM and additional space; 
           [0024]      FIG. 11  schematically depicts an STM with d&gt;½ but d&lt;l; 
           [0025]      FIG. 12  schematically depicts a sloped STM; 
           [0026]      FIG. 13  depicts a labeled nine-view STM image; 
           [0027]      FIG. 14  shows an STM image angle α; 
           [0028]      FIG. 15  schematically depicts an STM wherein the field of view of the camera covers only a portion of the STM; 
           [0029]      FIG. 16  schematically depicts an STM wherein the field of view of the camera covers more than the entire STM; 
           [0030]      FIG. 17  schematically depicts a corresponding point P in a virtual right view of an STM; 
           [0031]      FIG. 18  schematically depicts a situation wherein a corresponding point of P does not exist in the virtual right view of an STM image; 
           [0032]      FIG. 19  schematically depicts a projection of a point onto a virtual image plane with respect to an actual camera; 
           [0033]      FIG. 20  schematically depicts a point in a virtual right view of an STM image, and that points counterpart in the STM image right view; 
           [0034]      FIG. 21  schematically depicts a point in a virtual left view of an STM image, and that points counterpart in the STM image left view; 
           [0035]      FIG. 22  schematically depicts a patient mouth reproducing device; 
           [0036]      FIG. 23  schematically depicts an STM-based intraoral imaging device; 
           [0037]      FIG. 24  schematically depicts an STM-based Web camera for three-dimensional imaging; and 
           [0038]      FIG. 25  shows a nine-view image provided by the STM-based Web camera of  FIG. 24 . 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0039]    In the following detailed description of the illustrated embodiments, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Also, it is to be understood that other embodiments may be utilized and that process, materials, reagent, and/or other changes may be made without departing from the scope of the present invention. 
       Square-Tube Mirror-Based Imaging Device 
       [0040]    In one aspect, the present disclosure provides a Square Tube Mirror-based Imaging System  10  (hereinafter STMIS; schematically depicted in  FIG. 1 ) comprising a square tube mirror (STM)  12 , an imager  14  such as a digital camera, a computer  16  and a set of stereoscopic image generation, depth computation, modeling and rendering programs. 
         [0041]    In one embodiment (see  FIG. 2 ), the STM  12  comprises an interior component  20  and an outer frame  22 . The interior component comprises four substantially identically-shaped planar reflective surfaces  24   a, b, c, d  supported by a housing  26 , defining in cross-section a substantially square shape. The reflective surfaces will be selected from materials which do not create double reflections. The STM  12  may define in side view a quadrilateral such as a square, a rectangle, a pyramid, an isosceles trapezoid, or the like. The outer frame  22  is provided to support the housing  26 , and to cooperatively connect the interior component with the lens  18  of an imager  14  such as a digital camera. Typically, the outer frame  22  will also comprise a mechanism to adjust a distance between the interior, reflective surfaces  24   a, b, c, d  and a pinhole (not shown) of the imager  14 . That is, the housing  26  is adapted to be slidably displaceable within the outer frame  22 , to allow incremental adjustment of a distance between an end of the interior component  20  and the imager  14  pinhole. The STM is connected to the lens  18  of the imager  16  by substantially conventional means, such as by providing cooperating threaded fasteners  28 ,  30  at a rear portion of the STM  12  and at a front portion of the imager  14  or lens  18 . 
         [0042]    A major feature of the described STM  12  is that, when a user views a scene, nine discrete views of the scene are provided, as if the user is viewing at the scene from nine different view points and orientations concurrently. This is because in addition to the center view, the user also is provided eight reflections of the scene from the four reflective surfaces  24   a, b, c, d.  Four of the views are generated by reflecting the scene once and four of the views are generated by reflecting the scene twice. Therefore, each picture taken by the camera in the present STM-based imaging system is composed of nine different views of the scene. Such a picture is called an STM image. The nine different views, arranged in a 3×3 rectangular grid, are composed of a central view, a left view, a right view, a lower view and a upper view (reflected once) and four corner views (reflected twice).  FIG. 3  provides a representation of the described grid. 
         [0043]    As will be described in greater detail below, information from these different views can be used by the software of the system to generate stereoscopic images or image-plus-depths of the objects in the scene. Thus, a user can generate 3D images with only one camera and one picture. Broadly, an image-plus-depth is generated by combining the central view of an STM image with a depth map computed for the central view of that STM image. Once a region in the central view of an STM image is specified, a stereoscopic image is generated by taking appropriate regions from the left view and the right view of that STM image and interlacing these regions. 
         [0044]    The reflective interior component of the STM may be made of any suitably reflective surface, with the proviso that double image reflections are to be avoided. Without intending any limitation, in particular embodiments, the reflective surfaces may be fabricated of stainless steel, aluminum, or any suitable reflective material which does not create double images. Typically, use of glass is avoided due to the generation of double reflections thereby. The housing  26  for the reflective surfaces  24   a, b, c, d  may be made of any suitable material, such as metal, plastics, other polymers, and the like. In one embodiment, the housing  26  is molded as a unitary housing  26 , of polymethylmethacrylate (PMMA) or any other suitable plastic or polymer. 
       Stereoscopic Images 
       [0045]    Herein is described a technique to generate stereoscopic images using an STMIS as described. Broadly, the method comprises, for a specified region in a central view of an STM image (see  FIG. 3 ), there is identified the corresponding regions in the left view and the right view of the image (as a non-limiting example, the ‘Right image’ and the ‘Left image’ shown in  FIG. 3 ). Next, the images are interlaced, and the interlaced image can then be viewed with a pair of special glasses (shutter glasses or polarized glasses) on a 3D display panel designed for stereoscopic images. Such specialized glasses and 3D display panels are known in the art. 
         [0046]    For this particular application the left view and the right view first require rectification. The accuracy of the rectification process relies on accurate identification of the central view, the left view and the right view. In the following we show how to accurately identify the bounding edges of these views and then how to perform the rectification process. 
         [0047]    First, the bounding edges of the central view are identified via a focus/defocus process. A first image of a scene is acquired (see  FIG. 4 ). Typically, bounding edges of the central view appear blurry if the scene is not close to the front end of the STM. This is because the focus is on the scene of which an image is being taken, not on the front end of the STM. To accurately identify the bounding edges of the central view, it is necessary to acquire a second image of the scene, but with the smallest camera iris this time. This second image will be exactly the same as the first image except now the scene is not as clear but the bounding edges of the central view in the second image are very clear and sharp (see  FIG. 5 ). By clearly identifying the bounding edges of the central view in the second image, it is possible to compute the four corners of the central view in the second image. 
         [0048]    These four corners of the second view in the second image correspond to the four corners of the central view in the first image. The coordinates of these corners are defined as (x 1 ,y 1 ), (x 2 ,y 2 ), (x 3 ,y 3 ), and (x 4 ,y 4 ) (see  FIG. 6 ) with respect to the lower-left corner of the image, i.e., the lower-left corner is the origin of the coordinate system of the image. 
         [0049]    Next job is identification of (x 5 ,y 5 ), (x 6 ,y 6 ), (x 7 ,y 7 ) and (x 8 ,y 8 ) (see  FIG. 6 ). Without loss of generality, it is presumed that (x 1 ,y 1 ) and (x 2 ,y 2 ) lie on the same horizontal line. If this is not the case, the image must simply be rotated about the center of the central view for β degrees where β=tan −1 ((y 2 −y 1 )/(x 2 −x 1 )) and do a clipping after the rotation to ensure the image is an orthogonal rectangle (see  FIG. 7 ). A translation is also necessary if center of the new central view does not coincide with the center of the original image. 
         [0050]    The following theorem is used for the rectification process: 
         [0051]    THEOREM 1 If P=(X,Y,−l) is a point in the virtual central view corresponding to a given STM image and P′ is the corresponding point of P in the virtual right view, then the edge AP′ makes an angle of α degrees with the horizontal line y=Y and α is a function of Y 
         [0000]    
       
         
           
             
               
                 
                   α 
                   = 
                   
                     
                       tan 
                       
                         - 
                         1 
                       
                     
                     ( 
                     
                       
                         Y 
                          
                         
                             
                         
                          
                         
                           sin 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                       
                         
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                            
                           
                             cos 
                              
                             
                               ( 
                               
                                 2 
                                  
                                 θ 
                               
                               ) 
                             
                           
                         
                         + 
                         
                           h 
                            
                           
                               
                           
                            
                           
                             sin 
                              
                             
                               ( 
                               
                                 2 
                                  
                                 θ 
                               
                               ) 
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0052]    PROOF Since tan α=|P′B|/|AB|, we need to find |P′B| and |AB| in order to compute α. By Theorem 1, we have 
         [0000]    
       
         
           
             
               P 
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         X 
                         + 
                         
                           2 
                            
                           
                             σ 
                             r 
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     Y 
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
     
         [0053]    Hence, 
         [0000]    
       
         
           
             
               
                 
                   
                      
                     
                       
                         P 
                         ′ 
                       
                        
                       B 
                     
                      
                   
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         Y 
                       
                       
                         d 
                         + 
                         l 
                         - 
                         
                           2 
                            
                           
                             σ 
                             r 
                           
                            
                           sin 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                     
                     - 
                     Y 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         Y 
                       
                       - 
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         Y 
                       
                       + 
                       
                         2 
                          
                         Y 
                          
                         
                             
                         
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       2 
                        
                       
                         Y 
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         cos 
                          
                         
                             
                         
                          
                         θ 
                          
                         
                             
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                 
               
             
           
         
       
     
         [0054]    On the other hand, 
         [0000]    
       
         
           
             
               
                 
                   
                      
                     AB 
                      
                   
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             X 
                             + 
                             
                               2 
                                
                               
                                 σ 
                                 r 
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                       
                         d 
                         + 
                         l 
                         - 
                         
                           2 
                            
                           
                             σ 
                             r 
                           
                            
                           sin 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                     
                     - 
                     h 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             X 
                             + 
                             
                               2 
                                
                               
                                 σ 
                                 r 
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                       - 
                       
                         h 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                       
                       + 
                       
                         2 
                          
                         h 
                          
                         
                             
                         
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             X 
                             - 
                             h 
                           
                           ) 
                         
                       
                       + 
                       
                         2 
                          
                         
                           
                             σ 
                             r 
                           
                            
                           
                             ( 
                             
                               
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                   ) 
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 h 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                             ) 
                           
                         
                       
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             X 
                             - 
                             h 
                           
                           ) 
                         
                       
                       + 
                       
                         2 
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         cos 
                          
                         
                             
                         
                          
                         
                           θ 
                            
                           
                             ( 
                             
                               
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                   ) 
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 h 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                             ) 
                           
                         
                       
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             X 
                             - 
                             h 
                           
                           ) 
                         
                       
                       + 
                       
                         2 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         
                           cos 
                           2 
                         
                          
                         θ 
                       
                       + 
                       
                         2 
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         h 
                          
                         
                             
                         
                          
                         cos 
                          
                         
                             
                         
                          
                         θ 
                          
                         
                             
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             
                               2 
                                
                               
                                 cos 
                                 2 
                               
                                
                               θ 
                             
                             - 
                             1 
                           
                           ) 
                         
                       
                       + 
                       
                         2 
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         h 
                          
                         
                             
                         
                          
                         cos 
                          
                         
                             
                         
                          
                         θ 
                          
                         
                             
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         
                           cos 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                       + 
                       
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         h 
                          
                         
                             
                         
                          
                         
                           sin 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           σ 
                           r 
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                 
               
             
           
         
       
     
         [0055]    Therefore, 
         [0000]    
       
         
           
             
               
                 
                   
                     tan 
                      
                     
                         
                     
                      
                     α 
                   
                   = 
                   
                     
                       2 
                        
                       
                         Y 
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         
                           cos 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                       + 
                       
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         h 
                          
                         
                             
                         
                          
                         
                           sin 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       Y 
                        
                       
                           
                       
                        
                       
                         sin 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           cos 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                       + 
                       
                         h 
                          
                         
                             
                         
                          
                         
                           sin 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                     
                   
                 
               
             
           
         
       
     
         [0056]    And the theorem is proved. Ξ 
         [0057]    It is assumed that, after the steps of rotation, clipping and translation, the widths of the left view, the central view and the right view are m 3 , m and m 2 , respectively, and the heights of the lower view, the central view and the upper view are n 2 , n and n 3 , respectively (see  FIG. 6 ). Hence, for the right view, the rectification process is to restore it into an image of dimension m 2 ×n. For the left view, the rectification is to restore it into an image of dimension m 3 ×n. We will show the rectification for the right view only. The rectification process for the left view is similar. 
         [0058]    Let I R  be an image array of dimension m 2 ×n. The rectified right view is stored into I R . Another assumption is that the given STM image (after the rotation, clipping and translation steps) is stored in the image array I of dimension (m 3 +m+m 2 )×(n 2 +n+n 3 ). Hence, the question to be answered is:
       for J=0 to n−1   for i=0 to m 2 −1   I R (i,j)=?       
 
         [0062]    One method for calculation is as follows: 
         [0063]    First, for the given entry (i,j), its corresponding entry (M,N,−l) is found in the virtual right view of the virtual image plane. M and N are defined as follows: 
         [0000]    
       
         
           
             
               
                 
                   { 
                   
                     
                       
                         
                           M 
                           = 
                           
                             
                               
                                 ( 
                                 
                                   i 
                                   + 
                                   
                                     1 
                                     2 
                                   
                                 
                                 ) 
                               
                                
                               
                                 
                                   2 
                                    
                                   h 
                                 
                                 m 
                               
                             
                             + 
                             h 
                           
                         
                       
                     
                     
                       
                         
                           N 
                           = 
                           
                             
                               
                                 ( 
                                 
                                   j 
                                   + 
                                   
                                     1 
                                     2 
                                   
                                 
                                 ) 
                               
                                
                               
                                 
                                   2 
                                    
                                   h 
                                 
                                 n 
                               
                             
                             - 
                             h 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
         [0064]    Next is the step of finding the entry (X,N,−l) in the virtual central view such that 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       
                         [ 
                         
                           X 
                           + 
                           
                             2 
                              
                             
                               ( 
                               
                                 h 
                                 - 
                                 X 
                               
                               ) 
                             
                              
                             
                               cos 
                               2 
                             
                              
                             θ 
                           
                         
                         ] 
                       
                     
                     
                       d 
                       + 
                       l 
                       - 
                       
                         2 
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                          
                         cos 
                          
                         
                             
                         
                          
                         θ 
                          
                         
                             
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                   = 
                   M 
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0065]    Such an X is defined as follows: 
         [0000]    
       
         
           
             
               
                 
                   X 
                   = 
                   
                     
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         h 
                          
                         
                             
                         
                          
                         
                           cos 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                       + 
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             h 
                             - 
                             X 
                           
                           ) 
                         
                       
                       + 
                       
                         Mh 
                          
                         
                             
                         
                          
                         
                           sin 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                     
                     
                       
                         M 
                          
                         
                             
                         
                          
                         
                           sin 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                       + 
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           cos 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0066]    Then based on Theorem 1, the corresponding point of (X,N,−l) in the virtual right view is computed as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     ( 
                     
                       M 
                       , 
                       
                         N 
                         _ 
                       
                       , 
                       
                         - 
                         l 
                       
                     
                     ) 
                   
                   = 
                   
                     ( 
                     
                       M 
                       , 
                       
                         
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                            
                           N 
                         
                         
                           d 
                           + 
                           l 
                           - 
                           
                             2 
                              
                             
                               ( 
                               
                                 h 
                                 - 
                                 X 
                               
                               ) 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                       
                       , 
                       
                         - 
                         1 
                       
                     
                     ) 
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
         [0067]    Next, the corresponding location of (M,  N ,−l) in the right view of the STM image I is computed as follows: 
         [0000]      (m 3 +m+i,n 2 +  j )   (6) 
         [0068]    where 
         [0000]    
       
         
           
             
               
                 
                   
                     j 
                     _ 
                   
                   = 
                   
                     
                       
                         ( 
                         
                           
                             N 
                             _ 
                           
                           + 
                           h 
                         
                         ) 
                       
                        
                       
                         n 
                         
                           2 
                            
                           h 
                         
                       
                     
                     - 
                     
                       1 
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
         [0069]    By combining (7) with (5) and (4), we get the following expression for  j : 
         [0000]    
       
         
           
             
               
                 
                   
                     j 
                     _ 
                   
                   = 
                   
                     
                       
                         [ 
                         
                           
                             
                               N 
                                
                               
                                 [ 
                                 
                                   
                                     M 
                                      
                                     
                                         
                                     
                                      
                                     
                                       sin 
                                        
                                       
                                         ( 
                                         
                                           2 
                                            
                                           θ 
                                         
                                         ) 
                                       
                                     
                                   
                                   + 
                                   
                                     
                                       ( 
                                       
                                         d 
                                         + 
                                         l 
                                       
                                       ) 
                                     
                                      
                                     
                                       cos 
                                        
                                       
                                         ( 
                                         
                                           2 
                                            
                                           θ 
                                         
                                         ) 
                                       
                                     
                                   
                                 
                                 ] 
                               
                             
                             
                               
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                   ) 
                                 
                                  
                                 
                                   cos 
                                    
                                   
                                     ( 
                                     
                                       2 
                                        
                                       θ 
                                     
                                     ) 
                                   
                                 
                               
                               + 
                               
                                 h 
                                  
                                 
                                     
                                 
                                  
                                 
                                   sin 
                                    
                                   
                                     ( 
                                     
                                       2 
                                        
                                       θ 
                                     
                                     ) 
                                   
                                 
                               
                             
                           
                           + 
                           h 
                         
                         ] 
                       
                        
                       
                         n 
                         
                           2 
                            
                           h 
                         
                       
                     
                     - 
                     
                       1 
                       2 
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
         [0070]    where M and N are defined in (5-2).  j  is typically a real number, not an integer. 
         [0071]    Once we have the indices defined in (6) and the value of  j  defined in (8), we can compute I R (i,j) as follows: 
         [0072]    (a) if j=(n−1)/2 then I R (i,j)=I(m 3 +m+i,n 2 +j) 
         [0073]    (b) if j&gt;(n−1)/2 and l≦  j &lt;l+1 for some l≧j then 
         [0000]        I   R ( i,j )=(   j −l ) I ( m   3   +m+i,n   2   +l+ 1)+( l+ 1−   j   ) I ( m   3   +m+i,n   2   +l ) 
         [0074]    (c) if j&lt;(n−1)/2 and k−1&lt;  j ≦k for some k≦j then 
         [0000]        I   R ( i,j )=(   j + 1− k ) I ( m+m+i,n   2   +k )+( k−  j   ) I ( m   3   +m+i,n   2   +k− 1) 
         [0075]    An alternative method for computing  j  may be done via a shorter process. It can be seen that the right view of the STM image is similar to the virtual right view of the virtual image. Therefore, if Q=(i,j) and Q′=(i,  j ) in the right view of the STM image correspond to B=(M,N,−l) and P′=(M,  N ,−l) in the virtual right view, respectively, then the angle ∠P′ AB in  FIG. 8  and the angle ∠Q′DQ in  FIG. 9  must be the same. Consequently, by THEOREM 1, it is possible to compute the point Q′=(i,  j ) in the right view of the STM image by solving the following equation: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                        
                       
                         
                           Q 
                           ′ 
                         
                          
                         Q 
                       
                        
                     
                     
                        
                       DQ 
                        
                     
                   
                   = 
                   
                     tan 
                      
                     
                         
                     
                      
                     α 
                   
                 
               
               
                 
                   ( 
                   9 
                   ) 
                 
               
             
           
         
       
     
         [0076]    for Q′ where D=(−½,j). Note that in the right view of the STM image, it is D, not (0,j) (see  FIG. 9 ), that corresponds to A in the virtual right view (see  FIG. 8 ). Since the aspect ratio of the STM image is: 
         [0000]    
       
         
           
             
               Aspect 
                
               
                   
               
                
               ratio 
             
             = 
             
               
                 x 
                 y 
               
               = 
               
                 
                   
                     2 
                      
                     
                       h 
                       / 
                       m 
                     
                   
                   
                     2 
                      
                     
                       h 
                       / 
                       n 
                     
                   
                 
                 = 
                 
                   n 
                   m 
                 
               
             
           
         
       
     
         [0077]    (9) can be written as 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         j 
                         _ 
                       
                       - 
                       j 
                     
                     
                       
                         [ 
                         
                           i 
                           - 
                           
                             ( 
                             
                               - 
                               
                                 1 
                                 2 
                               
                             
                             ) 
                           
                         
                         ] 
                       
                        
                       
                         n 
                         m 
                       
                     
                   
                   = 
                   
                     
                       tan 
                        
                       
                           
                       
                        
                       α 
                     
                     = 
                     
                       
                         N 
                          
                         
                             
                         
                          
                         
                           sin 
                            
                           
                             ( 
                             
                               2 
                                
                               θ 
                             
                             ) 
                           
                         
                       
                       
                         
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                            
                           
                             cos 
                              
                             
                               ( 
                               
                                 2 
                                  
                                 θ 
                               
                               ) 
                             
                           
                         
                         + 
                         
                           h 
                            
                           
                               
                           
                            
                           
                             sin 
                              
                             
                               ( 
                               
                                 2 
                                  
                                 θ 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
         [0078]    From (2), we have 
         [0000]    
       
         
           
             
               i 
               = 
               
                 
                   
                     ( 
                     
                       M 
                       - 
                       h 
                     
                     ) 
                   
                    
                   
                     m 
                     
                       2 
                        
                       h 
                     
                   
                 
                 - 
                 
                   1 
                   2 
                 
               
             
             ; 
             
               j 
               = 
               
                 
                   
                     ( 
                     
                       N 
                       + 
                       h 
                     
                     ) 
                   
                    
                   
                     n 
                     
                       2 
                        
                       h 
                     
                   
                 
                 - 
                 
                   1 
                   2 
                 
               
             
           
         
       
     
         [0079]    Hence, from (10) we have 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           j 
                           _ 
                         
                         = 
                         
                           
                             
                               ( 
                               
                                 N 
                                 + 
                                 h 
                               
                               ) 
                             
                              
                             
                               n 
                               
                                 2 
                                  
                                 h 
                               
                             
                           
                           - 
                           
                             1 
                             2 
                           
                           + 
                           
                             
                               
                                 N 
                                  
                                 
                                     
                                 
                                  
                                 
                                   sin 
                                    
                                   
                                     ( 
                                     
                                       2 
                                        
                                       θ 
                                     
                                     ) 
                                   
                                 
                               
                               
                                 
                                   
                                     ( 
                                     
                                       d 
                                       + 
                                       l 
                                     
                                     ) 
                                   
                                    
                                   
                                     cos 
                                      
                                     
                                       ( 
                                       
                                         2 
                                          
                                         θ 
                                       
                                       ) 
                                     
                                   
                                 
                                 + 
                                 
                                   h 
                                    
                                   
                                       
                                   
                                    
                                   
                                     sin 
                                      
                                     
                                       ( 
                                       
                                         2 
                                          
                                         θ 
                                       
                                       ) 
                                     
                                   
                                 
                               
                             
                              
                             
                               ( 
                               
                                 M 
                                 - 
                                 h 
                               
                               ) 
                             
                              
                             
                               m 
                               
                                 2 
                                  
                                 h 
                               
                             
                              
                             
                               n 
                               m 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           
                             
                               ( 
                               
                                 N 
                                 + 
                                 h 
                                 + 
                                 
                                   
                                     N 
                                      
                                     
                                         
                                     
                                      
                                     
                                       sin 
                                        
                                       
                                         ( 
                                         
                                           2 
                                            
                                           θ 
                                         
                                         ) 
                                       
                                     
                                      
                                     
                                       ( 
                                       
                                         M 
                                         - 
                                         h 
                                       
                                       ) 
                                     
                                   
                                   
                                     
                                       
                                         ( 
                                         
                                           d 
                                           + 
                                           l 
                                         
                                         ) 
                                       
                                        
                                       
                                         cos 
                                          
                                         
                                           ( 
                                           
                                             2 
                                              
                                             θ 
                                           
                                           ) 
                                         
                                       
                                     
                                     + 
                                     
                                       h 
                                        
                                       
                                           
                                       
                                        
                                       
                                         sin 
                                          
                                         
                                           ( 
                                           
                                             2 
                                              
                                             θ 
                                           
                                           ) 
                                         
                                       
                                     
                                   
                                 
                               
                               ) 
                             
                              
                             
                               n 
                               
                                 2 
                                  
                                 h 
                               
                             
                           
                           - 
                           
                             1 
                             2 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           
                             
                               ( 
                               
                                 
                                   
                                     N 
                                      
                                     
                                       [ 
                                       
                                         
                                           M 
                                            
                                           
                                               
                                           
                                            
                                           
                                             sin 
                                              
                                             
                                               ( 
                                               
                                                 2 
                                                  
                                                 θ 
                                               
                                               ) 
                                             
                                           
                                         
                                         + 
                                         
                                           
                                             ( 
                                             
                                               d 
                                               + 
                                               l 
                                             
                                             ) 
                                           
                                            
                                           
                                             cos 
                                              
                                             
                                               ( 
                                               
                                                 2 
                                                  
                                                 θ 
                                               
                                               ) 
                                             
                                           
                                         
                                       
                                       ] 
                                     
                                   
                                   
                                     
                                       
                                         ( 
                                         
                                           d 
                                           + 
                                           l 
                                         
                                         ) 
                                       
                                        
                                       
                                         cos 
                                          
                                         
                                           ( 
                                           
                                             2 
                                              
                                             θ 
                                           
                                           ) 
                                         
                                       
                                     
                                     + 
                                     
                                       h 
                                        
                                       
                                           
                                       
                                        
                                       
                                         sin 
                                          
                                         
                                           ( 
                                           
                                             2 
                                              
                                             θ 
                                           
                                           ) 
                                         
                                       
                                     
                                   
                                 
                                 + 
                                 h 
                               
                               ) 
                             
                              
                             
                               n 
                               
                                 2 
                                  
                                 h 
                               
                             
                           
                           - 
                           
                             1 
                             2 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
         [0080]    (11) is exactly the same as (8). 
         [0081]    The computation process of I R (i,j) is the same as the one shown previously. 
         [0082]    Once the left view and the right view of the STM image are rectified as described herein, the generation of stereoscopic images is relatively straightforward. For any specified region in the central view, the corresponding regions in the rectified left view and the rectified right view are identified, divided by 78% and interlaced. Next, the interlaced image is output to a display panel designed for stereoscopic images. Such panels are known in the art. 
         [0083]    Consideration was given to the physical proportions of the STM, and to the relationship between the STM and the imager. Table 1 defines notations used subsequently. 
         [0000]    
       
         
               
               
             
           
               
                 TABLE 1 
               
               
                   
               
               
                 Nomenclature 
                   
               
               
                 Notation 
                 Meaning 
               
               
                   
               
             
             
               
                 1 
                 Length of the STM 
               
               
                 2r × 2r 
                 Dimension of the rear end (adjacent to the camera lens) 
               
               
                 2h × 2h 
                 Dimension of the front end 
               
               
                 1 × 2r × 2h 
                 Dimension of each mirror (each mirror is of the shape of 
               
               
                   
                 an           es trapezoid with 1, 2r and 2h being its height, top 
               
               
                   
                 side length           ttom side length, respectively) 
               
               
                 d 
                 Distance between pinhole of the camera and center point 
               
               
                   
                 of            rear end 
               
               
                 θ 
                 Slope of the interior of the hollow tube (angle between 
               
               
                   
                 the and the optical center of the tube) 
               
               
                 2α 
                 Field of view (or, angle of view) of the camera 
               
               
                 φ 
                 Effective field of view of the camera 
               
               
                   
               
               
                             indicates data missing or illegible when filed 
               
             
          
         
       
     
         [0084]    a. Parallel STM 
         [0085]    First was the case that the interior slope of the STM is zero, i.e., θ=0. In this case, we have r=h and the mirrors form two pairs of parallel sets: (top mirror, bottom mirror) and (left mirror, right mirror). Each mirror is a rectangle of dimension 2r×l. We refer to this case as parallel STM. 
         [0086]    Considering the situation of an STM with d&gt;l, where EF, HG, FG and EH are the top views of the left mirror, the right mirror, the front end and the rear end, respectively ( FIG. 9 ), the plane passing through the front end of the STM was defined as the projection plane or image plane. Any thing the real camera C can see (in the angular sector bounded by CF and CG) will be projected onto the image plane between F and G. Therefore, FG is also the top view of the scene image (central view). V l  and V r  are locations of the virtual cameras with respect to the left mirror and the right mirror, respectively. Anything the virtual camera V l  could see (in the angular sector bounded by V l E and V l F) was projected onto the image plane as the left view. KF is the top view of that image. Similarly, anything the virtual camera V r  can see (in the angular sector bounded by V r H and V r G) will be projected onto the image plane as the right view. GL′ is the top view of that image. 
         [0087]    Points I, J, Y and Z play important roles here. They are the four vertices of the trinocular region IZJY. If a point is outside this region, it can be seen by the real camera C, but not by virtual camera V l  or V r , or both. Such a point will not appear in the left view or the right view, or both. Consequently, one will not be able to find one of the to corresponding points (or both) for such a point in the generation of a stereoscopic image or in the computation of the depth value. In general, to ensure enough information is obtained for stereoscopic image generation or depth computation, the scene to be shot by the real camera should be inside the trinocular region. Hence, a good STM should make the distance between I and J long enough and the width between Y and Z wide enough. These points can be computed as follows. 
         [0088]    Let the distance between O and I be k and the distance between N and J be m. Since triangle V l CI is similar to triangle EOI, we have 
         [0000]    
       
         
           
             
               
                 2 
                  
                 r 
               
               r 
             
             = 
             
               
                 
                   d 
                   + 
                   k 
                 
                 k 
               
               . 
             
           
         
       
     
         [0000]    Hence, k=d. 
         [0089]    To compute J, note that triangle V l CJ is similar to triangle FNJ. Hence, we have 
         [0000]    
       
         
           
             
               
                 2 
                  
                 r 
               
               r 
             
             = 
             
               
                 d 
                 + 
                 l 
                 + 
                 m 
               
               m 
             
           
         
       
     
         [0000]    or m=d+l. Therefore, the distance between I and J is 2l. 
         [0090]    To compute Y, note that this is the intersection point of rays V l F and V r H which can be parameterized as follows: 
         [0000]        L ( t )= V   l   +t ( F−V   l ) t ∈ R    
         [0000]        L   1 ( s )= V   r   +s ( H−V   r ) s ∈ R    
         [0091]    The intersection point is a point where L(t 1 )=L 1 (s 1 ) for some t 1  and s 1 . By imposing a coordinate system on the STM with O as the origin, OH as the positive x-axis and OC as the positive z-axis, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           L 
                            
                           
                             ( 
                             
                               t 
                               1 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             V 
                             l 
                           
                           + 
                           
                             
                               t 
                               1 
                             
                              
                             
                               ( 
                               
                                 F 
                                 - 
                                 
                                   V 
                                   l 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           
                             ( 
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 r 
                               
                               , 
                               0 
                               , 
                               d 
                             
                             ) 
                           
                           + 
                           
                             
                               t 
                               1 
                             
                              
                             
                               [ 
                               
                                 
                                   ( 
                                   
                                     
                                       - 
                                       r 
                                     
                                     , 
                                     0 
                                     , 
                                     
                                       - 
                                       l 
                                     
                                   
                                   ) 
                                 
                                 - 
                                 
                                   ( 
                                   
                                     
                                       
                                         - 
                                         2 
                                       
                                        
                                       r 
                                     
                                     , 
                                     0 
                                     , 
                                     d 
                                   
                                   ) 
                                 
                               
                               ] 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           ( 
                           
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 r 
                               
                               + 
                               
                                 
                                   t 
                                   1 
                                 
                                  
                                 r 
                               
                             
                             , 
                             0 
                             , 
                             
                               d 
                               - 
                               
                                 
                                   t 
                                   1 
                                 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                   ) 
                                 
                               
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
             
             
               
                 and 
               
             
             
               
                 
                   
                     
                       
                         
                           
                             L 
                             1 
                           
                            
                           
                             ( 
                             
                               s 
                               1 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             V 
                             r 
                           
                           + 
                           
                             
                               s 
                               1 
                             
                              
                             
                               ( 
                               
                                 H 
                                 - 
                                 
                                   V 
                                   r 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           
                             ( 
                             
                               
                                 2 
                                  
                                 r 
                               
                               , 
                               0 
                               , 
                               d 
                             
                             ) 
                           
                           + 
                           
                             
                               s 
                               1 
                             
                              
                             
                               [ 
                               
                                 
                                   ( 
                                   
                                     r 
                                     , 
                                     0 
                                     , 
                                     0 
                                   
                                   ) 
                                 
                                 - 
                                 
                                   ( 
                                   
                                     
                                       2 
                                        
                                       r 
                                     
                                     , 
                                     0 
                                     , 
                                     d 
                                   
                                   ) 
                                 
                               
                               ] 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           ( 
                           
                             
                               
                                 2 
                                  
                                 r 
                               
                               - 
                               
                                 
                                   s 
                                   1 
                                 
                                  
                                 r 
                               
                             
                             , 
                             0 
                             , 
                             
                               d 
                               - 
                               
                                 
                                   s 
                                   1 
                                 
                                  
                                 d 
                               
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
             
           
         
       
     
         [0092]    For L(t 1 ) to be the same as L 1 (s 1 ), we must have 
         [0000]      −2 r+t   1   r= 2 r−s   1   r    
         [0000]        d−t   1 ( d+l )= d−s   1   d    
         [0093]    Solving this system of linear equations, we get t 1 =4d/(2d+l) and, consequently, 
         [0000]    
       
         
           
             
               
                 
                   Y 
                   = 
                   
                     L 
                      
                     
                       ( 
                       
                         t 
                         1 
                       
                       ) 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       
                         
                           
                             - 
                             2 
                           
                            
                           r 
                         
                         + 
                         
                           
                             4 
                              
                             rd 
                           
                           
                             
                               2 
                                
                               d 
                             
                             + 
                             l 
                           
                         
                       
                       , 
                       0 
                       , 
                       
                         d 
                         - 
                         
                           
                             4 
                              
                             
                               d 
                                
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                             
                           
                           
                             
                               2 
                                
                               d 
                             
                             + 
                             l 
                           
                         
                       
                     
                     ) 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               - 
                               2 
                             
                              
                             rl 
                           
                           
                             
                               2 
                                
                               d 
                             
                             + 
                             l 
                           
                         
                         , 
                         0 
                         , 
                         
                           
                             - 
                             
                               d 
                                
                               
                                 ( 
                                 
                                   
                                     2 
                                      
                                     d 
                                   
                                   + 
                                   
                                     3 
                                      
                                     l 
                                   
                                 
                                 ) 
                               
                             
                           
                           
                             
                               2 
                                
                               d 
                             
                             + 
                             l 
                           
                         
                       
                       ) 
                     
                     . 
                   
                 
               
             
           
         
       
     
         [0094]    Using property of symmetry, we have 
         [0000]    
       
         
           
             Z 
             = 
             
               ( 
               
                 
                   
                     2 
                      
                     rl 
                   
                   
                     
                       2 
                        
                       d 
                     
                     + 
                     l 
                   
                 
                 , 
                 0 
                 , 
                 
                   
                     - 
                     
                       d 
                        
                       
                         ( 
                         
                           
                             2 
                              
                             d 
                           
                           + 
                           
                             3 
                              
                             l 
                           
                         
                         ) 
                       
                     
                   
                   
                     
                       2 
                        
                       d 
                     
                     + 
                     l 
                   
                 
               
               ) 
             
           
         
       
     
         [0095]    Hence, the width between Y and Z is 4rl(2d+l). Summarizing the above results, we have 
         [0000]        I= (0,0, −d ); 
         [0000]        J= (0,0, −d− 2 l ) 
         [0000]      | IJ|= 2 l;    
         [0000]    
       
         
           
             
                
               YZ 
                
             
             = 
             
               
                 4 
                  
                 rl 
               
               
                 
                   2 
                    
                   d 
                 
                 + 
                 l 
               
             
           
         
       
     
         [0096]    These are important results because they tell us how a parallel STM should be designed. 
         [0097]    First, to ensure the trinocular region IZJY can be used for scene shooting as much as possible, point I should be inside the region GRQF (see  FIG. 9 ) instead of the STM, i.e., I should be to the right of N. Since z-component of 1 is −d and z-component of N is −l, this means that d should be greater than l. However, one should not make d too large because large d (not even excessively) could cause FOV of the camera to cover too much extra space other than the STM itself such as the areas to between K and W, and X and L in  FIG. 9 . These areas do not contain information related to the scene and, therefore, are of no use to the 3D image generation process. 
         [0098]    The distance between Y and Z and the locations of Y and Z actually are more critical in most of the applications because they determines if a scene can fit into the trinocular region IZJY. To ensure the widest part of the trinocular region can be used for the given scene, these points must be to the right of N, i.e., 
         [0000]    
       
         
           
             
               
                 - 
                 
                   d 
                    
                   
                     ( 
                     
                       
                         2 
                          
                         d 
                       
                       + 
                       
                         3 
                          
                         l 
                       
                     
                     ) 
                   
                 
               
               
                 
                   2 
                    
                   d 
                 
                 + 
                 l 
               
             
             &lt; 
             
               
                 - 
                 l 
               
                
               
                   
               
                
               or 
                
               
                   
               
                
               d 
             
             &gt; 
             
               
                 1 
                 2 
               
               . 
             
           
         
       
     
         [0099]    An example with d&gt;l/2 is shown in  FIG. 10 . Since d is smaller than l here, the apex of the trinocular region, I, is inside the STM while Y and Z are to the right of N. Actually it is easy to see that when l/2&lt;d&lt;l, we have 
         [0000]    
       
         
           
             
               2 
                
               r 
             
             &gt; 
             
               
                 4 
                  
                 rl 
               
               
                 
                   2 
                    
                   d 
                 
                 + 
                 l 
               
             
             &gt; 
             
               
                 4 
                  
                 r 
               
               3 
             
           
         
       
     
         [0100]    Hence, one can increase the distance between Y and Z (width of the trinocular region) by increasing the value of r (see  FIG. 11  for an example). However, it should be pointed out that increasing the value of r would reduce the coverage of the STM by the FOV of the camera (see  FIG. 11 ) and, consequently, reduce the size of the left view and the right view (actually the upper view and the lower view as well). Actually, if r satisfies the following condition 
         [0000]        r≧ ( d+l )tan α 
         [0101]    one will not get a left view or a right view at all because in such a ease 2α would be smaller than the effective FOV of the camera, φ. 
         [0102]    Based on the above analysis, we can see that when the scene is close to the STM, one can use most of the trinocular region IZJY for scene shooting if l/2&lt;d. One can increase the length of the trinocular region IZJY by increasing the length of the STM and increase its width by increasing the value of r. In general, a parallel STM was found suitable for imaging scenes close to the STM only. 
         [0103]    b. Sloped STM 
         [0104]    We next considered the case that the four interior sides of the STM make a positive angle θ with the optical center of the STM (and, therefore, the front of the STM closest to the image is larger than its rear closest to the camera). We refer to this case as sloped STM. An example is shown in  FIG. 12 . In the following, we will develop/show design criteria for STMs of this type. 
         [0105]    We assume O is the origin of the 3D coordinate system, i.e., O=(0,0,0), the optical center of the STM is the z-axis with C being in the positive direction, and OH is the positive x-axis. Hence, we have C=(0,0,d), E=(−r,0,0). In  FIG. 12 , CD′ is perpendicular to OC and D′E′ is perpendicular to OE. Therefore, |E′|=|D′E′|tan θ=d tan θ and, consequently, 
         [0000]      | CD′|=|EO|−|EE′|=r−d  tan θ 
         [0106]    Since |CD|=|CD′|cos θ, it follows that 
         [0000]        D =(−Δ cos θ,0, d+Δ  sin θ) 
         [0107]    where Δ=r cos θ−d sin θ. We get V l  as follows: 
         [0000]        V   l =(−2Δ cos θ,0, d+ 2Δ sin θ) 
         [0108]    because the length of V l C is twice the length of DC. 
         [0109]    With the location of V l  available, we can now compute the locations of I, J Y and Z. This can be done using properties of similar triangles or ray intersection. 
         [0110]    First note that triangle V l C′I is similar to triangle EOI. Therefore we have 
         [0000]    
       
         
           
             
               
                  
                 
                   
                     V 
                     t 
                   
                    
                   
                     C 
                     ′ 
                   
                 
                  
               
               
                  
                 EO 
                  
               
             
             = 
             
               
                 
                   
                      
                     
                       
                         C 
                         ′ 
                       
                        
                       I 
                     
                      
                   
                   
                      
                     OI 
                      
                   
                 
                  
                 
                     
                 
                  
                 or 
                  
                 
                     
                 
                  
                 
                   
                     2 
                      
                     Δ 
                      
                     
                         
                     
                      
                     cos 
                      
                     
                         
                     
                      
                     θ 
                   
                   r 
                 
               
               = 
               
                 
                   
                     2 
                      
                     Δ 
                      
                     
                         
                     
                      
                     sin 
                      
                     
                         
                     
                      
                     θ 
                   
                   + 
                   d 
                   + 
                   
                     - 
                     
                        
                       OI 
                        
                     
                   
                 
                 
                    
                   OI 
                    
                 
               
             
           
         
       
     
         [0111]    where Δ=r cos θ−d sin θ. A simple algebra shows that 
         [0000]    
       
         
           
             
                
               OI 
                
             
             = 
             
               
                 r 
                  
                 
                   ( 
                   
                     d 
                     + 
                     
                       2 
                        
                       Δ 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   ) 
                 
               
               
                 
                   2 
                    
                   Δ 
                    
                   
                       
                   
                    
                   cos 
                    
                   
                       
                   
                    
                   θ 
                 
                 - 
                 r 
               
             
           
         
       
     
         [0112]    Hence, 
         [0000]    
       
         
           
             I 
             = 
             
               ( 
               
                 0 
                 , 
                 0 
                 , 
                 
                   - 
                   
                     
                       r 
                        
                       
                         ( 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     
                       
                         2 
                          
                         Δ 
                          
                         
                             
                         
                          
                         cos 
                          
                         
                             
                         
                          
                         θ 
                       
                       - 
                       r 
                     
                   
                 
               
               ) 
             
           
         
       
     
         [0113]    To compute J, note that J exists only if the rays V l F and V r G intersect. This would happen only if the distance between the virtual cameras and the z-axis is bigger than h, i.e., 2Δ cos θ&gt;h. Otherwise, we have a trinocular region that is extended to infinity. Here we assume that 2Δ cos θ&gt;h. In this case, triangle V l C′J is similar to triangle FNJ. Hence, we have 
         [0000]    
       
         
           
             
               
                  
                 
                   
                     V 
                     l 
                   
                    
                   
                     C 
                     ′ 
                   
                 
                  
               
               
                  
                 
                   F 
                    
                   
                       
                   
                    
                   N 
                 
                  
               
             
             = 
             
               
                 
                   
                      
                     
                       
                         C 
                         ′ 
                       
                        
                       J 
                     
                      
                   
                   
                      
                     
                       N 
                        
                       
                           
                       
                        
                       J 
                     
                      
                   
                 
                  
                 
                     
                 
                  
                 or 
                  
                 
                     
                 
                  
                 
                   
                     2 
                      
                     Δcos 
                      
                     
                         
                     
                      
                     θ 
                   
                   h 
                 
               
               = 
               
                 
                   
                     2 
                      
                     Δsin 
                      
                     
                         
                     
                      
                     θ 
                   
                   + 
                   d 
                   + 
                   l 
                   + 
                   
                      
                     
                       N 
                        
                       
                           
                       
                        
                       J 
                     
                      
                   
                 
                 
                    
                   
                     N 
                      
                     
                         
                     
                      
                     J 
                   
                    
                 
               
             
           
         
       
     
         [0114]    where Δ=r cos θ−d sin θ. Again, a simple algebra gives us 
         [0000]    
       
         
           
             
                
               
                 N 
                  
                 
                     
                 
                  
                 J 
               
                
             
             = 
             
               
                 h 
                  
                 
                   ( 
                   
                     d 
                     + 
                     l 
                     + 
                     
                       2 
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   ) 
                 
               
               
                 
                   2 
                    
                   Δcos 
                    
                   
                       
                   
                    
                   θ 
                 
                 - 
                 h 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               
                 
                   
                      
                     
                       O 
                        
                       
                           
                       
                        
                       J 
                     
                      
                   
                   = 
                   
                     l 
                     + 
                     
                        
                       
                         N 
                          
                         
                             
                         
                          
                         J 
                       
                        
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         2 
                          
                         
                             
                         
                          
                         l 
                          
                         
                             
                         
                          
                         Δcos 
                          
                         
                             
                         
                          
                         θ 
                       
                       + 
                       dh 
                       + 
                       
                         2 
                          
                         
                             
                         
                          
                         h 
                          
                         
                             
                         
                          
                         Δsin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     
                       
                         2 
                          
                         Δcos 
                          
                         
                             
                         
                          
                         θ 
                       
                       - 
                       h 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       - 
                       d 
                     
                     + 
                     
                       
                         
                           2 
                            
                           Δcos 
                            
                           
                               
                           
                            
                           
                             θ 
                              
                             
                               ( 
                               
                                 d 
                                 + 
                                 l 
                                 + 
                                 
                                   h 
                                    
                                   
                                       
                                   
                                    
                                   tan 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                               
                               ) 
                             
                           
                         
                         
                           
                             2 
                              
                             Δcos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           h 
                         
                       
                       . 
                     
                   
                 
               
             
           
         
       
     
         [0115]    Therefore, we have 
         [0000]    
       
         
           
             
               
                 
                   J 
                   = 
                   
                     ( 
                     
                       0 
                       , 
                       0 
                       , 
                       
                         - 
                         
                           
                             
                               2 
                                
                               
                                   
                               
                                
                               l 
                                
                               
                                   
                               
                                
                               Δcos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             dh 
                             + 
                             
                               2 
                                
                               
                                   
                               
                                
                               h 
                                
                               
                                   
                               
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           
                             
                               2 
                                
                               Δcos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             h 
                           
                         
                       
                     
                     ) 
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       0 
                       , 
                       0 
                       , 
                       
                         d 
                         - 
                         
                           
                             2 
                              
                             Δcos 
                              
                             
                                 
                             
                              
                             
                               θ 
                                
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                   + 
                                   
                                     h 
                                      
                                     
                                         
                                     
                                      
                                     tan 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                           
                           
                             
                               2 
                                
                               Δcos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             h 
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0116]    Note that when θ=0, we have h=r and Δ=r. Hence, when θ=0, the above equations reduce to I=(0,0,−d) and J=(0,0,−d−2l), respectively. 
         [0117]    Y is computed as the intersection points of the ray V l F and ray V r H. These rays can he parameterized as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     L 
                      
                     
                       ( 
                       t 
                       ) 
                     
                   
                   = 
                   
                     
                       V 
                       l 
                     
                     + 
                     
                       t 
                        
                       
                         ( 
                         
                           F 
                           - 
                           
                             V 
                             l 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   
                     = 
                     
                       
                         
                           
                             
                               ( 
                               
                                 
                                   
                                     - 
                                     2 
                                   
                                    
                                   Δ 
                                    
                                   
                                       
                                   
                                    
                                   cos 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 , 
                                 0 
                                 , 
                                 
                                   d 
                                   + 
                                   
                                     2 
                                      
                                     Δsinθ 
                                   
                                 
                               
                               ) 
                             
                             + 
                           
                         
                       
                       
                         
                           
                             t 
                              
                             
                               [ 
                               
                                 
                                   ( 
                                   
                                     
                                       - 
                                       h 
                                     
                                     , 
                                     0 
                                     , 
                                     
                                       - 
                                       l 
                                     
                                   
                                   ) 
                                 
                                 - 
                                 
                                   ( 
                                   
                                     
                                       
                                         - 
                                         2 
                                       
                                        
                                       Δ 
                                        
                                       
                                           
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     , 
                                     0 
                                     , 
                                     
                                       d 
                                       + 
                                       
                                         2 
                                          
                                         Δsinθ 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               ] 
                             
                           
                         
                       
                     
                   
                    
                   
                       
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 t 
                                  
                                 
                                   ( 
                                   
                                     
                                       2 
                                        
                                       Δcos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     - 
                                     h 
                                   
                                   ) 
                                 
                               
                             
                             , 
                             0 
                             , 
                             
                               d 
                               + 
                             
                           
                         
                       
                       
                         
                           
                             
                               2 
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             
                               t 
                                
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                   + 
                                   
                                     2 
                                      
                                     Δsin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               
                 
                   
                     
                       L 
                       1 
                     
                      
                     
                       ( 
                       s 
                       ) 
                     
                   
                   = 
                   
                     
                       V 
                       r 
                     
                     + 
                     
                       s 
                        
                       
                         ( 
                         
                           H 
                           - 
                           
                             V 
                             r 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   
                     = 
                     
                       
                         
                           
                             
                               ( 
                               
                                 
                                   2 
                                    
                                   Δ 
                                    
                                   
                                       
                                   
                                    
                                   cos 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 , 
                                 0 
                                 , 
                                 
                                   d 
                                   + 
                                   
                                     2 
                                      
                                     Δsinθ 
                                   
                                 
                               
                               ) 
                             
                             + 
                           
                         
                       
                       
                         
                           
                             s 
                              
                             
                               [ 
                               
                                 
                                   ( 
                                   
                                     r 
                                     , 
                                     0 
                                     , 
                                     0 
                                   
                                   ) 
                                 
                                 - 
                                 
                                   ( 
                                   
                                     
                                       2 
                                        
                                       Δ 
                                        
                                       
                                           
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     , 
                                     0 
                                     , 
                                     
                                       d 
                                       + 
                                       
                                         2 
                                          
                                         Δsinθ 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               ] 
                             
                           
                         
                       
                     
                   
                    
                   
                       
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 2 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 s 
                                  
                                 
                                   ( 
                                   
                                     
                                       2 
                                        
                                       Δcos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     - 
                                     r 
                                   
                                   ) 
                                 
                               
                             
                             , 
                             0 
                             , 
                             
                               d 
                               + 
                             
                           
                         
                       
                       
                         
                           
                             
                               2 
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             
                               s 
                                
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   
                                     2 
                                      
                                     Δsin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0118]    We need to find parameters t 1  and s 1  such that L(t 1 )=L 1 (s 1 ). To have L(t 1 )=L 1 (s 1 ), we must have 
         [0000]      −2Δ cos θ+ t   1 (2Δ cos θ− h )=2Δ cos θ− s   1 (2Δ cos θ− r ) 
         [0000]        d+ 2Δ sin θ− t   1 ( d+l+ 2Δ sin θ)= d+ 2Δ sin θ− s   1 ( d+ 2Δ sin θ) 
         [0000]      or 
         [0000]        t   1 (2Δ cos θ− h )+ s   1 (2Δ cos θ− r )=4Δ cos θ 
         [0000]        −t   1 ( d+l+ 2Δ sin θ)+ s   1 ( d+ 2Δ sin θ)=0 
         [0119]    Solving this system of linear equations, we first get 
         [0000]    
       
         
           
             
               s 
               1 
             
             = 
             
               
                 
                   t 
                   1 
                 
                  
                 
                   ( 
                   
                     d 
                     + 
                     l 
                     + 
                     
                       2 
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   ) 
                 
               
               
                 d 
                 + 
                 
                   2 
                    
                   Δsin 
                    
                   
                       
                   
                    
                   θ 
                 
               
             
           
         
       
     
         [0120]    and then 
         [0000]    
       
         
           
             
               t 
               1 
             
             = 
             
               
                 4 
                  
                 Δcos 
                  
                 
                     
                 
                  
                 
                   θ 
                    
                   
                     ( 
                     
                       d 
                       + 
                       
                         2 
                          
                         Δsin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     ) 
                   
                 
               
               
                 
                   Δ 
                   1 
                 
                 + 
                 
                   Δ 
                   2 
                 
               
             
           
         
       
     
         [0121]    where 
         [0000]      Δ 1 =( d+ 2Δ sin θ)(2Δ cos θ− h ) 
         [0000]      Δ 2 =(2Δ cos θ− r )( d+l+ 2Δ sin θ) 
         [0122]    Note that Δ 1  and Δ 2  are the areas of the rectangles V l D′″E″E′″ and V l D″F′F″ respectively. Hence, Y can be expressed as follows: 
         [0000]    
       
         
           
             
               
                 
                   Y 
                   = 
                   
                     L 
                      
                     
                       ( 
                       
                         t 
                         1 
                       
                       ) 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       V 
                       l 
                     
                     + 
                     
                       
                         t 
                         1 
                       
                        
                       
                         ( 
                         
                           F 
                           - 
                           
                             V 
                             l 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δcos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   4 
                                    
                                   Δcos 
                                    
                                   
                                       
                                   
                                    
                                   
                                     θΔ 
                                     1 
                                   
                                 
                                 
                                   
                                     Δ 
                                     1 
                                   
                                   + 
                                   
                                     Δ 
                                     2 
                                   
                                 
                               
                             
                             , 
                             0 
                             , 
                             
                               d 
                               + 
                               
                                 2 
                                  
                                 Δsin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                             
                           
                         
                       
                       
                         
                           
                             
                               4 
                                
                               Δcos 
                                
                               
                                   
                               
                                
                               
                                 θ 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     
                                       2 
                                        
                                       Δsin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   ) 
                                 
                               
                                
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                   + 
                                   
                                     2 
                                      
                                     Δsin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                             
                               
                                 Δ 
                                 1 
                               
                               + 
                               
                                 Δ 
                                 2 
                               
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0123]    where Δ 1  and Δ 2  are defined as above. With the expression of Y available, we know the width of the trinocular region is 
         [0000]    
       
         
           
             
                
               YZ 
                
             
             = 
             
               
                 2 
                  
                 
                    
                   YA 
                    
                 
               
               = 
               
                 
                   2 
                    
                   
                      
                     
                       
                         ( 
                         Y 
                         ) 
                       
                       r 
                     
                      
                   
                 
                 = 
                 
                   
                     4 
                      
                     
                       Δ 
                        
                       
                         ( 
                         
                           
                             Δ 
                             2 
                           
                           - 
                           
                             Δ 
                             1 
                           
                         
                         ) 
                       
                     
                      
                     cos 
                      
                     
                         
                     
                      
                     θ 
                   
                   
                     
                       Δ 
                       1 
                     
                     + 
                     
                       Δ 
                       2 
                     
                   
                 
               
             
           
         
       
     
         [0124]    and it occurs at 
         [0000]    
       
         
           
             
               
                 ( 
                 Y 
                 ) 
               
               z 
             
             = 
             
               
                 
                   ( 
                   
                     d 
                     + 
                     
                       2 
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   ) 
                 
                  
                 
                   ( 
                   
                     
                       Δ 
                       1 
                     
                     - 
                     
                       Δ 
                       2 
                     
                     - 
                     
                       2 
                        
                       
                         Δ 
                         3 
                       
                     
                   
                   ) 
                 
               
               
                 
                   Δ 
                   1 
                 
                 + 
                 
                   Δ 
                   2 
                 
               
             
           
         
       
     
         [0125]    where 
         [0000]      Δ 3   =r ( d+l+ 2Δ sin θ) 
         [0126]    and Δ 1  and Δ 2  are defined as above. Δ 3  is the area of the rectangle D″C′NF′. 
         [0127]    An important criterion in the design of a sloped STM is: how do we want the trinocular region of the sloped STM to be? For a parallel STM, the length of the trinocular region is always finite because the rays V l F and V r G always intersect and, therefore, the point J always exist. This is not the case for sloped STMs. Consider, for example, a sloped STM. In this case, ray V l F and ray V r G do not intersect in the negative z direction. Therefore, the trinocular region is unbounded on the right hand side. This means when using a sloped STM to shoot a picture, one has the advantage of handling scenes with large depth. 
         [0128]    In the case of a bounded trinocular region, the distance between a virtual camera and the z-axis (optical center of the STM) must be bigger than h, i.e., 2Δ cos θ&gt;h. To ensure this is true, first note that r, d and α are related in the following sense: 
         [0000]    
       
         
           
             
               r 
               d 
             
             = 
             
               
                 tan 
                  
                 
                     
                 
                  
                 α 
                  
                 
                     
                 
                  
                 or 
                  
                 
                     
                 
                  
                 d 
               
               = 
               
                 r 
                 
                   tan 
                    
                   
                       
                   
                    
                   α 
                 
               
             
           
         
       
     
         [0129]    Therefore, for 2Δ cos θ&gt;h, we must have 
         [0000]      2( r  cos θ− d  sin θ)cos θ&gt; r+l  tan θ 
         [0130]    or 
         [0000]    
       
         
           
             
               r 
               l 
             
             &gt; 
             
               
                 sin 
                  
                 
                     
                 
                  
                 α 
                  
                 
                     
                 
                  
                 tan 
                  
                 
                     
                 
                  
                 θ 
               
               
                 sin 
                  
                 
                   ( 
                   
                     α 
                     - 
                     
                       2 
                        
                       θ 
                     
                   
                   ) 
                 
               
             
           
         
       
     
         [0131]    So, in this case, we expect 
         [0000]      α−2θ&gt;0 or α&gt;2θ 
         [0132]    It is easy to see that in this case we have 
         [0000]    
       
         
           
             
                
               
                 I 
                  
                 
                     
                 
                  
                 J 
               
                
             
             = 
             
               
                 
                   l 
                    
                   
                     ( 
                     
                       2 
                        
                       Δcos 
                        
                       
                           
                       
                        
                       θ 
                     
                     ) 
                   
                 
                  
                 
                   [ 
                   
                     
                       tan 
                        
                       
                           
                       
                        
                       
                         θ 
                          
                         
                           ( 
                           
                             d 
                             + 
                             
                               2 
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                     
                     + 
                     
                       2 
                        
                       Δcos 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     r 
                   
                   ] 
                 
               
               
                 
                   ( 
                   
                     
                       2 
                        
                       Δcos 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     r 
                   
                   ) 
                 
                  
                 
                   ( 
                   
                     
                       2 
                        
                       Δcos 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     h 
                   
                   ) 
                 
               
             
           
         
       
     
         [0133]    Hence, in this case, one can use the above equations to adjust the parameters r, d, θ and l to construct a trinocular region that would meet our requirements. 
         [0134]    In the case of an unbounded trinocular region, the distance between a virtual camera and the z-axis (optical center of the STM) must be smaller than h, i.e., 2Δ cos θ&lt;h. In this case one can still use the above equations to adjust the width and location of the trinocular region. However, since the relationship between r and d is fixed, one should mainly use the other two parameters (θ,l) to adjust the shape and location of the trinocular region. Actually the best parameter to use is l because adjusting this parameter will not affect the size of the left view and the right view much while adjusting the parameter θ will. 
         [0135]    Image-Plus-Depth 
         [0136]    A. Computing Corresponding Points 
         [0137]    a. Imager (Camera) Calibration 
         [0138]    For image reconstruction it was first necessary to effect camera calibration to obtain camera parameters for the reconstruction process. The calibration technique described follows a prior art approach [Z. Zhang. A flexible new technique for camera calibration.  IEEE Transactions on Pattern Analysis and Machine Intelligence,  22(11):1330-1334, 2000]. 
         [0139]    A 2D and a 3D point were denoted by m=[u,v] T  and M=[x,y,z] T , respectively. They can also be represented in homogeneous coordinates as {tilde over (m)}=[u,v,1] T  and {tilde over (M)}=[x,y,z,1] T , respectively. The camera was considered as a pinhole, so the relationship between a 3D point M and its projected image in was given by 
         [0000]      s{tilde over (m)}=A[R t]{tilde over (M)},   (12) 
         [0140]    where s is a scaling factor, [R,t] is the rotation and translation matrix which relates the world coordinate system with the camera coordinate system. R and t are called the extrinsic parameters. A, called the camera intrinsic matrix, is given by 
         [0000]    
       
         
           
             A 
             = 
             
               [ 
               
                 
                   
                     α 
                   
                   
                     γ 
                   
                   
                     
                       u 
                       0 
                     
                   
                 
                 
                   
                     0 
                   
                   
                     β 
                   
                   
                     
                       v 
                       0 
                     
                   
                 
                 
                   
                     0 
                   
                   
                     0 
                   
                   
                     1 
                   
                 
               
               ] 
             
           
         
       
     
         [0141]    with (u 0 ,v 0 ) being coordinates of the principal point. The principle point is the intersection of the optical axis and the image plane. α and β are scaling factors in u and v axes of the image plane and γ is the parameter describing the skewness of the two image axes. Note that α and β are related to the focal length f. 
         [0142]    In the calibration process, the camera needs to observe a planar pattern shown in a few different orientations. The plane in which the pattern lies is called the model plane, set to be the Z=0 plane of the world coordinate system. The i th  column of the rotation matrix R is denoted by r l . From (12), we have 
         [0000]    
       
         
           
             
               s 
                
               
                 [ 
                 
                   
                     
                       u 
                     
                   
                   
                     
                       v 
                     
                   
                   
                     
                       1 
                     
                   
                 
                 ] 
               
             
             = 
             
               
                 
                   A 
                    
                   
                     [ 
                     
                       
                         r 
                         t 
                       
                        
                       
                         r 
                         2 
                       
                        
                       
                         r 
                         3 
                       
                        
                       t 
                     
                     ] 
                   
                 
                  
                 
                   [ 
                   
                     
                       
                         X 
                       
                     
                     
                       
                         Y 
                       
                     
                     
                       
                         0 
                       
                     
                     
                       
                         1 
                       
                     
                   
                   ] 
                 
               
               = 
               
                 
                   A 
                    
                   
                     [ 
                     
                       
                         r 
                         1 
                       
                        
                       
                         r 
                         2 
                       
                        
                       t 
                     
                     ] 
                   
                 
                  
                 
                   [ 
                   
                     
                       
                         X 
                       
                     
                     
                       
                         Y 
                       
                     
                     
                       
                         1 
                       
                     
                   
                   ] 
                 
               
             
           
         
       
     
         [0143]    i.e., a point M in the model plane can be expressed as M=[X,Y] T  since Z is always 0. In turn, M=[X,Y,l] T . Therefore, a model point M and its image in are related by a homography H: 
         [0000]      s{tilde over (m)}=H {tilde over (M)} with H=A[r 1 ,r 2 ,t].   (13) 
         [0144]    The 3×3 matrix H is defined up to a scaling factor. 
         [0145]    The homography, denoted H=[h 1 ,h 2 ,h 3 ], was estimated with an image of the model plane. Note that from (13), we have 
         [0000]      [h 1  h 2  h 3 ]=λA[r 1  r 2  t], 
         [0146]    where λ is a scaling factor. Using the property that r 1  and r 2  are orthonormal, we have 
         [0000]        h   1   T   A   −T   A   −1   h   2 =0   (14) 
         [0000]        h   1   T   A   −T   A   −1   h   1   =h   2   T   A   −T   A   −1   h   2 .   (15) 
         [0147]    These are the two basic constraints on the intrinsic parameters, given one homography. Because a homography has 8 degrees of freedom and there are 6 extrinsic parameters (3 for rotation and 3 for translation), we can only obtain 2 constraints on the intrinsic parameters. 
         [0148]    Lens distortion was ignored to make the computation simpler. 
         [0149]    It is easy to see the inverse of A is 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       A 
                       
                         - 
                         1 
                       
                     
                     = 
                     
                       [ 
                       
                         
                           
                             
                               1 
                               / 
                               α 
                             
                           
                           
                             
                               
                                 - 
                                 γ 
                               
                               / 
                               
                                 ( 
                                 αβ 
                                 ) 
                               
                             
                           
                           
                             
                               
                                 ( 
                                 
                                   
                                     
                                       - 
                                       
                                         u 
                                         0 
                                       
                                     
                                      
                                     β 
                                   
                                   + 
                                   
                                     γ 
                                      
                                     
                                         
                                     
                                      
                                     
                                       v 
                                       0 
                                     
                                   
                                 
                                 ) 
                               
                               / 
                               
                                 ( 
                                 αβ 
                                 ) 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             
                               1 
                               / 
                               β 
                             
                           
                           
                             
                               
                                 - 
                                 
                                   v 
                                   0 
                                 
                               
                               / 
                               β 
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                    
                   
                     
 
                   
                    
                   Let 
                    
                   
                     
 
                   
                    
                   
                     
                       
                         
                           B 
                           = 
                           
                             
                               A 
                               
                                 - 
                                 T 
                               
                             
                              
                             
                               A 
                               
                                 - 
                                 1 
                               
                             
                           
                         
                       
                     
                     
                       
                         
                           = 
                           
                             [ 
                             
                               
                                 
                                   
                                     B 
                                     11 
                                   
                                 
                                 
                                   
                                     B 
                                     12 
                                   
                                 
                                 
                                   
                                     B 
                                     13 
                                   
                                 
                               
                               
                                 
                                   
                                     B 
                                     21 
                                   
                                 
                                 
                                   
                                     B 
                                     22 
                                   
                                 
                                 
                                   
                                     B 
                                     23 
                                   
                                 
                               
                               
                                 
                                   
                                     B 
                                     31 
                                   
                                 
                                 
                                   
                                     B 
                                     32 
                                   
                                 
                                 
                                   
                                     B 
                                     33 
                                   
                                 
                               
                             
                             ] 
                           
                         
                       
                     
                     
                       
                         
                           = 
                           
                             [ 
                             
                               
                                 
                                   
                                     1 
                                     
                                       α 
                                       2 
                                     
                                   
                                 
                                 
                                   
                                     
                                       - 
                                       γ 
                                     
                                     
                                       
                                         α 
                                         2 
                                       
                                        
                                       β 
                                     
                                   
                                 
                                 
                                   
                                     
                                       
                                         γ 
                                          
                                         
                                             
                                         
                                          
                                         
                                           v 
                                           0 
                                         
                                       
                                       - 
                                       
                                         
                                           u 
                                           0 
                                         
                                          
                                         β 
                                       
                                     
                                     
                                       
                                         α 
                                         2 
                                       
                                        
                                       β 
                                     
                                   
                                 
                               
                               
                                 
                                   
                                     
                                       - 
                                       γ 
                                     
                                     
                                       
                                         α 
                                         2 
                                       
                                        
                                       β 
                                     
                                   
                                 
                                 
                                   
                                     
                                       
                                         γ 
                                         2 
                                       
                                       
                                         
                                           α 
                                           2 
                                         
                                          
                                         
                                           β 
                                           2 
                                         
                                       
                                     
                                     + 
                                     
                                       1 
                                       
                                         β 
                                         2 
                                       
                                     
                                   
                                 
                                 
                                   
                                     
                                       
                                         - 
                                         
                                           γ 
                                            
                                           
                                             ( 
                                             
                                               
                                                 γ 
                                                  
                                                 
                                                     
                                                 
                                                  
                                                 
                                                   v 
                                                   0 
                                                 
                                               
                                               - 
                                               
                                                 
                                                   u 
                                                   0 
                                                 
                                                  
                                                 β 
                                               
                                             
                                             ) 
                                           
                                         
                                       
                                       
                                         
                                           α 
                                           2 
                                         
                                          
                                         
                                           β 
                                           2 
                                         
                                       
                                     
                                     - 
                                     
                                       
                                         v 
                                         0 
                                       
                                       
                                         β 
                                         2 
                                       
                                     
                                   
                                 
                               
                               
                                 
                                   
                                     
                                       
                                         γ 
                                          
                                         
                                             
                                         
                                          
                                         
                                           v 
                                           0 
                                         
                                       
                                       - 
                                       
                                         
                                           u 
                                           0 
                                         
                                          
                                         β 
                                       
                                     
                                     
                                       
                                         α 
                                         2 
                                       
                                        
                                       β 
                                     
                                   
                                 
                                 
                                   
                                     
                                       
                                         - 
                                         
                                           γ 
                                            
                                           
                                             ( 
                                             
                                               
                                                 γ 
                                                  
                                                 
                                                     
                                                 
                                                  
                                                 
                                                   v 
                                                   0 
                                                 
                                               
                                               - 
                                               
                                                 
                                                   u 
                                                   0 
                                                 
                                                  
                                                 β 
                                               
                                             
                                             ) 
                                           
                                         
                                       
                                       
                                         
                                           α 
                                           2 
                                         
                                          
                                         
                                           β 
                                           2 
                                         
                                       
                                     
                                     - 
                                     
                                       
                                         v 
                                         0 
                                       
                                       
                                         β 
                                         2 
                                       
                                     
                                   
                                 
                                 
                                   
                                     
                                       
                                         
                                           ( 
                                           
                                             
                                               γ 
                                                
                                               
                                                   
                                               
                                                
                                               
                                                 v 
                                                 0 
                                               
                                             
                                             - 
                                             
                                               
                                                 u 
                                                 0 
                                               
                                                
                                               β 
                                             
                                           
                                           ) 
                                         
                                         2 
                                       
                                       
                                         
                                           α 
                                           2 
                                         
                                          
                                         
                                           β 
                                           2 
                                         
                                       
                                     
                                     + 
                                     
                                       
                                         v 
                                         0 
                                         2 
                                       
                                       
                                         β 
                                         2 
                                       
                                     
                                     + 
                                     1 
                                   
                                 
                               
                             
                             ] 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   16 
                   ) 
                 
               
             
           
         
       
     
         [0150]    Note that B is symmetric. We define b, a 6D vector, as follows: 
         [0000]      b=[B 11 , B 12 , B 22 , B 13 , B 23 , B 33 ] T .   (17) 
         [0151]    Recall that h i  denotes the i th  column vector of H. Then we have 
         [0000]      h i   T Bh i =v ii   T b   (18) 
         [0000]      with 
         [0000]        v   ij   =[h   1i   h   1j   ,h   1i   h   2j   +h   2i   h   1j   ,h   2i   h   2j   ,h   3i   h   1j   +h   1i   h   3j   ,h   3i   h   2j   +h   2i   h   3j   ,h   3i   h   3j ] T    
         [0152]    Therefore, (14) and (15) can be rewritten as: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       [ 
                       
                         
                           
                             
                               v 
                               12 
                               T 
                             
                           
                         
                         
                           
                             
                               
                                 ( 
                                 
                                   
                                     v 
                                     11 
                                   
                                   - 
                                   
                                     v 
                                     22 
                                   
                                 
                                 ) 
                               
                               T 
                             
                           
                         
                       
                       ] 
                     
                      
                     b 
                   
                   = 
                   0. 
                 
               
               
                 
                   ( 
                   19 
                   ) 
                 
               
             
           
         
       
     
         [0153]    If we have n images of the model plane, by stacking n such equations as (19) we have 
         [0000]      Vb=0,   (20) 
         [0154]    where V is a 2n×6 matrix. If n≧3, we will have in general a unique solution b defined up to a scaling factor. Usually we take 7-15 pictures of the pattern and use around 10 images for calibration to obtain a more accurate result. The solution to (20) is the eigenvector of V T V associated with the smallest eigenvalue. 
         [0155]    Once b is estimated, A can be computed as follows: 
         [0000]      α=√{square root over (1/ B   11 )} 
         [0000]      β=1/√{square root over ( B   22 −(α B   12 ) 2 )} 
         [0000]      γ=−α 2   βB   12    
         [0000]      ν 0 =√{square root over (β 2 ( B   33 −(α B   13 ) 2 −1))} 
         [0000]        u   0 =(γ v   0   −α   2   βB   13 )/β. 
         [0156]    Once A is computed, we can compute the extrinsic parameters for each image: 
         [0000]        r   1   =λA   −1   h   1 ; 
         [0000]        r   2   =λA   −1   h   2 ; 
         [0000]        r   3   =r   1   ×r   2 ; 
         [0000]    
       
      
       t=λA 
       −1 
       h 
       3  
      
     
         [0157]    Here, 
         [0000]    
       
         
           
             λ 
             = 
             
               
                 1 
                 
                   
                     A 
                     
                       - 
                       1 
                     
                   
                    
                   
                     h 
                     1 
                   
                 
               
               = 
               
                 
                   1 
                   
                     
                       A 
                       
                         - 
                         1 
                       
                     
                      
                     
                       h 
                       2 
                     
                   
                 
                 . 
               
             
           
         
       
     
         [0158]    Since the virtual cameras had the same intrinsic parameters as the real camera, only one camera calibration was needed. Correspondences were selected by using a feature point based matching process. Surprisingly, it was found that building a 3D representation did not require calculating the depth of all pixels. 
         [0159]    b. Obtaining Correspondence Between Views 
         [0160]    With the camera parameters being known, the only challenge left was to find the correspondence between the views. Unfortunately, reliable identification of corresponding points between different views is a very difficult problem, especially with objects having solid colors or specular reflection, such as human teeth. To address this problem, in addition to the classic vision matching technique such as cross-correlation, feature points were also used in the matching process to achieve better results. Specular reflection was removed from each point of the given image, and intensity of each point in the left view, right view, upper view and lower view was divided by 78%. 
         [0161]    The Canny edge detection algorithm [Canny, J., A Computational Approach To Edge Detection,  IEEE Trans. Pattern Analysis and Machine Intelligence,  8:679-714, 1986] is considered in the art to be an optimal edge detector. The purpose of the method is to detect edges with noise suppressed at the same time. The Canny Operator has the following goals: 
         [0162]    (a) Good Detection: the ability to locate and mark all real edges. 
         [0163]    (b) Good Localization: minimal distance between the detected edge and real edge. 
         [0164]    (c) Clear Response: only one response per edge. 
         [0165]    The approach is based on convoluting the image function with Gaussian operators and their derivatives. This is a multi-step procedure. 
         [0166]    The Canny Operator sets two thresholds to detect the edge points. The six steps are as follows: 
         [0167]    Step 1. Noise Reduction 
         [0168]    First, the image is convolved with a discrete Gaussian filter to eliminate noise. The discrete Gaussian filter is typically a 5×5 matrix of the following form (for σ=1.4): 
         [0000]    
       
         
           
             G 
             = 
             
               
                 1 
                 159 
               
                
               
                 [ 
                 
                   
                     
                       2 
                     
                     
                       4 
                     
                     
                       5 
                     
                     
                       4 
                     
                     
                       2 
                     
                   
                   
                     
                       4 
                     
                     
                       9 
                     
                     
                       12 
                     
                     
                       9 
                     
                     
                       4 
                     
                   
                   
                     
                       5 
                     
                     
                       12 
                     
                     
                       15 
                     
                     
                       12 
                     
                     
                       5 
                     
                   
                   
                     
                       4 
                     
                     
                       9 
                     
                     
                       12 
                     
                     
                       9 
                     
                     
                       4 
                     
                   
                   
                     
                       2 
                     
                     
                       4 
                     
                     
                       5 
                     
                     
                       4 
                     
                     
                       2 
                     
                   
                 
                 ] 
               
             
           
         
       
     
         [0169]    If f(m,n) is the given image, then the smoothed image F(m,n) is computed as follows: 
         [0000]    
       
         
           
             
               F 
                
               
                 ( 
                 
                   m 
                   , 
                   n 
                 
                 ) 
               
             
             = 
             
               
                 
                   G 
                    
                   
                     ( 
                     
                       m 
                       , 
                       n 
                     
                     ) 
                   
                 
                 * 
                 
                   f 
                    
                   
                     ( 
                     
                       m 
                       , 
                       n 
                     
                     ) 
                   
                 
               
               = 
               
                 
                   ∑ 
                   
                     i 
                     = 
                     0 
                   
                   4 
                 
                  
                 
                   
                     ∑ 
                     
                       j 
                       = 
                       0 
                     
                     4 
                   
                    
                   
                     
                       G 
                        
                       
                         ( 
                         
                           i 
                           , 
                           j 
                         
                         ) 
                       
                     
                      
                     
                       f 
                        
                       
                         ( 
                         
                           
                             m 
                             - 
                             i 
                           
                           , 
                           
                             n 
                             - 
                             j 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
           
         
       
     
         [0170]    Step 2. Finding the Intensity Gradient of the Image 
         [0171]    This step finds the edge strength by taking the gradient of the image. This is done by performing convolution of F(m,n) with G x  and G y , respectively, 
         [0000]        E   x ( m,n )= G   x   *F ( m,n ); 
         [0000]        E   y ( m,n )= G   y   *F ( m,n ) 
         [0172]    where 
         [0000]    
       
         
           
             
               G 
               x 
             
             = 
             
               [ 
               
                 
                   
                     
                       - 
                       1 
                     
                   
                   
                     0 
                   
                   
                     1 
                   
                 
                 
                   
                     
                       - 
                       2 
                     
                   
                   
                     0 
                   
                   
                     2 
                   
                 
                 
                   
                     
                       - 
                       1 
                     
                   
                   
                     0 
                   
                   
                     1 
                   
                 
               
               ] 
             
           
         
       
       
         
           
             
               G 
               y 
             
             = 
             
               [ 
               
                 
                   
                     1 
                   
                   
                     2 
                   
                   
                     1 
                   
                 
                 
                   
                     0 
                   
                   
                     0 
                   
                   
                     0 
                   
                 
                 
                   
                     
                       - 
                       1 
                     
                   
                   
                     
                       - 
                       2 
                     
                   
                   
                     
                       - 
                       1 
                     
                   
                 
               
               ] 
             
           
         
       
     
         [0173]    and then computing the gradient 
         [0000]        A ( m,n )=√{square root over ( E   x ( m,n )) 2 +( E   y ( m,n )) 2 )}{square root over ( E   x ( m,n )) 2 +( E   y ( m,n )) 2 )} 
         [0174]    Step 3. Finding the Edge Direction 
         [0175]    This step is trivial once gradients in the X and Y directions are known. The direction is 
         [0000]    
       
         
           
             
               θ 
                
               
                 ( 
                 
                   m 
                   , 
                   n 
                 
                 ) 
               
             
             = 
             
               
                 tan 
                 
                   - 
                   1 
                 
               
                
               
                 ( 
                 
                   
                     
                       E 
                       y 
                     
                      
                     
                       ( 
                       
                         m 
                         , 
                         n 
                       
                       ) 
                     
                   
                   
                     
                       E 
                       x 
                     
                      
                     
                       ( 
                       
                         m 
                         , 
                         n 
                       
                       ) 
                     
                   
                 
                 ) 
               
             
           
         
       
     
         [0176]    However, we will generate an error whenever E x  is zero. So in the code, there has to be a restriction set whenever this takes place. Whenever E x  is zero, the edge direction is set to 90 degrees or 0 degrees, depending on what value E y  is equal to. If E y =0, the edge direction is set to 0. Otherwise, it is set to 90. 
         [0177]    Step 4. Rounding the Edge Directions 
         [0178]    This step relates each edge direction to a direction that can be traced in an image. Note that there are only four possible directions for each pixel: 0 degrees, 45 degrees, 90 degrees, or 135 degrees. So edge direction of each pixel has to be resolved into one of these four directions, depending on which direction it is closest to. An edge direction that is between 0 and 22.5 or 157.5 and 180 degrees is set to 0 degrees. An edge direction that is between 22.5 and 67.5 is set to 45 degrees. An edge direction that is between 67.5 and 112.5 degrees is set to 90 degrees. An edge direction that is between 112.5 and 157.5 degrees is set to 135 degrees. 
         [0179]    Step 5. Non-Maximum Suppression 
         [0180]    This step performs a search to determine if the gradient magnitude assumes a local maximum in the gradient direction. So, for example,
       if the rounded angle is zero degrees the point will be considered to be on the edge if its intensity is greater than the intensities in the north and south directions,   if the rounded angle is 90 degrees the point will be considered to be on the edge if its intensity is greater than the intensities in the west and east directions,   if the rounded angle is 135 degrees the point will be considered to be on the edge if its intensity is greater than the intensities in the north east and south west directions.   if the rounded angle is 45 degrees the point will be considered to be on the edge if its intensity is greater than the intensities in the north west and south east directions.       
 
         [0185]    This is worked out by passing a 3×3 grid over the intensity map. 
         [0186]    This step produces a set of edge points in the form of a binary image by suppressing any pixel value (setting it to 0) that is not considered to be an edge. 
         [0187]    Step 6. Edge Tracing Through Hysteresis Thresholding 
         [0188]    This step uses thresholding with hysteresis to trace edges. Thresholding with hysteresis requires 2 thresholds, high and low. The high threshold is used to select a start point of an edge and the low threshold is used to trace an edge from a start point. Points of the traced edges are then used as feature points subsequently to their corresponding points. 
         [0189]    The above process was improved to obtain edges with sub-pixel accuracy by using second-order and third-order derivatives computed from a scale-space representation in the non-maximum suppression step. 
         [0190]    Each view (area) of the grid image of  FIG. 3  was labeled ( FIG. 9 ). Area  0  was the real image; areas  1 ,  2 ,  3  and  4  were images reflected once by the upper, left, right and lower mirrors, respectively; areas  5 ,  6 ,  7  and  8  are the images reflected twice by the minors. The geometry of these reflections will be discussed below. On scanline L, there were two points A and B, with their corresponding points in area  3  being A 1  and B 1 , respectively, and A 2  and B 2  in area  2 , respectively. A 3  and A 4  are the corresponding point of A in areas  4  and  1 . The corresponding points in our system satisfy the following constraints: 
         [0191]    (1). Ordering Constraint: For opaque surfaces the order of neighboring correspondences on the corresponding epipolar line is always reversed. For example, if the indices of A and B on the scanline L satisfy the condition: (A) x &gt;(B) x , then we must have (B 1 ) x &gt;(A 1 ) x  and (B 2 ) x &gt;(A 2 ) x . This is because the mirror reflection reverses the image. 
         [0192]    (2). Disparity Limit: The search band is restricted along the epipolar line because the observed scene has only a limited depth range. For example, if we are looking for the corresponding point of A in area  2 , we don&#39;t need to search the entire scanline in area  2 , we only need to search pixels in a certain threshold depending on the depth range. 
         [0193]    (3). Variance limit: The differences of the depths computed using the corresponding points in the adjacent areas should be less than a threshold. For example, A 1 , A 2 , A 3 , A 4  each can be used as a corresponding point of A and to compute a depth of A. We compute the variances of the four depths and they must be smaller than a threshold. Otherwise at least one of the depths is wrong. 
         [0194]    After feature point determination, correspondences of these points were found using stereo matching approaches. Two intensity based approaches for stereo matching were considered: 
         [0195]    Normalized cross-correlation [J. P. Lewis, “Fast Template Matching”,  Vision Interface,  p. 120-123, 1995] is an effective and simple method to measure similarity. In our application, the reflected images have reduced intensity values than the central view because of the non-perfect reflection factors of the mirrors. But normalized cross-correlation is invariant to linear brightness and contrast variations. This approach provided good matching results for our feature points. 
         [0196]    The use of cross-correlation for template matching was motivated by squared Euclidean distance: 
         [0000]    
       
         
           
             
               
                 d 
                 
                   f 
                   , 
                   g 
                 
                 2 
               
                
               
                 ( 
                 
                   x 
                   , 
                   y 
                 
                 ) 
               
             
             = 
             
               
                 ∑ 
                 
                   i 
                   , 
                   j 
                 
               
                
               
                 
                   [ 
                   
                     
                       f 
                        
                       
                         ( 
                         
                           i 
                           , 
                           j 
                         
                         ) 
                       
                     
                     - 
                     
                       g 
                        
                       
                         ( 
                         
                           
                             i 
                             - 
                             x 
                           
                           , 
                           
                             j 
                             - 
                             y 
                           
                         
                         ) 
                       
                     
                   
                   ] 
                 
                 2 
               
             
           
         
       
     
         [0197]    where f is the source image in the region and the sum is over i,j under the region of destination image g positioned at (x,y). 
         [0198]    Here we expand d 2 : 
         [0000]    
       
         
           
             
               
                 d 
                 
                   f 
                   , 
                   g 
                 
                 2 
               
                
               
                 ( 
                 
                   x 
                   , 
                   y 
                 
                 ) 
               
             
             = 
             
               ∑ 
               
                 ⌊ 
                 
                   
                     
                       f 
                       2 
                     
                      
                     
                       ( 
                       
                         i 
                         , 
                         j 
                       
                       ) 
                     
                   
                   - 
                   
                     2 
                      
                     
                       f 
                        
                       
                         ( 
                         
                           i 
                           , 
                           j 
                         
                         ) 
                       
                     
                      
                     
                       g 
                        
                       
                         ( 
                         
                           
                             i 
                             - 
                             x 
                           
                           , 
                           
                             j 
                             - 
                             y 
                           
                         
                         ) 
                       
                     
                   
                   + 
                   
                     
                       g 
                       2 
                     
                      
                     
                       ( 
                       
                         
                           i 
                           - 
                           x 
                         
                         , 
                         
                           j 
                           - 
                           y 
                         
                       
                       ) 
                     
                   
                 
                 ⌋ 
               
             
           
         
       
     
         [0199]    The term 
         [0000]    
       
         
           
             ∑ 
             
               
                 g 
                 2 
               
                
               
                 ( 
                 
                   
                     i 
                     - 
                     x 
                   
                   , 
                   
                     j 
                     - 
                     y 
                   
                 
                 ) 
               
             
           
         
       
     
         [0000]    is a constant. If the term 
         [0000]    
       
         
           
             ∑ 
             
               
                 f 
                 2 
               
                
               
                 ( 
                 
                   i 
                   , 
                   j 
                 
                 ) 
               
             
           
         
       
     
         [0000]    is approximately a 
         [0200]    constant then the remaining cross-correlation term 
         [0000]    
       
         
           
             
               
                 
                   
                     c 
                      
                     
                       ( 
                       
                         x 
                         , 
                         y 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       ∑ 
                       
                         i 
                         , 
                         j 
                       
                     
                      
                     
                       
                         f 
                          
                         
                           ( 
                           
                             i 
                             , 
                             j 
                           
                           ) 
                         
                       
                        
                       
                         g 
                          
                         
                           ( 
                           
                             
                               i 
                               - 
                               x 
                             
                             , 
                             
                               j 
                               - 
                               y 
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   21 
                   ) 
                 
               
             
           
         
       
     
         [0201]    is a measure of the similarity between the source image and the destination image. 
         [0202]    Although (21) is a good measure, there are several disadvantages to use it for matching: 
         [0203]    1). If the image energy 
         [0000]    
       
         
           
             ∑ 
             
               
                 f 
                 2 
               
                
               
                 ( 
                 
                   i 
                   , 
                   j 
                 
                 ) 
               
             
           
         
       
     
         [0000]    varies with position, matching using (21) can fail. For example, the correlation between the destination image and an exactly matching region in the source image may be less than the correlation between the destination image and a bright spot. 
         [0204]    2). The range of c(x,y) is dependent on the size of the region. 
         [0205]    3). Equation (6-10) is not invariant to changes in image amplitude such as those caused by changing lighting conditions across the image sequence. 
         [0206]    The correlation coefficient overcomes these difficulties by normalizing the image and feature vectors to unit length, yielding a cosine-like correlation coefficient: 
         [0000]    
       
         
           
             
               
                 
                   
                     N 
                      
                     
                       ( 
                       
                         x 
                         , 
                         y 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       ∑ 
                       
                         
                           ⌊ 
                           
                             
                               f 
                                
                               
                                 ( 
                                 
                                   i 
                                   , 
                                   j 
                                 
                                 ) 
                               
                             
                             - 
                             
                               f 
                               
                                 x 
                                 , 
                                 y 
                               
                             
                           
                           ⌋ 
                         
                          
                         
                           [ 
                           
                             
                               g 
                                
                               
                                 ( 
                                 
                                   
                                     i 
                                     - 
                                     x 
                                   
                                   , 
                                   
                                     j 
                                     - 
                                     y 
                                   
                                 
                                 ) 
                               
                             
                             - 
                             t 
                           
                           ] 
                         
                       
                     
                     
                       
                         [ 
                         
                           ∑ 
                           
                             
                               
                                 [ 
                                 
                                   
                                     f 
                                      
                                     
                                       ( 
                                       
                                         i 
                                         , 
                                         j 
                                       
                                       ) 
                                     
                                   
                                   - 
                                   
                                     f 
                                     
                                       x 
                                       , 
                                       y 
                                     
                                   
                                 
                                 ] 
                               
                               2 
                             
                              
                             
                               ∑ 
                               
                                 
                                   [ 
                                   
                                     
                                       g 
                                        
                                       
                                         ( 
                                         
                                           
                                             i 
                                             - 
                                             x 
                                           
                                           , 
                                           
                                             j 
                                             - 
                                             y 
                                           
                                         
                                         ) 
                                       
                                     
                                     - 
                                     t 
                                   
                                   ] 
                                 
                                 2 
                               
                             
                           
                         
                         ] 
                       
                       
                         1 
                         / 
                         2 
                       
                     
                   
                 
               
               
                 
                   ( 
                   22 
                   ) 
                 
               
             
           
         
       
     
         [0207]    where t is mean of the destination image in the region and f x,y  is mean of f(i,j) in the region under the feature. (22) is what we referred to as the normalized cross-correlation. 
         [0208]    The corresponding points in the source image and the destination image did not lie on the same scanline, but satisfied certain condition. The intensity profiles from the corresponding segments of the image pair differed only by a horizontal shift and a local foreshortening. The similarity of the image pair was continuous, and therefore an optimization process was considered suitable. A prior art attempt to match parallel stereo images using simulated annealing [Barnard, S. T. (1987), Stereo Matching by Hierarchical, Microcanonical Annealing,  Int. Joint Conf. on Artificial Intelligence,  Milan, Italy, pp. 832-835] defined an energy function as: 
         [0000]        E   ij   =|I   L ( i,j )− I   R ( i,j )+ D ( i,j )|+λ|Δ D ( i,j )| 
         [0209]    where I L (i,j) denotes the intensity value of the source image at (i,j), and I R (i,k) denotes the intensity value of the destination image at the same row but at the k-th column; D(i,j) is the disparity value (or horizontal shift in this case) at the ij-position of the source image. So this was a constrained optimization problem in which the only constraint being used is a minimum change of disparity values D(i,j). 
         [0210]    c. Distance Between Imager (Camera) and STM 
         [0211]    A parameter that was important both in the design of an STM and in the 3D image computation is d, the distance between the pinhole of the camera and the STM. This distance was also needed in the computation of the locations of all virtual cameras. In practice, typically the bounding planes of a camera field of view (FOV) do not automatically pass through the boundary edges of the device rear because of hardware restrictions. Thus, it is necessary to compute the effective d. Two situations were considered: FOV of camera covers part of STM only or FOV of camera covers more than the entire STM. 
         [0212]    If the camera FOV of the camera does not cover the entire STM, but only a portion of it, the bounding planes of the camera&#39;s FOV do not pass through the boundary edges of the STM&#39;s rear end, but intersect the interior of the STM (see  FIG. 11 ). In  FIG. 11 , 2α is the FOV of the camera and Φ is the FOV of a virtual camera. E′ and H′ are the intersection points of the bounding planes of the Camera&#39;s FOV with (the horizontal cross-section of) the interior of the STM. O′ is the intersection point of E′H′ with the optical center of the STM. In this case, the vertical plane determined by E′H′ will be considered as the effective rear end of the STM and O′ is the center of the effective rear end of the STM. Hence, we need to compute the distance between C and O′ and the distance between O′ and N. These distances, denoted d′ and l′, are called the effective distance between the camera and the STM and the effective length of the STM. 
         [0213]    First was the step of determination of the distance between U and F. This is the horizontal length of the virtual left view in the virtual image plane. Given an image shot with this STMIS configuration, if the horizontal resolutions of the central view and the left view are m and m l , respectively, then since the length of the virtual central view is 2h, it follows that the horizontal dimension of each virtual pixel in the virtual central view is 2h/m. There are m l  virtual pixels between U and F. Therefore, the distance between U and F is 
         [0000]    
       
         
           
             t 
             = 
             
               
                 
                   2 
                    
                   h 
                 
                 m 
               
                
               
                 m 
                 l 
               
             
           
         
       
     
         [0214]    With α, t and h known, it was possible to compute L, the distance between the camera and the front end of the STM, and d, the distance between the camera and the real rear end of the STM, as follows: 
         [0000]    
       
         
           
             
               L 
               = 
               
                 
                   t 
                   + 
                   h 
                 
                 
                   tan 
                    
                   
                       
                   
                    
                   α 
                 
               
             
             ; 
           
         
       
       
      
       d=L−l  
      
     
         [0215]    Once we have d, we can compute the distance between O and I, and the distance between E and I: 
         [0000]      | OI|=d  tan α; 
         [0000]      | EI|=r−|OI|   
         [0216]    Using property of similar triangles, we have 
         [0000]    
       
         
           
             
               
                  
                 
                   DE 
                   ′ 
                 
                  
               
               
                 l 
                 ′ 
               
             
             = 
             
               
                 
                    
                   
                     EE 
                     ′ 
                   
                    
                 
                 
                    
                   
                     
                       E 
                       ′ 
                     
                      
                     F 
                   
                    
                 
               
               = 
               
                 
                    
                   EI 
                    
                 
                 t 
               
             
           
         
       
     
         [0217]    Hence, 
         [0000]    
       
         
           
             
                
               
                 DE 
                 ′ 
               
                
             
             = 
             
               
                 l 
                 ′ 
               
                
               
                 
                    
                   EI 
                    
                 
                 t 
               
             
           
         
       
     
         [0218]    Note that |DE′|+l′=l or l′=l−|DE′|, the above equation can be expressed as 
         [0000]    
       
         
           
             
                
               
                 DE 
                 ′ 
               
                
             
             = 
             
               
                 
                   ( 
                   
                     l 
                     - 
                     
                        
                       
                         DE 
                         ′ 
                       
                        
                     
                   
                   ) 
                 
                  
                 
                   
                      
                     EI 
                      
                   
                   t 
                 
               
               = 
               
                 
                   
                     l 
                     t 
                   
                    
                   
                      
                     EI 
                      
                   
                 
                 - 
                 
                   
                      
                     
                       DE 
                       ′ 
                     
                      
                   
                    
                   
                     
                        
                       EI 
                        
                     
                     t 
                   
                 
               
             
           
         
       
     
         [0219]    Consequently, 
         [0000]    
       
         
           
             
                
               
                 DE 
                 ′ 
               
                
             
             = 
             
               
                 
                    
                   EI 
                    
                 
                 
                   t 
                   + 
                   
                      
                     EI 
                      
                   
                 
               
                
               l 
             
           
         
       
     
         [0220]    And therefore, 
         [0000]        d′=d+|DE′|   (23) 
         [0221]    (23) includes the ideal case as a special case when |DE′| equals zero. 
         [0222]    The FOV of the camera may cover not only the entire STM, but also some extra space. In this situation, bounding planes of the FOV do not intersect the rear end or the interior of the STM, but an extension of the STM (see  FIG. 12 ). In  FIG. 12 , 2α is the FOV of the camera, E′ and H′ are the intersection points of the bounding planes of the camera&#39;s FOV with (the horizontal cross-section of) an extension of the STM. In this case since the left view (corresponding to region between M and F) contains information not related to the scene, we will consider the effective left view (corresponding to region between U and F) only. Hence, we need to compute the distance between C and O, instead of C and O′. We also need to compute s, horizontal dimension of the effective left view. In  FIG. 12 , Π was called the effective FOV of the camera. 
         [0223]    Given an image shot with this STMIS configuration, let the horizontal resolutions of the central view and the left view again be m and m l , respectively. Using a similar approach as above, we can again compute the horizontal dimension of the virtual left view (between M and F) as 
         [0000]      t= (2 h/m ) m   l . 
         [0224]    With α, t and h known to us, we can compute L, the distance between the camera and the front end of the STM as follows: 
         [0000]    
       
         
           
             L 
             = 
             
               
                 
                   t 
                   + 
                   h 
                 
                 
                   tan 
                    
                   
                       
                   
                    
                   α 
                 
               
               . 
             
           
         
       
     
         [0225]    Hence, 
         [0000]        d=L−l.    (24) 
         [0226]    With d available to us, we can compute the effective FOV as follows 
         [0000]    
       
         
           
             Π 
             = 
             
               
                 tan 
                 
                   - 
                   1 
                 
               
                
               
                 ( 
                 
                   r 
                   d 
                 
                 ) 
               
             
           
         
       
     
         [0227]    Since 
         [0000]    
       
         
           
             
               tan 
                
               
                   
               
                
               Π 
             
             = 
             
               
                 s 
                 + 
                 h 
               
               L 
             
           
         
       
     
         [0228]    Consequently, 
         [0000]        s=−h+L  tan Π.   (25) 
         [0229]    In the latter case, if the left edge of the effective left view between U and F is not easy to identify, one can consider a smaller effective left view. In one example, instead of using the angle Π as the effective FOV, a smaller angle such as Σ ( FIG. 7 ) was used as the effective FOV. The choice of the angle Σ (and, hence, the point V) is not unique, it basically depends on if it is easy to create an artificial left edge through V for the smaller effective left view. In this case, like in case 2, one needs to compute the parameters L, d, Π and s first, then compute u, l″, d″ and r″. u is the horizontal dimension of the smaller effective left view, and l″, d″ and r″ are effective values of l, d and r, respectively. 
         [0230]    Since Σ and L are known to us, by using the fact that tan Σ=(u+h)/L we have immediately that 
         [0000]        u=L  tan Σ− h    
         [0231]    and, consequently, V was known. 
         [0232]    To compute l″, note that triangle VE″J is similar to triangle VCN. Hence, we have 
         [0000]    
       
         
           
             
               
                 u 
                 + 
                 h 
               
               
                 u 
                 + 
                 v 
               
             
             = 
             
               L 
               
                 l 
                 ″ 
               
             
           
         
       
     
         [0233]    where v is the distance between F and J. On the other hand, since tan θ=v/l″, or v=l″ tan θ, we can solve the above equation with this information to get l″ as follows: 
         [0000]    
       
         
           
             
               l 
               ″ 
             
             = 
             
               
                 Lu 
                 
                   u 
                   + 
                   h 
                   - 
                   
                     L 
                      
                     
                         
                     
                      
                     tan 
                      
                     
                         
                     
                      
                     θ 
                   
                 
               
               . 
             
           
         
       
     
         [0234]    But then d″ and r″ are trivial: 
         [0000]        d″=L−l″  and  r″=d″  tan Σ. 
         [0235]    It was also necessary to determine the location of the pinhole (nodal point) of the camera, C, using a pan head on top of a tripod. This was done using a known method [http://www.4directions.org/resources/features/qtyr_tutorial/NodalPoint.htm]. 
         [0236]    d. Depth Computation 
         [0237]    Information obtained from the left view, the right view, the upper view and the bottom view was used to compute depth for each point of the central view of an STM image. This was possible because virtual cameras for these views can see the object point that projects to the given image point. Instead of the typical, two-stage computation process, i.e., computing the corresponding point and then the depth, the technique presented herein computes the corresponding point and the depth at the same time. 
         [0238]    Given a point A in the central view of an STM image, let P 1  be its corresponding point in the virtual central view. It was assumed that the scene shot by the camera was inside the trinocular region of the STM. For a trinocular region with a J-point, the scene must be before the J-point to avoid losing information. If the trinocular region does not have a J-point, then the scene must be before an artificial J-point defined as follows 
         [0000]        J= (0,0,( I ) z +λ[( Y ) z −( I ) z ]) 
         [0239]    where (I) z  and (Y) z  are defined in (3-7) and (3-13), respectively, and λ is a constant between 2 and 4. This setting was to avoid processing an infinite array in the corresponding point computing process. Since the existence of a J-point is characterized by the value of 2Δ cos θ−h, one can combine (3-8) with the above definition to define a general J-point as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           J 
                           = 
                           
                             ( 
                             
                               0 
                               , 
                               0 
                               , 
                               
                                 d 
                                 - 
                                 
                                   
                                     2 
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     
                                       θ 
                                        
                                       
                                         ( 
                                         
                                           d 
                                           + 
                                           l 
                                           + 
                                           
                                             h 
                                              
                                             
                                                 
                                             
                                              
                                             tan 
                                              
                                             
                                                 
                                             
                                              
                                             θ 
                                           
                                         
                                         ) 
                                       
                                     
                                   
                                   
                                     
                                       2 
                                        
                                       Δ 
                                        
                                       
                                           
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     - 
                                     h 
                                   
                                 
                               
                             
                             ) 
                           
                         
                         , 
                         
                           
                             if 
                              
                             
                                 
                             
                              
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           &gt; 
                           h 
                         
                       
                     
                   
                   
                     
                       
                         
                           = 
                           
                             ( 
                             
                               0 
                               , 
                               0 
                               , 
                               
                                 
                                   
                                     Δ 
                                     1 
                                   
                                    
                                   
                                     [ 
                                     
                                       
                                         2 
                                          
                                         λΔ 
                                          
                                         
                                             
                                         
                                          
                                         cos 
                                          
                                         
                                             
                                         
                                          
                                         
                                           θ 
                                            
                                           
                                             ( 
                                             
                                               
                                                 Δ 
                                                 1 
                                               
                                               - 
                                               
                                                 Δ 
                                                 2 
                                               
                                             
                                             ) 
                                           
                                         
                                       
                                       - 
                                       
                                         r 
                                          
                                         
                                           ( 
                                           
                                             
                                               Δ 
                                               1 
                                             
                                             + 
                                             
                                               Δ 
                                               2 
                                             
                                           
                                           ) 
                                         
                                       
                                     
                                     ] 
                                   
                                 
                                 
                                   
                                     ( 
                                     
                                       
                                         2 
                                          
                                         Δ 
                                          
                                         
                                             
                                         
                                          
                                         cos 
                                          
                                         
                                             
                                         
                                          
                                         θ 
                                       
                                       - 
                                       h 
                                     
                                     ) 
                                   
                                    
                                   
                                     ( 
                                     
                                       
                                         2 
                                          
                                         Δ 
                                          
                                         
                                             
                                         
                                          
                                         cos 
                                          
                                         
                                             
                                         
                                          
                                         θ 
                                       
                                       - 
                                       r 
                                     
                                     ) 
                                   
                                    
                                   
                                     ( 
                                     
                                       
                                         Δ 
                                         1 
                                       
                                       + 
                                       
                                         Δ 
                                         2 
                                       
                                     
                                     ) 
                                   
                                 
                               
                             
                             ) 
                           
                         
                         , 
                         
                           otherwise 
                           . 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   26 
                   ) 
                 
               
             
           
         
       
     
         [0240]    where Δ 1  and Δ 2  were defined as above, respectively. Therefore, if A is the image of a point P in the scene, then P must be a point between P 1  and P 2  where P 2  is the intersection point of the ray CP 1  with the J-plane (the plane that is perpendicular to the optical center (−z-axis) of the STM at the general J-point). If we know the 3D location of P then we know the depth of A. Unfortunately, with the central view alone, this is not possible because, for camera C, the entire line segment P 1 P 2  is mapped to one point and, therefore, A can be the image of any point between P 1  and P 2 . But this is not the case for the virtual cameras. 
         [0241]    Consider, for instance, virtual camera V r  ( FIG. 12 ). Virtual camera V r  can see all the points of the line segment P 1 P 2 . Therefore, for each point of P 1 P 2  there is a corresponding point in the virtual right view. If P 1 ′ and P 2 ′ are the corresponding points of P 1  and P 2  in the virtual right view, respectively, then the corresponding point of P must be a point between P 1 ′ and P 2 ′. If P′ in  FIG. 13  is the corresponding point of P between P 1 ′ and P 2 ′ then by following a simple inverse mapping process, we can find the location of P immediately and, consequently, the depth of A. 
         [0242]    There are cases where the corresponding points can not be found in some views. Consider the example shown in  FIG. 14 . In this case, A is the image of the scene point P. However, since virtual camera V r  can not see any points beyond P 3 , P will not be projected to the virtual right view. Hence, in this case, constructing a reflection of P 1 P 2  with respect to the mirror GH makes no sense at all. One needs to construct a reflection of P 1 P 3  instead if P is a point between P 1  and P 3  (see  FIG. 13 ). In the following, we show how to compute P 1  and P 2 , P 1 ′ and P 2 ′, and then P′. In some cases, such as the one shown in  FIG. 14 , we will also compute P 3 . 
         [0243]    If the coordinates of A are (x,y), 0≦x≦m−1, 0≦y≦n−1, where in m×n is the resolution of the central view, then coordinates of P 1  would be (X,Y,−l) where 
         [0000]    
       
         
           
             X 
             = 
             
               
                 
                   
                     ( 
                     
                       x 
                       - 
                       
                         m 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       h 
                     
                     m 
                   
                 
                 + 
                 
                   h 
                   m 
                 
               
               = 
               
                 
                   
                     ( 
                     
                       x 
                       + 
                       
                         1 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       h 
                     
                     m 
                   
                 
                 - 
                 h 
               
             
           
         
       
       
         
           
             Y 
             = 
             
               
                 
                   
                     ( 
                     
                       y 
                       - 
                       
                         n 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       h 
                     
                     n 
                   
                 
                 + 
                 
                   h 
                   n 
                 
               
               = 
               
                 
                   
                     ( 
                     
                       y 
                       + 
                       
                         1 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       h 
                     
                     n 
                   
                 
                 - 
                 h 
               
             
           
         
       
     
         [0244]    P 2  can be computed as follows. 
         [0245]    In  FIG. 13 , the ray CP 1  can be parameterized as follows: 
         [0000]        L ( t )= C+t ( P   1   −C )=( tX,tY,d−t ( d+l )) 
         [0246]    where C=(0,0,d) is the location of the camera. To compute P 2  we need to find a parameter t 2  such that z-component of L(t 2 ) is the same as the z-component of J, i.e., 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       d 
                       - 
                       
                         
                           t 
                           2 
                         
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                       
                     
                     = 
                     
                       d 
                       - 
                       
                         
                           2 
                            
                           Δ 
                            
                           
                               
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           
                             θ 
                              
                             
                               ( 
                               
                                 d 
                                 + 
                                 l 
                                 + 
                                 
                                   h 
                                    
                                   
                                       
                                   
                                    
                                   tan 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                               
                               ) 
                             
                           
                         
                         
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           h 
                         
                       
                     
                   
                   , 
                   
                     
                       if 
                        
                       
                           
                       
                        
                       2 
                        
                       Δ 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                     &gt; 
                     h 
                   
                 
               
             
             
               
                 
                   
                     = 
                     
                       
                         
                           Δ 
                           1 
                         
                          
                         
                           [ 
                           
                             
                               2 
                                
                               λΔ 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               
                                 θ 
                                  
                                 
                                   ( 
                                   
                                     
                                       Δ 
                                       1 
                                     
                                     - 
                                     
                                       Δ 
                                       2 
                                     
                                   
                                   ) 
                                 
                               
                             
                             - 
                             
                               r 
                                
                               
                                 ( 
                                 
                                   
                                     Δ 
                                     1 
                                   
                                   + 
                                   
                                     Δ 
                                     2 
                                   
                                 
                                 ) 
                               
                             
                           
                           ] 
                         
                       
                       
                         
                           ( 
                           
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             h 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             r 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             
                               Δ 
                               1 
                             
                             + 
                             
                               Δ 
                               2 
                             
                           
                           ) 
                         
                       
                     
                   
                   , 
                   otherwise 
                 
               
             
           
         
       
     
         [0000]    and then set P 2 =L(t 2 ). Solving the above equation we get 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           t 
                           2 
                         
                         = 
                         
                           
                             
                               
                                 
                                   2 
                                    
                                   Δ 
                                    
                                   
                                       
                                   
                                    
                                   cos 
                                    
                                   
                                       
                                   
                                    
                                   
                                     θ 
                                      
                                     
                                       ( 
                                       
                                         d 
                                         + 
                                         l 
                                         + 
                                         
                                           h 
                                            
                                           
                                               
                                           
                                            
                                           tan 
                                            
                                           
                                               
                                           
                                            
                                           θ 
                                         
                                       
                                       ) 
                                     
                                   
                                 
                                 
                                   
                                     ( 
                                     
                                       l 
                                       + 
                                       d 
                                     
                                     ) 
                                   
                                    
                                   
                                     ( 
                                     
                                       
                                         2 
                                          
                                         Δ 
                                          
                                         
                                             
                                         
                                          
                                         cos 
                                          
                                         
                                             
                                         
                                          
                                         θ 
                                       
                                       - 
                                       h 
                                     
                                     ) 
                                   
                                 
                               
                               . 
                               
                                   
                               
                                
                               if 
                             
                              
                             
                                 
                             
                              
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           &gt; 
                           h 
                         
                       
                     
                   
                   
                     
                       
                         
                           = 
                           
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               
                                 θ 
                                  
                                 
                                   [ 
                                   
                                     
                                       
                                         
                                           
                                             
                                               ( 
                                               
                                                 d 
                                                 + 
                                                 
                                                   r 
                                                    
                                                   
                                                       
                                                   
                                                    
                                                   tan 
                                                    
                                                   
                                                       
                                                   
                                                    
                                                   θ 
                                                 
                                               
                                               ) 
                                             
                                              
                                             
                                               ( 
                                               
                                                 
                                                   Δ 
                                                   1 
                                                 
                                                 + 
                                                 
                                                   Δ 
                                                   2 
                                                 
                                               
                                               ) 
                                             
                                           
                                           - 
                                         
                                       
                                     
                                     
                                       
                                         
                                           
                                             λ 
                                              
                                             
                                               ( 
                                               
                                                 d 
                                                 + 
                                                 
                                                   2 
                                                    
                                                   Δ 
                                                    
                                                   
                                                       
                                                   
                                                    
                                                   sin 
                                                    
                                                   
                                                       
                                                   
                                                    
                                                   θ 
                                                 
                                               
                                               ) 
                                             
                                           
                                            
                                           
                                             ( 
                                             
                                               
                                                 Δ 
                                                 1 
                                               
                                               - 
                                               
                                                 Δ 
                                                 2 
                                               
                                             
                                             ) 
                                           
                                         
                                       
                                     
                                   
                                   ] 
                                 
                               
                             
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   
                                     2 
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   - 
                                   r 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   
                                     Δ 
                                     1 
                                   
                                   + 
                                   
                                     Δ 
                                     2 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                         , 
                         otherwise 
                       
                     
                   
                 
               
               
                 
                   ( 
                   27 
                   ) 
                 
               
             
           
         
       
     
         [0247]    Hence, 
         [0000]        P   2   =C+t   2 ( P   1   −C )=( t   2   X,t   2   Y,d−t   2 ( d+l )) 
         [0248]    where t 2  is defined in (27). Note that t 2 &gt;2 if θ&gt;0 in both cases. 
         [0249]    However, there are occasions where computing P 2  is not necessary but rather computing P 3  is needed. In the following, we show how to compute P 3  for one case. The other cases can be done similarly. 
         [0250]    Note that P 3  is the intersection point of the ray CP 1  with the plane that passes through the virtual camera V r =(2Δ cos θ,0,d+2Δ sin θ) and the two front corners of the right side mirror, (h,h,−l) and (h,−h,−l). The normal of that plane is (−d−l−2Δ sin θ,0,−h+2Δ cos θ). Therefore, to find P 3 , we need to find a t 3  such that L(t 3 )−(h,0,−l) is perpendicular to (−d−l−2Δ sin θ,0,−h+2Δ cos θ). We have 
         [0000]        L ( t   3 )−( h, 0, −l )=( t   3   X−h,t   3   Y, ( d+l )(1− t   3 )) 
         [0251]    To satisfy the condition (L(t 3 )−(h,0,−l)·(−d−l−2Δ sin θ,0,−h+2Δ cos θ)=0, t 3  must be equal to 
         [0000]    
       
         
           
             
               
                 
                   
                     t 
                     3 
                   
                   = 
                   
                     
                       2 
                        
                       
                         Δ 
                          
                         
                           ( 
                           
                             
                               h 
                                
                               
                                   
                               
                                
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               d 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               l 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                     
                     
                       
                         X 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                             + 
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                       + 
                       
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                          
                         
                           ( 
                           
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             h 
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   28 
                   ) 
                 
               
             
           
         
       
     
         [0252]    And we have P 3  as 
         [0000]        P   3 =( t   3   X,t   3   Y,d−t   3 ( d+l )). 
         [0253]    In deciding when P 3  should be computed, if 2Δ cos θ&gt;h compute P 3  when X≠0. 
         [0254]    If 2Δ cos θ&lt;h, compute P 3  when |X|&gt;  X  where 
         [0000]    
       
         
           
             
               
                 X 
                 _ 
               
               = 
               
                 
                   
                     ( 
                     
                       d 
                       + 
                       l 
                     
                     ) 
                   
                    
                   
                     ( 
                     
                       d 
                       + 
                       l 
                       + 
                       
                         2 
                          
                         Δ 
                          
                         
                             
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     ) 
                   
                    
                   
                     
                       ( 
                       
                         
                           Δ 
                           1 
                         
                         - 
                         
                           Δ 
                           2 
                         
                       
                       ) 
                     
                      
                     
                       [ 
                       
                         
                           
                             ( 
                             
                               λ 
                               - 
                               1 
                             
                             ) 
                           
                            
                           
                             Δ 
                             1 
                           
                         
                         - 
                         
                           Δ 
                           2 
                         
                       
                       ] 
                     
                   
                 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         
                           r 
                            
                           
                               
                           
                            
                           tan 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         
                           Δ 
                           1 
                         
                         + 
                         
                           Δ 
                           2 
                         
                       
                       ) 
                     
                   
                   - 
                   
                     
                       λ 
                        
                       
                         ( 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                      
                     
                       ( 
                       
                         
                           Δ 
                           1 
                         
                         - 
                         
                           Δ 
                           2 
                         
                       
                       ) 
                     
                   
                 
               
             
             , 
           
         
       
     
         [0255]    Δ 1  and Δ 2  are defined as above, and λ is a constant between 2 and 4. 
         [0256]    To compute the corresponding points of P 1  and P 2  (designated P′) in the virtual right view, we need to find the reflections of these points with respect to the right side mirror (the one that passes through GH; see  FIG. 14 . For simplicity, we shall simply call that mirror “mirror GH”) and then project these reflections onto the virtual image plane. We show construction of the reflections of these points with respect to mirror GH first. 
         [0257]    The reflection of P 1 P 2  can be constructed as follows. First, compute reflections of C and P 1  with respect to mirror GH. The reflection of C with respect to mirror GH is the virtual camera V r . Hence, we need to compute V r  and Q 1 , the reflection of P 1 . The next step is to parameterize the ray V r Q 1  (see  FIG. 13 ) as follows: 
         [0000]        L   1 ( t )= V   r   +t ( Q   1   −V   r ),  t≧ 0 
         [0258]    The reflection of P 1 P 2  is the segment of L 1 (t) corresponding to the parameter subspace [1,t 2 ] where t 2  is defined in (27). More precisely, we have the following theorem. 
         [0259]    THEOREM 2 For each point P=C +t(P 1 −C), t ∈[1,t 2 ], of the segment P 1 P 2 , the reflection Q of P about the right mirror GH is 
         [0000]        Q=L   1 ( t )= V   r   +t ( Q   1   −V   r )   (29) 
         [0260]    for the same parameter t. 
         [0261]    PROOF In the following we will show that this is indeed the case by constructing V r  and Q 1  first. Note that virtual camera V r  is symmetric to virtual camera V l  with respect to the yz-plane and coordinates of V l  are (−2Δ cos θ,0,d+2Δ sin θ). Hence, it follows immediately that V r =(2Δ cos θ,0,d+2Δ sin θ). 
         [0262]    To compute Q 1  note that, from  FIG. 13 , the normalized normal of mirror GH is 
         [0000]    
       
         
           
             
               
                 
                   
                     N 
                     r 
                   
                   = 
                   
                     
                       
                         ( 
                         
                           l 
                           , 
                           0 
                           , 
                           
                             h 
                             - 
                             r 
                           
                         
                         ) 
                       
                       
                         
                           
                             l 
                             2 
                           
                           + 
                           
                             
                               ( 
                               
                                 h 
                                 - 
                                 r 
                               
                               ) 
                             
                             2 
                           
                         
                       
                     
                     = 
                     
                       ( 
                       
                         
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                         , 
                         0 
                         , 
                         
                           sin 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                   
                 
               
               
                 
                   ( 
                   30 
                   ) 
                 
               
             
           
         
       
     
         [0263]    Therefore, Q 1  can be expressed as P 1 +αN r  where α is the distance between P 1  and Q 1 . On the other hand, the distance between P 1  and the mirror GH is 
         [0000]      σ 1 =( h−X )cos θ  (31) 
         [0264]    and this distance is one half of the distance between P 1  and Q 1 . Hence, we have 
         [0000]        Q   1   =P   1 +2σ 1   N   r    (32) 
         [0265]    where σ 1  is defined in (31) and N r  is defined in (30). 
         [0266]    We now show that for a general point P=L(t)=C+t( P   1 −C) in the line segment P 1 P 2 , the reflection Q is defined in (29). To show this, note that 
         [0000]        P =( tX,tY,d−t ( d+l )) 
         [0267]    and the distance between P and the mirror GH is 
         [0000]      σ={ r+[t ( d+l )− d]  tan θ− tX}  cos θ  (33) 
         [0268]    Hence, the reflection Q is of the following form 
         [0000]        Q=P+ 2 σN   r   =C+t ( P   1   −C )+2σ N   r    (34) 
         [0269]    where σ is defined in (33). We claim that Q defined by (29) is exactly the same as 
         [0270]    the Q defined in (34). We need the following equation to prove this claim: 
         [0000]      Δ+ t (σ 1 −Δ)=σ  (35) 
         [0271]    where Δ=r cos θ 31  d sin θ and σ 1  and σ are defined in (31) and (33), respectively. 
         [0272]    The proof of (35) follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     Δ 
                     + 
                     
                       t 
                        
                       
                         ( 
                         
                           
                             σ 
                             1 
                           
                           - 
                           Δ 
                         
                         ) 
                       
                     
                   
                   = 
                   
                     
                       r 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       d 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       t 
                        
                       
                         { 
                         
                           
                             
                               ( 
                               
                                 h 
                                 - 
                                 X 
                               
                               ) 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           
                             r 
                              
                             
                                 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           + 
                           
                             d 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         } 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       { 
                       
                         r 
                         - 
                         
                           d 
                            
                           
                               
                           
                            
                           tan 
                            
                           
                               
                           
                            
                           θ 
                         
                         + 
                         
                           t 
                            
                           
                             [ 
                             
                               h 
                               - 
                               X 
                               - 
                               r 
                               + 
                               
                                 d 
                                  
                                 
                                     
                                 
                                  
                                 tan 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                             ] 
                           
                         
                       
                       } 
                     
                      
                     cos 
                      
                     
                         
                     
                      
                     θ 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       { 
                       
                         r 
                         - 
                         
                           d 
                            
                           
                               
                           
                            
                           tan 
                            
                           
                               
                           
                            
                           θ 
                         
                         - 
                         tX 
                         + 
                         
                           
                             t 
                              
                             
                               ( 
                               
                                 d 
                                 + 
                                 l 
                               
                               ) 
                             
                           
                            
                           tan 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       } 
                     
                      
                     cos 
                      
                     
                         
                     
                      
                     θ 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       { 
                       
                         r 
                         + 
                         
                           
                             [ 
                             
                               
                                 t 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                   ) 
                                 
                               
                               - 
                               d 
                             
                             ] 
                           
                            
                           tan 
                            
                           
                               
                           
                            
                           θ 
                         
                         - 
                         tX 
                       
                       } 
                     
                      
                     cos 
                      
                     
                         
                     
                      
                     θ 
                   
                 
               
             
             
               
                 
                   = 
                   σ 
                 
               
             
           
         
       
     
         [0273]    But then since V r =C+2ΔN r , we have 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       L 
                       1 
                     
                      
                     
                       ( 
                       t 
                       ) 
                     
                   
                   = 
                   
                     
                       V 
                       r 
                     
                     + 
                     
                       t 
                        
                       
                         ( 
                         
                           
                             Q 
                             1 
                           
                           - 
                           
                             V 
                             r 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     C 
                     + 
                     
                       2 
                        
                       Δ 
                        
                       
                           
                       
                        
                       
                         N 
                         r 
                       
                     
                     + 
                     
                       t 
                        
                       
                         ( 
                         
                           
                             P 
                             1 
                           
                           + 
                           
                             2 
                              
                             
                                 
                             
                              
                             
                               σ 
                               1 
                             
                              
                             
                               N 
                               r 
                             
                           
                           - 
                           C 
                           - 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             
                               N 
                               r 
                             
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     C 
                     + 
                     
                       t 
                        
                       
                         ( 
                         
                           
                             P 
                             1 
                           
                           - 
                           C 
                         
                         ) 
                       
                     
                     + 
                     
                       2 
                        
                       
                         ( 
                         
                           Δ 
                           + 
                           
                             t 
                              
                             
                               ( 
                               
                                 
                                   σ 
                                   1 
                                 
                                 - 
                                 Δ 
                               
                               ) 
                             
                           
                         
                         ) 
                       
                        
                       
                         N 
                         r 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     P 
                     + 
                     
                       2 
                        
                       
                           
                       
                        
                       σ 
                        
                       
                           
                       
                        
                       
                         N 
                         r 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   Q 
                 
               
             
           
         
       
     
         [0274]    Hence, the reflection of P=C+t(P 1 −C) with respect to mirror GH is indeed V r +t(Q 1 −V r ) and this completes the proof of the theorem. Ξ 
         [0275]    Representation (29) is an important observation. It shows to find the reflection of P 1 P 2  about a particular mirror, one needs two things only: location of the virtual camera for that mirror and reflection of P 1  about that mirror. In the following, we list reflections of P 1 P 2  about all mirrors. 
         [0276]    (1) Reflection for the right mirror: 
         [0000]        Q=V   r   +t ( Q   1   −V   r ) t ∈[ 1, t   2 ]       where V r =C+2ΔN r  and Q 1 =P 1 +2σ l N r  with Δ=r cos θ−d sin θ, σ r =(h−X)cos θ and N r =(cos θ,0, sin θ).         
         [0278]    (2) Reflection for the left mirror: 
         [0000]        Q=V   l   +t ( Q   1   −V   l ) t ∈[ 1, t   2 ]       where V l =C+2ΔN l  and Q 1 =P 1 +2σ l N l  with Δ=r cos θ−d sin θ, σ 1 =(h+X)cos θ and N l =(−cos θ,0, sin θ).         
         [0280]    (3) Reflection for the top mirror: 
         [0000]        Q=V   l   +t ( Q   1   −V   t ) t ∈[ 1, t   2 ]       where V t =C+2ΔN t  and Q 1 =P 1 +2σ t N t  with Δ=r cos θ−d sin θ, σ t =(h−Y)cos θ and N t =(0, cos θ, sin θ).         
         [0282]    (4) Reflection of the bottom minor: 
         [0000]        Q=V   b   +t ( Q   1   −V   b ) t ∈[ 1, t   2 ]       where V b =C+2ΔN b  and Q 1 =P 1 +2σ b N b  with Δ=r cos θ−d sin θ, σ b =( h+Y)cos θ and N   b =(0,−cos θ, sin θ).         
         [0284]    Note that in the above cases, 
         [0000]    
       
         
           
             X 
             = 
             
               
                 
                   
                     ( 
                     
                       x 
                       + 
                       
                         1 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       
                           
                       
                        
                       h 
                     
                     m 
                   
                 
                 - 
                 
                   h 
                    
                   
                       
                   
                    
                   and 
                    
                   
                       
                   
                    
                   Y 
                 
               
               = 
               
                 
                   
                     ( 
                     
                       y 
                       + 
                       
                         1 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       
                           
                       
                        
                       h 
                     
                     n 
                   
                 
                 - 
                 
                   h 
                   . 
                 
               
             
           
         
       
     
         [0285]    We now show how to find P 1 ′ and P 2 ′ (or, P 1 ′ and P 3 ′), the projections of Q 1  and Q 2  (or, Q 1  and Q 3 ) on the virtual image plane with respect to the real camera C. This is basically a process of finding the matrix representation of a perspective projection. 
         [0286]    Given a point (X,Y,Z), let (X′,Y′,Z′) be its projection on the virtual image plane with respect to the real camera C=(0,0,d). Recall that the virtual image plane is l units away from the origin of the coordinate system in the negative z direction. Hence, Z′=−l. Thus: 
         [0000]    
       
         
           
             
               
                 Y 
                 ′ 
               
               Y 
             
             = 
             
               
                 
                   
                      
                     
                       + 
                       l 
                     
                   
                   
                      
                     
                       - 
                       Z 
                     
                   
                 
                  
                 
                     
                 
                  
                 and 
                  
                 
                     
                 
                  
                 
                   
                     X 
                     ′ 
                   
                   X 
                 
               
               = 
               
                 
                    
                   
                     + 
                     l 
                   
                 
                 
                    
                   
                     - 
                     Z 
                   
                 
               
             
           
         
       
       
         
           or 
         
       
       
         
           
             
               Y 
               ′ 
             
             = 
             
               
                 
                   Y 
                   
                     1 
                     - 
                     
                       
                         ( 
                         
                           Z 
                           + 
                           l 
                         
                         ) 
                       
                       / 
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                     
                   
                 
                  
                 
                     
                 
                  
                 and 
                  
                 
                     
                 
                  
                 
                   X 
                   ′ 
                 
               
               = 
               
                 X 
                 
                   1 
                   - 
                   
                     
                       ( 
                       
                         Z 
                         + 
                         l 
                       
                       ) 
                     
                     / 
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                   
                 
               
             
           
         
       
     
         [0287]    Hence, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     [ 
                     
                       
                         X 
                         ′ 
                       
                       , 
                       
                         Y 
                         ′ 
                       
                       , 
                       
                         Z 
                         ′ 
                       
                       , 
                       1 
                     
                     ] 
                   
                   = 
                   
                     [ 
                     
                       
                         X 
                         
                           1 
                           - 
                           
                             
                               Z 
                               + 
                               l 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                       , 
                       
                         Y 
                         
                           1 
                           - 
                           
                             
                               Z 
                               + 
                               l 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                       , 
                       
                         - 
                         l 
                       
                       , 
                       1 
                     
                     ] 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         0 
                         , 
                         0 
                         , 
                         d 
                         , 
                         1 
                       
                       ] 
                     
                     + 
                     
                       [ 
                       
                         
                           X 
                           
                             1 
                             - 
                             
                               
                                 Z 
                                 + 
                                 l 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         , 
                         
                           Y 
                           
                             1 
                             - 
                             
                               
                                 Z 
                                 + 
                                 l 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         , 
                         
                           
                             - 
                             d 
                           
                           - 
                           l 
                         
                         , 
                         
                           - 
                           1 
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         0 
                         , 
                         0 
                         , 
                         d 
                         , 
                         1 
                       
                       ] 
                     
                     + 
                     
                       [ 
                       
                         X 
                         , 
                         Y 
                         , 
                         
                           Z 
                           - 
                           d 
                         
                         , 
                         
                           
                             d 
                             - 
                             Z 
                           
                           
                             d 
                             + 
                             l 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         [ 
                         
                           
                             
                               1 
                             
                             
                               0 
                             
                             
                               0 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               1 
                             
                             
                               0 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                             
                               d 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                           
                         
                         ] 
                       
                        
                       
                         [ 
                         
                           
                             
                               1 
                             
                             
                               0 
                             
                             
                               0 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               1 
                             
                             
                               0 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               
                                 
                                   - 
                                   1 
                                 
                                 
                                   d 
                                   + 
                                   l 
                                 
                               
                             
                             
                               0 
                             
                           
                         
                         ] 
                       
                     
                      
                     
                       [ 
                       
                         
                           
                             X 
                           
                         
                         
                           
                             Y 
                           
                         
                         
                           
                             
                               Z 
                               - 
                               d 
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         
                           [ 
                           
                             
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 1 
                               
                             
                           
                           ] 
                         
                          
                         
                           [ 
                           
                             
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 
                                   
                                     - 
                                     1 
                                   
                                   
                                     d 
                                     + 
                                     l 
                                   
                                 
                               
                               
                                 0 
                               
                             
                           
                           ] 
                         
                       
                        
                       
                         [ 
                         
                           
                             
                               1 
                             
                             
                               0 
                             
                             
                               0 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               1 
                             
                             
                               0 
                             
                             
                               0 
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                             
                               
                                 - 
                                 d 
                               
                             
                           
                           
                             
                               0 
                             
                             
                               0 
                             
                             
                               0 
                             
                             
                               1 
                             
                           
                         
                         ] 
                       
                     
                      
                     
                       [ 
                       
                         
                           
                             X 
                           
                         
                         
                           
                             Y 
                           
                         
                         
                           
                             Z 
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         M 
                         l 
                       
                        
                       
                         ( 
                         
                           0 
                           , 
                           0 
                           , 
                           d 
                         
                         ) 
                       
                     
                     * 
                     
                       
                         M 
                         per 
                       
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                     
                     * 
                     
                       
                         
                           M 
                           l 
                         
                          
                         
                           ( 
                           
                             0 
                             , 
                             0 
                             , 
                             
                               - 
                               d 
                             
                           
                           ) 
                         
                       
                        
                       
                         [ 
                         
                           
                             
                               X 
                             
                           
                           
                             
                               Y 
                             
                           
                           
                             
                               Z 
                             
                           
                           
                             
                               1 
                             
                           
                         
                         ] 
                       
                     
                   
                 
               
             
           
         
       
     
         [0288]    Consequently, matrix representation of the perspective projection is 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         M 
                         = 
                         
                           
                             
                               M 
                               l 
                             
                              
                             
                               ( 
                               
                                 0 
                                 , 
                                 0 
                                 , 
                                 d 
                               
                               ) 
                             
                           
                           * 
                           
                             
                               M 
                               per 
                             
                              
                             
                               ( 
                               
                                 d 
                                 + 
                                 l 
                               
                               ) 
                             
                           
                           * 
                           
                             
                               M 
                               l 
                             
                              
                             
                               ( 
                               
                                 0 
                                 , 
                                 0 
                                 , 
                                 
                                   - 
                                   d 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           [ 
                           
                             
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 1 
                               
                               
                                 0 
                               
                               
                                 0 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 
                                   l 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                 
                               
                               
                                 
                                   
                                     - 
                                     dl 
                                   
                                   
                                     d 
                                     + 
                                     l 
                                   
                                 
                               
                             
                             
                               
                                 0 
                               
                               
                                 0 
                               
                               
                                 
                                   
                                     - 
                                     1 
                                   
                                   
                                     d 
                                     + 
                                     l 
                                   
                                 
                               
                               
                                 
                                   d 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                 
                               
                             
                           
                           ] 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   36 
                   ) 
                 
               
             
           
         
       
     
         [0289]    To get P 1 ′ and P 2 ′ (or, P 1 ′ and P 3 ′) for a particular view, simply multiply the corresponding Q 1  and Q 2  (or, Q 1  and Q 3 ) by the above matrix M. 
         [0290]    (1) For the right view: to get P 1 ′, first compute the reflection of P 1  with respect to the right mirror: 
         [0000]    
       
         
           
             
               
                 
                   
                     Q 
                     1 
                   
                   = 
                   
                     
                       P 
                       1 
                     
                     + 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       
                         N 
                         r 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       ( 
                       
                         X 
                         , 
                         Y 
                         , 
                         
                           - 
                           l 
                         
                       
                       ) 
                     
                     + 
                     
                       2 
                        
                       
                         
                           σ 
                           r 
                         
                          
                         
                           ( 
                           
                             
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             , 
                             0 
                             , 
                             
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       
                         X 
                         + 
                         
                           2 
                            
                           
                             σ 
                             r 
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       , 
                       Y 
                       , 
                       
                         
                           - 
                           l 
                         
                         + 
                         
                           2 
                            
                           σ 
                         
                       
                       , 
                       
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0291]    where σ r =(h−X)cos θ. Then multiply the matrix representation of Q 1  by M: 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     1 
                     ′ 
                   
                   = 
                   
                     MQ 
                     1 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               l 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               
                                 - 
                                 dl 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               
                                 - 
                                 1 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               d 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               X 
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   r 
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         
                           
                             Y 
                           
                         
                         
                           
                             
                               
                                 - 
                                 l 
                               
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   r 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             X 
                             + 
                             
                               2 
                                
                               
                                 σ 
                                 r 
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                         
                       
                       
                         
                           Y 
                         
                       
                       
                         
                           
                             
                               - 
                               
                                 l 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     l 
                                     - 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         r 
                                       
                                        
                                       sin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   ) 
                                 
                               
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                       
                         
                           
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   r 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   X 
                                   + 
                                   
                                     2 
                                      
                                     
                                       σ 
                                       r 
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   r 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               Y 
                             
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   r 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                       
                       
                         
                           
                             - 
                             l 
                           
                         
                       
                       
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
             
           
         
       
     
         [0292]    To get P 2 ′, note that according to Theorem 2, the reflection of P 2  with respect to the right mirror can be computed as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     Q 
                     2 
                   
                   = 
                     
                    
                   
                     
                       V 
                       r 
                     
                     + 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ( 
                         
                           
                             Q 
                             1 
                           
                           - 
                           
                             V 
                             r 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       ( 
                       
                         
                           2 
                            
                           Δcos 
                            
                           
                               
                           
                            
                           θ 
                         
                         , 
                         0 
                         , 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δsinθ 
                           
                         
                       
                       ) 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     
                       t 
                       2 
                     
                      
                     
                       [ 
                       
                         
                           ( 
                           
                             
                               X 
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   r 
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                             , 
                             Y 
                             , 
                             
                               
                                 - 
                                 l 
                               
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   r 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                           ) 
                         
                         - 
                         
                           ( 
                           
                             
                               2 
                                
                               Δcos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             , 
                             0 
                             , 
                             
                               d 
                               + 
                               
                                 2 
                                  
                                 Δsinθ 
                               
                             
                           
                           ) 
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 2 
                                  
                                 Δcos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ( 
                                   
                                     X 
                                     + 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         r 
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     - 
                                     
                                       2 
                                        
                                       Δcos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   ) 
                                 
                               
                             
                             , 
                             
                               
                                 t 
                                 2 
                               
                                
                               Y 
                             
                             , 
                             
                               d 
                               + 
                             
                           
                         
                       
                       
                         
                           
                             
                               2 
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               
                                 t 
                                 2 
                               
                                
                               
                                 ( 
                                 
                                   
                                     - 
                                     d 
                                   
                                   - 
                                   l 
                                   + 
                                   
                                     2 
                                      
                                     
                                       σ 
                                       r 
                                     
                                      
                                     sin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   - 
                                   
                                     2 
                                      
                                     Δsin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0293]    Since 
         [0000]    
       
         
           
             
               
                 
                   
                     X 
                     + 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       Δcos 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   = 
                   
                     X 
                     + 
                     
                       2 
                        
                       
                         ( 
                         
                           h 
                           - 
                           X 
                         
                         ) 
                       
                        
                       
                         cos 
                         2 
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       
                         ( 
                         
                           
                             
                               
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   cos 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 - 
                               
                             
                           
                           
                             
                               
                                 d 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       X 
                        
                       
                         ( 
                         
                           1 
                           - 
                           
                             2 
                              
                             
                                 
                             
                              
                             
                               cos 
                               2 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     + 
                     
                       2 
                        
                       
                           
                       
                        
                       l 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θcos 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       
                           
                       
                        
                       d 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θcosθ 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         - 
                         X 
                       
                        
                       
                           
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     + 
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       
                         sin 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                   
                 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               
                 
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   = 
                     
                    
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         ( 
                         
                           h 
                           - 
                           X 
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     2 
                      
                     
                       ( 
                       
                         
                           r 
                            
                           
                               
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                         - 
                         
                           d 
                            
                           
                               
                           
                            
                           sin 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                      
                     sin 
                      
                     
                         
                     
                      
                     θ 
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       d 
                        
                       
                         ( 
                         
                           1 
                           - 
                           
                             2 
                              
                             
                               sin 
                               2 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     - 
                     
                       2 
                        
                       l 
                        
                       
                           
                       
                        
                       
                         sin 
                         2 
                       
                        
                       θ 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     l 
                     + 
                     
                       2 
                        
                       X 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       X 
                        
                       
                           
                       
                        
                       
                         sin 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     + 
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                   
                 
               
             
           
         
       
     
         [0294]    hence, we have 
         [0000]        Q   2 =(2Δ cos θ+ t   2 ρ 1r   ,t   2   Y,d+ 2Δ sin θ+ t   2 ρ 2r ) 
         [0000]      where 
         [0000]      ρ 1r   =−X  cos(2θ)+( d+l )sin(2θ) 
         [0000]      ρ 2r   =−X  sin(2θ)−( d+l )cos(2θ)   (37) 
         [0295]    and t 2  is defined in (27). Now multiply the matrix representation of Q 2  by M to get P 2 ′: 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     2 
                     ′ 
                   
                   = 
                   
                     MQ 
                     2 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               l 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               
                                 - 
                                 dl 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               
                                 - 
                                 1 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               d 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               
                                 2 
                                  
                                 Δcos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     1 
                                      
                                     
                                         
                                     
                                      
                                     r 
                                   
                                 
                               
                             
                           
                         
                         
                           
                             
                               
                                 t 
                                 2 
                               
                                
                               Y 
                             
                           
                         
                         
                           
                             
                               d 
                               + 
                               
                                 2 
                                  
                                 Δsin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     
                                         
                                     
                                      
                                     r 
                                   
                                 
                               
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               2 
                                
                               Δcos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               
                                 t 
                                 2 
                               
                                
                               
                                 ρ 
                                 
                                   1 
                                    
                                   
                                       
                                   
                                    
                                   r 
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               t 
                               2 
                             
                              
                             Y 
                           
                         
                       
                       
                         
                           
                             
                               
                                 l 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     
                                       2 
                                        
                                       Δsin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     + 
                                     
                                       
                                         t 
                                         2 
                                       
                                        
                                       
                                         ρ 
                                         
                                           2 
                                            
                                           
                                               
                                           
                                            
                                           r 
                                         
                                       
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               dl 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 - 
                                 d 
                               
                               - 
                               
                                 2 
                                  
                                 Δsin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     
                                         
                                     
                                      
                                     r 
                                   
                                 
                               
                               + 
                               d 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   
                                     2 
                                      
                                     Δcos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   + 
                                   
                                     
                                       t 
                                       2 
                                     
                                      
                                     
                                       ρ 
                                       
                                         1 
                                          
                                         
                                             
                                         
                                          
                                         r 
                                       
                                     
                                   
                                 
                                 ) 
                               
                             
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δsin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     
                                         
                                     
                                      
                                     r 
                                   
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 t 
                                 2 
                               
                                
                               Y 
                             
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δsin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     
                                         
                                     
                                      
                                     r 
                                   
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             - 
                             l 
                           
                         
                       
                       
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
             
           
         
       
     
         [0296]    where ρ 1r  and ρ 2r  defined in (37) and t 2  is defined in (27). This expression of P 2 ′ reduces to P 1 ′ when t 2 =1. Hence it includes P 1 ′ as a special case. 
         [0297]    The computation process of P 3 ′ for the right view is similar to the computation process of P 2 ′. First compute Q 3  as follows 
         [0000]        Q   3 =(2Δ cos θ+ t   3 ρ 1r   ,t   3   Y,d+ 2Δ sin θ+ t   3 ρ 2r ) 
         [0298]    where ρ 1r  and ρ 2r  are defined as above, and t 3  is defined as previously. Then multiply           by the matrix M defined in (36). The result is similar to P 2 ′ (simply replace t 2  with t 3  in the expression of P 2 ′). 
         [0299]    In the following, we show P 1 ′ and P 2 ′ for the left, the upper and the lower views. P 3 ′ will not be shown here because one can get P 3 ′ from P 2 ′ by replacing each t 2  in P 2 ′ with a t 3 . 
         [0300]    (2) For the left view: we have 
         [0000]        Q   1   =P   1 +2σ l   N   l =( X− 2σ l  cos θ, Y,−l+ 2σ l  sin θ) 
         [0301]    where N l =(−cos θ,0, sin θ) and σ l =(h+X)cos θ. Hence 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     1 
                     ′ 
                   
                   = 
                   
                     MQ 
                     1 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               l 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               
                                 - 
                                 dl 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               
                                 - 
                                 1 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               d 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               X 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   l 
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         
                           
                             Y 
                           
                         
                         
                           
                             
                               
                                 - 
                                 l 
                               
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   l 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   X 
                                   - 
                                   
                                     2 
                                      
                                     
                                       σ 
                                       l 
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   l 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               Y 
                             
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   l 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                       
                       
                         
                           
                             - 
                             l 
                           
                         
                       
                       
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
             
           
         
       
     
         [0302]    To get P 2 ′, we need to find Q 2  first. By Theorem 2, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     Q 
                     2 
                   
                   = 
                     
                    
                   
                     
                       V 
                       l 
                     
                     + 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ( 
                         
                           
                             Q 
                             1 
                           
                           - 
                           
                             V 
                             l 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       ( 
                       
                         
                           
                             - 
                             2 
                           
                            
                           Δcos 
                            
                           
                               
                           
                            
                           θ 
                         
                         , 
                         0 
                         , 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δsinθ 
                           
                         
                       
                       ) 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     
                       t 
                       2 
                     
                      
                     
                       [ 
                       
                         
                           ( 
                           
                             
                               X 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   l 
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                             , 
                             Y 
                             , 
                             
                               
                                 - 
                                 l 
                               
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   l 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                           ) 
                         
                         - 
                         
                           ( 
                           
                             
                               
                                 - 
                                 2 
                               
                                
                               Δcos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             , 
                             0 
                             , 
                             
                               d 
                               + 
                               
                                 2 
                                  
                                 Δsinθ 
                               
                             
                           
                           ) 
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δcos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ( 
                                   
                                     X 
                                     - 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         l 
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     + 
                                     
                                       2 
                                        
                                       Δcos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   ) 
                                 
                               
                             
                             , 
                             
                               
                                 t 
                                 2 
                               
                                
                               Y 
                             
                             , 
                             
                               d 
                               + 
                             
                           
                         
                       
                       
                         
                           
                             
                               2 
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             
                               
                                 t 
                                 2 
                               
                                
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                   - 
                                   
                                     2 
                                      
                                     
                                       σ 
                                       l 
                                     
                                      
                                     sin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   + 
                                   
                                     2 
                                      
                                     Δsin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0303]    Since 
         [0000]    
       
         
           
             
               
                 
                   
                     X 
                     - 
                     
                       2 
                        
                       
                         σ 
                         l 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       Δcos 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   = 
                   
                     X 
                     - 
                     
                       2 
                        
                       
                         ( 
                         
                           h 
                           + 
                           X 
                         
                         ) 
                       
                        
                       
                         cos 
                         2 
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       
                         ( 
                         
                           
                             
                               
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   cos 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 - 
                               
                             
                           
                           
                             
                               
                                 d 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       X 
                        
                       
                         ( 
                         
                           1 
                           - 
                           
                             2 
                              
                             
                                 
                             
                              
                             
                               cos 
                               2 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     - 
                     
                       2 
                        
                       
                           
                       
                        
                       l 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θcos 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       
                           
                       
                        
                       d 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θcosθ 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         - 
                         X 
                       
                        
                       
                           
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     - 
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       
                         sin 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                   
                 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               
                 
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         l 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   = 
                     
                    
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         ( 
                         
                           h 
                           + 
                           X 
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     2 
                      
                     
                       ( 
                       
                         
                           r 
                            
                           
                               
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                         - 
                         
                           d 
                            
                           
                               
                           
                            
                           sin 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                      
                     sin 
                      
                     
                         
                     
                      
                     θ 
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       d 
                        
                       
                         ( 
                         
                           1 
                           - 
                           
                             2 
                              
                             
                               sin 
                               2 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     - 
                     
                       2 
                        
                       
                         ( 
                         
                           h 
                           - 
                           r 
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     l 
                     - 
                     
                       2 
                        
                       X 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       d 
                        
                       
                           
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             
                                 
                             
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     + 
                     
                       l 
                        
                       
                         ( 
                         
                           1 
                           - 
                           
                             2 
                              
                             
                                 
                             
                              
                             
                               sin 
                               2 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     - 
                     
                       X 
                        
                       
                           
                       
                        
                       
                         sin 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       
                         - 
                         X 
                       
                        
                       
                           
                       
                        
                       
                         sin 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     + 
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                   
                 
               
             
           
         
       
     
         [0304]    Hence, 
         [0000]        Q   2 =(−2Δ cos θ+ t   2 ρ 1l   ,t   2   Y,d+ 2Δ sin θ+ t   2 ρ 2l ) 
         [0000]      where 
         [0000]      ρ 1l   =−X  cos(2θ)−( d+l )sin(2θ) 
         [0000]      ρ 2l   =X  sin(2θ)−( d+l )cos(2θ)   (38) 
         [0305]    Therefore, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     2 
                     ′ 
                   
                   = 
                   
                     MQ 
                     2 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               l 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               
                                 - 
                                 dl 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               
                                 - 
                                 1 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               d 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     1 
                                      
                                     l 
                                   
                                 
                               
                             
                           
                         
                         
                           
                             
                               
                                 t 
                                 2 
                               
                                
                               Y 
                             
                           
                         
                         
                           
                             
                               d 
                               + 
                               
                                 2 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     l 
                                   
                                 
                               
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   
                                     
                                       - 
                                       2 
                                     
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   + 
                                   
                                     
                                       t 
                                       2 
                                     
                                      
                                     
                                       ρ 
                                       
                                         1 
                                          
                                         l 
                                       
                                     
                                   
                                 
                                 ) 
                               
                             
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     l 
                                   
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 t 
                                 2 
                               
                                
                               Y 
                             
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     l 
                                   
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             - 
                             l 
                           
                         
                       
                       
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
             
           
         
       
     
         [0306]    where ρ 1l  and ρ 2l  are defined in (38) and t 2  is defined in (27). 
         [0307]    (3) For the upper view: we have 
         [0000]    
       
         
           
             
               
                 
                   
                     Q 
                     1 
                   
                   = 
                   
                     
                       P 
                       1 
                     
                     + 
                     
                       2 
                        
                       
                         σ 
                         t 
                       
                        
                       
                         N 
                         t 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       ( 
                       
                         X 
                         , 
                         Y 
                         , 
                         
                           - 
                           l 
                         
                       
                       ) 
                     
                     + 
                     
                       2 
                        
                       
                         
                           σ 
                           t 
                         
                          
                         
                           ( 
                           
                             0 
                             , 
                             
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             , 
                             
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       X 
                       , 
                       
                         Y 
                         + 
                         
                           2 
                            
                           
                             σ 
                             t 
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       , 
                       
                         
                           - 
                           l 
                         
                         + 
                         
                           2 
                            
                           
                             σ 
                             t 
                           
                            
                           sin 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0308]    where σ l =(h−Y)cos θ. Hence, 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     1 
                     ′ 
                   
                   = 
                   
                     MQ 
                     1 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               l 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               
                                 - 
                                 dl 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               
                                 - 
                                 1 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               d 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           
                             X 
                           
                         
                         
                           
                             
                               Y 
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   t 
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         
                           
                             
                               
                                 - 
                                 l 
                               
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   t 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           X 
                         
                       
                       
                         
                           
                             Y 
                             + 
                             
                               2 
                                
                               
                                 σ 
                                 t 
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 l 
                                  
                                 
                                   ( 
                                   
                                     
                                       - 
                                       l 
                                     
                                     + 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         t 
                                       
                                        
                                       sin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               dl 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                       
                         
                           
                             
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   t 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               d 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               X 
                             
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   t 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   Y 
                                   + 
                                   
                                     2 
                                      
                                     
                                       σ 
                                       t 
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   t 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                       
                       
                         
                           
                             - 
                             l 
                           
                         
                       
                       
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
             
           
         
       
     
         [0309]    To get P 2 ′, we need to find Q 2  first. By Theorem 2, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     Q 
                     2 
                   
                   = 
                     
                    
                   
                     
                       V 
                       t 
                     
                     + 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ( 
                         
                           
                             Q 
                             1 
                           
                           - 
                           
                             V 
                             t 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       ( 
                       
                         0 
                         , 
                         
                           2 
                            
                           Δ 
                            
                           
                               
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                         , 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                       
                       ) 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     
                       t 
                       2 
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   X 
                                   , 
                                   
                                     Y 
                                     + 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         t 
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   , 
                                   
                                     
                                       - 
                                       l 
                                     
                                     + 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         t 
                                       
                                        
                                       sin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                 
                                 ) 
                               
                               - 
                             
                           
                         
                         
                           
                             
                               ( 
                               
                                 0 
                                 , 
                                 
                                   2 
                                    
                                   Δ 
                                    
                                   
                                       
                                   
                                    
                                   cos 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 , 
                                 
                                   d 
                                   + 
                                   
                                     2 
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     sin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 t 
                                 2 
                               
                                
                               X 
                             
                             , 
                             
                               
                                 2 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ( 
                                   
                                     Y 
                                     + 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         t 
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     - 
                                     
                                       2 
                                        
                                       Δ 
                                        
                                       
                                           
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   ) 
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             d 
                             + 
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               
                                 t 
                                 2 
                               
                                
                               
                                 ( 
                                 
                                   
                                     - 
                                     d 
                                   
                                   - 
                                   l 
                                   + 
                                   
                                     2 
                                      
                                     
                                       σ 
                                       t 
                                     
                                      
                                     sin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   - 
                                   
                                     2 
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     sin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0310]    Since 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           Y 
                           + 
                           
                             2 
                              
                             
                               σ 
                               t 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         = 
                         
                           Y 
                           + 
                           
                             2 
                              
                             
                               ( 
                               
                                 h 
                                 - 
                                 Y 
                               
                               ) 
                             
                              
                             
                               cos 
                               2 
                             
                              
                             θ 
                           
                           - 
                           
                             2 
                              
                             
                               ( 
                               
                                 
                                   
                                     
                                       
                                         r 
                                          
                                         
                                             
                                         
                                          
                                         cos 
                                          
                                         
                                             
                                         
                                          
                                         θ 
                                       
                                       - 
                                     
                                   
                                 
                                 
                                   
                                     
                                       d 
                                        
                                       
                                           
                                       
                                        
                                       sin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                 
                               
                               ) 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           
                             Y 
                              
                             
                               ( 
                               
                                 1 
                                 - 
                                 
                                   2 
                                    
                                   
                                     cos 
                                     2 
                                   
                                    
                                   θ 
                                 
                               
                               ) 
                             
                           
                           + 
                           
                             2 
                              
                             l 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θcos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           + 
                           
                             2 
                              
                             d 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θcos 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           
                             
                               - 
                               Y 
                             
                              
                             
                                 
                             
                              
                             
                               cos 
                                
                               
                                 ( 
                                 
                                   2 
                                    
                                   θ 
                                 
                                 ) 
                               
                             
                           
                           + 
                           
                             
                               ( 
                               
                                 d 
                                 + 
                                 l 
                               
                               ) 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             
                               ( 
                               
                                 2 
                                  
                                 θ 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                 
               
             
             
               
                 
                   and 
                    
                   
                     
 
                   
                    
                   
                     
                       
                         
                           
                             d 
                             + 
                             l 
                             - 
                             
                               2 
                                
                               
                                 σ 
                                 t 
                               
                                
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           = 
                             
                            
                           
                             d 
                             + 
                             l 
                             - 
                             
                               2 
                                
                               
                                 ( 
                                 
                                   h 
                                   - 
                                   Y 
                                 
                                 ) 
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                           
                         
                       
                     
                     
                       
                         
                             
                            
                           
                             2 
                              
                             
                               ( 
                               
                                 
                                   r 
                                    
                                   
                                       
                                   
                                    
                                   cos 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 - 
                                 
                                   d 
                                    
                                   
                                       
                                   
                                    
                                   sin 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                               
                               ) 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                       
                     
                     
                       
                         
                           = 
                             
                            
                           
                             
                               d 
                                
                               
                                 ( 
                                 
                                   1 
                                   - 
                                   
                                     2 
                                      
                                     
                                       
                                         sin 
                                          
                                         
                                             
                                         
                                       
                                       2 
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                             - 
                             
                               2 
                                
                               l 
                                
                               
                                   
                               
                                
                               
                                 sin 
                                 2 
                               
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               2 
                                
                               Y 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                         
                       
                     
                     
                       
                         
                           = 
                             
                            
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 cos 
                                  
                                 
                                   ( 
                                   
                                     2 
                                      
                                     θ 
                                   
                                   ) 
                                 
                               
                             
                             + 
                             
                               Y 
                                
                               
                                   
                               
                                
                               
                                 sin 
                                  
                                 
                                   ( 
                                   
                                     2 
                                      
                                     θ 
                                   
                                   ) 
                                 
                               
                             
                           
                         
                       
                     
                   
                 
               
             
           
         
       
     
         [0311]    Hence, 
         [0000]        Q   2 =( t   2   X, 2Δ cos θ+ t   2 ρ 1r   ,d+ 2Δ sin θ+ t   2 ρ 2l ) 
         [0000]      where 
         [0000]      ρ 1t   =−Y  cos(2θ)+( d+l )sin(2θ) 
         [0000]      ρ 2t   =−Y  sin(2θ)−( d+l )cos(2θ)   (39) 
         [0312]    Therefore, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     2 
                     ′ 
                   
                   = 
                   
                     MQ 
                     2 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               l 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               
                                 - 
                                 dl 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               
                                 - 
                                 1 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               d 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               
                                 t 
                                 2 
                               
                                
                               X 
                             
                           
                         
                         
                           
                             
                               
                                 2 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     1 
                                      
                                     t 
                                   
                                 
                               
                             
                           
                         
                         
                           
                             
                               d 
                               + 
                               
                                 2 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     t 
                                   
                                 
                               
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               t 
                               2 
                             
                              
                             X 
                           
                         
                       
                       
                         
                           
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               
                                 t 
                                 2 
                               
                                
                               
                                 ρ 
                                 
                                   1 
                                    
                                   t 
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 l 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     
                                       2 
                                        
                                       Δ 
                                        
                                       
                                           
                                       
                                        
                                       sin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     + 
                                     
                                       
                                         t 
                                         2 
                                       
                                        
                                       
                                         ρ 
                                         
                                           2 
                                            
                                           t 
                                         
                                       
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               dl 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 - 
                                 d 
                               
                               - 
                               
                                 2 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     t 
                                   
                                 
                               
                               + 
                               d 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 t 
                                 2 
                               
                                
                               X 
                             
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     t 
                                   
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   
                                     2 
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   + 
                                   
                                     
                                       t 
                                       2 
                                     
                                      
                                     
                                       ρ 
                                       
                                         1 
                                          
                                         t 
                                       
                                     
                                   
                                 
                                 ) 
                               
                             
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     t 
                                   
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             - 
                             l 
                           
                         
                       
                       
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
             
           
         
       
     
         [0313]    where ρ 1t  and p 2t  are defined in (39) and t 2  is defined in (27). 
         [0314]    (4) For the lower view: we have 
         [0000]    
       
         
           
             
               
                 
                   
                     Q 
                     1 
                   
                   = 
                   
                     
                       P 
                       1 
                     
                     + 
                     
                       2 
                        
                       
                         σ 
                         b 
                       
                        
                       
                         N 
                         b 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       ( 
                       
                         X 
                         , 
                         Y 
                         , 
                         
                           - 
                           l 
                         
                       
                       ) 
                     
                     + 
                     
                       2 
                        
                       
                         
                           σ 
                           b 
                         
                          
                         
                           ( 
                           
                             0 
                             , 
                             
                               
                                 - 
                                 cos 
                               
                                
                               
                                   
                               
                                
                               θ 
                             
                             , 
                             
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       X 
                       , 
                       
                         Y 
                         - 
                         
                           2 
                            
                           
                             σ 
                             b 
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       , 
                       
                         
                           - 
                           l 
                         
                         + 
                         
                           2 
                            
                           
                             σ 
                             b 
                           
                            
                           sin 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0315]    where σ b =(h+Y)cos θ. Hence, 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     1 
                     ′ 
                   
                   = 
                   
                     MQ 
                     1 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               l 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               
                                 - 
                                 dl 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               
                                 - 
                                 1 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               d 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           
                             X 
                           
                         
                         
                           
                             
                               Y 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   b 
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         
                           
                             
                               
                                 - 
                                 l 
                               
                               + 
                               
                                 2 
                                  
                                 
                                   σ 
                                   b 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           X 
                         
                       
                       
                         
                           
                             Y 
                              
                             
                                 
                             
                             - 
                             
                               2 
                                
                               
                                 σ 
                                 b 
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 l 
                                  
                                 
                                   ( 
                                   
                                     
                                       - 
                                       l 
                                     
                                     + 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         b 
                                       
                                        
                                       sin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               dl 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                       
                         
                           
                             
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   b 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               d 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               X 
                             
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   b 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   Y 
                                   - 
                                   
                                     2 
                                      
                                     
                                       σ 
                                       b 
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                             
                               d 
                               + 
                               l 
                               - 
                               
                                 2 
                                  
                                 
                                   σ 
                                   b 
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                             
                           
                         
                       
                       
                         
                           
                             - 
                             l 
                           
                         
                       
                       
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
             
           
         
       
     
         [0316]    To get P 2 ′, we need to find Q 2  first. By Theorem 2, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     Q 
                     2 
                   
                   = 
                     
                    
                   
                     
                       V 
                       b 
                     
                     + 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ( 
                         
                           
                             Q 
                             1 
                           
                           - 
                           
                             V 
                             b 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       ( 
                       
                         0 
                         , 
                         
                           
                             - 
                             2 
                           
                            
                           Δ 
                            
                           
                               
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                         , 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                       
                       ) 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     
                       t 
                       2 
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   X 
                                   , 
                                   
                                     Y 
                                     + 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         b 
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   , 
                                   
                                     
                                       - 
                                       l 
                                     
                                     + 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         b 
                                       
                                        
                                       sin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                 
                                 ) 
                               
                               - 
                             
                           
                         
                         
                           
                             
                               ( 
                               
                                 0 
                                 , 
                                 
                                   
                                     - 
                                     2 
                                   
                                    
                                   Δ 
                                    
                                   
                                       
                                   
                                    
                                   cos 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 , 
                                 
                                   d 
                                   + 
                                   
                                     2 
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     sin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 t 
                                 2 
                               
                                
                               X 
                             
                             , 
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ( 
                                   
                                     Y 
                                     - 
                                     
                                       2 
                                        
                                       
                                         σ 
                                         b 
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     + 
                                     
                                       2 
                                        
                                       Δ 
                                        
                                       
                                           
                                       
                                        
                                       cos 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                   
                                   ) 
                                 
                               
                             
                             , 
                           
                         
                       
                       
                         
                           
                             d 
                             + 
                             
                               2 
                                
                               Δ 
                                
                               
                                   
                               
                                
                               sin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               
                                 t 
                                 2 
                               
                                
                               
                                 ( 
                                 
                                   
                                     - 
                                     d 
                                   
                                   - 
                                   l 
                                   + 
                                   
                                     2 
                                      
                                     
                                       σ 
                                       b 
                                     
                                      
                                     sin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   - 
                                   
                                     2 
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     sin 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0317]    Since 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       Y 
                       - 
                       
                         2 
                          
                         
                           σ 
                           b 
                         
                          
                         cos 
                          
                         
                             
                         
                          
                         θ 
                       
                       + 
                       
                         2 
                          
                         Δ 
                          
                         
                             
                         
                          
                         sin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     = 
                     
                       Y 
                       - 
                       
                         2 
                          
                         
                           ( 
                           
                             h 
                             + 
                             Y 
                           
                           ) 
                         
                          
                         
                           cos 
                           2 
                         
                          
                         θ 
                       
                       + 
                       
                         2 
                          
                         
                           ( 
                           
                             
                               
                                 
                                   
                                     r 
                                      
                                     
                                         
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   - 
                                 
                               
                             
                             
                               
                                 
                                   d 
                                    
                                   
                                       
                                   
                                    
                                   sin 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                               
                             
                           
                           ) 
                         
                          
                         cos 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                   
                    
                   
                       
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       - 
                       
                         Y 
                          
                         
                           ( 
                           
                             
                               
                                 
                                   
                                     2 
                                      
                                     
                                       cos 
                                       2 
                                     
                                      
                                     θ 
                                   
                                   - 
                                 
                               
                             
                             
                               
                                 1 
                               
                             
                           
                           ) 
                         
                       
                     
                     - 
                     
                       2 
                        
                       l 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θcos 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       d 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θcos 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       
                         - 
                         Y 
                       
                        
                       
                           
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     - 
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       
                         ( 
                         
                           2 
                            
                           θ 
                         
                         ) 
                       
                     
                   
                 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               
                 
                   
                     
                       - 
                       d 
                     
                     - 
                     l 
                     + 
                     
                       2 
                        
                       
                         σ 
                         b 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       Δ 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                   = 
                     
                    
                   
                     
                       - 
                       d 
                     
                     - 
                     l 
                     + 
                     
                       2 
                        
                       
                         ( 
                         
                           h 
                           + 
                           Y 
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     2 
                      
                     
                       ( 
                       
                         
                           r 
                            
                           
                               
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                         - 
                         
                           d 
                            
                           
                               
                           
                            
                           sin 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                      
                     sin 
                      
                     
                         
                     
                      
                     θ 
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       - 
                       
                         d 
                          
                         
                           ( 
                           
                             1 
                             - 
                             
                               2 
                                
                               
                                 
                                   sin 
                                    
                                   
                                       
                                   
                                 
                                 2 
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                     
                     + 
                     
                       2 
                        
                       l 
                        
                       
                           
                       
                        
                       
                         sin 
                         2 
                       
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     l 
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     Y 
                      
                     
                         
                     
                      
                     sin 
                      
                     
                         
                     
                      
                     
                       ( 
                       
                         2 
                          
                         θ 
                       
                       ) 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       Y 
                        
                       
                           
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       
                         ( 
                         
                           2 
                            
                           θ 
                         
                         ) 
                       
                     
                     - 
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             θ 
                           
                           ) 
                         
                       
                     
                   
                 
               
             
           
         
       
     
         [0318]    Hence, 
         [0000]        Q   2 =( t   2   X,− 2Δ cos θ+ t   2 ρ 1b   ,d+ 2Δ sin θ+ t   2 ρ 2b ) 
         [0000]      where 
         [0000]      ρ 1b   =−Y  cos(2θ)−( d+l )sin(2θ) 
         [0000]      ρ 2b   =Y  sin(2θ)−( d+l )cos(2θ)   (40) 
         [0319]    Therefore, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     2 
                     ′ 
                   
                   = 
                   
                     MQ 
                     2 
                   
                 
               
             
             
               
                 
                   = 
                   
                     
                       [ 
                       
                         
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             1 
                           
                           
                             0 
                           
                           
                             0 
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               l 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                           
                             
                               
                                 - 
                                 dl 
                               
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                         
                           
                             0 
                           
                           
                             0 
                           
                           
                             
                               
                                 - 
                                 1 
                               
                               
                                 d 
                                 + 
                                 L 
                               
                             
                           
                           
                             
                               d 
                               
                                 d 
                                 + 
                                 l 
                               
                             
                           
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           
                             
                               
                                 t 
                                 2 
                               
                                
                               X 
                             
                           
                         
                         
                           
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 cos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     1 
                                      
                                     b 
                                   
                                 
                               
                             
                           
                         
                         
                           
                             
                               d 
                               + 
                               
                                 2 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     b 
                                   
                                 
                               
                             
                           
                         
                         
                           
                             1 
                           
                         
                       
                       ] 
                     
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               t 
                               2 
                             
                              
                             X 
                           
                         
                       
                       
                         
                           
                             
                               
                                 - 
                                 2 
                               
                                
                               Δ 
                                
                               
                                   
                               
                                
                               cos 
                                
                               
                                   
                               
                                
                               θ 
                             
                             + 
                             
                               
                                 t 
                                 2 
                               
                                
                               
                                 ρ 
                                 
                                   1 
                                    
                                   b 
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 l 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     
                                       2 
                                        
                                       Δ 
                                        
                                       
                                           
                                       
                                        
                                       sin 
                                        
                                       
                                           
                                       
                                        
                                       θ 
                                     
                                     + 
                                     
                                       
                                         t 
                                         2 
                                       
                                        
                                       
                                         ρ 
                                         
                                           2 
                                            
                                           b 
                                         
                                       
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               dl 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 - 
                                 d 
                               
                               - 
                               
                                 2 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     b 
                                   
                                 
                               
                               + 
                               d 
                             
                             
                               d 
                               + 
                               l 
                             
                           
                         
                       
                     
                     ] 
                   
                 
               
             
             
               
                 
                   = 
                   
                     [ 
                     
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 t 
                                 2 
                               
                                
                               X 
                             
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     b 
                                   
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                                
                               
                                 ( 
                                 
                                   
                                     
                                       - 
                                       2 
                                     
                                      
                                     Δ 
                                      
                                     
                                         
                                     
                                      
                                     cos 
                                      
                                     
                                         
                                     
                                      
                                     θ 
                                   
                                   + 
                                   
                                     
                                       t 
                                       2 
                                     
                                      
                                     
                                       ρ 
                                       
                                         1 
                                          
                                         b 
                                       
                                     
                                   
                                 
                                 ) 
                               
                             
                             
                               
                                 
                                   - 
                                   2 
                                 
                                  
                                 Δ 
                                  
                                 
                                     
                                 
                                  
                                 sin 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               - 
                               
                                 
                                   t 
                                   2 
                                 
                                  
                                 
                                   ρ 
                                   
                                     2 
                                      
                                     b 
                                   
                                 
                               
                             
                           
                         
                       
                       
                         
                           
                             - 
                             l 
                           
                         
                       
                       
                         
                           1 
                         
                       
                     
                     ] 
                   
                 
               
             
           
         
       
     
         [0320]    where ρ 1b  and ρ 2b  are defined in (40) and t 2  is defined in (27). 
         [0321]    Once we have P 1 ′ and P 2 ′ (or, P 1 ′ and P 3 ′) in a particular virtual view of the virtual image plane, the next step was to find their counterparts in the STM image. We need their counterparts for the subsequent matching process to identify A&#39;s corresponding point A′. It is sufficient to show the process for a general point in the virtual right view. 
         [0322]    Let P=(X,Y,−l) be an arbitrary point in the virtual right view of the virtual image plane. The lower-left, upper-left, lower-right and upper-right corners of the virtual right view are 
         [0000]        D′= ( h,−h,−l ) 
         [0000]        A′= ( h,h,−l ), 
         [0000]    
       
         
           
             
               
                 C 
                 ′ 
               
               = 
               
                 ( 
                 
                   
                     
                       r 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                     
                     d 
                   
                   , 
                   
                     
                       
                         - 
                         
                           h 
                            
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                         
                       
                        
                       
                         ( 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     
                       d 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                           + 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                   
                   , 
                   
                     - 
                     l 
                   
                 
                 ) 
               
             
             , 
             
               
 
             
              
             and 
           
         
       
       
         
           
             
               
                 B 
                 ′ 
               
               = 
               
                 ( 
                 
                   
                     
                       r 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                     
                     d 
                   
                   , 
                   
                     
                       
                         h 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                       
                        
                       
                         ( 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     
                       d 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                           + 
                           
                             2 
                              
                             Δ 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                   
                   , 
                   
                     - 
                     l 
                   
                 
                 ) 
               
             
             , 
           
         
       
     
         [0323]    respectively, where d, r, l, h, and θ are parameters of the STMIS defined as before and Δ=r cos θ−d sin θ is the distance between the real camera and each of the mirror plane. 
         [0324]    Let G=(x,y) be the counterpart of P in the right view of the STM image, x and y are real numbers (see  FIG. 15 ). Here we assume a real number coordinate system has been imposed on the right view of the STM image, whose x- and y-axes coincide with the x- and y-axes of the original integer coordinate system of the right view. Therefore, it makes sense to consider points with real number coordinates in the right view of the STM image. The lower-left, upper-left, lower-right and upper-right corners of the right view of the STM image with this real number coordinate system are now: 
         [0000]    
       
         
           
             
               D 
               = 
               
                 ( 
                 
                   
                     - 
                     
                       1 
                       2 
                     
                   
                   , 
                   
                     - 
                     
                       1 
                       2 
                     
                   
                 
                 ) 
               
             
             , 
             
               
 
             
              
             
               A 
               = 
               
                 ( 
                 
                   
                     - 
                     
                       1 
                       2 
                     
                   
                   , 
                   
                     n 
                     - 
                     
                       1 
                       2 
                     
                   
                 
                 ) 
               
             
             , 
             
               
 
             
              
             
               C 
               = 
               
                 ( 
                 
                   
                     
                       m 
                       1 
                     
                     - 
                     
                       1 
                       2 
                     
                   
                   , 
                   
                     
                       - 
                       q 
                     
                     - 
                     
                       1 
                       2 
                     
                   
                 
                 ) 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               B 
               = 
               
                 ( 
                 
                   
                     
                       m 
                       1 
                     
                     - 
                     
                       1 
                       2 
                     
                   
                   , 
                   
                     n 
                     + 
                     q 
                     - 
                     
                       1 
                       2 
                     
                   
                 
                 ) 
               
             
             , 
           
         
       
     
         [0325]    respectively, where m l &gt;0 is the resolution of the right view in x direction and n+2q (q&gt;0) is the resolution of the right view&#39;s right edge BC. 
         [0326]    The x- and y-coordinates of G can be computed as follows. Note that the shape of the virtual right view and the shape of the right view of the STM image are similar. This implies that the shape of the rectangle A′E′F′D′ is also similar to the shape of the rectangle AEFD (see  FIG. 16 ). Therefore, when we use ‘aspect ratio preserving property’ to compute x and y, we can simply consider aspect rations of P in the rectangle A′E′F′D′ and aspect ratios of G in the rectangle AEFD. By using the aspect ration preserving property in the x direction, we have 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           X 
                           - 
                           h 
                         
                         
                           x 
                           - 
                           
                             ( 
                             
                               
                                 - 
                                 1 
                               
                               / 
                               2 
                             
                             ) 
                           
                         
                       
                       = 
                       
                         
                           
                             
                               r 
                                
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                             
                             / 
                             d 
                           
                           - 
                           h 
                         
                         
                           
                             m 
                             1 
                           
                           - 
                           
                             1 
                             / 
                             2 
                           
                           - 
                           
                             ( 
                             
                               
                                 - 
                                 1 
                               
                               / 
                               2 
                             
                             ) 
                           
                         
                       
                     
                     , 
                     
                       
 
                     
                      
                     or 
                   
                    
                   
                     
 
                   
                    
                   
                     
                       
                         
                           X 
                           - 
                           h 
                         
                         
                           x 
                           + 
                           
                             1 
                             / 
                             2 
                           
                         
                       
                       = 
                       
                         
                           
                             
                               
                                 r 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                   ) 
                                 
                               
                               / 
                               d 
                             
                             - 
                             h 
                           
                           
                             m 
                             1 
                           
                         
                         = 
                         
                           
                             
                               
                                 r 
                                  
                                 
                                   ( 
                                   
                                     d 
                                     + 
                                     l 
                                   
                                   ) 
                                 
                               
                               / 
                               d 
                             
                             - 
                             h 
                           
                           
                             
                               ( 
                               
                                 
                                   
                                     r 
                                      
                                     
                                       ( 
                                       
                                         d 
                                         + 
                                         l 
                                       
                                       ) 
                                     
                                   
                                   / 
                                   d 
                                 
                                 - 
                                 h 
                               
                               ) 
                             
                              
                             
                               ( 
                               
                                 
                                   m 
                                   / 
                                   2 
                                 
                                  
                                 h 
                               
                               ) 
                             
                           
                         
                       
                     
                     , 
                     
                       
 
                     
                      
                     or 
                   
                    
                   
                      
                   
                    
                   
                     
                       
                         
                           X 
                           - 
                           h 
                         
                         
                           x 
                           + 
                           
                             1 
                             / 
                             2 
                           
                         
                       
                       = 
                       
                         
                           2 
                            
                           h 
                         
                         m 
                       
                     
                     , 
                     
                       
 
                     
                      
                     or 
                   
                    
                   
                     
 
                   
                    
                   
                     
                       
                         x 
                         + 
                         
                           1 
                           2 
                         
                       
                       = 
                       
                         
                           m 
                            
                           
                             ( 
                             
                               X 
                               - 
                               h 
                             
                             ) 
                           
                         
                         
                           2 
                            
                           h 
                         
                       
                     
                     , 
                     
                       
 
                     
                      
                     or 
                   
                    
                   
                      
                   
                    
                   
                     
                       x 
                       = 
                       
                         
                           
                             - 
                             
                               1 
                               2 
                             
                           
                           + 
                           
                             
                               m 
                                
                               
                                 ( 
                                 
                                   X 
                                   - 
                                   h 
                                 
                                 ) 
                               
                             
                             
                               2 
                                
                               h 
                             
                           
                         
                         = 
                         
                           
                             
                               
                                 - 
                                 h 
                               
                               + 
                               
                                 m 
                                  
                                 
                                   ( 
                                   
                                     X 
                                     - 
                                     h 
                                   
                                   ) 
                                 
                               
                             
                             
                               2 
                                
                               h 
                             
                           
                           . 
                           
                             
 
                           
                            
                           Hence 
                         
                       
                     
                     , 
                     
                        
                     
                      
                     
                       x 
                       = 
                       
                         
                           
                             mX 
                             - 
                             
                               
                                 ( 
                                 
                                   m 
                                   + 
                                   1 
                                 
                                 ) 
                               
                                
                               h 
                             
                           
                           
                             2 
                              
                             h 
                           
                         
                         . 
                       
                     
                   
                 
               
               
                 
                   ( 
                   41 
                   ) 
                 
               
             
           
         
       
     
         [0327]    By using the aspect ration preserving property in the y direction in the rectangle A′E′F′D′ and in the rectangle AEFD we have 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           y 
                           + 
                           
                             1 
                             / 
                             2 
                           
                         
                         
                           Y 
                           + 
                           h 
                         
                       
                       = 
                       
                         
                           
                             n 
                             - 
                             
                               1 
                               / 
                               2 
                             
                             - 
                             
                               ( 
                               
                                 
                                   - 
                                   1 
                                 
                                 / 
                                 2 
                               
                               ) 
                             
                           
                           
                             h 
                             - 
                             
                               ( 
                               
                                 - 
                                 h 
                               
                               ) 
                             
                           
                         
                         = 
                         
                           n 
                           
                             2 
                              
                             h 
                           
                         
                       
                     
                     , 
                     
                       
 
                     
                      
                     or 
                   
                    
                   
                     
 
                   
                    
                   
                     
                       
                         y 
                         + 
                         
                           1 
                           2 
                         
                       
                       = 
                       
                         
                           nY 
                           + 
                           nh 
                         
                         
                           2 
                            
                           h 
                         
                       
                     
                     , 
                     
                       
 
                     
                      
                     or 
                   
                    
                   
                      
                   
                    
                   
                     
                       y 
                       = 
                       
                         
                           
                             - 
                             
                               1 
                               2 
                             
                           
                           + 
                           
                             
                               nY 
                               + 
                               nh 
                             
                             
                               2 
                                
                               h 
                             
                           
                         
                         = 
                         
                           
                             
                               nY 
                               + 
                               nh 
                               - 
                               h 
                             
                             
                               2 
                                
                               h 
                             
                           
                           . 
                           
                             
 
                           
                            
                           Hence 
                         
                       
                     
                     , 
                     
                        
                     
                      
                     
                       y 
                       = 
                       
                         
                           
                             nY 
                             + 
                             
                               
                                 ( 
                                 
                                   n 
                                   - 
                                   1 
                                 
                                 ) 
                               
                                
                               h 
                             
                           
                           
                             2 
                              
                             h 
                           
                         
                         . 
                       
                     
                   
                 
               
               
                 
                   ( 
                   42 
                   ) 
                 
               
             
           
         
       
     
         [0328]    The computation of counterparts for other virtual views can be done similarly. The results are listed below. 
         [0329]    Let P=(X,Y,−l) be an arbitrary point in the virtual left view of a virtual image plane (see  FIG. 16 ) whose lower-left, upper-left, lower-right and upper-right corners are 
         [0000]    
       
         
           
             
               
                 C 
                 ′ 
               
               = 
               
                 ( 
                 
                   
                     
                       - 
                       
                         r 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                       
                     
                     d 
                   
                   , 
                   
                     
                       
                         - 
                         
                           h 
                            
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                         
                       
                        
                       
                         ( 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     
                       d 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                           + 
                           
                             2 
                              
                             Δsinθ 
                           
                         
                         ) 
                       
                     
                   
                   , 
                   
                     - 
                     l 
                   
                 
                 ) 
               
             
             , 
             
               
 
             
              
             
               
                 B 
                 ′ 
               
               = 
               
                 ( 
                 
                   
                     
                       - 
                       
                         r 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                       
                     
                     d 
                   
                   , 
                   
                     
                       
                         h 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                       
                        
                       
                         ( 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     
                       d 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                           + 
                           
                             2 
                              
                             Δsinθ 
                           
                         
                         ) 
                       
                     
                   
                   , 
                   
                     - 
                     l 
                   
                 
                 ) 
               
             
             , 
             
               
 
             
              
             
               
                 D 
                 ′ 
               
               = 
               
                 
                   ( 
                   
                     
                       - 
                       h 
                     
                     , 
                     
                       - 
                       h 
                     
                     , 
                     
                       - 
                       l 
                     
                   
                   ) 
                 
                  
                 
                     
                 
                  
                 and 
               
             
           
         
       
       
         
           
             
               
                 A 
                 ′ 
               
               = 
               
                 ( 
                 
                   
                     - 
                     h 
                   
                   , 
                   h 
                   , 
                   
                     - 
                     l 
                   
                 
                 ) 
               
             
             , 
           
         
       
     
         [0330]    respectively. If G=(x,y) is the counterpart of P in the left view of the STM image whose lower-left, upper-left, lower-right and upper-right corners are 
         [0000]        C= (− m   1 +½,− q− ½), 
         [0000]        B= (− m   1 +½, n+q− ½), 
         [0000]        D= (½,−½) and 
         [0000]        A= (½, n− ½), 
         [0331]    respectively, then x and y are real numbers of the following values 
         [0000]    
       
         
           
             
               
                 
                   x 
                   = 
                   
                     
                       mX 
                       + 
                       
                         
                           ( 
                           
                             m 
                             + 
                             1 
                           
                           ) 
                         
                          
                         h 
                       
                     
                     
                       2 
                        
                       h 
                     
                   
                 
               
               
                 
                   ( 
                   43 
                   ) 
                 
               
             
             
               
                 
                   y 
                   = 
                   
                     
                       nY 
                       + 
                       
                         
                           ( 
                           
                             n 
                             - 
                             1 
                           
                           ) 
                         
                          
                         h 
                       
                     
                     
                       2 
                        
                       h 
                     
                   
                 
               
               
                 
                   ( 
                   44 
                   ) 
                 
               
             
           
         
       
     
         [0332]    Let P=(X,Y,−l) be an arbitrary point in the virtual upper view of a virtual image plane (see  FIG. 17 ) whose lower-left, upper-left, lower-right and upper-right corners are 
         [0000]    
       
         
           
             
               
                 D 
                 ′ 
               
               = 
               
                 ( 
                 
                   
                     - 
                     h 
                   
                   , 
                   h 
                   , 
                   
                     - 
                     l 
                   
                 
                 ) 
               
             
             , 
             
               
 
             
              
             
               
                 C 
                 ′ 
               
               = 
               
                 ( 
                 
                   
                     
                       
                         - 
                         
                           h 
                            
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                         
                       
                        
                       
                         ( 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                     
                       d 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                           + 
                           
                             2 
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                   
                   , 
                   
                     
                       r 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                     
                     d 
                   
                   , 
                   
                     - 
                     l 
                   
                 
                 ) 
               
             
             , 
             
               
 
             
              
             
               
                 B 
                 ′ 
               
               = 
               
                 
                   ( 
                   
                     
                       
                         
                           h 
                            
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                         
                          
                         
                           ( 
                           
                             d 
                             + 
                             
                               2 
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                       
                         d 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                             + 
                             
                               2 
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                           
                           ) 
                         
                       
                     
                     , 
                     
                       
                         r 
                          
                         
                           ( 
                           
                             d 
                             + 
                             l 
                           
                           ) 
                         
                       
                       d 
                     
                     , 
                     
                       - 
                       l 
                     
                   
                   ) 
                 
                  
                 
                     
                 
                  
                 and 
               
             
           
         
       
       
         
           
             
               
                 A 
                 ′ 
               
               = 
               
                 ( 
                 
                   h 
                   , 
                   
                     h 
                     - 
                     l 
                   
                 
                 ) 
               
             
             , 
           
         
       
     
         [0333]    respectively. If G=(x,y) is the counterpart of P in the upper view of the STM image whose lower-left, upper-left, lower-right and upper-right corners are 
         [0000]        D =(−½,−½), 
         [0000]        C =(− p− ½, n   1 −½), 
         [0000]        B =( m− ½+ p,n   1 −½) and 
         [0000]        A =( m− ½,−½), 
         [0334]    respectively, then x and y are real numbers of the following values 
         [0000]    
       
         
           
             
               
                 
                   x 
                   = 
                   
                     
                       mX 
                       + 
                       
                         
                           ( 
                           
                             m 
                             - 
                             1 
                           
                           ) 
                         
                          
                         h 
                       
                     
                     
                       2 
                        
                       h 
                     
                   
                 
               
               
                 
                   ( 
                   45 
                   ) 
                 
               
             
             
               
                 
                   y 
                   = 
                   
                     
                       nY 
                       - 
                       
                         
                           ( 
                           
                             n 
                             + 
                             1 
                           
                           ) 
                         
                          
                         h 
                       
                     
                     
                       2 
                        
                       h 
                     
                   
                 
               
               
                 
                   ( 
                   46 
                   ) 
                 
               
             
           
         
       
     
         [0335]    For the lower view case, we will simply give the x- and y-coordinates of the counterpart G=(x,y) in the lower view of the STM image for the given point P=(X,Y,−l) in the virtual lower view of the virtual image plane: 
         [0000]    
       
         
           
             
               
                 
                   x 
                   = 
                   
                     
                       mX 
                       + 
                       
                         
                           ( 
                           
                             m 
                             - 
                             1 
                           
                           ) 
                         
                          
                         h 
                       
                     
                     
                       2 
                        
                       h 
                     
                   
                 
               
               
                 
                   ( 
                   47 
                   ) 
                 
               
             
             
               
                 
                   y 
                   = 
                   
                     
                       nY 
                       + 
                       
                         
                           ( 
                           
                             n 
                             + 
                             1 
                           
                           ) 
                         
                          
                         h 
                       
                     
                     
                       2 
                        
                       n 
                     
                   
                 
               
               
                 
                   ( 
                   48 
                   ) 
                 
               
             
           
         
       
     
         [0336]    With the computation processes developed above, we are ready for initial screening of the corresponding points now. The concept is described as follows. 
         [0337]    For each point A=(x,y), 0≦x≦m−1, 0≦y≦n−1, in the central view of the given STM image where in x n is the resolution of the central view, first identify its corresponding point P 1 =(X,Y,−l) in the virtual central view of the virtual image plane with 
         [0000]    
       
         
           
             X 
             = 
             
               
                 
                   
                     ( 
                     
                       x 
                       - 
                       
                         m 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       h 
                     
                     m 
                   
                 
                 + 
                 
                   h 
                   m 
                 
               
               = 
               
                 
                   
                     ( 
                     
                       x 
                       + 
                       
                         1 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       h 
                     
                     m 
                   
                 
                 - 
                 h 
               
             
           
         
       
       
         
           
             Y 
             = 
             
               
                 
                   
                     ( 
                     
                       y 
                       - 
                       
                         n 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       h 
                     
                     n 
                   
                 
                 + 
                 
                   h 
                   n 
                 
               
               = 
               
                 
                   
                     ( 
                     
                       y 
                       + 
                       
                         1 
                         2 
                       
                     
                     ) 
                   
                    
                   
                     
                       2 
                        
                       h 
                     
                     n 
                   
                 
                 - 
                 h 
               
             
           
         
       
     
         [0338]    where l is the length of the STM and 2h×2h is the dimension of the front opening of the STM. 
         [0339]    We then compute the location of the point P 2  by using (27) to compute the values of t 2  first and then using the following equation to get coordinates of P 2 : 
         [0000]        P   2   =C+t   2 ( P   1   −C )=( t   2   X,t   2   Y,d−t   2 ( d+l )) 
         [0340]    where C=(0,0,d) is the location of the pinhole of the camera. The value of d can be determined using the technique described previously. 
         [0341]    Note that if the following condition is satisfied:
       (i) 2Δ cos θ&gt;h and X≠0, or   (ii) 2Δ cos θ&lt;h and |X|&gt;  X  where       
 
         [0000]    
       
         
           
             
               
                 X 
                 _ 
               
               = 
               
                 
                   
                     ( 
                     
                       d 
                       + 
                       l 
                     
                     ) 
                   
                    
                   
                     ( 
                     
                       d 
                       + 
                       l 
                       + 
                       
                         2 
                          
                         Δsin 
                          
                         
                             
                         
                          
                         θ 
                       
                     
                     ) 
                   
                    
                   
                     
                       ( 
                       
                         
                           Δ 
                           1 
                         
                         - 
                         
                           Δ 
                           2 
                         
                       
                       ) 
                     
                      
                     
                       [ 
                       
                         
                           
                             ( 
                             
                               λ 
                               - 
                               1 
                             
                             ) 
                           
                            
                           
                             Δ 
                             1 
                           
                         
                         - 
                         
                           Δ 
                           2 
                         
                       
                       ] 
                     
                   
                 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         
                           r 
                            
                           
                               
                           
                            
                           tan 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         
                           Δ 
                           1 
                         
                         + 
                         
                           Δ 
                           2 
                         
                       
                       ) 
                     
                   
                   - 
                   
                     
                       λ 
                        
                       
                         ( 
                         
                           d 
                           + 
                           
                             2 
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                     
                      
                     
                       ( 
                       
                         
                           Δ 
                           1 
                         
                         - 
                         
                           Δ 
                           2 
                         
                       
                       ) 
                     
                   
                 
               
             
             , 
           
         
       
     
         [0344]    Δ=r cos θ−d sin θ, Δ 1  and Δ 2  are defined above, 2r×2r is the dimension of the rear end of the STM and λ is a constant between 2 and 4 specified by the user for the location of the general J-point (see (26) for the definition of the general J-point), then t 2  should be computed using (27). 
         [0345]    Next, we compute the projections of P 1  and P 2  in the virtual right view, virtual left view, virtual upper view and virtual lower view. These projections will be called P 1r ′ and P 2r ′, P 1l ′ and P 2l ′, P 1r ′ and P 2r ′and P 1b ′ and P 2b ′, respectively. They are listed below: 
         [0000]    
       
         
           
             
               P 
               
                 1 
                  
                 r 
               
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         
                           X 
                           + 
                           
                             2 
                              
                             σ 
                           
                         
                         , 
                         
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     Y 
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
       
         
           
             
               P 
               
                 2 
                  
                 r 
               
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         
                           2 
                            
                           Δcos 
                            
                           
                               
                           
                            
                           θ 
                         
                         + 
                         
                           
                             t 
                             2 
                           
                            
                           
                             ρ 
                             
                               1 
                                
                               r 
                             
                           
                         
                       
                       ) 
                     
                   
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ρ 
                         
                           2 
                            
                           r 
                         
                       
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       t 
                       2 
                     
                      
                     Y 
                   
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ρ 
                         
                           2 
                            
                           r 
                         
                       
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
       
         
           
             
               P 
               
                 1 
                  
                 l 
               
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         X 
                         - 
                         
                           2 
                            
                           
                             σ 
                             l 
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         l 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     Y 
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         l 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
       
         
           
             
               P 
               
                 2 
                  
                 l 
               
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         
                           
                             - 
                             2 
                           
                            
                           Δcos 
                            
                           
                               
                           
                            
                           θ 
                         
                         + 
                         
                           
                             t 
                             2 
                           
                            
                           
                             ρ 
                             
                               1 
                                
                               l 
                             
                           
                         
                       
                       ) 
                     
                   
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ρ 
                         
                           2 
                            
                           l 
                         
                       
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       t 
                       2 
                     
                      
                     Y 
                   
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ρ 
                         
                           2 
                            
                           l 
                         
                       
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
       
         
           
             
               P 
               
                 1 
                  
                 i 
               
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     X 
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         i 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         Y 
                         + 
                         
                           2 
                            
                           
                             σ 
                             i 
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         i 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
       
         
           
             
               P 
               
                 2 
                  
                 i 
               
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       t 
                       2 
                     
                      
                     X 
                   
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ρ 
                         
                           2 
                            
                           i 
                         
                       
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         
                           2 
                            
                           Δcosθ 
                         
                         + 
                         
                           
                             t 
                             2 
                           
                            
                           
                             ρ 
                             
                               1 
                                
                               i 
                             
                           
                         
                       
                       ) 
                     
                   
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ρ 
                         
                           2 
                            
                           i 
                         
                       
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
       
         
           
             
               P 
               
                 1 
                  
                 b 
               
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     X 
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         b 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         Y 
                         - 
                         
                           2 
                            
                           
                             σ 
                             b 
                           
                            
                           cos 
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                   
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         b 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
       
         
           
             
               P 
               
                 2 
                  
                 b 
               
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       t 
                       2 
                     
                      
                     X 
                   
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ρ 
                         
                           2 
                            
                           b 
                         
                       
                     
                   
                 
                 , 
                 
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         
                           
                             - 
                             2 
                           
                            
                           Δcos 
                            
                           
                               
                           
                            
                           θ 
                         
                         + 
                         
                           
                             t 
                             2 
                           
                            
                           
                             ρ 
                             
                               1 
                                
                               b 
                             
                           
                         
                       
                       ) 
                     
                   
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       
                         t 
                         2 
                       
                        
                       
                         ρ 
                         
                           2 
                            
                           b 
                         
                       
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
     
         [0346]    where 
         [0000]      σ r =( h−X )cos θ; 
         [0000]      σ l =( h+X )cos θ; 
         [0000]      σ i =( h−Y )cos θ; 
         [0000]      σ b =( h+Y )cos θ; 
         [0347]    and ρ iv , ρ il , ρ ir  and ρ ib , i=1,2, are defined as set forth above. 
         [0348]    We then set a step size for the digitization parameter t of the line segment P 1 P 2  as follows: 
         [0000]      Δ t =δ/( d+l ) 
         [0000]      where 
         [0000]      δ=min{2 h/m, 2 h/n}   
         [0349]    This step size will ensure that each digitized element of P 1 P 2  is of length δ, the minimum of the dimension 
         [0000]    
       
         
           
             
               
                 2 
                  
                 h 
               
               m 
             
             × 
             
               
                 2 
                  
                 h 
               
               n 
             
           
         
       
     
         [0000]    of a virtual pixel in the virtual image plane. The digitization process starts at P 1 (t=1) and proceeds by the step size 
         [0000]      Δ P =Δ l ( P   1   −C ). 
         [0350]    The number of digitized elements of the line segment P 1 P 2  is 
         [0000]    
       
         
           
             N 
             = 
             
               
                 [ 
                 
                   
                     
                       t 
                       2 
                     
                     - 
                     1 
                   
                   
                     Δ 
                     t 
                   
                 
                 ] 
               
               . 
             
           
         
       
     
         [0351]    The basic idea of the searching process for corresponding points in the right view can be described as follows. The searching process for corresponding points in the other views is similar. 
         [0352]    For the i-th digitized element P of P 1 P 2    
         [0000]        P=C+ (1 +iΔ   t )( P   1   −C )= P   1   +iΔ   p    
         [0353]    we find its projection P′ in the virtual right view 
         [0000]    
       
         
           
             
               P 
               ′ 
             
             = 
             
               
                 ( 
                 
                   
                     X 
                     _ 
                   
                   , 
                   
                     Y 
                     _ 
                   
                   , 
                   
                     - 
                     l 
                   
                 
                 ) 
               
               = 
               
                 ( 
                 
                   
                     
                       
                         
                           
                             
                               ( 
                               
                                 d 
                                 + 
                                 l 
                               
                               ) 
                             
                              
                             
                               ( 
                               
                                 
                                   
                                     
                                       
                                         2 
                                          
                                         Δcosθ 
                                       
                                       + 
                                     
                                   
                                 
                                 
                                   
                                     
                                       
                                         ( 
                                         
                                           1 
                                           + 
                                           
                                             i 
                                              
                                             
                                                 
                                             
                                              
                                             Δ 
                                              
                                             
                                                 
                                             
                                              
                                             t 
                                           
                                         
                                         ) 
                                       
                                        
                                       
                                         ρ 
                                         
                                           1 
                                            
                                           r 
                                         
                                       
                                     
                                   
                                 
                               
                               ) 
                             
                           
                           
                             
                               
                                 - 
                                 2 
                               
                                
                               Δsin 
                                
                               
                                   
                               
                                
                               θ 
                             
                             - 
                             
                               
                                 ( 
                                 
                                   1 
                                   + 
                                   
                                     i 
                                      
                                     
                                         
                                     
                                      
                                     
                                       Δ 
                                       i 
                                     
                                   
                                 
                                 ) 
                               
                                
                               
                                 ρ 
                                 
                                   2 
                                    
                                   r 
                                 
                               
                             
                           
                         
                         , 
                       
                     
                   
                   
                     
                       
                         
                           
                             
                               ( 
                               
                                 d 
                                 + 
                                 l 
                               
                               ) 
                             
                              
                             
                               ( 
                               
                                 1 
                                 + 
                                 
                                   i 
                                    
                                   
                                       
                                   
                                    
                                   
                                     Δ 
                                     i 
                                   
                                 
                               
                               ) 
                             
                              
                             Y 
                           
                           
                             
                               
                                 - 
                                 2 
                               
                                
                               Δsinθ 
                             
                             - 
                             
                               
                                 ( 
                                 
                                   1 
                                   + 
                                   
                                     i 
                                      
                                     
                                         
                                     
                                      
                                     
                                       Δ 
                                       i 
                                     
                                   
                                 
                                 ) 
                               
                                
                               
                                 ρ 
                                 
                                   2 
                                    
                                   r 
                                 
                               
                             
                           
                         
                         , 
                         
                           - 
                           l 
                         
                       
                     
                   
                 
                 ) 
               
             
           
         
       
     
         [0354]    and the counterpart of P′ in the STM image&#39;s right view 
         [0000]    
       
         
           
             G 
             = 
             
               
                 ( 
                 
                   α 
                   , 
                   β 
                 
                 ) 
               
               = 
               
                 
                   ( 
                   
                     
                       
                         
                           m 
                            
                           
                             X 
                             _ 
                           
                         
                         - 
                         
                           
                             ( 
                             
                               m 
                               + 
                               1 
                             
                             ) 
                           
                            
                           h 
                         
                       
                       
                         2 
                          
                         h 
                       
                     
                     , 
                     
                       
                         
                           n 
                            
                           
                             Y 
                             _ 
                           
                         
                         + 
                         
                           
                             ( 
                             
                               n 
                               - 
                               1 
                             
                             ) 
                           
                            
                           h 
                         
                       
                       
                         2 
                          
                         h 
                       
                     
                   
                   ) 
                 
                 . 
               
             
           
         
       
     
         [0355]    A matching process is then performed on a patch centered at A=(α,β) in the STM image&#39;s central view and a patch centered at G=(α,β) in the STM image&#39;s right view. This matching process will return the difference of the intensity values of the patches. Whichever P′ whose returned difference result is the smallest and is smaller than a given tolerance is considered the corresponding point of P 1  (or, A=(x,y)). 
         [0356]    In the above process, P can be computed using an incremental method. Actually P′ can be computed using an incremental method as well. Note that the start point of P is P 1  and the start point of P′ is P 1 ′. Hence, it is sufficient to show that the second point of P′ (t=1+Δ l ) can be computed from P 1 ′ incrementally. First, note that 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       2 
                        
                       Δcos 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       ρ 
                       
                         1 
                          
                         r 
                       
                     
                   
                   = 
                     
                    
                   
                     
                       2 
                        
                       
                         ( 
                         
                           
                             r 
                              
                             
                                 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           
                             d 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       X 
                        
                       
                           
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             
                                 
                             
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     + 
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       
                         sin 
                          
                         
                           ( 
                           
                             2 
                              
                             
                                 
                             
                              
                             θ 
                           
                           ) 
                         
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       2 
                        
                       r 
                        
                       
                           
                       
                        
                       
                         cos 
                         2 
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       d 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       X 
                        
                       
                           
                       
                        
                       
                         cos 
                         2 
                       
                        
                       θ 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     
                       X 
                        
                       
                           
                       
                        
                       
                         sin 
                         2 
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       2 
                        
                       
                         ( 
                         
                           r 
                           + 
                           
                             l 
                              
                             
                                 
                             
                              
                             tan 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                        
                       
                         cos 
                         2 
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       X 
                        
                       
                           
                       
                        
                       
                         cos 
                         2 
                       
                        
                       θ 
                     
                     + 
                     X 
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     X 
                     + 
                     
                       2 
                        
                       h 
                        
                       
                           
                       
                        
                       
                         cos 
                         2 
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       X 
                        
                       
                           
                       
                        
                       
                         cos 
                         2 
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     X 
                     + 
                     
                       
                         2 
                          
                         
                           [ 
                           
                             
                               ( 
                               
                                 h 
                                 - 
                                 X 
                               
                               ) 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           ] 
                         
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     X 
                     + 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
           
         
       
       
         
           and 
         
       
       
         
           
             
               
                 
                   
                     
                       
                         - 
                         2 
                       
                        
                       Δsinθ 
                     
                     - 
                     
                       ρ 
                       
                         2 
                          
                         r 
                       
                     
                   
                   = 
                     
                    
                   
                     
                       
                         - 
                         2 
                       
                        
                       
                         ( 
                         
                           
                             r 
                              
                             
                                 
                             
                              
                             cos 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           
                             d 
                              
                             
                                 
                             
                              
                             sin 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       X 
                        
                       
                           
                       
                        
                       
                         sin 
                          
                         
                           ( 
                           
                             2 
                              
                             
                                 
                             
                              
                             θ 
                           
                           ) 
                         
                       
                     
                     + 
                     
                       
                         ( 
                         
                           d 
                           + 
                           l 
                         
                         ) 
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             2 
                              
                             
                                 
                             
                              
                             θ 
                           
                           ) 
                         
                       
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     
                       
                         - 
                         2 
                       
                        
                       r 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       d 
                        
                       
                           
                       
                        
                       
                         sin 
                         2 
                       
                        
                       θ 
                     
                      
                     
                         
                     
                     + 
                     
                       2 
                        
                       X 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsinθ 
                     
                     + 
                   
                 
               
             
             
               
                 
                     
                    
                   
                     
                       ( 
                       
                         d 
                         + 
                         l 
                       
                       ) 
                     
                      
                     
                       ( 
                       
                         
                           
                             cos 
                             2 
                           
                            
                           
                               
                           
                            
                           θ 
                         
                         - 
                         
                           
                             sin 
                             2 
                           
                            
                           
                               
                           
                            
                           θ 
                         
                       
                       ) 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       r 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     - 
                     
                       2 
                        
                       l 
                        
                       
                           
                       
                        
                       
                         sin 
                         2 
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       X 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         ( 
                         
                           r 
                           + 
                           
                             l 
                              
                             
                                 
                             
                              
                             tan 
                              
                             
                                 
                             
                              
                             θ 
                           
                         
                         ) 
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       X 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       h 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                     + 
                     
                       2 
                        
                       X 
                        
                       
                           
                       
                        
                       cos 
                        
                       
                           
                       
                        
                       θsin 
                        
                       
                           
                       
                        
                       θ 
                     
                   
                 
               
             
             
               
                 
                   = 
                     
                    
                   
                     d 
                     + 
                     l 
                     - 
                     
                       2 
                        
                       
                         σ 
                         r 
                       
                        
                       sin 
                        
                       
                           
                       
                        
                       
                         θ 
                         . 
                       
                     
                   
                 
               
             
           
         
       
     
         [0357]    The second point of P′ (t=1+Δ t ) can be expressed as 
         [0000]    
       
         
           
             
               
                 
                   
                     P 
                     ′ 
                   
                   = 
                   
                     ( 
                     
                       
                         
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                            
                           
                             ( 
                             
                               
                                 2 
                                  
                                 Δcos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 
                                   ( 
                                   
                                     1 
                                     + 
                                     
                                       Δ 
                                       i 
                                     
                                   
                                   ) 
                                 
                                  
                                 
                                   ρ 
                                   
                                     1 
                                      
                                     r 
                                   
                                 
                               
                             
                             ) 
                           
                         
                         
                           
                             
                               - 
                               2 
                             
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           
                             
                               ( 
                               
                                 1 
                                 + 
                                 
                                   Δ 
                                   i 
                                 
                               
                               ) 
                             
                              
                             
                               ρ 
                               
                                 2 
                                  
                                 r 
                               
                             
                           
                         
                       
                       , 
                       
                         
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                            
                           
                             ( 
                             
                               1 
                               + 
                               
                                 Δ 
                                 i 
                               
                             
                             ) 
                           
                            
                           Y 
                         
                         
                           
                             
                               - 
                               2 
                             
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           
                             
                               ( 
                               
                                 1 
                                 + 
                                 
                                   Δ 
                                   i 
                                 
                               
                               ) 
                             
                              
                             
                               ρ 
                               
                                 2 
                                  
                                 r 
                               
                             
                           
                         
                       
                       , 
                       
                         - 
                         l 
                       
                     
                     ) 
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       
                         
                           
                             ( 
                             
                               d 
                               + 
                               l 
                             
                             ) 
                           
                            
                           
                             ( 
                             
                               
                                 2 
                                  
                                 Δcos 
                                  
                                 
                                     
                                 
                                  
                                 θ 
                               
                               + 
                               
                                 ρ 
                                 
                                   1 
                                    
                                   r 
                                 
                               
                               + 
                               
                                 
                                   Δ 
                                   i 
                                 
                                  
                                 
                                   ρ 
                                   
                                     1 
                                      
                                     r 
                                   
                                 
                               
                             
                             ) 
                           
                         
                         
                           
                             
                               - 
                               2 
                             
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           
                             ρ 
                             
                               2 
                                
                               r 
                             
                           
                           - 
                           
                             
                               Δ 
                               i 
                             
                              
                             
                               ρ 
                               
                                 2 
                                  
                                 r 
                               
                             
                           
                         
                       
                       , 
                       
                         
                           
                             
                               ( 
                               
                                 d 
                                 + 
                                 l 
                               
                               ) 
                             
                              
                             Y 
                           
                           + 
                           
                             
                               
                                 Δ 
                                 i 
                               
                                
                               
                                 ( 
                                 
                                   d 
                                   + 
                                   l 
                                 
                                 ) 
                               
                             
                              
                             Y 
                           
                         
                         
                           
                             
                               - 
                               2 
                             
                              
                             Δsin 
                              
                             
                                 
                             
                              
                             θ 
                           
                           - 
                           
                             ρ 
                             
                               2 
                                
                               r 
                             
                           
                           - 
                           
                             
                               Δ 
                               i 
                             
                              
                             
                               ρ 
                               
                                 2 
                                  
                                 r 
                               
                             
                           
                         
                       
                       , 
                       
                         - 
                         l 
                       
                     
                     ) 
                   
                 
               
             
             
               
                 
                   = 
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 
                                   
                                     ( 
                                     
                                       d 
                                       + 
                                       l 
                                     
                                     ) 
                                   
                                    
                                   
                                     ( 
                                     
                                       X 
                                       + 
                                       
                                         2 
                                          
                                         
                                           σ 
                                           r 
                                         
                                          
                                         cos 
                                          
                                         
                                             
                                         
                                          
                                         θ 
                                       
                                     
                                     ) 
                                   
                                 
                                 + 
                                 
                                   
                                     
                                       Δ 
                                       i 
                                     
                                      
                                     
                                       ( 
                                       
                                         d 
                                         + 
                                         l 
                                       
                                       ) 
                                     
                                   
                                    
                                   
                                     ρ 
                                     
                                       1 
                                        
                                       r 
                                     
                                   
                                 
                               
                               
                                 d 
                                 + 
                                 l 
                                 - 
                                 
                                   2 
                                    
                                   
                                     σ 
                                     r 
                                   
                                    
                                   sin 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 - 
                                 
                                   
                                     Δ 
                                     i 
                                   
                                    
                                   
                                     ρ 
                                     
                                       2 
                                        
                                       r 
                                     
                                   
                                 
                               
                             
                             , 
                           
                         
                       
                       
                         
                           
                             
                               
                                 
                                   
                                     ( 
                                     
                                       d 
                                       + 
                                       l 
                                     
                                     ) 
                                   
                                    
                                   Y 
                                 
                                 + 
                                 
                                   
                                     
                                       Δ 
                                       i 
                                     
                                      
                                     
                                       ( 
                                       
                                         d 
                                         + 
                                         l 
                                       
                                       ) 
                                     
                                   
                                    
                                   Y 
                                 
                               
                               
                                 d 
                                 + 
                                 l 
                                 - 
                                 
                                   2 
                                    
                                   
                                     σ 
                                     r 
                                   
                                    
                                   sin 
                                    
                                   
                                       
                                   
                                    
                                   θ 
                                 
                                 - 
                                 
                                   
                                     Δ 
                                     i 
                                   
                                    
                                   
                                     ρ 
                                     
                                       2 
                                        
                                       r 
                                     
                                   
                                 
                               
                             
                             , 
                             
                               - 
                               l 
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0358]    Hence, if we define 
         [0000]        A= ( d+l )( X+ 2σ, cos θ); 
         [0000]        B=d+l− 2σ r  sin θ; 
         [0000]        E= ( d+l ) Y;    
         [0000]      and 
         [0000]      Δ A =Δ t ( d+l )ρ 1r ; 
         [0000]      Δ B =Δ t ρ 2r ; 
         [0000]      Δ E =Δ l ( d+l ) Y    
         [0359]    then P′ for t=1+Δ l  can be written as 
         [0000]    
       
         
           
             
               P 
               ′ 
             
             = 
             
               ( 
               
                 
                   
                     A 
                     + 
                     
                       Δ 
                       A 
                     
                   
                   
                     B 
                     - 
                     
                       Δ 
                       B 
                     
                   
                 
                 , 
                 
                   
                     E 
                     + 
                     
                       Δ 
                       E 
                     
                   
                   
                     B 
                     - 
                     
                       Δ 
                       B 
                     
                   
                 
                 , 
                 
                   - 
                   l 
                 
               
               ) 
             
           
         
       
     
         [0360]    Note that A/B is the x-coordinate of P 1 ′ and E/B is the y-coordinate of P 2 ′. Hence P′ can indeed be computed incrementally. 
         [0361]    The skilled artisan will appreciate that the above calculations may be embodied in software code for converting an STM image as described herein into an image-plus-depth, and from there to a 3D image. Example software code is set forth herein in Appendices 1 and 2. 
         [0362]    The skilled artisan will further appreciate that the above-described devices, and methods and software therefore, are adaptable to a variety of applications, including document cameras, endoscopy, three-dimensional Web cameras, and the like. Representative designs for devices for providing three-dimensional intraoral images are shown in  FIGS. 22 and 23 , having an STM  12  according to the present disclosure, an external structured light source  40 , and an imager (digital camera  42  in  FIG. 22 , shutter/CCD  52  in  FIG. 23 . The device of  FIG. 22  is contemplated for obtaining three-dimensional images of an entirety of a patient&#39;s oral cavity, whereas the device of  FIG. 23  is contemplated for obtaining three-dimensional images of discrete regions of a patient&#39;s oral cavity, such as tooth  54 . 
         [0363]    Likewise,  FIG. 24  shows an embodiment of a three-dimensional Web camera  60 , comprising an STM  12  according to the present disclosure. The camera  60  includes a housing  62 , a lens/aperture  64 , an external light source  66 , and a shutter/CCD  68 . A concave lens  70  is provided at a front opening  72  of the STM  12  to reduce the size of the Web camera  60  without reducing the necessary field of view.  FIG. 25  presents a nine-view image provided by a three-dimensional Web camera as shown in  FIG. 24 , demonstrating that the field of view for the STM of the present disclosure is indeed larger. 
         [0364]    The foregoing description is presented for purposes of illustration and description of the various aspects of the invention. One of ordinary skill in the art will recognize that additional embodiments of the invention are possible without departing from the teachings herein. This detailed description, and particularly the specific details of the exemplary embodiments, is given primarily for clarity of understanding, and no unnecessary limitations are to be imported, for modifications will become obvious to those skilled in the art upon reading this disclosure and may be made without departing from the spirit or scope of the invention. Relatively apparent modifications, of course, include combining the various features of one or more figures with the features of one or more of other figures. All such modifications and variations are within the scope of the to invention as determined by the appended claims when interpreted in accordance with the breadth to which they are fairly, legally and equitably entitled. 
       APPENDIX 1 
       [0365]      
         [0000]    
       
         
               
             
               
               
             
               
               
               
             
               
               
               
             
               
               
             
           
               
                   
               
             
             
               
                 [Computing Corresponding Point P r ′] 
               
               
                 [Initialization] 
               
               
                 σ r  := (h − X)cosθ; 
               
             
          
           
               
                   
                 ρ 1r  := −X cos(2θ) + (d + l)sin(2θ); 
               
               
                   
                 ρ 2r  := −X sin(2θ) − (d + l)cos(2θ); 
               
               
                   
                 C := (0,0,d); /* Location of the camera */ 
               
               
                   
                 P 1  = (X,Y,−l); /* Corresponding point of A in virtual central view */ 
               
               
                   
                 t := 1; /* Start point of the digitization parameter of P 1 P 2  */ 
               
               
                   
                 Δ t  := δ/(d + l); /* Step size for digitization parameter of P 1 P 2  */ 
               
               
                   
                 P := P 1 ; /* Start point of the digitization */ 
               
               
                   
                 Δ P  := Δ t  (P 1  − C); /* Step size for digitization of P 1 P 2  */ 
               
               
                   
                 t min  := 1; /* Initialize the parameter for P r ′ */ 
               
               
                   
                 P min  := P 1 ; /* Initialize the location of P r ′ */ 
               
               
                   
                 A := (X + 2σ r  cosθ)(d + l); /* Numerator of x-component of P 1 ′ */ 
               
               
                   
                 B := d + l − 2σ r  sinθ; /* Denominator of x-component of P 1 ′ */ 
               
             
          
           
               
                   
                 E := (d + l)Y; 
                 /* Numerator of y-component of P 1 ′ */ 
               
             
          
           
               
                   
                 Δ := Δ t ρ 1r  (d + l); 
                 /* Step size for updating A */ 
               
               
                   
                 Δ B  := Δ t ρ 2r ; 
                 /* Step size for updating B */ 
               
               
                   
                 Δ E  := ΔE; 
                 /* Step size for updating E */ 
               
             
          
           
               
                   
                 ε t  := 1; /* Initialize the error tolerance for intensity comparison */ 
               
               
                   
                   
               
             
          
         
       
     
         [0000]    
       
         
               
             
               
               
             
               
               
             
               
               
             
           
               
                   
               
             
             
               
                 [Corresponding Point Identification] 
               
             
          
           
               
                   
                 for i := 1 to N − 1 do { 
               
             
          
           
               
                   
                 t := t + Δ t ; /* Update the digitization parameter */ 
               
               
                   
                 P := P + Δ P ; /* Update current location of digitization process */ 
               
               
                   
                 A := A + Δ A ; /* Update numerator of x-component of P′ */ 
               
               
                   
                 B := B − Δ B ; /* Update denominator of x-component of P′ */ 
               
               
                   
                 E := E + Δ E ; /* Update numerator of y-component of P′ */ 
               
               
                   
                   X  := A/B; /* Compute x-component of P′ */ 
               
               
                   
                   Y  := E/B; /* Compute y-component of P′ */ 
               
               
                   
                 α := [m  X  − (m + 1)h]/(2h); 
               
               
                   
                 β := [n  Y  + (n − 1)h]/(2h); 
               
               
                   
                 /* (α,β) is the counterpart of P′ in the STM image&#39;s right 
               
               
                   
                 view */ 
               
               
                   
                 ε := Match(x,y,α,β); 
               
               
                   
                 if (ε &lt; ε r ) then {ε r  := ε; t min  := t; P min  := P;} 
               
               
                   
                 /* this step keeps track of the digitized element whose projection 
               
               
                   
                 in the */ 
               
             
          
           
               
                   
                 /* virtual right view has the smallest difference on intensity value 
               
               
                   
                 with */ 
               
               
                   
                 /* that of A = (x,y) in the central view */ 
               
               
                   
                 } 
               
               
                   
                 Depth_of_A := (P min ) z ; 
               
               
                   
                   
               
             
          
         
       
     
         [0366]    In the above code, Match(x,y,α,β) is a function that will compare intensities of a patch centered at (x,y) in the central view with the intensities of a same dimension patch centered at (α,β) in the right view of the STM image. match( ) can use one of the techniques described in herein or a technique of its own. This function returns a positive real number as the difference of the intensity values. 
         [0367]    Note also that the parameters α and β are real numbers, not integers. When computing the intensity at (α,β), one shouldn&#39;t simply round α and β to the nearest integers, but use the following formula instead to get a more appropriate approximation: 
         [0000]        I (α,β)= C   i,j   I ( i,j )+ C   i+1,j   I ( i+ 1, j )+ C   i,j+1   I ( i+ 1, j )+ C   i+1 j+1   I ( i+ 1, j+ 1) 
         [0368]    where i≦α&lt;i+1, j≦β&lt;j+1 and C i,j , C i+1,j , C i,j+1  and C i+1,j+1  are real number coefficients defined as follows: 
         [0000]        C   i,j =( i+ 1−α)( j+ 1−β); 
         [0000]        C   i+1,j =(α− i )( j+ 1−β); 
         [0000]        C   i,j+1 =( i+ 1−α)(β− j ); 
         [0000]        C   i+1,j+1 =(α− i )(β− j ). 
       APPENDIX 2 
       [0369]    One can easily extend the software code of Appendix 1 to compute P r ′, P l ′, P t ′ and P b ′ at the same time. The code is shown below. 
         [0000]    
       
         
               
             
               
               
             
               
               
               
             
               
               
             
               
               
             
               
               
             
               
             
               
               
             
               
             
               
               
             
               
             
               
               
             
               
               
               
             
               
             
               
               
             
               
             
               
               
             
               
               
               
             
               
             
               
               
             
               
               
               
             
               
               
             
               
               
               
             
               
               
               
               
               
             
               
               
               
             
               
               
             
           
               
                   
               
             
             
               
                 [Computing Corresponding Points P r ′, P l ′, P t ′ and P b ′] 
               
               
                 [Initialization] 
               
               
                 σ r  := (h − X)cosθ; σ l  := (h + X)cosθ; 
               
               
                 σ t  := (h − Y)cosθ; σ b  := (h + Y)cosθ; 
               
             
          
           
               
                   
                 ρ 1r  := −X cos(2θ) + (d + l)sin(2θ); 
               
               
                   
                 ρ 2r  := −X sin(2θ) − (d + l)cos(2θ): 
               
               
                   
                 ρ 1l  := −X cos(2θ) − (d + l)sin(2θ); 
               
               
                   
                 ρ 2l  := X sin(2θ) − (d + l)cos(2θ); 
               
               
                   
                 ρ 1t  := −Y cos(2θ) + (d + l)sin(2θ); 
               
               
                   
                 ρ 2t  −Y sin(2θ) − (d + l)cos(2θ); 
               
               
                   
                 ρ 1b  := −Y cos(2θ) − (d + l)sin(2θ); 
               
               
                   
                 ρ 2b  := Y sin(2θ) − (d + l)cos(2θ); 
               
               
                   
                 C := (0,0,d); /* Location of the camera */ 
               
               
                   
                 P 1  = (X,Y,−l); /* Corresponding point of A in virtual central view */ 
               
             
          
           
               
                   
                 t := 1; 
                 /* Start point of the digitization parameter of P 1 P 2  */ 
               
             
          
           
               
                   
                 Δ 1  := δ/(d + l); /* Step size for digitization parameter of P 1 P 2  */ 
               
               
                   
                 P := P 1 ; /* Start point of the digitization */ 
               
               
                   
                 Δ P  := Δ t (P 1  − C); /* Step size for digitization of P 1 P 2  */ 
               
               
                   
                 t min.r  := 1; P min.r  := P 1 ; t min.l  := 1; P min.l  := P 1 ; 
               
               
                   
                 /* Initialize the tracking parameters and pointers for P r ′ and P l ′ */ 
               
               
                   
                 t min.t  := 1; P min.t  := P 1 ; t min.b  := 1; P min.b  := P 1 ; 
               
             
          
           
               
                   
                 /* Initialize the tracking parameters and pointers for P t ′ and P b ′*/ 
               
             
          
           
               
                   
                 t min.rl  := 1; P min.rl  := P 1 ; t min.tb  := 1; P min.tb  := P 1 ; 
               
               
                   
                 /* Initialize the tracking parameters and pointers for cross-matching on */ 
               
             
          
           
               
                 /* P r ′ and P l ′, and on P t ′ and P b ′*/ 
               
             
          
           
               
                   
                 t min.rt  := 1; P min.rt  := P 1 ; t min.lb  := 1; P min.lb  := P 1 ; 
               
               
                   
                 /* Initialize the tracking parameters and pointers for cross-matching on */ 
               
             
          
           
               
                 /* P r ′ and P t ′, and on P l ′ and P b ′*/ 
               
             
          
           
               
                   
                 t min.lt  := 1; P min.lt  := P 1 ; t min.rb  := 1; P min.rb  := P 1 ; 
               
               
                   
                 /* Initialize the tracking parameters and pointers for cross-matching on */ 
               
             
          
           
               
                 /* P l ′ and P t ′, and on P r ′ and P b ′*/ 
               
               
                 AR := (X + 2σ r  cosθ)(d + l); /* Numerator of x-component of P 1 ′ in RV*/ 
               
             
          
           
               
                   
                 BR := d + l − 2σ r  sinθ; /* Denominator of x-component of P 1 ′ in RV*/ 
               
             
          
           
               
                   
                 ER := (d + l)Y; 
                 /* Numerator of y-component of P 1 ′ in RV*/ 
               
             
          
           
               
                 AL := (X − 2σ l  cosθ)(d + l); /* Numerator of x-component of P′ 1  in LV*/ 
               
             
          
           
               
                   
                 BL := d + l − 2σ sinθ; /* Denominator of x-component of P 1 ′ in LV*/ 
               
             
          
           
               
                 AT := (Y + 2σ t  cosθ)(d + l); /* Numerator of y-component of P 1 ′ in UV*/ 
               
             
          
           
               
                   
                 BT := d + l − 2σ t  sinθ; /* Denominator of y-component of P 1 ′ in UV*/ 
               
             
          
           
               
                   
                 ET := (d + l)X; 
                 /* Numerator of x-component of P 1 ′ in UV*/ 
               
             
          
           
               
                 AB := (Y − 2σ b  cosθ)(d + l); /* Numerator of x-component of P 1 ′ in BV*/ 
               
             
          
           
               
                   
                 BB := d + l − 2σ b  sinθ; /* Denominator of x-component of P 1 ′ in BV*/ 
               
             
          
           
               
                   
                 Δ AR  := Δ t (d + l)ρ 1r ; 
                 /* Step size for updating AR*/ 
               
               
                   
                 Δ BR  := Δ t ρ 2t ; 
                 /* Step size for updating BR */ 
               
               
                   
                 Δ ER  := Δ t (d + l)Y; 
                 /* Step size for updating ER */ 
               
               
                   
                 Δ AL  := Δ t (d + l)ρ 1l ; 
                 /* Step size for updating AL*/ 
               
               
                   
                 Δ BL  := Δ t ρ 2l ; 
                 /* Step size for updating BL */ 
               
               
                   
                 Δ AT  := Δ t (d + l)ρ lt ; 
                 /* Step size for updating AT*/ 
               
               
                   
                 Δ BT . := Δρ 2t ; 
                 /* Step size for updating BT */ 
               
               
                   
                 Δ ET  := Δ t (d + l)X; 
                 /* Step size for updating ET */ 
               
             
          
           
               
                 Δ AB  := Δ t (d + l}ρ 1b ; 
                 /* Step size for updating AB*/ 
               
             
          
           
               
                   
                 Δ BB  := Δ t ρ 2b ; 
                 /* Step size for updating BB */ 
               
             
          
           
               
                   
                 ε r  := 1; 
                 ε l  := 1; 
                 ε t  := 1; 
                 ε b  := 1; 
               
               
                   
                 ε rl  := 1 
                 ε tb  := 1; 
                 ε rt  := 1; 
                 ε rb  := 1; 
               
             
          
           
               
                   
                 ε lt  := 1; 
                 ε lb  := 1; 
               
             
          
           
               
                   
                 /* Initialize the error tolerances for intensity comparison */ 
               
               
                   
                   
               
             
          
         
       
     
         [0000]    
       
         
               
             
               
               
             
               
               
             
               
               
               
             
               
             
               
               
             
               
             
               
               
             
               
             
               
               
             
               
             
               
               
             
               
             
               
               
             
               
             
               
               
             
               
             
               
               
               
             
               
               
             
               
               
               
             
               
               
             
               
               
               
             
               
               
             
               
               
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
               
             
               
               
             
               
               
             
           
               
                   
               
             
             
               
                 [Corresponding Point Identification] 
               
             
          
           
               
                   
                 for i := 1 to N − 1 do { 
               
             
          
           
               
                   
                 t := t + Δ t ; /* Update the digitization parameter */ 
               
               
                   
                 P := P + Δ P ; /* Update corrent location of digitization process */ 
               
             
          
           
               
                   
                 AR := AR + Δ AR ; 
                 AL := AL + Δ AL ; 
               
             
          
           
               
                 /* Update numerator of x-component of P′ in RV and LV*/ 
               
             
          
           
               
                 BR := BR + Δ BR ; 
                 BL := BL + Δ BL ; 
               
             
          
           
               
                 /* Update denominator of x-component of P′ in RV and LV*/ 
               
             
          
           
               
                   
                 ER := ER + Δ ER ; 
               
             
          
           
               
                 /* Update numerator of y-component of P′ in RV and LV */ 
               
             
          
           
               
                 AT := AT + Δ AT ; 
                 AB := AB + Δ AB ; 
               
             
          
           
               
                 /* Update numerator of y-component of P′ in TV and BV*/ 
               
             
          
           
               
                 BT := BT + Δ BT ; 
                 BB := BB + Δ BB ; 
               
             
          
           
               
                 ;/* Update denominator of y-component of P′ in TV and BV*/ 
               
             
          
           
               
                   
                 ET := ET + Δ ET ; 
               
             
          
           
               
                 /* Update numerator of x-component of P′ in TV and BV */ 
               
             
          
           
               
                   
                   X   R  := AR/BR;  Y   R  := ER/BR;  X   L  := AL/BL;  Y   L  := ER/BL; 
               
             
          
           
               
                 /* Compute x- and y-components of P′ in RV and LV*/ 
               
               
                   X   T  := ET/BT;  Y   T  := AT/BT;  X   B  := ET/BB;  Y   B  := AB/BB; 
               
               
                 /* Compute x- and y-components of P′ in TV and BV*/ 
               
             
          
           
               
                   
                 α R  := [m  X   R  − (m + 1)h]/(2h); 
                 β R  := [n  Y   R  + (n − 1)h]/(2h); 
               
             
          
           
               
                   
                 /* (α R ,β R ) is the counterpart of P′ in the STM image&#39;s right view */ 
               
             
          
           
               
                   
                 α L  := [m  X   L  + (m + 1)h]/(2h); 
                 β L  := [n  Y   L  + (n − 1)h]/(2h); 
               
             
          
           
               
                   
                 /* (α L ,β L ) is the counterpart of P′ in the STM image&#39;s left view */ 
               
             
          
           
               
                   
                 α T  := [m  X   T  + (m − 1)h]/(2h); 
                 β T  := [n  Y   T  − (n + 1)h]/(2h); 
               
             
          
           
               
                   
                 /* (α T ,β T ) is the counterpart of P′ in the STM image&#39;s upper view */ 
               
             
          
           
               
                   
                 α B  := [m  X   B  + (m − 1)h]/(2h); 
                 β B  := [n  Y   B  + (n + 1)h]/(2h); 
               
             
          
           
               
                   
                 /* (α B ,β B ) is the counterpart of P′ in the STM image&#39;s lower view */ 
               
               
                   
                 ε := Match(x,y,α R ,β R ); 
               
               
                   
                 if (ε &lt; ε r ) then {ε r  := ε; t min.r  := t; P min.r  := P;} 
               
               
                   
                 /* this step keeps track of the digitized element whose projection in the */ 
               
             
          
           
               
                   
                 /* virtual right view has the smallest difference on intensity value with */ 
               
               
                   
                 /* A = (x,y) in the central view */ 
               
             
          
           
               
                 ε := Match(x,y,α L ,β L ); 
               
             
          
           
               
                   
                 if (ε &lt; ε l  ) then {ε l  := ε; t min.l  := t; P min.l  := P;} 
               
               
                   
                 /* this step keeps track of the digitized element whose projection in the */ 
               
             
          
           
               
                   
                 /* virtual left view has the smallest difference on intensity value with */ 
               
               
                   
                 /* A = (x,y) in the central view */ 
               
             
          
           
               
                 ε := Match(x,y,α T ,β T ); 
               
             
          
           
               
                   
                 if (ε &lt; ε t ) then {ε t  := ε; t min.t  := t; P min.t  := P;} 
               
               
                   
                 /* this step keeps track of the digitized element whose projection in the */ 
               
             
          
           
               
                   
                 /* virtual upper view has the smallest difference on intensity value with */ 
               
               
                   
                 /* A = (x,y) in the central view */ 
               
             
          
           
               
                 ε := Match(x,y,α B ,β B ); 
               
             
          
           
               
                   
                 if (ε &lt; ε b  ) then { ε b  := ε b ; t min.b  := t; P min.b  := P; } 
               
               
                   
                 /* this step keeps track of the digitized element whose projection in the */ 
               
             
          
           
               
                   
                 /* virtual lower view has the smallest difference on intensity value with */ 
               
               
                   
                 /* A = (x,y) in the central view */ 
               
               
                   
                 } 
               
               
                   
                 Depth_of_A := (P min ) z ;