Abstract:
A method for characterizing a geometric element of an object includes positioning a calibrated target adjacent a calibration geometric element, the calibrated target having at least two differentially detectable features having a known geometric relationship to each other. Next a relationship of the calibrated target to the calibration geometric element is determined, and the calibrated target is moved adjacent a geometric feature of an object desired to be characterized. Then photogrammetry is applied to the calibrated target features and the desired geometric feature to spatially characterize the desired geometric feature. A system for characterizing a geometric element of an object includes the movable calibrated target and a photogrammetric analysis system for determining a relationship of the calibrated target to a calibration geometric element and for spatially characterizing the desired geometric feature using the calibrated target features.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
   This application claims priority from provisional application No. 60/221,702, filed Jul. 31, 2000, entitled “Photogrammetric Measurement System and Method.” 

   BACKGROUND OF THE INVENTION 
   1. Field of the Invention 
   The present invention relates to systems and methods for remote measurement of geometric features on an object, and, more particularly, to such systems and methods that are photogrammetrically based. 
   2. Description of Related Art 
   Photogrammetry is a fast, accurate three-dimensional measurement method based on photographic principles. In a single-camera or sequential mode of photogrammetry, a single high-resolution digital camera takes multiple pictures of an object from different locations. These pictures are then automatically processed to yield three-dimensional coordinates of points on an object (Ganci and Handley, 1998). 
   The sequential mode is not real time; so it can only measure static objects and targeted points. However, it is well suited to high-accuracy measurement of large, complicated objects, since virtually any number of pictures can be taken and processed. In addition, the photography is usually quick; so production downtime is low, and temperature effects are minimized. The use of the sequential method of photogrammetry on a typical measurement is illustrated in  FIG. 1 , wherein the user  90  is shown taking pictures of an object  91  from three locations  71 , 72 , 73  sequentially. 
   Sequential photogrammetry has other attractive attributes, including high accuracy, portability, and the ability to perform measurements in unstable or vibrating environments (Brown, 1998). All these attributes combined with the present invention are believed to make sequential photogrammetry a compelling choice for numerous applications in the automotive industry. 
   Three-dimensional inspection within an automotive environment has been known to be conducted within a coordinate measurement machine (CMM) room. The CMM room typically has several stationary gantry-type CMMs dedicated to a certain class of dimensional inspection. This class of inspection is most notably characterized by the need to bring the part to be measured to the CMM. Such a constraint instantly disqualifies traditional CMMs from inspecting the tooling fixtures that are located on the assembly line. These fixtures are in fact the most important items measured in the factory, as they control the dimensions and fit-up of the subassemblies throughout the manufacturing process. In-line production measurements are also impossible using CMM systems. 
   It would thus be desirable to have a portable system that can measure items in place for a number of applications. Although numerous portable, in-place measurement systems exist, they typically rely on touching the desired features with some type of probe or measuring adapter that is usually held in place by an operator. For many applications, access to the object is limited, making setup of the instrument difficult. Furthermore, touching the desired features with the measuring device can be difficult, tedious, and prone to lead to measurement errors. 
   For many automotive inspection tasks it is necessary quickly to collect and process feature data. One limiting factor in the use of photogrammetry for feature measurement in the past has been the need for point-of-interest targeting. For example, if a plane is desired to be measured, then at least three points (&gt;3 for redundancy) need to be applied to the surface that defines that plane. Similar requirements apply to other geometric features, such as circles and lines. In some instances, point-of-interest targeting cannot be used to define the feature. 
   An edge is one example of a feature that cannot be directly targeted. Typically, a special target adapter is needed to define the edge  92 , or the edge is determined indirectly via the intersection of two planes  93 , 94  ( FIG. 2 ). The example of an edge can be extended to a corner via the addition of another plane and a line—line intersection to produce the corner. Traditionally, target adapters have been used to bypass some of the problems associated with measuring difficult features such as edges and corners. The time penalty for these adapters comes at the processing stage, when they need to be identified and reduced to yield the desired feature. 
   An alternative to stick-on targeting when using photogrammetry is the use of hand-held probes to touch points of interest (Ganci and Brown, 2000). Three exemplary probes  95 – 97  ( FIGS. 3A–3C ) each have a standard tip  98 , similar to that used on conventional CMMs. In addition, each probe  95 – 97  has five permanent targets  99 . In use, two or more cameras simultaneously view these targets  99  and calculate their xyz position. From the xyz position of the five targets  99 , the probe tip  98  position can be calculated. 
   Probes have been successfully used in automotive applications for many years. However, in some applications setting up the cameras for measuring the object is difficult. Furthermore, many features are difficult to probe accurately, especially in the tight spaces and difficult conditions encountered in many measurements. 
   Sequential photogrammetry can often acquire data rapidly since photography is very quick. However, sometimes the need to target the desired features is difficult and time consuming. If, instead, a probing tool is used, there is typically a setup time for the instrument followed by laborious probing and analysis of each feature. The present invention is directed to reducing setup time and also to removing the requirement for probing features. 
   SUMMARY OF THE INVENTION 
   It is therefore an object of the present invention to provide a system and method for measuring geometric features of an object. 
   It is an additional object to provide such a system and method particularly adapted for difficult-to-measure features. 
   It is a further object to provide such a system and method that use a targeted adaptor. 
   It is another object to provide such a system and method that incorporate digital photogrammetry. 
   It is yet an additional object to provide such a system and method that are adapted for use in the automotive industry. 
   These objects and others are attained by the present invention, an embodiment of which is a method for characterizing a geometric element of an object. The method comprises the step of positioning a calibrated target adjacent a calibration geometric element. The calibrated target comprises at least two differentially detectable features having a known geometric relationship to each other. 
   Next a relationship of the calibrated target to the calibration geometric element is determined, and the calibrated target is moved adjacent a geometric feature of an object desired to be characterized. Then photogrammetry is applied to the calibrated target features and the desired geometric feature to spatially characterize the desired geometric feature. 
   Another embodiment of the invention is a system for characterizing a geometric element of an object. The system comprises a movable calibrated target comprising at least two differentially detectable features having a known geometric relationship to each other. The system also comprises a photogrammetric analysis system for determining a relationship of the calibrated target to a calibration geometric element. The analysis system is also for spatially characterizing a geometric feature of an object desired to be characterized using the calibrated target features. 
   The features that characterize the invention, both as to organization and method of operation, together with further objects and advantages thereof, will be better understood from the following description used in conjunction with the accompanying drawing. It is to be expressly understood that the drawing is for the purpose of illustration and description and is not intended as a definition of the limits of the invention. These and other objects attained, and advantages offered, by the present invention will become more fully apparent as the description that now follows is read in conjunction with the accompanying drawing. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  (prior art) illustrates the process of sequential photogrammetry. 
       FIG. 2  (prior art) shows the use of a plane—plane intersection to derive the location of an edge. 
       FIGS. 3A–3C  (prior art) illustrate three exemplary configurations of hand-held probes. 
       FIGS. 4A–4C  illustrate a plane target&#39;s top view ( FIG. 4A ), rear view ( FIG. 4B ), and side view ( FIG. 4C ). 
       FIG. 5  illustrates an exemplary calibration setup. 
       FIG. 6  illustrates a calibration file for the setup of  FIG. 5 . 
       FIGS. 7A–7L  illustrate four types of feature targets (Table 1): plane ( FIGS. 7A–7C ); edge ( FIGS. 7D–7F ); corner ( FIGS. 7G–7I ); and circle ( FIGS. 7J–7L ). 
       FIGS. 8A–8H  illustrate sample FT solutions: for measuring a plane (FIGS.  8 A, 8 B), an edge ( FIG. 8C ), a corner ( FIGS. 8D , alone, and  8 E, 8 F, together), a complex corner ( FIG. 8G ), and a centerline ( FIG. 8H ). 
     FIGS.  9 A, 9 B illustrate two case studies (Table 3):  FIG. 9A  illustrates fixture measurement;  FIG. 9B , an in-line car body inspection. 
       FIG. 10  is a side perspective view of a sample clamping mechanism. 
       FIG. 11  illustrates a camera station network for the system of  FIG. 10 . 
     FIGS.  12 A, 12 B show the measurement of two features on the system of  FIG. 10 . 
       FIG. 13  illustrates a test unit for measuring features of an axle carrier on an automobile. 
       FIG. 14  shows the camera station network for  FIG. 13 . 
       FIG. 15A  shows key measurement features of the system of  FIG. 13 . 
       FIG. 15B  shows geometric reductions for the features of  FIG. 12A . 
   

   DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
   A description of the preferred embodiments of the present invention will now be presented with reference to  FIGS. 4A–15B . 
   Feature targets (FTs) are identified with the use of a “code target,” which comprises a unique, calibratable pattern of geometric shapes, such as, but not intended to be limited to, squares, and a substantially central dot. The important element in the shapes is that there be at least two different shapes, preferably a unitary shape comprising a first shape and a plurality of shapes comprising a second shape. 
   Means are provided for automatically detecting the patterns and for identifying and measuring them by image processing techniques (Fraser  1997   a,b ). 
   The addition of such a target to an adaptor makes it possible to identify that adaptor in a measurement. There are advantages to automating the use and reduction of these adaptors. Once an FT is identified, a calibration file associated therewith is used to determine the type and function of that particular target adaptor. 
   A first embodiment of the FT  10  is for measuring a plane. This FT ( FIGS. 4A–4C ) comprises a planar base  11 , comprising, for example, a flat piece of aluminum. The top side  121  of the base  11  has a plurality of coded targets  12 , 12 ′ thereon ( FIG. 4A ); the bottom side  122  of the base  11  has at least one magnet  13  embedded therein to hold the FT  10  in place. This positioning device is exemplary and is not intended to be limiting, and it will be understood by one of skill in the art that alternate affixing methods are able to be substituted therefor, such as an adhesive for affixing to nonmetallic portions of objects. The relationship between the targets  12 , 12 ′ on the top side  121  of the base  11  and the plane represented by the base  11  is determined by a one-time calibration. 
   The FT  10  is placed on a flat surface  89  such as granite ( FIG. 5 ). Retroreflective targets  14  are placed adjacent the FT  10  and are used to define a “local” plane  15 . A photogrammetric measurement is then made to determine the location of the face targets  12  relative to the local plane  15 , which thus serves as a calibration geometric element for a planar surface. 
   With the relationship between the targets  12 , 14  and the local plane  15  known, the coded target and its adaptor form a feature target  10  capable of directly measuring any plane to which it is attached. The relationship between the targets  12  and the plane  15  is shown in  FIG. 6 , as measured by a camera at station  24 . 
   In an alternate embodiment, if a right-angle bracket is used instead of a flat plate, the resulting FT  10  after calibration can measure a plane at right angles to the face targets. Multiple FTs can be placed on a block, and the feature planes they measure intersected to form a line  92  as shown in  FIG. 2 . 
   The face targets  12  can be in any orientation to the plane that needs to be defined. For example, it might be desired to have the FT at a 45° angle to the plane. In fact, the targets  12  can be in any relative orientation providing that they can be calibrated and are thus predetermined at the time of the desired characterization. 
   At run time, a local transformation is performed for each of the FTs found. This transformation uses the calibrated and measured values of the face targets  12  to transfer the plane, line, or feature desired into the global coordinate system of the measured object. Photogrammetric software  51  is known in the art that can be resident on a processor  50  for performing the requisite calculations, the processor  50  in electronic communication with the camera stations  24 . 
   Four exemplary types of FTs are given in Table 1 and  FIGS. 7A–7K . Exemplary advantageous features of FTs are given in Table 2. In  FIGS. 7A–7C  planar FTs  10   a , 10   b , 10   c  are used to define contact planes  123   a , 123   b , 123   c , with the bases  11   a , 11   b , 11   c  shown at 0°, 90°, and 45°, respectively. 
   Edge definitions are provided ( FIGS. 7D–7F ) by using FTs  10   d , 10   e , 10   f  to define two contact planes  123   d , 123   d ′; 123   e , 123   e ′; 123   f , 123   f ′ each, and then deriving their intersection, which comprises the desired edge  124   a , 124   b , 124   c . FT  10   d  is a mirror image of FT  10   e , both having angles of substantially 90° between the contact planes. FT  10   f  is a tilted view of a target having an angle between the planes less than 90°. 
   Corner definitions are provided ( FIGS. 7G–7I ) with three contact planes  123   g , 123   g ′, 123   g ″ to define two edges  123   h , 123   h ′, from which in turn are derived the corresponding corner  123   i.    
   A center point  128  of a circle, or of a cylindrical object  125 , for example, may be defined with the use of an FT  10   j  having two planes  123   j , 123   j ′ at substantially right angles (FIGS.  7 J, 7 K) or an FT  10   k  having three planes  123   k , 123   k ′, 123   k ″ at substantially right angles to each other ( FIG. 7L ). In both cases the cylindrical object  125  is surrounded by two of the planes, creating thereby two contact points  126 , 126 ′; 127 , 127 ′ at tangent points. From the positions of these contact points  126 , 126 ′; 127 , 127 ′, as determined photogrammetrically, the radius and thereby the center point  128  may be derived. 
   
     
       
             
           
             
             
             
           
         
             
               TABLE 1 
             
           
           
             
                 
             
             
               Exemplary Types of Feature Targets 
             
           
        
         
             
               Type 
               FIG. 
               Function 
             
             
                 
             
             
               Plane 
               FIGS. 7A–7C 
               contact plane defined 
             
             
               Edge 
               FIGS. 7D–7F 
               two contact planes and their intersection are 
             
             
                 
                 
               defined 
             
             
               Corner 
               FIGS. 7G–7I 
               three contact planes, two edges, and the 
             
             
                 
                 
               corresponding corner are defined 
             
             
               Circle 
               FIGS. 7J–7L 
               the center point of a circle with known radius is 
             
             
                 
                 
               defined 
             
             
                 
             
           
        
       
     
   
   
     
       
             
           
             
             
           
         
             
               TABLE 2 
             
             
                 
             
             
               Feature Target Features 
             
             
                 
             
           
           
             
                 
             
           
        
         
             
               Fast 
               Measurement with FTs is much faster than with other 
             
             
                 
               methods. The setup time for sequential photogrammetry is 
             
             
                 
               very low compared with other in-place measurement 
             
             
                 
               systems. Using FTs to measure features is faster and easier 
             
             
                 
               in many applications than probing the feature. Photography 
             
             
                 
               is usually very quick. Finally, analysis is completely 
             
             
                 
               automated so that it is both fast and error-free. 
             
             
               Higher 
               The accuracy obtained is higher than normal probed data 
             
             
               accuracy 
               because the FT is measured from multiple locations, and 
             
             
                 
               many points are used to define the feature. 
             
             
               Line of Sight 
               FTs can be configured and used to eliminate many line-of- 
             
             
                 
               sight problems. 
             
             
               Versatile 
               The method is extremely versatile in both its use and 
             
             
                 
               application. FTs can be configured to meet many measuring 
             
             
                 
               requirements. 
             
             
               Minimal 
               The fast overall measurement and analysis time means 
             
             
               downtime 
               critical production downtime is minimized. 
             
             
               Enhanced 
               FTs can measure features that cannot be directly targeted. 
             
             
               targeting 
             
             
               Repeat 
               The ability to automate measurements through construction 
             
             
               measure- 
               macros (Ganci and Handley, 1998) makes FTs ideal for 
             
             
               ment 
               repeat production measurements. 
             
             
                 
             
           
        
       
     
   
   As an example of a use of the system and method of the present invention, FTs can be used to measure NC blocks on automotive fixtures. A target is placed on each block or surface desired to be measured. The target is preferably selected depending upon the data desired and the viewing angles available to the block. For example, for blocks having nonperpendicular faces, it is customary to use a combination of FTs. For pin or circle measurements it is necessary to specify the correct radius at the calibration phase, which may be obtained, for example, from the design data. 
   Sample FT solutions are given in  FIGS. 8A–8H . As indicated above, a plane  40  may be measured with one FT  10  positioned on the plane  40 , so long as the FT  10  has been properly calibrated (FIGS.  8 A, 8 B); an edge  41  may be measured with an FT  10   g  having a first face  410  of the base  11   g  at right angles to a second face  411  of the base  11   g  ( FIG. 8C ); a corner  42  may be measured in one step with two FTs  10   h  ( FIG. 8D ) or in two steps with a single FT  10   i  ( FIGS. 8E and 8F ); a complex corner  43  may be measured in one step with three FTs  10   j  ( FIG. 8G ); and a centerline  44  of a cylindrical object  45 , or a cylindrical portion of an object, may be measured with in one step with two FTs  10   k  ( FIG. 8H ). 
   Two exemplary test case studies are illustrated in Table 3 and  FIGS. 9A and 9B . For these two studies the FT concept of the present invention was believed preferable to other known prior art measurements. In  FIG. 9A , a plurality of FTs  10  are positioned around the components of the device  46 , permitting complex feature analysis without a hand-held probe. In  FIG. 9B , an axle carrier  47  also has a plurality of FTs  10  positioned thereon. 
   
     
       
             
           
             
             
             
           
         
             
               TABLE 3 
             
           
           
             
                 
             
             
               Overview of Two Case Studies 
             
           
        
         
             
                 
               Object, 
                 
             
             
               Case study 
               FIG. 
               Objectives 
             
             
                 
             
             
               Case 1: Fixture 
               FIG. 9A 
               1. Use FTs to inspect the fixture 
             
             
               measurement 
                 
               2. Complete the inspection in less time 
             
             
                 
                 
               than the current inspection system 
             
             
               Case 2: In-line car 
               FIG. 9B 
               1. Use FTs to inspect the underbody of a 
             
             
               body inspection 
                 
               vehicle during assembly (in-line) 
             
             
                 
                 
               2. Complete the measurement in less than 
             
             
                 
                 
               the production time of 5 min 
             
             
                 
             
           
        
       
     
   
   In Case 1 ( FIG. 9A ), analysis of production tooling requires a fast and efficient measurement system. The driving factors are time and availability for analysis. Case 2 ( FIG. 9B ) illustrates the need for immediate information for decision making during production. In both cases the need for uninterrupted production schedules is deemed extremely important. 
   In the case of production tooling measurement, unavailability of tooling during after-shift hours due to routine maintenance needs, and constantly changing production schedules due to high product demand, has forced inspection personnel into working during scheduled break times. This means finding a way to measure a tool during a normal 40-min lunch break with the goal of eventually completing the measurement during a 10-min coffee break. 
   The ability to flexibly analyze product deviation in-line is the focus of the second case study. Increasingly, inspection is needed at the point of origin in order to efficiently determine the cause of the deviation and set corrective action. Typical problems facing dimensional inspection include flexibility and time. The system needs to be set up quickly and measure in unstable environments, such as constantly moving assembly lines. It also needs to be able to complete the measurement between assembly processes. 
   Case 1. In this study a 1.8-m-long panel-holding fixture ( FIG. 9A ) is desired to be measured, to determine the location of features such as corners, edges, and planes on the fixture. 
   An FT  10  was placed on each of the desired features. Where possible, one was used. For features without a suitable target, the necessary data were created using a combination of FTs. 
   An exemplary sample clamping mechanism is illustrated in  FIG. 10 , wherein four FTs  20 – 23  are shown. The FT  20  at the back of the clamp  88  is used to define a contact plane. The three remaining FTs  21 – 23  define the hard corner of the clamping surface. The fixture measurement required a total of 40 FTs to measure all the desired features. 
   After targeting a total of 60 photographs were taken of the fixture  88 . The number of photographs taken depends on the complexity of the measurement and accuracy requirements. The photography for the fixture  89  was completed in approximately 5 min. Camera station locations  24  for the measurement are shown in  FIG. 11 . 
   Statistics from the measurement of the fixture  89  are as follows: 
                                                   No. of photos   60               No. of FTs   40           No. of scales   2           Scale agreement   0.01   mm           Accuracy rms (mm) xyz   x   0.010               y   0.009               z   0.008                        
Some of the features measured on the fixture  89  are shown in  FIGS. 12A and 12B , along with the corresponding FT analyses  87 , 87 ′.
 
   To obtain a better idea of the time savings of the FTs of the present invention, the same measurement was completed using conventional stick-on targeting and a multicamera probe system. The probes were needed to collect data on the features that could not be targeted. A comparison of the two measurements is given in Table 4, wherein it is clear that the FT system is much faster. The automated analysis of the FTs especially saves time while also eliminating measurement errors. 
   
     
       
             
           
             
             
             
             
             
             
           
             
             
             
             
             
             
           
         
             
               TABLE 4 
             
           
           
             
                 
             
             
               Comparison of the Two Measurements 
             
           
        
         
             
                 
                 
               FT 
                 
               Stick-on 
                 
             
             
                 
               Measurement type 
               solution 
                 
               and probes 
                 
             
             
                 
                 
             
           
        
         
             
                 
               Number of targets 
               40 
               FTs 
               320 
                 
             
             
                 
               Targeting 
               5 
               min 
               25 
               min 
             
             
                 
               Probing 
                 
               — 
               15 
               min 
             
             
                 
               Target removal 
               3 
               min 
               20 
               min 
             
             
                 
               Photography 
               2 
               min 
               2 
               min 
             
             
                 
               Processing 
               2 
               min 
               3 
               min 
             
             
                 
               Analysis 
               3 
               min 
               30 
               min 
             
             
                 
               Total time 
               15 
               min 
               95 
               min 
             
             
                 
                 
             
           
        
       
     
   
   Case 2. This case study is a production measurement, wherein FTs were used to measure a front axle carrier  86  of an automobile, here, a BMW Z3. This test was performed to examine whether in-line measurements could be used to identify cars with bushing angle problems. Rectifying these problems early in the production process would ultimately result in significant scrap value savings farther down the line. The test unit  30  is shown in  FIG. 13 . 
   The measurement was desired to be performed within a time period the car would be idle at a station. A total of 5 min was set aside for targeting, photography, and tear down. Processing the images to yield the desired data was not necessary during the 5-min time limit. 
   Two operators performed the measurement. The targeting and tear down were completed in approximately 2 min. A total of 18 photographs were collected in less than 2 min with the network  31  used is shown in  FIG. 14 . A summary of statistics from the measurement follows: 
   
     
       
             
             
             
             
           
         
             
                 
                 
             
           
           
             
                 
               No. of photos 
               18 
                 
             
             
                 
               No. of FTs 
               8 
             
             
                 
               No. of scales 
               8 
             
             
                 
               Scale agreement 
               0.01 
               mm 
             
             
                 
               Accuracy rms (mm) xyz 
               x 
               0.008 
             
             
                 
                 
               y 
               0.007 
             
             
                 
                 
               z 
               0.017 
             
             
                 
                 
             
           
        
       
     
   
   Once the processing is finished, the desired features are automatically generated. In this case six planes, eight circles, and four lines were created. The feature generation is shown in  FIGS. 15A and 15B , wherein the key measurement features are shown in  FIG. 15A , with plane template  31  and two rod templates  32  attached to the axle carrier  30  and the corresponding geometric reductions  33  in  FIG. 15B . 
   It can be seen that all the objectives of the study were met, illustrating how FTs could be used to automate and significantly reduce the amount of time needed to complete a measurement. 
   REFERENCES 
   
       
       Brown, J., 1998. V-STARS/S Acceptance Results. Boeing Large Scale Optical Metrology Seminar, Seattle. 
       Fraser, C. S., 1997a. Automation in Digital Close Range Photogrammetry. First Trans Tasmin Surveyors Conf., 12–18. 
       Fraser, C. S., 1997b. Innovations in Automation for Vision Metrology Systems.  Photogrammetric Record  15(90): 901–11. 
       Ganci, G., and Brown, J., 2000. Developments in Non-Contact Measurement Using Videogrammetry. Boeing Large Scale Optical Metrology Seminar, Long Beach. 
       Ganci, G., and Handley, H. B., 1998. Automation in Videogrammetry,  Intl. Arch. Photogrammetry and Remote Sensing , Hakodate 32(5): 53–58. 
     
  
   It may be appreciated by one skilled in the art that additional embodiments may be contemplated, including alternate shapes and configurations of the feature target and camera positions. 
   In the foregoing description, certain terms have been used for brevity, clarity, and understanding, but no unnecessary limitations are to be implied therefrom beyond the requirements of the prior art, because such words are used for description purposes herein and are intended to be broadly construed. Moreover, the embodiments of the apparatus illustrated and described herein are by way of example, and the scope of the invention is not limited to the exact details of construction.