Abstract:
A method for determining 3D distances on a 2D pixelized image of a part or object includes acquiring a real 2D pixelized image of the object, creating a simulated image of the object using the 3D CAD model and the 2D pixelized image, determining a specified cost function comparing the simulated image with the real 2D pixilated image and repositioning the simulated image in accordance with iterated adjustments of a relative position between the CAD model and the XID pixilated image to change the simulated image until the specified cost function is below a specified value. Then, the workstation is used to generate a 3D distance scale matrix using the repositioned simulated image, and to measure and display distances between selected pixels on a surface of the real image using 2D distances on the 2D pixelized image of the object and the 3D distance scale matrix.

Description:
BACKGROUND OF THE INVENTION 
       [0001]    The field of the invention relates generally to methods and apparatus for measuring 3 dimensions (3D) on  2  dimensional (2D) images, and more particularly to methods and apparatus to conduct adaptive inspection for obtaining data that may include information in external and/or internal dimensions, surface condition information, and/or internal defect information. 
         [0002]    Many components must be manufactured or maintained without surface or interior flaws. CMC (ceramic matrix composite) material is often used in fabricating aircraft components and other types of components. Sometimes, these components have, or develop, surface and/or interior flaws. Inspection for such flaws may be performed using known Infrared (IR) inspection systems in which an IR camera is used to provide 2D thickness images, along projection directions. The size of each defect is measured on a 2D IR image, which provides only information on the plane orthogonal to the direction of projection of the IR system. In addition, diffusivity information, which cannot be derived solely from 2D IR images, cannot be used to help characterize defect types on IR images. Also, as a result of optimization for their various applications, CMC components often have complex shapes and surfaces. Thus it may be difficult to measure a defect in three dimensions (3D) directly from an IR image. 
         [0003]    Known methods for linking IR to CAD data have relied upon attaching a fixture to the part during inspection, as described in, for example, “[a] novel method for internal feature reconstruction based on Infrared thickness measurement” at the Third International Symposium on Multispectral Image Processing and Pattern Recognition, Proceedings of the SPIE, Volume 5286, pp. 230-237 (2003). Known methods for unrolling UT (ultrasonic tomographic) images use a mesh model rather than a solid CAD model. For example, U.S. Patent Application No. US20070217672A1 to Shaimon et al., entitled “Combined 2D and 3D nondestructive examination” discloses a technique of using 3D information as a visual aid for NDE (non-destructive examination) inspection. However, Shannon et al. does not describe nor suggest a specific method of unrolling IR images using 3D information. Rather, the methods disclosed therein are used to map 2D image onto 3D geometry, rather than mapping 3D information onto a 2D image. 
         [0004]    A need thus exists for a device that can conveniently measure such defects in 3D directly from an acquired image. A need also exists for an inspection system in which acquired images and CAD data are linked to allow diffusivity compensation to improve inspection accuracy and/or to characterize defect types. 
       BRIEF DESCRIPTION OF THE INVENTION 
       [0005]    In one aspect, a method for determining 3D distances on a 2D pixelized image of a part or object is provided. The method includes acquiring a real 2D pixelized image of the object using the camera and operating the workstation to create a simulated image of the object using the 3D CAD model and the 2D pixelized image. The method further includes using the workstation to determine a specified cost function comparing the simulated image with the real 2D pixilated image and repositioning the simulated image in accordance with interated adjustments of a relative position between the CAD model and the 2D pixilated image to change the simulated image until the specified cost function is below a specified value. Once below the specified value, the method further includes using the workstation to generate a 3D distance scale matrix using the repositioned simulated image, and using the workstation to measure and display distances between selected pixels on a surface of the real image using 2D distances on the 2D pixelized image of the object and the 3D distance scale matrix. 
         [0006]    In another aspect, there is provided an apparatus for determining 3D distances on a 2D pixelized image of a part or object. The apparatus comprises a camera, a computer workstation configured to receive images from the camera and having a 3D CAD model of the object, and a display. The apparatus is configured to acquire a real 2D pixelized image of the object, create a simulated image of the object using the 3D CAD model, and determine a specified cost function comparing the simulated image with the real 2D pixilated image. The apparatus is further configured to reposition the simulated image in accordance with interated adjustments of a relative position between the CAD model and the 2D pixilated image to change the simulated image until the specified cost function is below a specified value, generate a 3D distance scale matrix using the repositioned simulated image, and measure and display distances between selected pixels on a surface of the real image using 2D distances on the pixelized image of the object and the 3D distance scale matrix. 
         [0007]    In yet another aspect, there is provided a machine-readable medium or media having recorded thereon instructions configured to instruct a computer workstation in an apparatus comprising a camera, a computer workstation configured to receive images from said camera and having a 3D CAD model of an object, and a display, to acquire a real 2D pixelized image of the object using the camera and to create a simulated image of the object using the 3D CAD model. The instructions are further configured to instruct the computer workstation to determine a specified cost function comparing the simulated image with the real 2D pixilated image, reposition the simulated image in accordance with interated adjustments of a relative position between the CAD model and the 2D pixilated image to change the simulated image until the specified cost function is below a specified value, generate a 3D distance scale matrix using the repositioned simulated image, and measure and display distances between selected pixels on a surface of the real image using 2D distances on the pixelized image of the object and the 3D distance scale matrix, 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0008]      FIG. 1  is a pictorial block diagram of an exemplary apparatus for use in determining 3D distances from a 2D image of an object. 
           [0009]      FIG. 2  is a representation of a portion of a pixelized image obtained from the apparatus shown in  FIG. 1 , wherein the image includes a fault or defect present in an object. 
           [0010]      FIG. 3  is a flowchart of a portion of an exemplary process performed by the apparatus shown in  FIG. 1 . 
           [0011]      FIG. 4  is a side cut-away view of an exemplary CAD image of a part represented by the 2D image shown in  FIG. 2  and taken along line  4 - 4  in  FIG. 2 . 
           [0012]      FIG. 5  is a portion of a 3D distance scale matrix containing Z-axis distances determined using an unrolled CAD model and the distance of its surface from pixels in a focal plane. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0013]    A technical effect of some embodiments of the present invention is to link a real inspection image with a simulated CAD (computer aided design) model and to display dimensional information that otherwise would not be available. This linking can be used, for example, to enable 3D measurements of defects from 2D images and diffusivity compensation for IR inspection. 
         [0014]      FIG. 1  is a pictorial block diagram of an exemplary apparatus  10  for use in determining 3D distances from a 2D image of an object. In the exemplary exhibit, apparatus ( 10 ) includes an IR camera  12  that scans or otherwise produces IR (thermal) images  14  of a part  16  under inspection. One or more IR lamps  18  are used to initially heat part  16  before being de-energized. In some embodiments, lamps  18  arc only energized for several seconds to heat part  16 . The acquision of a first thermal image  14  occurs after lamp  18  is de-energized. In some embodiments  10 , thermal images  14  can be rapidly and repeatedly acquired as the temperature of part  16  diminishes. For example, images  14  may be acquired up to one hundred times per second, at every microsecond, at every several microseconds, or at any other rate, depending upon the thickness of part  16  and the speed of its cool down. Each image  14  acquired in such a manner is different, in that the temperature varies somewhat from image to image. Although images  14  are related to what is shown in  FIG. 1  as a “film plane,” those of ordinary skill in the art will recognize that this “film plane” in most embodiments represents an array of IR sensors. 
         [0015]    Images  14  are processed to determine a thickness of part  16  at various points  20  on the image. The thickness is readily determined when part  16  has a uniform composition because in such embodiments, areas of greater thickness will have a greater heat capacity and will thus cool down more slowly. The thicknesses of parts with known, non-uniform compositions, such as those parts that include coatings or laminations, can also be determined using a more detailed application of this principle. The processing required for those determinations, as well as control of apparatus  10 , input of camera IR image  14 , and processing of a CAD image can be performed in a computer workstation  21  under control of a program which can be in internal storage of workstation  21  or on a machine readable medium  23  or media of any suitable type, such as a CD-ROM. 
         [0016]    In some embodiments and referring to  FIG. 2 , a defect may be found on some portion  15  of a pixilated image  14 . More specifically,  FIG. 2  illustrates a representation of a portion of a pixelized image obtained from the apparatus shown in  FIG. 1 , wherein the image includes a fault or defect present in an object. A defect, such as a crack, may manifest itself as, for example, a dark line or curve  30 , because the portion of part  16  in the immediate vicinity of the crack is thinner and hence cools more rapidly than other portions of part  16  that do not include defects. In some instances, depending on the orientation of part  16  relative to the “film plane,” the crack may be projected on image  14  as a straight line  30  rather than as a curve, even though the portion of part  16  that is cracked is actually curved. Linking the IR thermal image  14  to the 3D geometry of part  16 , enables the actual 3D dimensions of the crack or other defect on part  16  to be determined and enables a determination of whether the defect is curved, and if so, whether it is an arc or a spline, etc. 
         [0017]      FIG. 3  is a flowchart of a portion of an exemplary process performed by the apparatus shown in  FIG. 1 . In flow chart  100 , 3D distances are determined between pairs of pixels. At block  102 , the real 2D pixilated IR (thermal) image  14  is read in, as well as a view angle and distance for the real IR image. (The view angle can be determined based on the relative orientations of IR camera  12  and part  16 , and may itself be relative to a standard position.) At block  104 , a simulated IR image is generated from a CAD model of part  16  in accordance with the view angle of the real 2D pixilated IR image  14  and based on the temperature to which part  16  is heated. The simulated IR image may be generated by a computer projecting the CAD model through a simulated IR camera at the view angle and distance of the real part  16  from real camera  12 . In many embodiments, block  106  is subsumed by the making of this computer projection. In other embodiments, an additional adjustment may be made at block  106 . The adjustment at block  106  may be made either manually, e.g., by visual inspection of images on a screen followed by a manual adjustment, automatically by a program in a computer or workstation, or by some combination thereof, 
         [0018]    Next, at block  108 , a suitable cost function is used to compare the simulated IR image with the real 2D pixilated IR (thermal) image  14 . The cost function can be evaluated by establishing a correspondence between pixels on the 2D pixilated IR image  14  with points on the 3D CAD model. On suitable cost function is related to a grayscale difference between the two images. For example, an embodiment may have pixels that are assigned grayscale values between 0 and 255 depending upon real or simulated IR emission intensity or interpreted thickness value. Such an embodiment may also have an image size of, for example, 500×500 pixels. An example of a cost function suitable for this embodiment is one in which a statistic is calculated that depends upon the difference of grayscale values of corresponding pixels in the real 2D pixilated IR (thermal) image  14  and the simulated IR image. The statistic may be, for example, the sum of the absolute values of the differences in grayscale values. 
         [0019]    If, at block  108 , the value of the cost function is at or above a selected threshold value, a suitable optimization is performed  110  to determine a repositioning of the simulated IR image (i.e., an adjustment of a relative position between the CAD model and the 2D pixilated image to change the IR image), and the simulated IR image is repositioned in accordance with this determination. One suitable optimization method utilizes an LM (Levenberg-Marquardt) algorithm, which provides a numerical solution to the problem of minimizing a function, generally non-linear, over a space of parameters of the function. The algorithm interpolates between the Gauss-Newton method and the method of gradient descent. However, any other optimization method that works can be employed in other embodiments. For example, a Newton algorithm can be used, but the Newton algorithm does not necessarily guarantee a group or optimization solution. 
         [0020]    If an optimization is necessary, it is performed and the comparison at block  108  is performed again. The loop comprising blocks  108  and  110  is performed until the cost function is below the selected threshold value. When this occurs, or if no optimization is necessary, a 3D distance scale matrix between two pixels along the x-axis and the y-axis is generated at block  112  in accordance with the simulated and optimized IR image and the CAD model used to produce the simulated and optimized IR image. More particularly, in an embodiment having a thermal image  14  of 500 by 500 pixels, it is assumed that the pixels are linearly spaced in the x and y direction, e.g., that the pixels represent 100 mm×100 mm portions of part  16 . (Some, but not all embodiments, apply a correction to image  14  based on the view angle projection for each pixel to compensate for pincushion, barrel, or other known types of distortion in image  14 .) In this embodiment, the 3D distance scale matrix comprises a value for each pixel (x,y) that represents the distance (depth) of the surface of the CAD model simulated pixel relative to a distance to an imputed 2D surface representing thermal image  14 . It is thus possible to determine a distance along the surface of a part  16  using the matrix values contained in the 3D distance scale matrix as offsets from a fat surface. At block  114 , assuming that neither IR camera  12  nor part  16  are moved or otherwise adjusted, additional thermal images  14  can be read repeatedly (e.g., acquired every several microseconds, or as otherwise discussed above) and analyzed for defects and such defects measured using the same 3D distance scale matrix generated in block  112 . 
         [0021]    For example and referring to  FIG. 2  and  FIG. 4  (the latter figure being a cross section along line  4 - 4  of a part  16  depicted in  FIG. 2 ), a crack manifesting itself as line  30  in portion  15  of  FIG. 4  includes pixels representing portions of part  16  that are not coincident with focal plane  17  of camera  12 . Examples of such distances in the z-direction relative to focal plane  17  are shown in  FIG. 4  as d 1 , d 2 , d 3 , d 4 , and d 5 , with d 4 ≈0. Therefore, if a straight crack extends, for example, from pixel  202  to pixel  204  as shown in  FIG. 2 , the distance determined solely by the x-y coordinates of pixels  202  and  204  does not give an accurate representation of the length or geometry of the crack because it such a distance ignores the z-axis distance of the projection of the crack onto line  30 . Referring to  FIG. 5 , a portion of the 3D distance scale matrix  206  containing z-axis distances of the unrolled CAD model from focal plane  17  is shown. The numbers contained in the matrix each represent the z-axis distance representative of the corresponding pixel in image  14 . Thus, a good approximation to the real distance represented by any specific pixels on image  14  along the surface of a part  16  is readily determined using x-y coordinates and z-axis distances in the 3D distance scale matrix  206  corresponding to selected x-y coordinates. For example, the distance along a curved portion of the surface of part  16  represented by a line between pixels  202  and  204  can be determined by obtaining the geodesic distance between two points in the 3D image corresponding to the adjacent pixels along the length of the line. If the curve is known from the CAD model to follow a specific parametric curve, an even better, non-linear estimate can be obtained. 
         [0022]    The unrolling of the CAD model and the IR (thermal) image thus allows a workstation to use CAD information to improve inspection accuracy, such as thermal diffusivity correction for IR inspection. The CAD information can also be used to obtain accurate inspection analysis on a 2D inspection images (such as defect sizing on an IR image). Furthermore, IR inspection data can be visualized and/or analyzed on the 3D geometry of the CAD model and the inspection directly validated based upon engineering specifications of the CAD model. Some configurations can be extended by incorporating X-ray CT (computed tomographic) information with the CAD model to link the IR (thermal) image to the more accurate 3D geometry information from the CAD model and CT X-ray inspection data. 
         [0023]    in place of a 2D pixilated IR (thermal) image, some embodiments utilize a UT (ultrasound tomographic) image. Other types of real 2D images may be used along with the 3D CAD model. Linking 2D images with 3D CAD models enables 3D world measurements of defects on 2D IR images. One embodiment enables measurements of CMC composite blades, vanes, and shrouds used in aircraft. CAD information can also be used in some embodiments for diffusivity compensation, for better defect characterization and for more accurate defect sizing. 
         [0024]    This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.