Patent Publication Number: US-2021192670-A1

Title: System and method for robotic inspection

Description:
BACKGROUND 
     The disclosure relates to robotic inspection of a part and, more particularly, to an automated system and method for inspecting mechanical components, especially turbine blades. 
     It is well known that mechanical components may incur imperfections during manufacturing and may suffer wear and damage during operation. These components, therefore, are episodically or periodically inspected for imperfections, damage, deformation, and wear. In particular, mechanical components such as turbine disks and airfoils have complex mechanical shapes that are difficult to manufacture and are operated under stresses that cause damage, e.g., leading edge erosion, hot corrosion (sulfidation), cracks, dents, nicks, gouges, etc. from foreign object damage. These turbine disks and airfoils are currently inspected manually by visual inspection or by non-destructive evaluation (NDE) techniques such as eddy current, ultrasound, or fluorescent penetrant inspection. These inspections are tedious, time consuming, imprecise, and error prone. Techniques to automate these types of inspection have been emerging, but the automated detection and operation can be improved. 
     Manual turbine blade damage detection is known in the prior art, e.g. using embedded vibroacoustic sensing and strain sensing. This prior art uses embedded or contacting sensors and is intended for wind turbines where the blades to not suffer the impact, erosional, or corrosion that gas turbine blades suffer. Also, blade damage detection for gas turbines is known, e.g. using eddy current sensors or by using mechanical touch probes, e.g., a coordinate measuring machine (CMM), but these require slow, laborious scanning of the blade surface. 
     It is known to process borescope video of blades in an engine to determine damage. This approach analyzes two dimensional (2D) images for differences between the current image and a model learned from other 2D images in a blade row. This is not suitable for determining damage in absolute units for components outside an engine. There is additional prior art for 2D (image-based) inspection systems wherein many steps are performed to determine the pose of an inspection device (a camera) with respect to the part and an a priori engineering model so that differences between the part and the model may be determined. This particular approach is unnecessarily inefficient and error prone. Some methods use X-rays, which requires special, shielded equipment. 
     SUMMARY 
     In accordance with the present disclosure, there is provided a method for robotic inspection of a part, which comprises the steps of: supporting the part with a robot mechanism; obtaining part-related sensor input with a sensor positioned to inspect the part supported by the robot mechanism; controlling movement of the robot mechanism relative to the sensor, wherein the controlling is done by a feedback control unit which receives the sensor input, and the feedback control unit is configured to control the robot mechanism based upon the sensor input. 
     In accordance with a further non-limiting embodiment, the method further comprises the steps of storing the part-related sensor input, or information derived therefrom, as past sensor input in a storage in communication with the feedback control unit and controlling movement of the robot mechanism based upon current sensor input and the past sensor input. 
     In a further non-limiting embodiment, the feedback control unit is further configured to plan a path of movement, relative to the sensor, of the part supported by the robot mechanism, wherein the path of movement is determined based upon the past sensor input. 
     In a further non-limiting embodiment, the sensor has a controllable lens and the feedback control unit is configured to control the lens based upon the part-related sensor input. 
     In a further non-limiting embodiment, the sensor further comprises a plurality of lenses and an automated lens changing system for positioning a lens of said plurality of lenses along a line of sight from the sensor to the part, and the method further comprises the step of changing the lens along the line of sight based upon the part-related sensor input. 
     In a further non-limiting embodiment, at least one filter is positioned along a line of sight from the sensor to the part, and the method further comprises the step of operating the filter based upon the part-related sensor input. 
     In a further non-limiting embodiment, the method further comprises illuminating the part with an illumination mechanism, and the feedback control unit is configured to control the illumination mechanism based on the part-related sensor input. 
     In a further non-limiting embodiment, at least one filter is positioned along a line of sight from the light mechanism to the part, and the method further comprises the step of operating the filter based upon the part-related sensor input. 
     In a further non-limiting embodiment, the illumination mechanism further comprises a controllable lens, and the feedback control unit is configured to control the lens based upon the part-related sensor input. 
     In a further non-limiting embodiment, the feedback control unit is configured to control the robot mechanism based upon the sensor input and manual input. 
     In a further non-limiting embodiment, the method further comprises the step of annotating the part based upon the part-related sensor input. 
     In a further non-limiting embodiment, an inspection system for robotic inspection of a part, comprises a robot mechanism configured to support the part, the robot mechanism being moveable to adjust position and pose of the part; a sensor positioned to obtain part-related sensor input of the part supported by the robot mechanism; and a feedback control unit in communication with the sensor to receive the part-related sensor input, the feedback control unit being configured to control movement of the robot mechanism based on the part-related sensor input. 
     In a further non-limiting embodiment, the system further comprises a storage in communication with at least one of the sensor and the feedback control unit, the storage being configured to receive and store the part-related sensor input or information derived therefrom. 
     In a further non-limiting embodiment, the feedback control unit is further configured to plan a path of movement, relative to the sensor, of the part supported by the robot mechanism, wherein the path of movement is determined based upon the past sensor input. 
     In a further non-limiting embodiment, the sensor has a controllable lens and the feedback control unit is configured to control the lens based upon the part-related sensor input. 
     In a further non-limiting embodiment, the sensor further comprises a plurality of lenses and an automated lens changing system for positioning a lens of said plurality of lenses along a line of sight from the sensor to the part, and the feedback control unit is configured to change the lens along the line of sight based upon the part-related sensor input. 
     In a further non-limiting embodiment, at least one filter is positioned along a line of sight from the sensor to the part, and the feedback control unit is configured to operate the filter based upon the part-related sensor input. 
     In a further non-limiting embodiment, an illumination mechanism is provided for illuminating the part, and the feedback control unit is configured to control the illumination mechanism based on the part-related sensor input. 
     In a further non-limiting embodiment, at least one filter is positioned along a line of sight from the illumination mechanism to the part, and the feedback control unit is configured to operate the filter based upon the part-related sensor input. 
     In a further non-limiting embodiment, the illumination mechanism further comprises a controllable lens, and the feedback control unit is configured to control the lens based upon the part-related sensor input. 
     In a further non-limiting embodiment, the feedback control unit is configured to control the robot mechanism based upon the sensor input and manual input. 
     Other details of the process are set forth in the following detailed description and the accompanying drawings wherein like reference numerals depict like elements. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  illustrates a non-limiting embodiment of an automated system and process for automated component inspection; 
         FIG. 2  illustrates a further non-limiting embodiment; and 
         FIG. 3  illustrates a further non-limiting embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     The present disclosure relates to the automated inspection of a part such as a turbine blade or the like. 
     Various mechanical system components such as turbine blades, disks, and airfoils, require inspection for damage, defects, the need for repair and/or maintenance, and the like. One form of such inspection is automated inspection. During such inspection, the part is supported by a robot mechanism in a position relative to a sensor such that the position and pose of the part relative to the sensor can be adjusted primarily through movement of the robot mechanism. As disclosed herein, the position and pose of the part relative to the sensor are adjusted based upon part-related sensor input so that control of the position and pose of the part can be coupled to automated detection or inspection results. Thus, following this approach, an inspection can be conducted to focus more specifically on a location of a part which the current and/or past inspection has indicated a reason for further and/or more focused inspection to be conducted. Thus, according to the disclosure, robot motion is coupled to automated detection results. Another approach would be a “hybrid” approach, wherein the system is mostly automatic, but is configured to reach out to some other system or person for either confirmation, for example to help avoid re-certifying the inspection process, for annotation, and for guidance such as, for example, where to pay attention next, or for further analysis. For example, the system could reach out to a human on site or at a remote station, humans in the cloud or a more powerful image processing system in the cloud. 
       FIG. 1  shows a system  10  including a robot mechanism  12 , a sensor mechanism  14 , and a feedback control unit  16 . As will be discussed further below, feedback control unit  16  controls operation of robot mechanism  12  to control the position and pose of a part  18  held by robot mechanism  12  relative to sensor mechanism  14 . 
     Robot mechanism  12  can have various support mechanisms to hold part  18 , such support mechanisms being schematically illustrated by graspers  19  in  FIG. 1 . Such support mechanisms can hold part  18  mechanically, or with suction or vacuum, or through any other manner suitable for firmly holding the part for inspection as desired. 
     Robot mechanism  12  is typically a fully articulated arm configured to allow multi direction adjustment of the position of a part, and particularly including rotation and orientation of the part at a particular location, which is referred to herein as the pose of the part. The position in which robot mechanism  12  holds part  18  is with respect to sensor mechanism  14 , and the different positioning and pose allows for full inspection of the part. 
     Sensor mechanism  14  can be any of a wide variety of different sensors, such as image sensors, thermal sensors, or the like. Sensor mechanism  14  can, by way of further non-limiting example, comprise one or more two-dimensional (2D) cameras, three-dimensional (3D) depth sensors, and/or sonic sensor arrays, operating in any portion of the electromagnetic spectrum or acoustic spectrum (as relevant), to capture current information of a part under inspection. In the non-limiting embodiment of  FIG. 1 , sensor mechanism  14  is an image sensor or camera, which obtains as sensor input a sequence of images or video of a part supported in robot mechanism  12 . This sensor input is referred to herein as part-related sensor input, and can be image data of the part, for example. The part-related sensor input can provide input to feedback control unit  16 , for example through an image processing unit  20 , and/or can be input to a storage unit  22 . Components which pass data such as sensor input or control instructions are said herein to be communicated with each other, and/or in communication with each other. Thus, sensor mechanism  14  can be in communication with feedback control unit  16 , optionally through image processing unit  20 , and can also be in communication with storage unit  22 . 
     The information or sensor input is processed and stored in a database, for example in storage unit  22 , in such a way that relationships between the current inspection and previous inspections are established. The results of the current and previous inspections are provided to feedback control unit  16 . One way to establish relationships is to add metadata about each inspection to the data stored in the database such as the individual part type, serial number, inspection date and time, inspection software version number, and the like. 
     Feedback control unit  16  can be any of a wide variety of processing units configured to execute and/or send various machine language commands including but not limited to commands which can be sent to robot mechanism  12  for controlling movement thereof. In addition, feedback control unit  16  can be configured to process sensor input from sensor mechanism  14  or receive processed sensor input from image processing unit  20 . Feedback control unit  16  can also be in communication with storage unit  22  for accessing past part-related sensor input, for example. Feedback control unit  16  can also be in communication with a separate storage unit  24 , which may contain various programming and machine executable instructions for controlling operation of the feedback control unit  16 . 
     Of particular interest in the present disclosure, feedback control unit  16  is advantageously configured to control movement of the robot mechanism based on feedback from the sensor mechanism of a current inspection. Thus, if the part-related sensor input received in real time from the sensor mechanism indicates an area of a part needs to be inspected more closely or at a different pose, feedback control unit  16  is configured to send instructions to the robot mechanism to make such adjustments and enhance the inspection. In one non-limiting embodiment, feedback control  16  may implement a feedback control algorithm designed to reduce uncertainty or resolve a non-binary probability in damage detection of part  18 . That is, image processing unit  20  may use a statistical damage detection algorithm that provides a probability of detection. If in any instance this probability is not zero or one, feedback control unit  16  may iteratively change the pose, illumination, and/or sensing of part  18  to drive the probability or detection to either zero or one. The feedback control may be considered or implemented as an optimization process with uncertainty as its objective function. In one case, the optimization may comprise a complete exploration of the parameter space. 
     Also of interest, since feedback control unit  16  has access to past part-related sensor input, which for example may be stored in storage unit  22 , the initial movement path of robot mechanism  12  can be set or adjusted to focus specifically on areas of already identified interest, such as defects already detected in a disk, blade row, or the like. Thus, feedback control unit  16  provides path planning based on prior information such as past part-related sensor input. 
     A further aspect of interest with respect to interaction of feedback control unit  16  in system  10 , related to the path planning mentioned above, is to design the initial movement path of robot mechanism based on past part-related sensor input such that a maximum amount of information is gained by the inspection. In other words, the position and pose of the part can be selected by feedback control unit  16  such that knowledge of a particular part being inspected is maximized per each movement path of robot mechanism  12  and part  18  supported thereby relative to sensor mechanism  14 . 
     In a further non-limiting aspect of the present disclosure, an illumination mechanism  26  can be provided for illuminating part  18  supported by robot mechanism  12  such that sensor mechanism  14  can obtain more clear and well illuminated part-related sensor input. Illumination mechanism  26  can be any suitable source of light which is suitable for enhancing inspection of a part. Such light can be in the visible spectrum, or in other spectra suitable to various types of inspection which may be desired. Further, illumination mechanism  26  can be controlled by feedback control unit  16  in a basic sense to power on or off, and also to increase or decrease intensity and/or type of illumination, such as wavelength, band of wavelengths, polarization, spatial structure, and the like. Controllable filters can be included in both the illumination mechanism  26  and sensor mechanism  14  to allow automated selection of wavelength and polarization. 
     All such control can be based upon current or past part-related sensor input. For example, if past part-related sensor input indicated a portion of a complex structure was not properly illuminated during inspection, position and pose of the part can be adjusted, as can intensity of light from illumination mechanism  26 , to obtain better part-related sensor input in the next inspection and/or for the remainder of the current inspection. 
     Returning to sensor mechanism  14 , a further non-limiting embodiment includes sensor mechanism  14  having a controllable lens schematically illustrated at  28 . Controllable lens  28  can allow for enhanced specific inspection of a particular area of a part, thus enhancing proper positioning and pose of the part relative to the sensor by allowing focus from the sensor side as well. Lens  28  can be controlled by feedback control unit  16  as with other components of system  10 , again so that results of current and past part inspection can be used to enhance movement of robot mechanism  12  and focus of sensor mechanism  14  to produce more accurate and effective inspection. It should also be appreciated that within the scope of a controllable lens  28  is a lens system with physical lens changes, for example where the desired or intended change (e.g. magnification) is more than can be accomplished with a single adjustable lens. In this configuration, the system can be configured to operate an automated lens changing system.  FIG. 2  further illustrates certain aspects of system  10 , including sensor mechanism  14  having an interchangeable lens system schematically illustrated at  29  and being configured to switch out lens  28  with one or more different lenses  28   a ,  28   b  . . .  28   x . Interchangeable lens system could be controlled by feedback control unit  16 , again as schematically illustrated, and could utilize conveyors, robots, or any other mechanism suitable for removing and replacing lens  28 . 
     As set forth above, one or more filters can also be incorporated into system  10  to modify operation of either sensor mechanism  14  and lens  28 , or illumination mechanism  26  and/or lens  30 .  FIG. 2  shows filters  31 ,  33  positioned for this purpose, in this case along a line of sight from either lens  28  or lens  30 , or both, to part  18 . As with lens  28 ,  30 , filters  31 ,  33  can be controllable and/or interchangeable to adapt the system to produce useful imaging and analysis.  FIG. 2  also shows schematic control connections, or communications, between feedback control unit  16  and components of sensor mechanism  14 , illumination mechanism  26  and interchangeable lens system  29 . 
     Similarly, illumination mechanism  26  can have a controllable lens  30  which can be used to focus, diffuse, or otherwise adjust light being directed to part  18  during inspection. Feedback control unit  16  is configured and in communication with illumination mechanism  26  to control lens  30  to focus light from illumination mechanism  26  in a way which is directed by results of current or past inspection and part-related sensor input. Again, by way of non-limiting example, if a portion of a part is not clearly seen in images obtained during either a prior or current inspection, this will be identified by feedback control unit  16  and instructions sent to robot mechanism  12  to change position or pose of the part relative to sensor mechanism  14 , and also in this case to adjust focus of light emitted by illumination mechanism  26  to more fully illuminate a formerly obscured or not clearly inspected area of the part. Lens  28  and/or filters  31 ,  33  could also be adjusted. 
     Still referring to  FIG. 1 , feedback control unit  16  can also be configured to produce controllable annotation  32 . Annotation  32  may be accomplished by any device capable of depositing ink or other indicia onto part  18 . For example, a deposition device may comprise a controllable inkjet print head schematically illustrated at  34  ( FIG. 3 ). A fiducial mark  36  denoting a detected damage location may be deposited onto part  18  by the deposition device. The fiducial mark may comprise any unique and/or distinguishable mark capable of being detected by image processing unit  20  via sensor mechanism  14 . For example, a fiducial mark may comprise a dot, line, circle, square, polygon, numerical values, Roman numerals, alphabetical characters, or any other distinguishable marks. The ink may comprise a high temperature resistant marking agent. In general, the ink may comprise any subsequently detectable material, e.g., the ink may be visibly detectable as described, may contain infrared fluorescent constituents, radioactive tracers, and the like. Similarly, sensor mechanism  14  may be additionally sensitive to non-human-visible portions of the electromagnetic spectrum, radiation, and the like. In one non-limiting embodiment, the ink could be a high-temperature, non-reactive blue ink such as DYKEM® High Temp  44  Medium Blue (Item #44094 from American Marking, Inc.), or the ink described in US Application No. 2003/0101899 to Hartanto. The deposition device may be spatially controllable to move relative to part  18 , may be spatially stationary while robot mechanism  12  moves part  18 , or a combination of the above. 
     The method of operation of system  10  can be further discussed and understood through consideration of  FIG. 1  as a process scheme. System  10  begins operation with a part  18  supported by robot mechanism  12 . Initial inspection may already be programmed and stored in either storage unit  22  or  24  for conducting an initial inspection of the part. Such initial inspection data may be the result of an earlier inspection using system  10 , could be generated based on expected part-related data, could be based on mimicking a one-time human inspection, and the like. Once inspection starts, part-related sensor input is obtained with sensor mechanism  14  and communicated to image processing unit  20 . Sensor input is processed and the processed input is sent to and stored in storage unit  22 . In addition, processed sensor input is received by feedback control unit  16 . Feedback control unit  16  analyzes such input and generates control instructions which are sent to robot mechanism  12 , and also to sensor mechanism and/or illumination mechanism  26  as desired. These control instructions adjust the positon and pose, as well as sensor and/or light focus, to obtain additional sensor input related to part  18 , which is obtained and fed to image processing unit  20  and feedback control unit  16  for further analysis and issuance of control commands by feedback control unit  16 . This can continue until inspection is considered complete, either through substantive passing of inspection, or obtaining a predetermined amount of additional part-related sensor data, or the like. 
     As disclosed herein, feedback control unit  16  operates to provide one or more of the following: 
     Repositioning based on the current inspection, e.g., to re-inspect from a different relative location or pose and combine the new inspection results with the previous results using, for instance, a Bayesian damage estimator, to produce a first fused inspection result. The re-inspection process may now continue with the first fused inspection result used as the previous inspection result. This iterative process can terminate when the fused result is the same as the previous result or when a predetermined number of iterations is reached, for example. 
     Inspection path planning can be based on prior information such as already detected defects in a disk, blade row, and the like. That is, a default sequence of location and pose robot controls may be modified based on the spatial probability of damage and the probability that the current part is related to previous parts. This path planning may exploit UAV probabilistic search path planning as modified for part similarity. 
     Inspection path planning can also be based on expected information gain. The expected information gain is the change in information from a prior state, or previous inspection, to a state that takes some information as known (current inspection). That is, a default sequence of location and pose robot controls may be modified to make a next inspection where the expected information gain is maximized. The information gain may be learned from previous inspections, which can serve as training examples, and may be fixed after training, or may be adapted continuously during inspection. 
     There has been provided a system and method for automated inspection of a part which provides for a coupling of robot motion with current and/or past inspection results, thereby reducing uncertainty in detection and producing a more fully automated and reliable part inspection. While the system and method have been described in the context of specific embodiments thereof, other unforeseen alternatives, modifications, and variations may become apparent to those skilled in the art having read the foregoing description. Accordingly, it is intended to embrace those alternatives, modifications, and variations which fall within the broad scope of the appended claims.