Automatic detection and robot-assisted machining of surface defects

A method for automated detection of defects in a workpiece surface and generation of a robot program for the machining of the workpiece is described. In accordance with one embodiment, the method comprises the localization of defects in a surface of a workpiece as well as determining a three-dimensional topography of the localized defects and categorizing at least one localized defect based on its topography. Dependent on the defect category of the at least one defect, a machining process is selected and, in accordance with the selected machining process, a robot program for the robot-assisted machining of the at least one defect is generated with the assistance of a computer.

TECHNICAL FIELD

The present disclosure generally relates to the field of industrial robots, in particular to a system and a method for the automated detection of defects in surfaces (e.g. painting defects on a car body) and the robot-assisted machining thereof, in particular by grinding and polishing.

BACKGROUND

In automated robot-assisted manufacturing, for example in the automotive sector, the problem arises, inter alia, of automatedly detecting defects in surfaces of a workpiece (for example defects in the paint layer after the painting of the workpiece) and, if necessary, to repair them by means of robots (e.g. by gridding or polishing). Systems and methods for robot-assisted detection of surface defects have been known for some time. For example, an inspection apparatus movably arranged on a robot arm with an illumination unit and a camera unit is known from publication WO 87/00629 A1. The camera unit receives the light of the illumination unit reflected from the surface to be inspected and, in doing so, identifies surface defects. A method for detecting surface defects in bodies-in-white in a portal unit with a conveyor is known from DE 197 30 885 A1, wherein detected surface defects are marked in a successive marking apparatus. For this purpose controllably movable and triggerable marking nozzles are installed on a portal, which are equipped with water-soluble paint for the marking of relevant surface defects. A distance adjustment regulated according to the contour is provided for the marking nozzles.

Most of the systems employed today are limited to detecting and marking surface defects. Frequently the defects are then individually checked and repaired manually by a skilled worker. A system for detecting and repairing defects, particularly on painted surfaces, is known from the publication U.S. Pat. No. 6,714,831 B2, wherein the positions of the surface defects are determined in the coordinate system of the inspected object, a repair strategy is developed, and, based on this repair strategy, a repair system is controlled using the object coordinates of the positions of the defects. The “repair strategy” thereby includes the selection of the path along which the defects are approached, as well as the selection of the tools and the robots. However, as not all surface defects can be machined in the same manner, and some defects need not be machined at all, there is a need for improvement.

The application discloses a method and a system which is capable of automatedly detecting surface defects and to repairing them with the help of robots. In doing so, the machining of the surface defects as part of the robot-assisted repair should be adapted to the type (the characteristics) of the defect.

SUMMARY

Some exemplary embodiments are summarized below. Various other embodiments and further developments are discussed further below in the Detailed Description.

In the following a method for the automated detection and robot-assisted machining of defects in a workpiece surface is described. In accordance with one embodiment, the method includes an optical inspection of the surface to detect defects as well as a three dimensional measurement of the workpiece surface in the area of detected defects by means of optical sensors. The method further includes the determination of the topography of the workpiece surface in the area of at least one defect and the determination of a parameter set that characterizes the at least one defect. At least one of the defects is categorized based on the determined parameter set. That is, the defect is assigned to a defect category. Dependent on the defect category of the at least one defect a machining process is selected. When doing so, each machining process is associated with at least one template of a machining path along which the defect is to be machined. At least one machining path is determined for the at least one defect by means of projection of the at least one template onto the workpiece surface in accordance with a CAD model of the workpiece. Subsequently, the computer-assisted generation of a robot program for the robot-assisted machining of the at least one defect can be carried out.

Furthermore, a method for automated detection of defects in a workpiece surface and for the generation of a robot program for the machining of the workpiece is described. In accordance with one further embodiment, the method comprises the localization of defects in a surface of a workpiece as well as determining a three-dimensional topography of the localized defects and categorizing at least one localized defect based on its topography. Dependent on the defect category of the at least one defect, a machining process is selected and, in accordance with the selected machining process, a robot program for the robot-assisted machining of the at least one defect is generated with computer assistance.

In one embodiment, a parameter set may be determined which characterizes the topography of the localized defects. The categorization of the at least one localized defect is carried out based on the determined parameter set, wherein the defect may be unambiguously assigned to a defect category. The determination of the three-dimensional topography of the localized defects includes, for example, the determination of 3D-coordinates of a point cloud as well as a three-dimensional reconstruction of the workpiece surface in the area of the respective defect.

Each machining process may be associated with at least one template of a machining path along which the defect is to be machined. A machining path for the at least one defect may then be determined by means of projection of the at least one template onto the workpiece surface in accordance with a CAD model of the workpiece.

Moreover a system for automated detection and robot-assisted machining of defects in a workpiece surface is described. In accordance with one embodiment, the system includes an optical inspection and measurement system for the inspection of the surface, both for defecting defects as well as for the three-dimensional measurement of the workpiece surface in the area of detected defects with the use of optical sensors. The system further comprises at least one industrial robot for machining the workpiece surface, as well as a data processing device that is configured to determine the topography of the workpiece surface in the area of at least one defect, as well as a parameter set that characterizes the at least one defect. The at least one defect is categorized based on the determined parameter set. That is, the defect is assigned to a defect category. A machining process stored in a database is selected in dependency of the defect category of the at least one defect. Each machining process is associated with at least one template of a machining path along which the defect is to be machined. A specific machining path for the at least one defect is subsequently determined by means of projection of the at least one template onto the workpiece surface in accordance with a CAD model of the workpiece. Subsequently, a robot program for the robot-assisted machining of the at least one defect by at least one industrial robot may be generated.

Furthermore, a system for the automated detection of defects in a workpiece surface and generation of a robot program for the machining of the workpiece is described. In accordance with one embodiment, the system includes an optical inspection system for the localization of defects in a surface of a workpiece, as well as a data processing device configured to determine a three-dimensional topography of the localized defects, assign at least one localized defect to a defect category based on its topography and select a machining process dependent on the defect category of the at least one defect. Subsequently, a robot program for the robot-assisted machining of the at least one defect may be generated in accordance with the selected machining process

DETAILED DESCRIPTION

The following description relates basically to the detection of surface defects in painted workpiece surfaces. The application of the method described herein is, however, not limited to the inspection of painting processes, but may also be used for the detection and machining (with regard to a repair, spot-repair) of surface defects resulting from causes different from an imperfect painting.

During a painting process, various surface defects such as dirt or fiber inclusions, PVC remnants or “craters” may occur after each painting step. Today, in many production plants defects of that kind are detected by qualified personnel and repaired by manual grinding. Despite the fact that, today, in the field of painting the majority of activities are automated, the correction of any defects is a very personnel and time consuming activity, the result of which heavily depends on the person carrying it out. Due to the subjective assessment of the responsible person who evaluates whether and, as the case may be, how a paint defect is to be eliminated in accordance with applicable quality standards, maintaining a uniform quality proves to be difficult.

The methods described herein are intended to allow for a full automation of the surface inspection, of the evaluation of the detected surface defects and of their machining. The automated, computer-assisted evaluation of the measurement results would allow reproducible quality, and the specifiable quality standards can be constantly complied with.

Various measurement systems for the three-dimensional measurement of workpiece surfaces are known. In the examples described herein, the measurement system (optical inspection system) operates based on the technique of deflectometry which allows to detect and localize defects starting from a lateral (i.e. along the surface) extent of about 100 μm on painted surfaces.FIG. 1shows an example of a measurement system with a plurality of sensors, guided by manipulators (industrial robots), for the optical inspection, with the use of cameras, of the surface of a workpiece10, for example, a car body painted with base coat and primer. The purpose of the surface inspection is a detection (this includes a localization) of surface defects and a three-dimensional measurement of at least those areas of the workpiece surface in or on which a defect has been detected. In the present example, manipulators31,32, and33, equipped with sensor heads21,22, and23, are employed in a robot cell and perform the surface inspection simultaneously. Dependent on the time available for the surface inspection, two or more manipulators may be employed. In specific applications, a single robot with a sensor head may be sufficient.

In the present example, each of the sensor heads includes an LCD monitor (for illumination), a plurality of (e.g. four) cameras, and a controller unit. With the use of the LCD monitor structured light may be generated for the illumination of the workpiece surface, which is imaged by high-resolution cameras. The structured light generated by the LCD monitor has a stripe pattern with a sinusoidal brightness modulation which is projected onto the workpiece. The resulting reflected pattern is captured—for different phase shifts of the stripe pattern—by the cameras of the respective sensor heads21,22, and23, and the captured images are evaluated to determine the coordinates of surface defects (“defect candidates”, to be precise) on the surface of the workpiece. When using the measurement system described herein, a three-dimensional measurement of the whole workpiece surface is not needed for the determination of defect candidates. The defect candidates may have already been localized in a two-dimensional camera image (with the mentioned stripe pattern) using a CAD model of the workpiece. Subsequently, a three-dimensional measurement need only be done for those areas in which a defect candidate has been localized by use of a deflectometric measurement technique. Whether a defect candidate actually is a surface defect to be machined may then be evaluated based on the three-dimensional measurement. In the present example, no separate image acquisition is required for the three-dimensional measurement, but instead only a digital evaluation of the two-dimensional camera images (curvature images, the curvature information is in the gray values of the individual pixels); from these, point clouds of 3D coordinates of points on the surface of the workpiece (in the areas of defects/defect candidates) can be calculated.

Using a best fit approach characteristic features (e.g. edges, holes, corners, etc.) distributed throughout the workpiece surface are considered before each measurement with one of the sensor heads21,22,23. From these, the exact position of the workpiece relative to a desired position (based on a CAD model of the workpiece) is determined. The manipulators31,32, and33may then be controlled such that the determined position deviations are compensated. In doing so, it is ensured that the positions of the sensor heads21,22, and23relative to the workpiece surface to be inspected are always the same for various workpieces of the same kind and independent of any position tolerances. This allows for a very precise localization of a defect on the CAD model of the workpiece. This accuracy of the positioning may also be important for the automated machining of the workpiece for repair of the surface defects as explained further below.

The first result of a three-dimensional measurement of a defect candidate is a point cloud that describes the three-dimensional structure (the topography) of the relevant surface area. For each defect candidate, for example, its lateral extension (across the surface) and its height or depth (extension perpendicular to the surface) can be determined with great precision from the point clouds provided by the sensor heads21,22, and23(see alsoFIG. 3) using surface reconstruction. When, as shown in the example ofFIG. 1, the sensor heads for the optical inspection are moved by use of manipulators, the measurement values (point coordinates) determined by a sensor head must undergo a coordinate transformation into a global coordinate system. Naturally, this coordinate transformation depends on the position of the respective sensor head and thus on the joint angles of the manipulator that carries the sensor head. Accordingly, in the coordinates of a point cloud (of a surface defect or a defect candidate) the positions of the sensor heads21,22, and23during the measurements are taken into account. A suitable measurement system is, for example, the system reflectCONTROL of Micro-Epsilon Messtechnik. Dependent on the application, other systems may be used for the three-dimensional measurement of surfaces. As such measurement systems are well known, they will not be described in further detail here.

The system shown inFIG. 1includes a data processing device50which, in one embodiment, is configured to (inter alia) localize defects and determine the mentioned three-dimensional topography of the localized defects (or defect candidates). Basically, the data processing device may be any entity including hardware, software or any combination thereof, which is capable of performing the automated processing of the data (i.e. the mentioned point clouds) provided by the measurement system (e.g. by the sensors21,22, and23) in order to obtain machining paths and a corresponding robot program suitable for repairing the defects. For this purpose, the data processing device50may include one or more processors with a memory containing instructions that, when executed, cause the optical inspection system to perform the activities described herein. In one example, the data processing device50may include a workstation computer or a personal computer including interface modules (hardware and software) allowing communication with the optical inspection system, e.g. with the robots31,32, and33, and the sensors21,22, and23.

Before explaining the processing of the surface measurement data that is detected by the measurement system ofFIG. 1in greater detail, the robot-assisted repair of the detected surface flaws should be briefly discussed.FIG. 2shows a robot cell with a manipulator34that is equipped with a grinding tool24(e.g. an orbital grinding machine). The manipulator34may here include a handling device (not shown, cf. actuator25inFIG. 10), which is arranged between the tool center point (TCP) of the manipulator34and the grinding tool24and which is configured to practically arbitrarily adjust (within certain limits) and, e.g., keep the contact force with which the grinding tool24is pressed against the surface of the workpiece10at a constant level or segment-wise at a constant level. The controller40does not only set the trajectory of the robot but also the tool-dependent parameters relevant to the repair process such as, e.g., contact pressure of the grinding tool24, rotational speed or velocity of the abrasives and the like.

FIG. 3illustrates, by means of a flow chart, one example of a method with which surface defects may automatedly be detected (identified as such and localized) and automatedly classified in accordance with specifiable criteria. The further machining of the surface for repairing (sport-repair) the defect depends on the classification of the defect (cf. the explanations concerningFIG. 4). In a first step, the automated surface inspection is performed (seeFIG. 3, step S1) in order to detect potential surface defects (defect candidates) (e.g. by use of image processing techniques as such known) and to obtain, for each defect candidate and with use of 3D measurement of the workpiece surface, a point cloud (seeFIG. 3, step S2), which represents the workpiece surface in the area of a surface defect. In this way a set D of N surface defects is determined (D={D1, D2, . . . , DN}). A measurement system, that is suitable for this has already been discussed with reference toFIG. 1. In a further step, a surface reconstruction is carried out, i.e. a three-dimensional reconstruction of the workpiece surface (seeFIG. 3, step S3) to determine the structure (topography) of the respective defect candidate.

When a defect Diis detected on the surface of a workpiece10, it is parametrized in accordance with the method described herein (seeFIG. 3, step S4). That is, a set Piof characteristic parameters which abstractly describe the topography of the defect Diis assigned to each defect Di. In a simple case, the parameter set Pimay include the lateral (along the workpiece surface) extension dias well as the extension tiperpendicular to the workpiece surface (Pi={di, ti}). The lateral extension dimay designate, e.g., the length of a scratch or the diameter of an (approximately circular) bulge (e.g. due to a drop of paint) on the painted surface. The extension timight designate the depth of a scratch or the height of a bulge (with reference to the ideal workpiece surface). A more complex parametrization is possible dependent on the application. In addition to the extension of a surface defect, the steepness of a defect may also be relevant to the subsequent machining. This may be, e.g., characterized by a parameter of the set Piand may be, for example, the ratio of the area of a surface defect with respect to its height or depth tior the ratio ti|dior an average gradient (slope) of the surface structure in the area of the defect. Further, the position and the orientation of a defect Diare represented by a point Oion the surface of the workpiece and the respective normal vector ni. The point Oimay designate, e.g., approximately the “center” (e.g. the centroid) of a surface defect. The normal vector nidefines a plane Ei, which is also referred to as defect plane (see alsoFIGS. 4 and 6).

A, so to speak, evaluation of the surface defects with regard to various criteria is carried out with the categorization of the surface defects (defect candidates). In practice, relevant or useful criteria for the categorization of surface defects may be, e.g., the distinction of defects with regard to size categories (e.g. very small, small, medium, large), the distinction of defects with regard to their lateral extension (e.g. defined by the average or maximum radius of the defect), the distinction of flaws with regard to their extension perpendicular to the workpiece surface (e.g. an encapsulation (bulge) with a height of more than 5 μm, a crater (dent) with a depth of more than 10 μm, etc.).

Whether or not a detected surface defect (defect candidate) needs to be machined at all may also be made to depend on various criteria. Possible criteria for this are, e.g. the number of flaws of a specific category within a defined zone of the workpiece. For example, a single surface defect may be accepted, while, when a plurality of surface defects appear (or a specific number of surface defects), at least so many of these must be machined until the maximum allowable number is achieved. Similarly, a machining of surface defects may be made dependent on whether they appear cumulatively (i.e. when more than a specific number of defects appear within a spatially confined area of the workpiece surface). Seen individually, a very small defect would not be relevant. When, however, too many (not relevant if seen individually) very small defects are within a specific distance to each other, then these together are no longer irrelevant and have to be considered in the machining process. Based on these criteria, for example, some defect candidates may be removed from the list of defects to be machined. The method steps illustrated by inFIG. 3may be performed at least partly by the data processing device50shown inFIG. 1. In particular, the data processing device50may be configured to assign the localized defects D1, D2, . . . , DN, based on their three-dimensional topography, to specific defect categories K1, K2, . . . , KN.

As mentioned above, each defect category Kjis associated with exactly one machining process Rjwhich may include one or more machining steps, wherein in each machining step the tool is moved, by use of a manipulator, along at least one machining path (seeFIG. 2, tool24, manipulator34). These machining paths are stored (e.g. in the mentioned database) in the form of templates, which are defined in a plane (the defect plane) independently from the actual geometry of the workpiece. A template Xiis composed of a plurality of points Xi1, Xi2, etc. which—in order to obtain the actual machining path Xi′ from the template Xi—are projected from the defect plane Eionto the workpiece surface (in accordance with the CAD model). The projected points Xi1′, Xi2, etc. form the actual machining path Xi′ for a specific machining step of a machining process Rjfor the machining of a defect Diof category Kj. A normal vector ni1′, ni2′, etc. is associated with each point Xi1′, Xi2′, etc. Between two projected points the path may be completed, e.g. by use of spline interpolation. This approach is outlined inFIG. 4. During the machining, the machining tool is always pressed onto the workpiece10perpendicular to the workpiece surface with a defined, adjustable force.

The flow chart ofFIG. 5shows one example of the generation of a robot program for the machining of surface defects starting at the selection of a machining process Rjdependent on the defect category Kj(seeFIG. 5, step S6). A machining process Rjmay include one or more machining steps each with one or more respective machining path templates Xi. Each of the templates Xiis composed of a set of points (at least two points) Xi1, Xi2, etc. To calculate the actual machining path Xithe points of the template are projected (seeFIG. 5, step S7) from the defect plane Eionto the workpiece surface (in accordance with the CAD model). The projected points Xi1′, Xi2′, etc. and interjacent points, e.g. intermediate points determined by interpolation, render the desired machining path (seeFIG. 5, step S8). The transition paths between two machining paths (within one or more machining steps of a process Rjor between the last path of a process for machining the defect Diand the first path for machining the next defect Di+1) may be calculated using well-known automated path planning methods (seeFIG. 5, step S9). From the thus planned machining and transition paths one or more robot programs may be automatedly generated with computer assistance using well-known techniques (seeFIG. 5, step S10). The method steps illustrated by inFIG. 5may be performed at least partly by the data processing device50shown inFIG. 1. In particular, the data processing device50may be configured to perform the mentioned projection of the templates, the mentioned interpolation, the automated path planning to obtain the transition paths, as well as the automated robot program generation.

FIG. 6schematically shows a template Xifor the determination of a machining path Xi′ (cf. projection in accordance withFIG. 4) of a machining process Rjfor the machining of a defect Diof category Kj(the line A-A′ represents the sectional plane illustrated inFIG. 4). The template may be adapted to the defect Didependent on its lateral extension, e.g. by means of transformation by shifting, rotating, scaling or skewing or an arbitrary combination of shifting, rotating, scaling and skewing. A problem may occur when two defects Di, Dklie so closely side by side that the machining paths of the processes for the machining of the two defects Di, Dkintersect. The machining area of a machining process Rjis that area of the workpiece surface which is actually machined by the tool during the machining process Rj. When the machining paths belonging to different machining processes RjRk, lie too closely side by side such an overlap may occur. Whether an overlap (i.e. a collision of two machining processes) will occur can be determined during the projection (FIG. 5, step S8). In the event of an overlap, two options exist: in the event of two neighboring defects Di, Dkof the same category Kj, it may be checked (with the use of software), whether both defects Di, Dkcan be repaired simultaneously in one process by applying a transformation (shift, rotation, scaling, and/or skew) to the template (seeFIG. 7); in the event of two neighboring defects Di, Dkof different categories, it may be checked (with the use of software), whether an overlap can be avoided when applying a transformation to the respective templates (seeFIG. 8).

Dependent on the geometry of the workpiece, certain areas of the workpiece surface may not be able to be machined (e.g. design edges and the like). Such “forbidden areas” of the workpiece surface may be marked in the CAD model, for example, as a set of edges (depicted as spread lines), which must not overlap with a machining area (seeFIG. 9, edge11). Whether this is the case (i.e. an overlap exists) may be checked during the projection of the template onto the surface of the CAD model (FIG. 5, step S8). Also in this case, an attempt may be made to avoid an overlap by use of a transformation (shift, rotation, scaling, skew) of the respective template. This situation is illustrated inFIG. 9. The defect Diis far enough away from the edge11so that a transformation of the template is not necessary. To calculate the machining path of the process for machining the defect Dk, the machining path has been shifted and skewed in the present example to avoid an overlap with edge11. The approach is substantially the same as before in the example ofFIG. 8.

FIG. 10exemplarily shows a portion of a machining process for machining a surface defect Di. The surface geometry corresponds to the illustration ofFIG. 6. One can see the points Xi1and Xi2of the machining path projected onto the surface and the respective position of the tool24(at point Xi1at time t1and at point Xi2at time t2). The tool is aligned by the manipulator24such that the force F, which is exerted by tool24onto the surface of workpiece10, is always effective normal to the direction of the respective surface (ni1′ or ni2′). An actuator25, acting between tool24and TCP of the manipulator34, allows for an arbitrary regulation of the force F in accordance with specifications which are stored in the mentioned database for a specific machining process.

FIG. 11is a block diagram illustrating one example structure of a system for the automated detection of defects in a workpiece surface and generation of a robot program for the machining of the workpiece.FIG. 11partly corresponds to the system ofFIG. 1. However, emphasis is placed on the interaction between the components and the data processing device50. As already mentioned above, the optical inspection system is, as such, known and commercially available, e.g. the system reflectCONTROL of Micro-Epsilon Messtechnik. As inFIG. 1, the robots31,32,33carry the sensors21,22, and23, which were already explained in detail above and, therefore, reference is made to these explanations. As mentioned, the data processing device50may be an entity including hardware, software or any combination thereof which is suitable to perform and/or control the methods described herein, in particular with reference toFIGS. 3 and 5. The data processing device50may be a workstation computer or a personal computer. As such, the data processing device50may include one or more processors52as well as a memory53configured to store data including processor instructions54and other data (e.g. the measured point clouds obtained from the sensors21,22,23, data derived therefrom and related data).

The data processing device can communicate with the sensors21,22, and32as well as with the robots31,32, and33(e.g. via the robot controller40). For this purpose the processing device50may include one or more communication interfaces51, which allow data transmission to and from the sensors21,22, and32, e.g. via a communication bus25, and to and from the robot controller40, e.g. via communication bus41. The term “communication bus” includes any known hardware and a respective communication protocol that allows the data processing device to communicate with the sensors and the robot controller40. For example, the communication busses may be implemented as field busses or serial busses, such as Universal Seral Bus, or packed based communication busses such as Ethernet or the like. Alternatively, wireless communication may be used instead of wired connections. Although the present example shows different busses for the communication with the sensors and the robot controller, a single bus system (e.g. a network) may be used instead.