Method for calibrating a camera-laser-unit in respect to a calibration-object

The invention refers to a method for calibrating a camera-laser-unit (1) with respect to at least one calibration-object (12) disposed at a given position and orientation in a three-dimensional space (13). The camera-laser-unit (1) comprises a laser (4) and a camera (3), wherein the laser (4) and the camera (3) are disposed at a given distance with respect to each other. An optical axis (9) of the laser (4) and an optical axis (8) of the camera (3) subtend a given angle (α). The camera-laser-unit (1) is adapted to record the location, form and/or dimensions of a measurement-object (5). The method has the advantage that the same calibration-object (12) can be used for the calibration of the camera (3) and of the laser (4), wherein the camera (3) is first calibrated using a Tsai-algorithm and then the laser (4) is calibrated with respect to, and by making use of, the already calibrated camera (3).

This application is the national stage of PCT/EP2004/09892 filed on Sep. 4, 2004 and also claims Paris Convention priority of EP 03023402.5 filed on Oct. 17, 2003

BACKGROUND OF THE INVENTION

The present invention refers to a method for calibrating a camera-laser-unit with respect to at least one calibration-object disposed at a given position and orientation in a three-dimensional space. The camera-laser-unit comprises at least one laser (light-amplification by stimulated emission of radiation) and at least one camera. The laser and the camera are disposed in a given distance with respect to one another. An optical axis of the laser and an optical axis of the camera subtend a given angle. The camera-laser-unit is adapted to record the location, form and/or dimensions of a measurement-object.

The invention also concerns a calibration-object used for calibrating a camera-laser-unit and disposed at a given position and orientation within a three-dimensional space. The camera-laser-unit comprises at least one laser and at least one camera. The laser and the camera are disposed at a given distance and an optical axis of the laser and an optical axis of the camera subtend a given angle. The camera-laser-unit is adapted to record the location, form and/or dimensions of a measurement-object.

Camera-laser-units of the kind mentioned above are well known in the art. For example, such a camera-laser-unit is produced and sold by Perceptron, Inc., 47827 Halyard Drive, Plymouth, Mich. 48170, USA. Furthermore, such a camera-laser-unit and the use of such a unit is described in detail in the U.S. Pat. Nos. 4,645,348 and 6,166,811 both issued to Perceptron, Inc. A preferred embodiment of a camera-laser-unit of the kind mentioned above is described in detail in the German patent application 103 11 247, filed on Mar. 14, 2003 by the applicant of the present patent application. The publications of the two US-patents and of the German patent application are hereby incorporated by reference.

According to the methods known in the art, calibration of a camera-laser-unit is very complex and time-consuming. To begin with, a first calibration-object has to be disposed with respect to the camera-laser-unit and the camera is calibrated with respect to the first calibration-object by, for example, making use of a Tsai algorithm, which is well known in the art. Thereafter, a second calibration-object has to be disposed with respect to the camera-laser-unit and the laser is also calibrated with respect to the second calibration-object.

The second conventional calibration-object typically comprises a planar object with an array of pins protruding from the plane. The pins represent a non-coplanar calibration-pattern used for calibrating the laser. The dimensions and distances of the pins are known. The problem is that the pins, when illuminated by the laser or any other light source, form shadows on the adjacent pins, the shadows rendering very complicated and even impossible an accurate calibration of the laser.

Therefore, it is known to make the pins retractable and to extract them one by one from the planes of the cubic second calibration-object. After the extraction of each pin, a calibration routine is run and then the next pin is extracted with another calibration routine run thereafter, and so on. Thus, the calibration of the laser consists of numerous calibration routines. It may be well understood that in this way calibration of the laser is very cumbersome and requires a relatively long time. Typically, the calibration of the laser with the described method will take a couple of hours. Furthermore, the known calibration-method may be used only in laboratories or other places where the calibration assembly required for the calibration process may be built up. It is certainly not suited for calibration of lasers during the intended use of the camera-laser-unit.

One way for simplifying the calibration of the laser could be to reduce the number of pins. However, reducing the number of pins would reduce accuracy of the calibration.

It is an object of the present invention to simplify the process of calibration of a camera-laser-unit without reducing the associated accuracy.

SUMMARY OF THE INVENTION

This object is achieved by a method for calibrating a camera-laser-unit comprising the features of the independent method claim. In particular, a method of the above mentioned kind is suggested comprising the steps of:selecting a calibration-object having at least two planes disposed in a given angle with respect to each other and provided with a given non-coplanar calibration-pattern;disposing the calibration-object with respect to the camera-laser-unit at a given position and orientation in the three-dimensional space, wherein the orientation of the calibration-object is such that light emitted by the laser is visible on at least two planes of the calibration-object;calibrating the camera with respect to the calibration-object using a Tsai algorithm;activating the laser to emit light visible on the at least two planes of the calibration-object;recording the light on the planes by the camera;determining the laser-properties from the light recorded by the camera; andcalibrating the laser according to the determined laser-properties.

It is a main aspect of the present invention that the same calibration-object is used for the calibration of the camera as well as for the calibration of the laser. Furthermore, after calibration of the camera with respect to the calibration-object, the laser is calibrated with respect to the (already calibrated) camera. In other words, the camera is used for calibrating the laser.

The whole procedure for calibrating a camera-laser-unit takes very little time and can be accomplished within a few seconds. This makes it the ideal procedure for calibration of a camera-laser-unit during its intended field use. For example, the application of the camera-laser-unit (recording the location, form and/or dimensions of a measurement-object) could be interrupted for a short period, an industrial robot carrying the camera-laser-unit could move the unit to the calibration-object and dispose it in front of the object. Then the calibration process according to the present invention is run. Thereafter, the robot would move the camera-laser-unit away from the calibration-object back to the measurement-object and the intended use of the unit would be resumed.

According to a preferred embodiment of the invention it is suggested that calibrating the laser comprises the step of defining a relative position and orientation of the laser with respect to a coordinate frame associated with the calibration-object, wherein the coordinate frame associated with the calibration-object is in a given position and orientation in the three-dimensional space.

Concerning the calibration of the laser, it is suggested thatthe light emitted by the laser is visible on the planes of the calibration-object as a line on each plane, the lines intersecting on a contact line of the two planes; andthe laser-properties are determined from the lines recorded by the camera by means of a line detection algorithm.

Visible in the sense of the present invention means that the light emitted by the laser is visible to the camera. So this definition would comprise, for example, a case where the laser emits infrared (IR) light, invisible to the human eye but visible to an IR camera.

In other words, three points can be defined for the calibration of the laser, the two end points of the lines on the two planes of the calibration-object and the point of intersection of the two lines.

Further, it is suggested thatthe light emitted by the laser is visible on the planes of the calibration-object as a line on each plane, the lines intersecting on a contact line of the two planes;a laser-plane is defined by the optical axis of the laser and the lines visible on the planes of the calibration-object; andin order to calibrate the laser according to the determined laser-properties, the position and orientation of the laser-planes with respect to a coordinate frame associated with the calibration-object is defined.

Concerning the calibration of the camera it is suggested that the method comprises the step of defining a relative position and orientation of the camera with respect to a coordinate frame associated with the calibration-object, wherein the coordinate frame associated with the calibration-object is at a given position and orientation in three-dimensional space.

According to another embodiment of the present invention, a transformation matrix is defined in dependence on the relative position and orientation of the camera with respect to a coordinate frame associated with the calibration-object, the relative position and orientation of the laser with respect to a coordinate frame associated with the calibration-object, and optionally on internal camera parameters. The picture taken from the measurement-object by the camera is superposed or multiplied with the transformation matrix in order to obtain the actual location, form and/or dimensions of the measurement-object.

According to yet another embodiment of the present invention, the camera-laser-unit to be calibrated is grasped by an industrial robot and disposed at a given position and orientation in the three-dimensional space with respect to the calibration-object, wherein the orientation of the camera-laser-unit is such that light emitted by the laser is visible on at least two planes of the calibration-object.

The object of the present invention is also achieved by a calibration-object of the above-mentioned kind characterized in that it comprises two planes disposed at a given angle to each other, provided with a non-coplanar calibration-pattern on each plane comprising an array of features, and the calibration-object is used for calibration of the camera as well as for calibration of the laser. The array of features represents a grid of calibration points used for calibrating the camera as well as the laser of the camera-laser-unit.

According to a preferred embodiment of the present invention, the angle between the two planes is a right angle. The light emitted by the laser is visible on the planes of the calibration-object as a line on each plane, the lines intersecting on a contact line of the two planes. In this way, the maximum possible volume in space can be calibrated.

The features of the calibration-object may have recesses, in particular cavities with a circular cross section, or prints on the planes of the calibration-object. The features may have any shape or design. For example, the features may have rectangular recesses or prints with a square or a circular cross section, as viewed from above. The prints on the planes have a different color or a different surface than the planes of the calibration-object, so that the features and the calibration pattern can be detected by the camera during the calibration process.

Further embodiments as well as further advantages of the present invention are outlined in the following description of the enclosed figures.

DESCRIPTION OF THE PREFERRED EMBODIMENT

The present invention refers to a camera-laser-unit for recording the location, form and/or dimensions of a measurement-object. The camera-laser-unit shown inFIG. 2as a whole is designated with reference sign1. It comprises mounting means2with a camera3and a light source4, in particular a laser, mounted thereon. The laser4illuminates part of the measurement-object5and the camera3takes an image of the illuminated measurement-object5. The camera3comprises a CCD (charged coupled device)-chip6or other means (e.g. a CMOS-chip) for converting optical signals of the image into electrical signals for further processing within the camera-laser-unit1. Furthermore, the camera3comprises an optical system7, in particular a lens, for reproducing the image on the CCD-chip6. The camera3and the laser4are disposed on the mounting means2such that an optical axis8of the camera3intersects an optical axis9of the laser4at a measurement point10. The measurement point10is not necessarily disposed on a contact line between the two planes14,15. Preferably the optical axis8and9are disposed in an angle α of 45° with respect to one another. The laser4generates a line11on the measurement object5, which is recorded by the camera3and reproduced on the CCD-chip6by the lens7.

For example, a camera-laser-unit of the mentioned kind is produced and sold by Perceptron, Inc., 47827 Halyard Drive, Plymouth, Mich. 48170, USA. Furthermore, such a camera-laser-unit and the use of such a unit is described in detail in the U.S. Pat. Nos. 4,645,348 and 6,166,811 both issued to Perceptron, Inc. A preferred embodiment of a camera-laser-unit of the kind mentioned above is described in detail in the German patent application 103 11 247, filed on Mar. 14, 2003 by the applicant of the present patent application. The publications of the two US-patents and of the German patent application are hereby incorporated by reference into the present specification, in particular for more detailed information on possible structures of camera-laser-units and on the process of determining the location, form and/or dimensions of a measurement-object by means of a camera-laser-unit.

However, before the camera-laser-unit1can be used for recording the location, form and/or dimensions of a measurement-object5, it has to be calibrated. During the calibration process the camera3and the laser4are put into relationship with the coordinates of a calibration-object12(seeFIG. 1) and a three-dimensional space13, with the calibration-object12being disposed at a given position and orientation within the three-dimensional space13. In particular, camera parameters are determined and a relative position of one or more laser planes is defined with respect to a coordinate frame19related to the calibration-object12. A laser plane is defined by the optical axis9and the line11of light emitted by the laser4(seeFIG. 2or3). Calibration of the camera-laser-unit1is necessary in order to allow the camera-laser-unit1to determine absolute values for the location, form and/or dimensions of the measurement-object5.FIG. 1also shows that the camera-laser-unit1to be calibrated can be grasped by an industrial robot50and disposed relative to the calibration-object12in a given position and orientation in three-dimensional space13. The orientation of the camera-laser-unit1is such that light11emitted by the laser4is visible for the camera3on at least two planes14.15of the calibration-object12.

FIG. 3shows the calibration-object12according to the present invention with its two planes14,15facing the camera-laser-unit1. Any object12comprising two calibration planes14,15is suitable for the calibration process according to the invention. A non-coplanar calibration-pattern comprising an array of recesses17is disposed on the two planes14,15. Of course, the calibration-object12can comprise an array of any kind of features, such as simple prints on the planes14,15of any desired shape and cross section, instead of the recesses17.

In the embodiment shown inFIG. 3the calibration-pattern comprises a six by six array of 36 grid-like disposed recesses17on each plane14,15. Each recess17of the calibration-pattern is designed as a cavity having a circular cross section viewed from above. The same calibration-object12is used for calibration of the camera3as well as for calibration of the laser4. The two planes14,15are disposed at a known angle β, preferably 90°. The closer to perpendicular the two planes14,15are, the larger is the volume over which the calibration is valid, so larger angles β could be used if volume is not a critical factor.

The present invention refers to a particularly advantageous process for calibrating the camera-laser-unit1with respect to the calibration-object12disposed at a given position and orientation within the three-dimensional space13. The process comprises the following steps:Selecting a calibration-object12comprising at least two planes14,15disposed at a given angle β with respect to each other and provided with a given non-coplanar calibration-pattern (seeFIG. 3). Preferably, the angle β is 90°.Disposing the calibration-object12at a given position and orientation in three-dimensional space13with respect to the camera-laser-unit1, wherein the orientation of the calibration-object12is such that light emitted by the laser4is visible for the camera3on at least two planes14,15of the calibration-object12.In a first step, calibrating the camera3with respect to the calibration-object12using a Tsai algorithm. The Tsai algorithm is well known for camera calibration. It is described in detail in M. Tapper, Ph. J. McKerrow, J. Abrantes: “Problems Encountered in the Implementation of Tsai's Algorithm for Camera Calibration”, Proc. 2002 Australasian Conference on Robotics and Automation, Auckland, 27-29 Nov. 2002, which is hereby incorporated by reference.In a second step calibrating the laser4with respect to the (already calibrated) camera3, and activating the laser4in order to make it emit light (line11) visible as two lines11a,11bon the two planes14,15of the calibration-object12,recording the light (lines11a,11b) on the planes14,15by the camera3and reproducing the image (lines16a,16b) on the CCD-chip6by the camera lens7,determining the laser-properties from the light16recorded by the camera3, andcalibrating the laser4according to the determined laser-properties.

The laser-properties are determined from the properties (length and angle γ) of the lines16a,16breproduced on the CCD-chip6of the already calibrated camera3by using a line detection algorithm. The two lines11a,11bcan be easily reconstructed in space from the images16a,16busing the camera parameters and the known geometry of the calibration object12. Once the two three dimensional lines11a,11bare determined, the information is adequate to reconstruct the laser plane(s)

According to the present invention, calibration of the camera-laser-unit1is performed in two steps:First, standard camera calibration is performed to define the internal parameters of the camera3and the position of the camera3with reference to a coordinate system.Second, the position of the laser plane (defined by the optical axis9and the line11) is defined with reference to the same coordinate system.

These two sets of parameters—camera parameters and laser plane equations—provide all the information necessary for performing projection and reconstruction using the camera-laser-unit1(e.g. transformation from image pixels18of the CCD-chip5(seeFIG. 3) into millimeters (mm) and vice versa.

Making reference to the flow chart ofFIG. 4, the method for calibrating the camera-laser-unit1according to the present invention will now be described in detail. The first step of the calibration process is to calibrate the camera3(step20) using an image21and the geometry22of the calibration object12and nominal camera parameters23. The calibration of the camera is done by using a non-coplanar Tsai calibration algorithm. As a result of the camera calibration the camera parameters24are obtained.

The second step of the calibration process is to determine the laser plane(s) (step25) defined by the optical axis9and the line11of light emitted by the laser4. This can be accomplished by detecting the points belonging to the laser lines11a,11bvisible on the calibration-object12. The laser line(s)11a,11bare extracted (step26) from the laser line image27taken by the camera3. The geometry22of the calibration object12, the camera parameters24and the extracted laser lines26are used for determining the laser plane(s)25. As a result of the determination of the laser planes(s) the laser plane(s)28are obtained. Output29is calculated from the camera parameters24and the laser plane(s)28. The output is, for example, a transformation matrix.

FIG. 5shows an example for a calibration-image comprising two laser planes each with lines11aand11bon the respective planes14and15of the calibration object12. Provided that the scene illumination is not too bright (to avoid saturation) and that the surface of the calibration object12is not too reflective, the laser lines11aand11bcan be easily distinguished.

The detection of the laser lines11aand11bcomprises the following steps:Automatically defining or detecting the regions in the image recorded by the camera3covered by each plane14,15;For each region:scanning the region horizontally or vertically, depending on the relative positions of camera3and laser4and detecting the brightest pixel along each line or column. For multiple laser planes (seeFIG. 5), multiple points can be detected, provided that some estimation for the minimum and maximum distance between the lines is known (in pixels). This estimation can either be defined by the user, or calculated automatically from the angle between two adjacent laser planes and the calibration planes'14,15positions.For each detected pixel, detecting the center of intensity on its neighborhood, using the weighted average of pixel intensities.For each laser plane, fitting a line on the detected points, using a linear regression algorithm.Calculating the distance of each detected point from all detected lines11a,11b. If the point is closer to some other line11a,11b, then this point is misclassified, so it has to be moved to the correct set of points.

One approach to define a laser plane once a pair of laser lines11a,11bhas been extracted (step26), is to reconstruct the lines in three dimensional space, using the camera calibration parameters24and the plane equations of the calibration planes14,15. The two reconstructed lines have adequate information to determine the plane of the laser4in the same coordinate system. A second approach that has proved to be more accurate, is to reconstruct all detected points (using the camera parameters24and the equations of the calibration planes14,15), and iteratively fitting a plane to the reconstructed set of points, discarding at each step the most distant points. The position and orientation of the two planes14,15can be easily defined with respect to a common coordinate system, e.g. centered at the point of intersection10of the two lines11a,11b.

As described above, calibration of a camera-laser-unit1with any number of laser planes can be performed using a three dimensional calibration-object12, comprising at least two calibration planes14,15. Of course, the complexity of the procedure would increase if objects12with more than two planes were considered. The main advantages of the method according to the present invention are:The same camera model that is used for the calibration of the camera3can be used for the calibration of the laser4allowing a straight forward integration of the laser calibration into existing camera calibration processes.Calibration of the camera3is performed by using an already existing, well known and highly tested Tsai-based algorithm and appropriate software.The topology of the calibration-object12is easy to construct with very high accuracy.High accuracy of calibration can be guaranteed over an entire volume covered by the calibration object12.The camera-laser-unit1can be easily and directly calibrated in any coordinate frame19(if the position of point of origin0of the frame19is known in the global coordinate frame13), without the need for time-consuming alignment processing.