Patent Description:
During the operation of an assembly line, the pose of a particular part on the body of different vehicles may over time drift away from a target pose. Such offsets from a target pose may be caused by part variations of parts (provided e.g. by a separate supplier), mechanical tolerances within the assembly line, notably by mechanical tolerances of an assembly robot, mechanical tolerances of an end-of-arm gripper frame and tooling variation over time and/or of a camera used for controlling the positioning process.

<CIT> describes a method for calibrating a robot system, in order to enable the robot system to grab a workpiece. <CIT> describes a method for positioning error compensation during manufacturing of complex-shaped gas turbine engine parts. Academic publication titled "<NPL> refers to the problem of compliance of the components in robotic assembly tasks.

The present document is directed at the technical problem of increasing the precision, with which parts are mounted to the body of a product, notably a vehicle, within an assembly line.

According to an aspect a method for controlling an assembly step of a part onto a body of a product, which is performed by a robot, is described. The method comprises determining deviation data at a plurality of measurement points of an already assembled part, wherein the deviation data indicates for each measurement point a deviation in a measurement direction from a target position of the measurement point. Furthermore, the method comprises determining offset data based on the deviation data and based on a transformation matrix for the plurality of measurement points and the plurality of measurement directions. In addition, the method comprises adjusting operation of the robot for assembling a subsequent part based on the offset data, in order to reduce the deviation for at least one of the measurement points.

According to a further aspect, a control unit for controlling an assembly robot which is configured to attach a part onto a body of a product is described. The control unit is configured to determine deviation data at a plurality of measurement points of an already assembled part, wherein the deviation data indicates for each measurement point a deviation in a measurement direction from a target position of the measurement point. Furthermore, the control unit is configured to determine offset data based on the deviation data and based on a transformation matrix for the plurality of measurement points and the plurality of measurement directions. In addition, the control unit is configured to adjust operation of the robot for assembling a subsequent part based on the offset data, such that the deviation for at least one of the measurement points is reduced (for the subsequent part, compared to the already assembled part).

It should be noted that the methods and systems including its preferred embodiments as outlined in the present document may be used stand-alone or in combination with the other methods and systems disclosed in this document. In addition, the features outlined in the context of a system are also applicable to a corresponding method. Furthermore, all aspects of the methods and systems outlined in the present document may be arbitrarily combined. In particular, the features of the claims may be combined with one another in an arbitrary manner.

As outlined in the introductory section, the present document is directed at increasing the assembling precision of an assembly line and/or at maintaining a high level of assembling precision for an assembly line. In this context, <FIG> shows the body <NUM> of a vehicle <NUM>, notably of a car, as an example for a general product which is assembled within an assembly line. Typically, an assembly line is used for assembling a sequence (e.g. thousands) of different vehicles <NUM> (which are at least partially identical to one another and/or which are of the same type).

Furthermore, <FIG> highlights a place <NUM> within the body <NUM>, where a particular part (e.g. a door) is to be fixed to the body <NUM> within the assembly line. In addition, <FIG> shows a vehicle coordinate system <NUM> which is positioned at a vehicle reference point <NUM>. The vehicle reference point <NUM> may be the center of the front axis of the vehicle <NUM>. The vehicle coordinate system <NUM> may comprise an x-axis (pointing from the front to the back of the vehicle <NUM>), a y-axis (pointing from the left to the right of the vehicle <NUM>) and a z-axis (pointing from the bottom to the top of the vehicle <NUM>).

<FIG> shows an excerpt of an assembly line. In particular, <FIG> illustrates a robot <NUM> which is configured to position and/or to fix a part <NUM> (notably a door) at a particular place <NUM> within the body <NUM> of the vehicle <NUM>. The part <NUM> may be fixed within an actual pose that is offset with regards to a target pose <NUM> for the part <NUM>. In particular, the actual pose may be rotated <NUM> and/or translated <NUM> with regards to the target pose <NUM>. Hence, the offset between the actual pose and the target pose <NUM> may comprise K = <NUM> components or degrees of freedom, notably three components for the rotation <NUM> and three components <NUM> for the translation.

The assembly step of the robot <NUM> may be supervised and/or guided by data provided by one or more assembly sensors <NUM> (notably one or more cameras). The one or more assembly sensors <NUM> and/or the robot <NUM> may use an assembly coordinate system <NUM> which may deviate in an undetermined manner from the vehicle coordinate system <NUM> (e.g. due to mispositioning of the body <NUM> of the vehicle <NUM> relative to the robot <NUM>). Such a mismatch between the coordinate systems <NUM>, <NUM> may cause an offset between the actual pose and the target pose <NUM> of a part <NUM>. Further reasons for an offset between the actual pose and the target pose <NUM> of an assembled part <NUM> may be mechanical tolerances of the robot <NUM> and/or of the orientation and/or position of the one or more assembly sensors <NUM>.

The actual pose of an assembled part <NUM> may be controlled using CMM (coordinate measuring machine) measurements. This is illustrated in <FIG>. A plurality of measurement devices <NUM> may be used to measure a deviation of a particular measurement point <NUM> from a target position (wherein the target position of a particular measurement point <NUM> is defined by the target pose <NUM> for the assembled part <NUM>). A measurement device <NUM> may be configured to determine the deviation of a measurement point <NUM> within a particular measurement direction <NUM> (which may be described e.g. by a three-dimensional vector). By way of example, a measurement device <NUM> may be configured to determine the deviation of a measurement point <NUM> using a laser, or radar or a movable needle.

Hence, using a plurality of measurement devices <NUM> for a plurality of different measurement points <NUM>, which are positioned on the assembled part <NUM> (wherein the plurality of measurement devices <NUM> may exhibit different measurement directions <NUM>), the actual pose of the assembled part <NUM> may be determined in a precise manner. This measurement data, i.e. the (scalar) deviations measured by the plurality of measurement devices <NUM>, may be used to adjust the operation of the robot <NUM>, in order to improve the assembling precision of a part <NUM> which is handled subsequently by the robot <NUM>.

As indicated above, the offset between the actual pose and the target pose <NUM> of an assembled part <NUM> may comprise a rotation <NUM>. Such a rotation can typically not be described in a precise manner relative to the the vehicle coordinate system <NUM> and/or relative to the vehicle reference point <NUM> (due to the relatively large distance between the assembled part <NUM> and the vehicle reference point <NUM>). In view of this, a part coordinate system <NUM> relative to a part reference point <NUM> may be defined. The part reference point <NUM> may correspond to a center point (e.g. to a center of gravity) of the assembled part <NUM>. The part coordinate system <NUM> may have preferably the same orientation as the vehicle coordinate system <NUM>. Hence, coordinates within the vehicle coordinate system <NUM> may be transformed into the part coordinate system <NUM> using a fixed (three-dimensional) coordinate offset. The measurement points <NUM> may be described within the part coordinate system <NUM>, thereby allowing the pose offset between the actual pose and the target pose <NUM> to be determined in a precise and efficient manner.

In the following, it is outlined how the pose offset may be determined based on the measurement data using a transformation, notably a Jacobian, matrix which is dependent on the location of the measurement points <NUM> and on the measurement directions <NUM> of the measurement devices <NUM>. The transformation matrix, notably an inverse of the transformation matrix, may be used to transform the deviations which are provided by the measurement devices <NUM> into a (six-dimensional) pose offset P. The pose offset P may then be used to adjust the operation of the robot <NUM>. By making use of a transformation matrix which depends on all available measurement points <NUM> and all available measurement directions <NUM>, the pose offset P may be determined in a precise manner.

It can be shown that for relatively small translations, the (translational) offset can be described as <MAT> wherein the partial deviations may be given by <MAT>.

In a similar manner, the (rotational) offset for a relatively small rotation may be given by <MAT> wherein × denotes the cross-product operation, and wherein the vector [x,y,z]T denotes the position of a measurement point <NUM> of a measurement device <NUM>.

As outlined above, a measurement device <NUM> typically only provides a (scalar) deviation measurement within one particular measurement direction <NUM>. In view of this, the Jacobian component should be projected onto the respective measurement direction <NUM>, as shown in the following equations: <MAT> <MAT> wherein <MAT> is a vector indicative of the measurement direction <NUM> of a measurement device <NUM> and wherein ° is the projection operator.

Using the above mentioned formula, a partial transformation and/or Jacobian matrix Jn may be determined for each measurement device <NUM>, using the coordinates <MAT> of the measurement point <NUM> of the measurement device <NUM>. The coordinates may be given relative to the part reference point <NUM>. For n = <NUM>,. , N measurement devices <NUM>, N Jacobian matrices Jn may be determined. Furthermore, the N Jacobian matrices Jn may be projected onto the measurement directions <NUM> of the respective measurement devices <NUM> using the respective direction vectors <MAT> describing the respective measurement direction <NUM>, using CnT = VnTJn. The transformation and/or Jacobian matrix J which may be used for transforming the N dimensional vector of the scalar deviations dn of the N measurement devices <NUM>, i.e. D = [d<NUM>,. , dN]T, may be determined as J = <MAT>. This transformation and/or Jacobian matrix J may be inversed using a known inversion method to provide a (pseudo) inverse matrix J-<NUM>. The pose offset P = [Δx, Δy, Δz, Δrx, Δry, Δrz]T may then be determined as <MAT>.

The (pseudo) inverse matrix J-<NUM> for a particular type of part <NUM> may be determined in advance. Typically, different types of parts <NUM> have different (pseudo) inverse matrices J-<NUM>.

The pose offset P may be used to adjust the assembly process, notably the operation of the robot <NUM>, in order to ensure that the actual pose of an assembled part <NUM> is within a given tolerance band around the target pose <NUM> for the part <NUM>.

<FIG> shows a flow chart of an example method <NUM> for controlling an assembly step of a part <NUM> onto the body <NUM> of a product <NUM> (notably of a vehicle or a component of a vehicle), wherein the assembly step is performed by one or more robots <NUM>. The method <NUM> may be performed by a control unit <NUM> for controlling the one or more robots <NUM> (see <FIG>). The method <NUM> may be used to increase the quality of an assembly process. Alternatively or in addition, the method <NUM> may be used to control the supply quality of a particular part <NUM> (which may be provided by a particular supplier). The offset data which is determined in the context of method <NUM> may be used to reduce variations of a particular part <NUM>. For this purpose, the offset data may be provided to the supplier of the particular part <NUM>.

The method <NUM> comprises determining <NUM> deviation data at a plurality of measurement points <NUM> of an already assembled part <NUM>. The deviation data may comprise coordinate measuring machine (CMM) measurement data. In particular, the deviation data may indicate for each measurement point <NUM> a (scalar) deviation dn in a measurement direction <NUM> from a target position of the measurement point <NUM>. In other words, for each measurement point <NUM>, it may be indicated how much the measurement point <NUM> deviates from a target position of the measurement point <NUM>. The deviation may be indicated for the measurement direction <NUM> of the measurement device <NUM> which is measuring the deviation.

The measurement directions <NUM> may at least partially differ for the different measurement points <NUM>. By using different measurement directions <NUM> different degrees of freedom of a pose offset may be determined. In particular K different measurements with K different degrees of freedom may be used for determining a pose offset P with K degrees of freedom.

Furthermore, the method <NUM> comprises determining <NUM> offset data based on the deviation data and based on a transformation matrix for the plurality of measurement points <NUM> and the plurality of measurement directions <NUM>. The transformation matrix may comprise or may depend on a Jacobian matrix (with partial deviations with regards to the different degrees of freedom of the pose offset). In particular, the transformation matrix may be configured to determine the deviation dn in a measurement direction <NUM> at a measurement point <NUM>, which is caused by a pose offset P of the part <NUM> relative to the target pose <NUM> of the part <NUM>. The target position of the measurement point <NUM> typically dependents on the target pose. Typically, the transformation matrix is dependent on coordinates Qn of the plurality of measurement points <NUM> and on the plurality of measurement directions <NUM>.

In addition, the method <NUM> comprises adjusting <NUM> operation of the robot <NUM> for assembling a subsequent part <NUM> (of a subsequent product <NUM>) based on the offset data, in order to reduce the deviation for at least one of the measurement points <NUM> (for the subsequent part <NUM>). The subsequent part <NUM> may be fixed to the body <NUM> of a subsequent product <NUM> of the same product type as the previously assembled product <NUM>. Adjusting <NUM> the operation of the robot <NUM> may comprise adjusting a movement trajectory of the robot <NUM> when assembling a part <NUM>, and/or adjusting, notably rotating and/or translating, a coordinate system <NUM> used by the robot <NUM> when assembling a part <NUM>.

By making use of a transformation matrix which takes into account the position of the different measurement points <NUM> and the different measurement directions <NUM>, offset data may be determined in an efficient and precise manner. The offset data may be directly applied for adjusting the operation of the one or more assembly robots <NUM>, thereby achieving and maintaining a high level of assembly quality.

A measurement point <NUM> may be described by coordinates within a part coordinate system <NUM> relative to a part reference point <NUM>. The part reference point <NUM> may be located closer to the assembled part <NUM> than a general product reference point <NUM> of a product coordinate system <NUM> which is used for describing the position of various different parts <NUM> within a product <NUM>. In particular, the part reference point <NUM> may be located within the assembled part <NUM>, notably at a midpoint and/or a center of gravity of the assembled part <NUM>, in case the assembled part <NUM> is fixed to the body <NUM> of the product <NUM> according to the target pose <NUM>. By making use of coordinates which are relative to a part reference point <NUM>, the offset data may be determined with an increased precision (compared to a situation where the coordinates are provided relative to the product reference point <NUM>).

Deviation data may be determined for N measurement points <NUM>, with N > <NUM>,<NUM>, or <NUM>. Furthermore, a pose of the assembled part <NUM> may comprise K degrees of freedom, with K = <NUM>,<NUM>,<NUM>, or <NUM>. In such a case, the transformation matrix may be a N × K matrix, thereby providing the N deviation values dn from the K components of the pose offset P.

Qn, n = <NUM>,. , N, may be coordinate vectors indicating positions of the N measurement points <NUM>, respectively. The transformation matrix may be dependent on Qn, n = <NUM>,.

Furthermore, Jn, n = <NUM>,. , N, may be partial transformation matrices for the N measurement points <NUM>, respectively. The transformation matrix may be dependent on Jn, n = <NUM>,. , N, wherein Jn may be a <NUM> × K Matrix. In particular, the partial transformation matrix Jn may be determined as <MAT> for K = <NUM>, wherein × is the cross product operator. As a result of this, the offset data may be determined in a precise manner.

Vn, n = <NUM>,. , N, may be vectors indicating the N measurement directions <NUM>, respectively. The vectors VnTJn may then indicate a projection of the partial transformation matrix Jn onto the measurement direction <NUM> which is indicated by Vn. The vector Vn TJn may be a <NUM> × K vector. The transformation matrix may be dependent on VnTJn, n = <NUM>,. As a result of this, the offset data may be determined in a precise manner.

In particular, the transformation matrix may be given by <MAT>.

The offset data may be determined based on an estimate of an inverse of the transformation matrix, wherein the estimate of the inverse of the transformation matrix may be denoted as J-<NUM>. The estimate of the inverse may e.g. by a Moore-Penrose inverse. In this case, the offset data P may be determined from the deviation data D using P = J-<NUM>D.

A product <NUM> may comprise a plurality of different parts <NUM> which are placed at different positions on the body <NUM> of the product <NUM>. The deviation data for the plurality of different parts <NUM> may be determined using at least partially different (differently located) measurement points <NUM>. Furthermore, the transformation matrices for determining the offset data for the plurality of different parts <NUM> may be different from one another. The different transformation matrices may be determined prior to execution of the method <NUM>, thereby enabling a resource efficient execution of the method <NUM>.

The method <NUM> may comprise determining the offset data for a sequence of different products <NUM> (of the same product type) that are assembled using the robot <NUM>. Furthermore, the method <NUM> may comprise determining, based on the offset data for the sequence of different products <NUM>, whether there is a systematic offset of the assembled part <NUM> from a target pose <NUM>. In particular, it may be determined whether an average of actual poses of the assembled parts <NUM> differs from the target pose <NUM> for the part <NUM> (by more than a predetermined tolerance). If this is the case, it may be determined that a systematic offset occurs. The step of adjusting <NUM> the operation of the robot <NUM> may be performed (notably only) if it is determined that there is a systematic offset. By doing this, the efficiency and the precision of the assembly process may be further increased.

Hence, the method <NUM> may be repeated as indicated by the closed loop in <FIG>. Consequently, the offset data may be determined for a sequence of parts <NUM> and/or products <NUM>, in order to continuously and/or repeatedly adjust the assembly process. By doing this, the precision of the assembly process may be increased in a robust manner.

Claim 1:
A method (<NUM>) for controlling an assembly step of a part (<NUM>) onto a body (<NUM>) of a product (<NUM>), which is performed by a robot (<NUM>); wherein the method (<NUM>) comprises,
- determining (<NUM>) deviation data at a plurality of measurement points (<NUM>) of an already assembled part (<NUM>); wherein the deviation data indicates for each measurement point (<NUM>) a deviation in a measurement direction (<NUM>) from a target position of the measurement point (<NUM>); wherein the measurement directions (<NUM>) at least partially differ for the different measurement points (<NUM>);
- determining (<NUM>) offset data based on the deviation data and based on a transformation matrix for the plurality of measurement points (<NUM>) and the plurality of measurement directions (<NUM>); and
- adjusting (<NUM>) operation of the robot (<NUM>) for assembling a subsequent part (<NUM>) based on the offset data, in order to reduce the deviation for at least one of the measurement points (<NUM>).