Patent ID: 12246458

DETAILED DESCRIPTION OF THE EMBODIMENTS

In reality in this technical field, there is a time lag between the time when the camera captures an image and the time when the control device takes in the image data due to a delay in communication of the image data. Hence, when image capturing is performed while a robot is operated, the position information of the robot at the time when the image data is obtained and the position information of the robot at the time when the control device obtains the image data are different.

Accordingly, there is a demand for accurately obtaining the position information of the robot at the time when the camera captures an image, even with a communication delay.

A robot system1according to an embodiment of the present disclosure will be described below with reference to the drawings.

As shown inFIG.1, the robot system1according to this embodiment includes a robot20that performs an operation on workpieces W, an image capturing device30fixed to the distal end of an arm21of the robot20, and a control device40. InFIG.1, reference sign10denotes a transport device, such as a conveyor, for transporting the workpieces W.

In the example shown inFIG.1, the robot20is a vertical six-axis articulated-type robot and has, at the distal end of the wrist provided at the distal end of the arm21, a hand22capable of gripping a workpiece W.

The control device40includes at least one processor (not shown) including hardware and controls the robot20and the image capturing device30according to multiple commands in an operation program, by executing the operation program taught in advance.

As shown inFIGS.2and3, the control device40includes a first clock unit (clock)41that clocks the time in the control device40, and a first communication unit42that transmits an image-capturing instruction to the image capturing device30and receives image data from the image capturing device30.

The control device40also includes a first storage unit43that stores angle information (position information), detected by encoders (not shown), of joint axes of the robot20.

The control device40also includes an image processing unit44that processes the image data received from the image capturing device30to extract the workpiece W in the image and calculates the position thereof, and a driving unit45that controls the robot20on the basis of the calculated position of the workpiece W and the angle information stored in the first storage unit43.

As shown inFIGS.2and3, the image capturing device30includes a second clock unit (clock)31that clocks the time in the image capturing device30, and a second communication unit32. The second communication unit32receives the image-capturing instruction transmitted from the control device40and transmits image data of the workpiece W, obtained by image capturing, to the control device40.

The image capturing device30also includes a camera33that captures an image of the workpiece W according to the image-capturing instruction from the control device40, and a second storage unit34that stores a preset difference time Δt needed to transfer the image-capturing instruction. Herein, the difference time Δt is determined on the basis of the result of measurement and evaluation of the difference time Δt that actually occurred in transferring the image-capturing instruction in an experiment performed in advance.

The image capturing device30also includes a calculation unit35that subtracts the difference time Δt stored in the second storage unit34from a predetermined waiting time Td included in the image-capturing instruction from the control device40.

The first clock unit41in the control device40and the second clock unit31in the image capturing device30are synchronized so as to have the same zero points.

The first clock unit41and the second clock unit31are synchronized by, for example, a time synchronization protocol (PTP) defined by IEEE1588.

In this embodiment, the control device40transmits, to the image capturing device30, an image-capturing instruction including the predetermined waiting time Td, which is the time from the transmission time To of the image-capturing instruction to the image capturing time Tc. The image-capturing instruction transmitted from the control device40is received by the second communication unit32. The calculation unit35calculates the required time T, which is the time from the receiving time Ti of the image-capturing instruction to the image capturing time Tc, on the basis of the waiting time Td included in the received image-capturing instruction and the difference time Δt stored in the second storage unit34.
T=Td−Δt

Then, the image capturing device30captures an image of the workpiece W at the time when the required time T has elapsed from the time Ti, when the second communication unit32received the image-capturing instruction from the control device40.

The image data of the workpiece W obtained by image capturing is transmitted to the control device40by the second communication unit32.

The operation of the thus-configured robot system1according to this embodiment will be described below.

First, the second clock unit31that clocks the time in the image capturing device30and the first clock unit41that clocks the time in the control device40are synchronized. This equalizes the zero point of the second clock unit31in the image capturing device30and the zero point of the first clock unit41in the control device40.

In other words, the image capturing device30and the control device40are controlled on the basis of the synchronized time base.

Next, as shown inFIGS.2and3, the first communication unit42in the control device40transmits, at the transmission time To, an image-capturing instruction to the image capturing device30to capture an image of the workpiece W.

Furthermore, the control device40stores, in the first storage unit43, the angle information of the joint axes detected by the encoders of the robot20at the time Tc, which is the time after the waiting time Td has elapsed from the transmission time To, clocked by the first clock unit41.

Meanwhile, in the image capturing device30, the second communication unit32receives the image-capturing instruction transmitted from the control device40at time Ti, which is the time delayed by the difference time Δt from the transmission time To.

Furthermore, the calculation unit35calculates the required time T by subtracting the difference time Δt, preliminarily stored in the second storage unit34, from the waiting time Td included in the image-capturing instruction.

Then, the camera33captures an image of the workpiece W at the time Tc, which is the time after the required time T has elapsed from the receiving time Ti of the image-capturing instruction, clocked by the second clock unit31.

Then, the image data of the workpiece W obtained by the camera33is transmitted to the control device40by the second communication unit32.

As a result, while the receiving time Ti of the image-capturing instruction is delayed by the difference time Δt from the transmission time To, the image capturing device30performs image capturing after the required time T, which is shorter than the waiting time Td by the difference time Δt, has elapsed from the receipt of the image-capturing instruction.

More specifically, even with the difference time Δt between transmission and receiving of the image-capturing instruction, the image capturing device30accurately performs image capturing of the workpiece W at the time Tc, which is the time after the waiting time Td has elapsed from the transmission time To of the image-capturing instruction.

When image data of the workpiece W is transmitted to the control device40, the image processing unit44processes the image data of the workpiece W to extract the workpiece W in the image and detects the position and orientation of the workpiece W.

Then, the driving unit45calculates the angles of the joint axes of the robot20for gripping the workpiece W with the hand22on the basis of the position information of the robot20at the time Tc and the information of the workpiece W photographed at the time Tc.

Thus, it is possible to make the robot20perform an accurate operation on the workpiece W.

Although the control device40adds the waiting time Td to the image-capturing instruction in this embodiment, instead, the control device40may add the required time T, obtained by subtracting the difference time Δt between the receiving time Ti of the image-capturing instruction and the transmission time To from the waiting time Td, to the image-capturing instruction.

In that case, the difference time Δt is stored in the first storage unit43in the control device40, together with the waiting time Td. Furthermore, the control device40calculates the required time T by subtracting the difference time Δt from the waiting time Td. The calculated required time T is added to the image-capturing instruction and is transmitted to the image capturing device30.

In the image capturing device30, the camera33captures an image of the workpiece W at the time Tc, which is the time after the required time T included in the image-capturing instruction has elapsed from the receiving time Ti of the image-capturing instruction.

More specifically, in this case, because the image capturing device30does not need to calculate the required time T, the second storage unit34for storing the difference time Δt can be omitted, which further simplifies the structure of the image capturing device30.

Furthermore, in this embodiment, although the control device40transmits the image-capturing instruction including the waiting time Td alone to the image capturing device30, instead, it is possible to add the transmission time To to the image-capturing instruction to be transmitted, in addition to the waiting time Td. In that case, the difference time Δt does not need to be stored in the second storage unit34.

As shown inFIG.4, when the second communication unit32in the image capturing device30receives an image-capturing instruction including the waiting time Td and the transmission time To, the second clock unit31obtains the receiving time Ti′ of the image-capturing instruction.

Then, the calculation unit35calculates the actual difference time Δt′ by subtracting the transmission time To from the receiving time Ti′, thus making it possible to accurately calculate the image capturing time Tc. As a result, it is possible to capture an image of the workpiece W at the accurately calculated image capturing time Tc and to accurately obtain the position information of the robot20at the image capturing time Tc.

In this embodiment, as shown inFIG.5, the image capturing device30may transmit the actual image capturing time Tc′ to the control device40. The control device40may correct the waiting time Td to be added to the next image-capturing instruction using the received actual image capturing time Tc′ and the time Tc, at which the position information of the robot20was obtained.

More specifically, when the control device40receives the actual image capturing time Tc′ from the image capturing device30, the control device40calculates a time lag (difference) x between the received image capturing time Tc′ and the time Tc, at which the control device40obtained the position information of the robot20. Then, the control device40calculates the waiting time Td′ in which the time lag x has been corrected, using the expression below:
Td′=Td+x,
where Td′ is the waiting time after the correction.

Furthermore, the image capturing device30subtracts the difference time Δt stored in the second storage unit34from the corrected waiting time Td′ included in the image-capturing instruction. By doing so, in the next image capturing, an image of the workpiece W can be captured at the correct image capturing time Tc′, in which the time lag x has been corrected.

The correction of the waiting time Td may be performed by using either the most-recent prior image capturing time Tc′ alone or an average time of previously obtained multiple image capturing times Tc′. When changes of the actual difference time Δt′ are mild, the former correcting method is effective, and when the actual difference time Δt′ varies each time, the latter correcting method is effective.

Furthermore, in this embodiment, the control device40may correct the required time T to be added to the next image-capturing instruction on the basis of the actual image capturing time Tc′ received from the image capturing device30.

In that case, the control device40calculates the time lag x between the actual image capturing time Tc′ received from the image capturing device30and the time Tc, at which the position information of the robot20was obtained. Then, the control device40calculates the difference time Δt′, in which the time lag x has been corrected, from the time lag x and the difference time Δt stored in the first storage unit43.

Thereafter, the control device40corrects the required time T by the time lag x by subtracting the corrected difference time Δt′ from the waiting time Td stored in the first storage unit43.

By doing so, in the next image capturing, an image of the workpiece W can be captured at the correct image capturing time Tc′, in which the time lag x has been corrected.

The correction of the required time T may be performed by using either the most-recent prior image capturing time Tc′ alone or the average time of previously obtained multiple image capturing times Tc′. When changes of the actual difference time Δt′ are mild, the former correcting method is effective, and when the actual difference time Δt′ varies each time, the latter correcting method is effective.

In another aspect, the control device40may add the image capturing time Tc to the image-capturing instruction to be transmitted to the image capturing device30.

In that case, in the image capturing device30, when the second communication unit32receives an image-capturing instruction from the control device40, the camera33captures an image of the workpiece W at the time when the second clock unit31has clocked the image capturing time Tc.

Meanwhile, in the control device40, when the first clock unit41clocks the image capturing time Tc, the position information of the robot20is obtained and stored in the first storage unit43.

This way, the image capturing device30and the control device40perform image capturing of the workpiece W and acquisition of the position information of the robot20at the same image capturing time Tc.

In other words, there is an advantage in that it is possible to perform image capturing of the workpiece W with the image capturing device30and acquisition of the position information of the robot20with the control device40at the same time, without the influence of the difference time Δt occurring in the transfer of the image-capturing instruction.

Furthermore, although the first clock unit41and the second clock unit31have the same zero points in this embodiment, the zero point of the first clock unit41and the zero point of the second clock unit31may differ by a known amount.

In that case, the image capturing device30needs to increase or decrease the waiting time Td or the required time T by the time corresponding to the time lag.

Although a vertical six-axis articulated-type robot has been described as an example in this embodiment, the robot is not limited thereto, and the present invention may also be applied to any other types of robots, such as horizontal articulated robots and parallel-link robots.