Source: https://patents.google.com/patent/DE102015015638A1/en
Timestamp: 2020-01-22 22:42:06
Document Index: 714574523

Matched Legal Cases: ['art 24', 'art 24', 'art 24', 'art 24', 'art 24', 'arts 34']

DE102015015638A1 - Apparatus and method for transferring articles using a robot - Google Patents
Apparatus and method for transferring articles using a robot
DE102015015638A1
DE102015015638A1 DE102015015638.1A DE102015015638A DE102015015638A1 DE 102015015638 A1 DE102015015638 A1 DE 102015015638A1 DE 102015015638 A DE102015015638 A DE 102015015638A DE 102015015638 A1 DE102015015638 A1 DE 102015015638A1
DE102015015638.1A
DE102015015638B4 (en
Kentarou Koga
2014-12-02 Priority to JP2014-244004 priority Critical
2014-12-02 Priority to JP2014244004A priority patent/JP6042860B2/en
2015-12-02 Application filed by FANUC Corp filed Critical FANUC Corp
2016-10-13 Publication of DE102015015638A1 publication Critical patent/DE102015015638A1/en
2018-11-29 Publication of DE102015015638B4 publication Critical patent/DE102015015638B4/en
An article transfer device with a robot. An image processing area includes an article detecting area for performing image pickup and detection of articles moving in accordance with a conveying movement of a conveying device, a first period allowing all of the plurality of articles to be picked up and detected, and obtaining information on the starting position of each the item; and an article tracking section for performing image capturing and detecting the plurality of articles moving in accordance with the conveying movement of the conveying device, wherein a second period is shorter than the first period, and repeatedly obtaining information about the displaced position of each of the plurality of articles of the second period, wherein the information about the displaced position is based on the information about the initial position. A robot control section is configured to control the robot using the information about the displaced position to cause the robot to transmit each of the plurality of articles and while tracking the conveying movement of the conveying device.
The present invention relates to an article transfer device and an article transfer method for transferring articles using a robot.
The provision of a system in which a vision sensor detects a plurality of articles being conveyed on a conveyer, and a robot holds the article while tracking the conveyance of the conveyance device based on positional information about each detected article Transferring items to another location is known.
Japanese Unexamined Patent Publication No. H08-063214 ( JP H08-063214 A ) discloses, for example, a visual tracking method in which a vision sensor detects a workpiece to be conveyed on a conveyor, and a robot holds the workpiece based on the position of the detected workpiece while tracking the conveyor. In this visual tracking method, the conveyor is equipped with a pulse encoder to detect the extent to which the workpiece has been carried. After recognizing that the workpiece has been conveyed a predetermined distance based on the output signal of the pulse coder, the vision sensor acquires an image of the conveyed workpiece. After it has been detected based on the output of the pulse encoder that the workpiece has been conveyed the predetermined distance, the tracking movement of the robot is started under the control of a robot controller. During the tracking movement, the robot controller controls the robot in a tracking coordinate system that moves with the workpiece, and causes the robot to acquire the workpiece to be conveyed on the conveyor in consideration of the position of the workpiece obtained from the image acquired by the vision sensor will, hold.
In a system in which a robot transmits an article by holding the article while tracking the conveyance movement of a conveyance device based on the positional information of the article detected by a vision sensor, it is desirable that the robot be configured to: to be able to hold the article while following the conveyor by accurately identifying the current position of the article without having to equip the conveyor with a sensor such as an encoder for detecting the amount of movement. In such a system, it is also desirable that the robot be configured to hold the article while tracking its position, even if the position of the article on the conveyor is displaced by the vision sensor after detection, or even if the article could not be detected by the vision sensor because the article is, for example, obscured by another article.
One aspect of the present invention provides an article transferring device comprising a conveying device configured to convey an article; a robot configured to hold and transfer the article; an image capture area configured to capture an image of the article; an image processing area configured to control the image pickup area and to detect the article based on data picked up by the image pickup area; and a robot control area configured to control the robot using information about the article detected by the image processing area, the image processing area comprising: an article detection area configured to perform image capture and detection of a plurality of articles, moving in accordance with a conveying movement of the conveying device, a first period allowing all of the plurality of articles to be picked up and detected, and obtaining information about the starting positions of each of all the articles; and an article tracking section configured to perform image pickup and detection of the plurality of articles moving in accordance with the conveying movement of the conveying device, a second period shorter than the first period, and information on the displaced position of each of the plurality of articles repeatedly obtaining the second period, the shifted position information being based on the initial position information; and wherein the robot control section is configured to control the robot using the shifted position information so that the robot holds and transmits each of the plurality of articles while tracking the conveyance movement of the conveyance device.
Another aspect of the present invention provides an article transfer method configured to cause a robot to hold and transfer a plurality of articles conveyed by a conveyer, wherein the The method includes: performing image pickup and detection of the plurality of articles moving in accordance with a conveyance movement of the conveyance device, a first period allowing all of the plurality of articles to be picked up and detected, and information about the initial positions of each of the To obtain articles; Performing image pickup and detection of the plurality of articles moving in accordance with the conveying movement of the conveying device, wherein a second period is shorter than the first period, and repeatedly obtaining information on the displaced position of each of the plurality of articles with the second period, wherein the information about the displaced position is based on the information about the starting position; and controlling the robot using the displaced position information so that the robot holds and transmits each of the plurality of articles while tracking the conveying movement of the conveying device.
According to the article transfer device of one aspect, after the article detecting area in the image processing area has detected the information on the starting position of the article conveyed on the conveying device, the article tracking area in the image processing area obtains the information about the displaced position of the article, and the robot control area controls the robot using the robot Information about the relocated position; this obviates the need to equip the conveyor with a sensor such as an encoder for detecting the amount of movement, and the robot can detect the current position of the article being conveyed by the conveyor and hold the article M while following the conveyor. Accordingly, the system in which the robot holds each article while tracing the conveyance of the conveyance device and transfers the article to another location can be constructed even in a situation where it is difficult to convey the conveyance device with a sensor such as for example, an encoder for detecting the amount of movement to equip. In addition, the article tracking area may be configured since the article tracking area is configured to obtain the information about the displaced position of each article repeatedly with the second period when the position of the article on the conveyor is shifted after the article detecting area acquires the information about the initial position of the article has to update the information about the relocated position by tracking the position of the article and can the robot 14 capture the current position of the article after the position shift using the updated displaced position information and hold the article.
According to the article transfer method of the other aspect, the same advantageous effect as that of the article transfer device described above can be achieved.
These objects, features and advantages of the present invention will become more apparent from the following description of the embodiments taken in conjunction with the accompanying drawings. In which are:
1 a diagram schematically showing the configuration of an article transfer device according to an embodiment;
2 a diagram schematically showing a modified example of the article transfer device;
3 a diagram schematically showing another modified example of the article transfer device;
4 a diagram schematically showing yet another modified example of the article transfer device;
5 a diagram schematically showing the configuration of an article transfer device according to another embodiment;
6A Fig. 12 is a diagram for explaining an example of an arrangement pattern showing articles arranged according to this arrangement pattern;
6B a diagram showing an arrangement form, the arrangement pattern from 6A Are defined;
7 a block diagram showing an example of the hardware configuration of a robot control area;
8th Fig. 10 is a block diagram showing an example of the hardware configuration of an image processing area;
9 10 is a block diagram showing another example of the hardware configuration of the robot control area and the image processing area;
10 10 is a block diagram showing still another example of the hardware configuration of the robot control area and the image processing area;
11 12 is a functional block diagram illustrating the robot control area and the image processing area in the embodiment of FIG 5 shows;
12 a diagram schematically showing an example of the work of holding and transmitting an article;
13 a diagram for explaining an example of an article tracking process;
14 a diagram for explaining another example of an article tracking process;
15 Fig. 10 is a flow chart showing processing executed by an article management area in the image processing area;
16 Fig. 10 is a flow chart showing processing executed by an article tracking area in the image processing area;
17 a flowchart showing the details of processing executed by the article tracking area;
18 a flowchart showing the details of processing executed by the article tracking area;
19 a flowchart showing the details of processing executed by the article tracking area;
20 Fig. 10 is a flowchart showing processing executed by an information management area in a first control area;
21 Fig. 10 is a flow chart showing the details of processing executed by the information management area; and
22 Fig. 10 is a flowchart showing the details of processing executed by the information management area.
Embodiments of the present invention will be described below with reference to the accompanying drawings. In all drawings, corresponding components are designated by like reference numerals.
1 shows an article transfer device 10 according to one embodiment. The article transfer device 10 includes a conveyor 12 configured to convey an article M, a robot 14 configured to hold and transmit the article M, an image pickup area 16 configured to capture an image of the article M, an image processing area 18 which is configured for the image pickup area 16 to control and the article M based on the data of the image pick-up area 16 detected image and a robot control area 20 which is configured for the robot 14 using information from that of the image processing area 18 controlled article M to head.
The conveyor 12 comprises a known conveying element capable of holding a plurality of articles M and conveying them in one direction (in the direction indicated by arrow W in the drawing), and a known driving mechanism for driving the conveying element to a continuous one or intermittent way. The variety of on the conveyor 12 conveyed articles M may have various shapes, dimensions, appearance (colors), etc., or may have the same shape, dimension, appearance (color), etc. In addition, the articles M may be articles such as fruits or vegetables, the shapes of which hold the articles M on the conveyor 12 makes it unstable.
The plurality of articles M will be in any arrangement on the conveyor 12 transported and enters a predetermined work space 22 a (the area where the items M are held and picked up) (indicated by half-dashed lines) to allow the robot 14 can access it. The term "arbitrary arrangement" refers to an arrangement in which the articles M are not intended to be arranged in an order but are arranged in a random manner so that the articles M have different positions and orientations in a two-dimensional space assume (for example, in one of the image pickup area 16 taken picture) when the items M are viewed directly from above. In the present embodiment, the conveying device 12 not equipped with a sensor for detecting the amount of movement, such as an encoder, which can detect the position or speed of the conveying element or the driving mechanism.
The robot 14 may include a mechanical portion (not shown) suitably selected from known mechanical ranges (for example, manipulators) of different types, such as a hinge type, a gantry type, and a parallel link type, and a hand suitably among known hands of various types, such as suction type or grip type is selected.
The robot 14 is in a predetermined position on one side of the conveyor 12 arranged and works so that the mechanical area or the hand the item M in the working space 22 of the robot 14 stops and picks up while on the conveyor 12 Carried article M is tracked, and transfers the article M to another location. In the present embodiment, the robot is 14 equipped with a mechanical area.
The image pickup area 16 has a predetermined field of vision 24 (shown by dashed lines) extending longitudinally in the conveying direction of the conveyor 12 to include a portion thereof and laterally across the entire width of the conveyor 12 extends. The workroom 22 of the robot 14 is inside the field of view 24 , The image pickup area 16 acquires two-dimensional image data by capturing an image of the articles M and the conveyor 12 in the field of view 24 directly from the top of the conveyor 12 , In the present embodiment, the image pickup area includes 16 a camera (sometimes referred to as the camera 16 designated). The camera 16 can be a digital camera and, for example, the resolution and image pickup area can be set as desired. In the present embodiment, the camera 16 between a mode, which is an image of a part 24a of the field of view 24 at the upstream end in the conveying direction of the conveying device, and a mode taking a picture in the entire field of view 24 receives, be switched. The camera 16 is via a camera cable 26 with the image processing area 18 connected.
The image processing area 18 applies appropriate image processing to the two-dimensional data that the recording area 16 by taking a picture of the article M and the conveyor 12 in the field of view 24 and thereby detects the presence of each article M and obtains information representing the position (coordinate values) and orientation (rotation angle) of each article M in a predetermined two-dimensional coordinate system. As will be described below, the "position information" (initial position information, displaced position information, etc.) includes the image processing area 18 obtained, usually information representing the position and orientation of the article M, but if the article M is, for example, circular, the "position information" may not contain the information representing the orientation of the article M. In the present specification, the information representing the position and orientation of the article M and the information representing the position of the article M are both referred to as the "position information". The image processing area 18 may also obtain information concerning the two-dimensional outer shape, color, etc. of each article M (hereinafter referred to as the information about the external appearance characteristics). The of the image processing area 18 Obtained position information of the article M can be displayed in the form of an image on a screen, not shown.
The image processing area 18 is via a communication cable 28 with the robot control area 20 connected so that the position information of the article M, which from that of the image pickup area 16 obtained two-dimensional data is detected, as needed to the robot control area 20 can be transferred. Alternatively, the image processing area 18 and the robot control area 20 be integrated in a single common control device, as will be described later.
A two-dimensional conveyor coordinate system (stationary coordinate system) 30 can for the conveyor 12 be defined by taking their conveying direction as the X-axis. In the present embodiment, the robot 14 in the conveyor coordinate system 30 according to one in the conveyor coordinate system 30 be operated defined command. In addition, the image processing area 18 the position information of the detected article M in the conveyor coordinate system 30 by performing the calibration of the image pickup area 16 in the conveyor coordinate system 30 define. The robot control area 20 controls the robot 14 using the position information of the image processing area 18 detected article M and causes the robot 14 Perform a holding action that corresponds to the type of hand.
The image processing area 18 includes an article detection area 32 configured to perform image acquisition and detection of a plurality of articles M that conform to a conveying motion of the conveyor 12 wherein a first period T1 enables all of the plurality of articles M to be picked up and detected, and to obtain information about the initial positions of each of all the articles M; and an article tracking area 34 configured to perform image capture and detection of the plurality of articles M that conform to the conveying motion of the conveyor 12 wherein a second period T2 is shorter than the first period T1, and repeats information about the displaced position D2 of each of the plurality of articles M to be obtained with the second period T2, wherein the information on the displaced position D2 based on the information on the initial position D1; and wherein the robot control area 20 configured for the robot 14 using the information about the displaced position D2 to control so that the robot 14 holds and transmits each of the plurality of articles M while it is conveying the conveyor 12 tracked.
In the present embodiment, the initial position information D1 and the shifted position information D2 are respectively represented by the coordinate values and the rotation angle (or only by the coordinate values) in the conveyor coordinate system 30 shown.
The article detection area 32 For example, the function of initial recognition and detection (hereinafter sometimes referred to as "initial detection") has each of the plurality of articles M in the field of view 24 of the image pickup area 16 if the items M on the conveyor 12 to the workroom 22 of the robot 14 be transported. In the present embodiment, the article detection area is 32 configured to initially detect each item M when the item M in the upstream end portion 24a of the field of view 24 of the image pickup area 16 entry. The first period T1 is the period at which the image pickup area 16 an entire image of each article M at least once in the upstream end part 24a of the field of view 24 can record.
In a specific example, where the length of the upstream end part 24a of the field of view 24 in the conveying direction W of the conveying device 12 (that is, in the X-axis direction in the conveyor coordinate system 30 ) is 400 mm, the maximum width of the article M is 100 mm and the conveying speed of the conveyor 12 200 mm / s, the first period T1 required for this is the image of each particular one on the conveyor 12 once conveyed article M during the interval from the moment when the entire article M in the upstream end part 24a occurs until the moment when a part of the article M from the upstream end part 24a resign, take up, given as follows: T1 = (400 (mm) - 100 (mm)) / 200 (mm / s) = 1.5 (s)
In the above-mentioned specific example, by performing image pickup of the plurality of articles M having the first period T1 when the articles M on the conveyor 12 to the workroom 22 9, the image of each article M is received and initially detected while the article M is within the upstream end portion 24a and thus the information about the initial position D1 of each of the articles M can be obtained. The article tracking area 34 has the function of sequentially detecting and detecting (hereinafter sometimes referred to as "tracking") each of the plurality of articles M in the field of view 24 of the image pickup area 16 After the items M are initially received by the item detection area 32 be detected. In the present embodiment, the article tracking area is 34 configured each article M in the entire field of view 24 of the image pickup area 16 to pursue. The second period T2 is the period with which the image pickup area 16 an entire image of each article M in the field of view 24 can repeatedly record a number of times sufficient to track the article M without erroneously detecting another article M.
For example, if two articles M next to each other in exactly the same orientation (rotation angle) are to be detected correctly without being confused with each other, it is assumed that the image acquisition of each article M by the image pick-up area 16 and detection by the article tracking area 34 should be completed within a period in which the article M on the conveyor 12 a distance equal to one half of the minimum width of the article M is conveyed. Under this condition, when the minimum width of the article M is 60 mm in the above specific example, the second period T2 is given as follows. T2 = (60 (mm) / 2) / 200 (mm / s) = 0.15 (s)
In the above specific example, by performing image pickup of the plurality of articles M with the second period T2, when the articles M on the conveyor 12 be conveyed, the image of each article M repeated in the field of view 24 can be taken and tracked without being confused with another article M, and thus the information about the displaced position D2 of each of the articles M can be obtained. When the second period T2 is set as short as the processability of the image processing area 18 by neglecting the condition "one half of the minimum width of the article M" may allow the reliability of the information about the displaced position D2 and thus the accuracy with which the robot 14 works to keep the item M maximized while tracking its position.
The image processing area 18 performs the image capture and detection (ie, tracking) with the second period T2 through the article tracking area 24 simultaneously with the image acquisition and detection (ie the initial detection) with the first Period T1 through the article detection area 32 off (ie performs so-called multitasking). At this time, the initial values of the position and alignment values (or only the position value) included in the information on the initial position D1 of the article M are obtained from the article detecting area 32 is obtained, the article tracking area 34 continuously and repeatedly repeats the information about the displaced position D2, which represents constantly changing position and orientation values (or only the position value) (ie, the amount of change) of the article M, by repeatedly performing the image pick-up and detection of the article M with the second period T2. The camera 16 taking the picture-taking area 16 forms the image pickup with the first period T1 and the image pickup with the second period T2 at appropriate times and in an appropriate order.
In the image processing area 18 is the information about the starting position D1 of each article M, that of the article detection area 32 is obtained for article tracking through the article tracking area 34 At the same time, the information about the initial position D1 is combined into a single packet (hereinafter referred to as a packet α) and together with other information necessary to cause the robot 14 every article M stops at the robot control area 20 cleverly. The information carried in the packet α comprises a packet identifier indicating that the information is the packet α coming from the article detection area 32 acquired information about the initial position D1, the image capturing time of the article corresponding to the information on the initial position D1, and an article identifier as a sequence number indicating the article M. For each of the variety of items M, the on the conveyor 12 to the workroom 22 of the robot 14 the package α is created only once and sent to the robot control area 20 cleverly.
In the image processing area 18 is the information about the displaced position D2 of each article M, repeated from the article tracking area 34 Also, each time the information is obtained, it is assembled into a single packet (hereinafter referred to as a packet β) and along with other information necessary to cause the robot 14 every article M stops at the robot control area 20 cleverly. The information carried in the packet β comprises a packet identifier indicating that the information is the packet β that is from the article tracking area 34 acquired information about the displaced position D2, the image capturing time of the article corresponding to the displaced position information D2, and an article identifier as a sequence number indicating the article M. For each of the variety of items M, the on the conveyor 12 to the workroom 22 of the robot 14 the packet β is repeatedly created and sent to the robot control section 20 cleverly.
Based on the image processing area 18 received information of the packet α creates the robot control area 20 Article information to cause the robot 14 holds each article M, and assembles the article information into a single package (hereinafter referred to as a package γ). The information carried in the packet γ comprises, in addition to the information of the packet α, the information on the displaced position D2 and the article image recording time carried in the information of the packet β, that of the image processing area 18 an arbitrary number of times is received up to the time that contains the last received. In addition, the information concerning the moving speed of the article M (that is, the conveying speed of the conveying device 12 ) obtained from the time interval over which the information on the displaced position D2 and the article image recording time has been received any number of times may also be included in the packet γ. The number of times the packet β must be received for the creation of the packet γ may be determined by the user of the article transfer device as desired 10 be set. If every article M from the robot 14 picked up and from the conveyors 12 is removed, the robot control area clears 20 the information of the package γ, which relates to this article M.
The robot control area 20 compares the information of the packet β which is constant from the image processing area 18 with the presently held information of the packet γ to detect the presence of the article M being in the workspace 22 of the robot 14 enter, or the work space 22 currently traversed. Then the robot control area controls 20 the robot 14 using the information conveyed in the packet β about the displaced position D2 of the article M corresponding to the article M whose presence has been detected. Under control of the robot control area 20 stops the robot 14 the article M and attacks him while he is the conveying movement of the conveyor 12 followed, and transmits the article M from the conveyor 12 to a predetermined other place.
According to the article transfer device 10 with the configurations described above after the article detection area 32 in the image processing area 18 the information about the initial position D1 of the on the conveyor 12 article M, the article tracking area 14 in the image processing area 18 the information about the relocated Position D2 of the article M and controls the robot control area 20 the robot 14 using the information about the displaced position D2; this eliminates the need for the conveyor 12 with a sensor such as an encoder for detecting the amount of movement, and the robot 14 can the current position of the on the conveyor 12 Carrying article M and holding the article while holding the conveyor 12 tracked. Accordingly, the system in which the robot holds each article while tracing the conveyance of the conveyance device and transfers the article to another location can be constructed even in a situation where it is difficult to convey the conveyance device with a sensor such as for example, an encoder for detecting the amount of movement to equip. Also, since the article tracking area 34 is configured to repeat the information about the displaced position D2 of each article M with the second period T2 when the position of the article M on the conveyor 12 is displaced after the article detection area 32 has obtained the information about the starting position of the article M, the article tracking area 34 the information about the displaced position D2 can be updated by tracking the position of the article M and the robot 14 using the updated displaced position information D2, detect the current position of the article M after the positional shift and hold the article M.
The above-described configuration of the article transfer device 10 may be described in terms of an article transfer process according to another aspect of the present invention. The article transfer method relates to a method for causing the robot 14 each of a variety of in the conveying direction 12 conveyed article M and transfer the article M to another location, and includes the steps of obtaining information on the initial position D1 for each of the plurality of on the conveyor 12 conveyed articles M by performing image capturing and detection of the plurality of articles M with a first period T1 enabling the acquisition and detection of all of the plurality of articles M; Obtaining information on the displaced position D2 relative to the information on the initial position D1 for each of the plurality of on the conveyor 12 conveyed article M repeats with a second period T2 shorter than the first period T1 by performing the image pickup and detection of the plurality of articles M with the second period T2; and based on the information on the displaced position D2 controlling the robot 14 such that the robot 14 the article M transmits by holding the plurality of articles M, while he the conveying movement of the conveyor 12 tracked.
2 shows a modified example of the article transfer device 10 , In this modified example, the image pickup area includes 16 a first camera 16A and a second camera 16B that are configured to work independently of each other. The field of vision of the first camera 16A includes the above-mentioned upstream end portion 24a of the field of view 24 considered in the conveying direction of the conveyor. The field of view of the second camera 16B is the same as the field of view 24 , The article detection area 32 is configured to cause the first camera 16A takes the picture of the article M, which is in the field of vision 24a located, and the article M based on the data of the first camera 16A detected image to be detected. The article tracking area 34 is configured to cause the second camera 16B the image of the article M, which is within the field of view 24 is located, and the article M based on the data of the second camera 16B detected image to be detected. Both the first camera 16A as well as the second camera 16B may be digital cameras and, for example, resolution and image pickup area can be adjusted as desired. The first and the second camera 16A and 16B are via appropriate camera cable 26 with the image processing area 18 connected. In this configuration, those from the first camera 16A Image taken with the first period T1 and that of the second camera 16A With the second period T2 executed image recording to be performed simultaneously.
3 shows another modified example of the article transfer device 10 , In this modified example, the robot includes 14 a first mechanical area 14A and a second mechanical area 14B configured to operate independently of each other, and includes the robot control section 20 a first control area 20A which is configured for the first mechanical area 14A to control and a second control area 20B which is configured for the second mechanical area 14B to control. The first and the second mechanical area 14A and 14B may each have the configuration of any of various known mechanical ranges (e.g., manipulators) and may be equipped with any of various known hands (not shown). The first mechanical area 14A is in a predetermined position on one side of the conveyor 12 arranged and works such that he the article M in a working space 22A stops and picks up while on the conveyor 12 transported article M, and the article M to another Place transfers. The second mechanical area 14B is in a predetermined position on one side of the conveyor 12 and on the downstream side of the first mechanical section 14A arranged in the conveying direction of the conveying device and operates in such a way that he the article M in a working space 22B stops and picks up while on the conveyor 12 transported article M, and transfers the article M to another location. The workrooms 22A and 22B are set such that they do not overlap one another to prevent the first mechanical area from being obstructed 14A and the second mechanical area 14B hinder each other.
In addition, there are the workrooms 22A and 22B both in the field of vision 24 of the image pickup area 16 ,
The first tax area 20A controls the first mechanical area 14A based on the information about the displaced position D2 such that the first mechanical region 14A the article M transfers by holding each of the plurality of articles M, while it the conveying movement of the conveyor 12 tracked. The second tax area 20B controls the second mechanical area 14B based on the information about the displaced position D2 such that the second mechanical range 14B The article M transmits by placing each of the plurality of articles M (other than those of the first mechanical section 14A picked up article) while holding the conveyor movement of the conveyor 12 tracked. The first and the second tax area 20A and 20B are connected to each other via a communication cable 28 connected and are together with the image processing area 18 connected. In this configuration, the first and second control areas 20A and 20B perform such control that the first and the second mechanical area 14A and 14B respectively hold predetermined numbers of articles M according to a predetermined duty ratio (that is, work pitch ratio). The control performed in consideration of the work division ratio will be described in detail in connection with a further embodiment described later.
4 shows yet another modified example of the article transfer device 10 , This modified example is a combination of the modified example according to FIG 2 and the modified example according to 3 and includes the first and the second camera 16A and 16B as the image pickup area 16 , the first and the second mechanical area 14A and 14B as the robot 14 and the first and second control areas 20A and 20B as the robot control area 20 ,
As described above with reference to the various modified examples, the number of cameras that can capture the image pickup area 16 form, and the number of mechanical areas that the robot 14 form according to various factors, such as the total number of on the conveyor 12 conveyed articles M, the conveying speed of the conveyor 12 , the time, the robot 14 needed to transfer the items M to another location, and the required working accuracy can be selected suitably. For example, there may be three or more cameras, three or more mechanical robot sections, and three or more robot control sections. When the cameras and robot mechanical areas are arranged along the conveyance direction of the conveyance apparatus, the downstream-side camera and robot mechanical areas may be configured to perform the work that the upstream-side camera and robot mechanical area did not perform.
5 shows an article transfer device 40 according to a further embodiment. The article transfer device 40 has substantially the same basic configuration as the article transfer device described above 10 except that the number of cameras that the image processing area 16 form, the number of mechanical areas that the robot 14 form, and the number of control areas, the robot control area 20 form, are different, and that a second conveyor is provided, to which the robot 14 the article M transmits. In the following description, components are the same as those in the article transfer device 10 may be denoted by the same reference numerals, and further description of such components may be omitted. In addition, the configuration, the details of the article transfer device 10 is assigned and omitted from the previous description, in connection with the configuration of the article transfer device 40 will be described in detail below.
The article transfer device 40 includes a conveyor 12 configured to convey an article M, a robot 14 configured to hold and transmit the article M, an image pickup area 16 configured to capture an image of the article M, an image processing area 18 which is configured for the image pickup area 16 and to control the article M based on data from the image pickup area 16 detected image and a robot control area 20 which is configured for the robot 14 using the information of the image processing area 18 detected To control article M. The article transfer device 40 also includes a second conveyor 42 (hereinafter referred to as a discharge conveyor) to which the robot 14 that of the conveyor 12 adopted article M transmits.
The robot 14 includes a first mechanical area 14A , a second mechanical area 14B and a third mechanical area 14C that are configured to work independently of each other. The first mechanical area 14A is in a predetermined position on one side of the conveyor 12 arranged and works such that he the article M in a working space 22A stops and picks up while on the conveyor 12 transported article M, and the article M to the discharge conveyor 42 transfers. The second mechanical area 14B is in a predetermined position on one side of the conveyor 12 and on the downstream side of the first mechanical section 14A arranged in the conveying direction of the conveying device and operates in such a way that he the article M in a working space 22B stops and picks up while on the conveyor 12 transported article M, and the article M to the discharge conveyor 42 transfers. The third mechanical area 14C is in a predetermined position on one side of the conveyor 12 and on the downstream side of the second mechanical area 14B arranged in the conveying direction of the conveying device and operates in such a way that he the article M in a working space 22C stops and picks up while on the conveyor 12 transported article M, and the article M to the discharge conveyor 42 transfers. The workrooms 22A . 22B and 22C are set so that they do not overlap each other, to prevent the adjacent mechanical areas 14A . 14B and 14C hinder each other. The first to third mechanical area 14A . 14B and 14C have essentially the same hardware configuration.
The image pickup area 16 includes a first camera 16A , a second camera 16B and a third camera 16C that are configured to work independently of each other. The first camera 16A has a predetermined field of vision 44 (indicated by dashed lines) extending longitudinally in the conveying direction of the conveyor 12 so that it comprises part of it and laterally across the entire width of the conveyor 12 extends. The field of vision 44 is set in the conveying direction of the conveying device in a position which corresponds to the upstream end portion described above 24a of the field of view 24 corresponds (compare 1 ), but shifted upstream in the conveying direction of the conveying device by a predetermined distance. The field of view of the second camera 16B is the same as the field of view 24 , The field of vision 44 the first camera 16A and the field of vision 24 the second camera 16B partially overlap each other. The third camera 16C has a predetermined field of vision 46 (shown by dashed lines) extending longitudinally in the conveying direction of the conveyor 12 so as to comprise part of it, and laterally across the entire width of the conveyor 12 extends. The field of vision 46 is set so that it is 24 in the conveying direction of the conveyor considered further downstream of the downstream end of the field of view 24 extends. The field of vision 46 the third camera 16C and the field of vision 24 the second camera 16B partially overlap. The workrooms 22A and 22B the first and the second mechanical area 14A and 14B are within the field of view 24 the second camera 16B , The workroom 22C the third camera 14C is in the field of view 46 the third camera 16C , The first to third camera 16A . 16B and 16C are via appropriate camera cable 26 with the image processing area 18 connected. The first to third camera 16A . 16B and 16C have essentially the same hardware configuration.
The robot control area 20 includes a first control area 20A which is responsible for controlling the first mechanical area 14A is configured, a second control area 20B which controls the second mechanical area 14B is configured, and a third control area 20C which controls the third mechanical area 14C is configured. The first tax area 20A controls the first mechanical area 14A based on the information about the displaced position D2, so that the first mechanical area 14A the article M transfers by holding each of the plurality of articles M, while it the conveying movement of the conveyor 12 tracked. The second tax area 20B controls the second mechanical area 14B based on the information about the displaced position D2, so that the second mechanical range 14B The article M transmits by placing each of the plurality of articles M (other than those of the first mechanical section 14 picked up item M) while he is the conveying movement of the conveyor 12 tracked. The third tax area 20C controls the third mechanical area 14C based on the additional information about the displaced position D3 (described later), so that the third mechanical range 14C The article M transmits by giving each of the more variety of articles M (other than those of the first mechanical section 14 picked up item M) while he is the conveying movement of the conveyor 12 tracked. The first to third tax area 20A . 20B and 20C are connected to each other via a communication cable 28 and a network hub 48 connected and are together with the image processing area 18 connected. The first to third tax area 20A . 20B and 20C can essentially have the same hardware configuration.
The image processing area 18 includes an article detection area 32 that is configured to cause the first camera 16A takes the picture of the article M, which is in the field of vision 44 located, and the article M based on the data of the first camera 16A to detect (or initially detect) the captured image, an article tracking area 34 that is configured to cause the second camera 16B takes the picture of the article M, which is in the field of vision 24 located, and the article M on the basis of the data of the second camera 16B recorded image and an auxiliary tracking area 50 that is configured to cause the third camera 16C takes the picture of the article M, which is in the field of vision 46 located, and the article M based on the data of the third camera 16C detected image to be detected. The auxiliary tracking area 50 is configured to perform image pick-up and detection of a plurality of articles M that conform to the conveying motion of the conveyor 12 , wherein a third period T3 is shorter than the first period T1, and additional information about the displaced position D3 of each of the plurality of articles M repeatedly obtained with the third period T3, wherein the additional information on the displaced position D3 on the information based on the displaced position D2.
The auxiliary tracking area 50 has the function of each of the plurality of articles M in the field of view 46 the third camera 16C continue to track after the article M in the field of view 24 the second camera 16B from the article tracking area 34 be followed. The third camera 16C and the auxiliary tracking area 50 are provided to cause the third mechanical area 14C the article M picks up the first and the second mechanical area 14A and 14B in their workrooms 22A and 22B from the articles M, the article tracking area 34 in the field of view 24 pursued, did not take up. The third period T3 may be set in accordance with the same condition as the second period T2, and may be set the same as the second period T2.
The image processing area 18 Performs the image acquisition and detection (ie the initial detection) with the first period T1 through the article detection area 32 , the image capture and detection (ie, tracking) with the second period T2 through the article tracking area 34 and the image pickup and detection (ie, tracking) with the third period T3 through the auxiliary tracking area 50 at the same time. The first to third camera 16A . 16B and 16C can simultaneously perform the image pickup with the first to third periods T1, T2 and T3. At this time, the auxiliary tracking area acquires 50 taking the initial values of the position and orientation values (or only the position value) included in the displaced position information D2 repeatedly through the article tracking area 34 from the article M, and more precisely the final information about the displaced position D2 obtained by the article M, which is in the field of view 46 the third camera 16C has occurred, continuously and repeatedly repeats the additional information about the displaced position D3 representing constantly changing position and orientation values (or only the position value) (ie, the amount of change) of the article M by repeatedly performing image acquisition and detection of the article Article M with the third period T3.
In the image processing area 18 becomes that of the article detection area 32 obtained information about the starting position D1 of each article M for article tracking through the article tracking area 34 at the same time, the information about the initial position D1 composed in the form of the previously described packet α is used only at the first control area 20A cleverly. In this embodiment, in addition to the above-described information, the packet α carries a camera ID including the first camera 16A indicating that the image data has been obtained on the basis of which the information on the initial position D1 has been created. In addition, in the image processing area 18 the Information about the displaced position D2 of each article M repeated through the article tracking area 34 and the additional information about the displaced position D3 of each article M repeated from the auxiliary tracking area 50 is obtained each time the information is obtained is composed in the form of the above-described packet β and is applied to all of the first to third control areas 20A . 20B and 20C cleverly. In this embodiment, in addition to the above-described information, the packet β carries a camera ID including the second or third camera 16B or 16C indicating that the image data has been obtained on the basis of which the information on the displaced position D2 or the additional information on the displaced position D3 has been created.
Based on the information of the image processing area 18 received packet α creates the first control area 20A Article information to cause the first to third mechanical area 14A . 14B and 14C Article M and composes the article information in the form of the above-described package γ. The information carried in the packet γ comprises, in addition to the information of the packet α, the information on the displaced position D2 or the additional information on the displaced position D3 and the article image recording time carried in the information of the packet β transmitted by the image processing area 18 any number of times has been received up to the time that the last received received. Since every article M from the first mechanical area 14A picked up and from the conveyor 12 is removed, deletes the first control area 20A the information of the packet γ concerning the article M; on the other hand, does not transmit to everyone from the first mechanical area 14A from the conveyor 12 removed item M the first control area 20A the information of the packet γ to the second control area 20B , The second tax area 20B containing the information of the packet γ from the first control area 20A receives, clears the information of the packet γ when the article M from the second mechanical area 14B picked up and from the conveyor 12 Will get removed; on the other hand, does not transmit for each of the second mechanical area 14B from the conveyor 12 removed item M the second control area 20B the information of the packet γ to the third control area 20bC ,
The first to third tax area 20A . 20B and 20C respectively compare the information of the packet β which is constant from the image processing area 18 is transmitted, with the information currently held by the packet γ, to detect the presence of the article M that is in the working space 22A . 22B or 22C the corresponding one of the first to third mechanical areas 14A . 14B or 14C enter, or currently the work space 22A . 22B or 22C crosses. Then control the first and the second control area 20A and 20B each the first or the second mechanical area 14A respectively. 14B using the information about the displaced position D2 of the article M transported in the packet β, which corresponds to the article M whose presence has been detected. The third tax area 20C controls the third mechanical area 14C using the additional information about the displaced position D3 of the article M transported in the packet β, which corresponds to the article M whose presence has been detected. Under control of the corresponding control area, the first to third mechanical areas hold 14A . 14B and 14C respectively the article M and attack it while it is the conveying movement of the conveyor 12 track and transfer the article M from the conveyor 12 to the discharge conveyor 42 ,
The discharge conveyor 42 is side by side with the conveyor 12 arranged in a substantially parallel relationship in a position in which the first to third mechanical region 14A . 14B and 14C perform the task of transmitting the article M. The discharge conveyor 42 comprises a known conveying element capable of conveying a plurality of articles M in one direction (the direction indicated by arrow W in the drawing) by holding the articles M thereon, and a known driving mechanism for driving the conveying element on one continuous or intermittent way. The discharge conveyor 42 is with an encoder 52 equipped to detect the amount of movement of the conveyor element. That of the encoder 52 Detected range of motion is to the first control area 20A and is sent from the first to third tax area 20A . 20B and 20C shared, which are interconnected via a communication network.
A variety of bowls 54 each capable of holding a predetermined number of articles M in a predetermined arrangement are formed on the discharge conveyor 42 promoted. The discharge conveyor 42 is with a photocell sensor 56 to detect the arrival of each tub 54 equipped at a fixed position. Upon receiving a detection signal of a pan 54 from the photocell sensor 56 The first control area reads 20A the amount of movement of the discharge conveyor 42 from the encoder 52 and stores the read value as the initial value; then, by comparing the constant updated detection value of the encoder 52 with the stored initial value the current position of the shell 54 be determined.
Every shell 54 is capable of a plurality of articles M according to a predetermined arrangement pattern 58 to keep. 6A shows an example of the arrangement pattern 58 the article M in the shell 54 , The arrangement pattern 58 comprises four pattern elements P1, P2, P3 and P4, each representing the position of an article M. In the in 6A The example shown is an arrangement form 59 ( 6B ) representing the pattern elements P1, P2, P3 and P4 by their coordinate values (X, Y) and relative rotation angles Φ in a coordinate system 57 defined, whose origin is at the upper right corner of the shell 54 located in a two-dimensional image. As in the illustrated arrangement form 59 12, the coordinate values of P1, P2, P3 and P4 are (X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2), and the rotation angles of P1, P2, P3 and P4 are 0 ° C , -90 ° C, 90 ° C and 180 ° C.
The first to third tax area 20A . 20B and 20C control the first to third mechanical range 14A . 14B and 14C such that the articles M are arranged one after the other according to the arrangement pattern 58 with the above-described pattern elements P1, P2, P3 and P4 in the shell 54 to be placed. In 6A four arrowhead-shaped articles M are shown, which are in accordance with the arrangement pattern 58 are arranged in the shell. In the drawing, the pattern elements P1, P2, P3 and P4 are respectively indicated by a character "+" for better understanding, but the pattern elements P1, P2, P3 and P4 themselves need not have shapes. In addition, in the drawing, the four items M are in the arrangement pattern 58 arranged by plotting the geometric centers of the two-dimensional outer shapes of the respective articles M with the pattern elements P1, P2, P3 and P4, but alternatively, other common points defined suitably on the respective articles M may be associated with the corresponding pattern elements P1, P2, P3 and P4 be registered.
7 shows an example of the hardware configuration of each of the first to third control areas 20A . 20B and 20C or the robot control area 20 ( 1 ). Each of the first to third tax areas 20A . 20B and 20C or the robot control area 20 is with a CPU 60 equipped with a microprocessor. A ROM 62 , a ram 64 , an SRAM 66 , a digital signal processor (DSP) data storage 68 and a digital signal processor (DSP) 70 are with the CPU 60 over a bus 71 connected. The ROM 62A stores a program for controlling the entire system and the RAM 64 temporarily saves from the CPU 60 data to be processed. The SRAM 66 stores an operation program and setting data for each of the first to third control areas 20A . 20B and 20C or the robot 14 , The DSP 70 is a processor for processing by the encoder 52 output signals and the DSP data memory 68 saves setting parameters and from the DSP 70 processed data. The DSP 70 has the function of detecting an output from the encoder 52 at a given time according to an instruction from the CPU 60 and writing the result to an associated area in the DSP data store 68 ,
Each of the first to third tax areas 20A . 20B and 20C or the robot control area 20 includes an axis control area 72 for controlling the corresponding one of the mechanical areas 14A . 14B and 14C or the robot 14 , The axis control area 72 is with the corresponding of the mechanical areas 14A . 14B and 14C or the robot 14 via a servo circuit 74 connected. With this configuration, each of the first to third control areas can 20A . 20B and 20C or the robot control area 20 the corresponding one of the mechanical areas 14A . 14B and 14C or the robot 14 Taxes. Each of the first to third tax areas 20A . 20B and 20C or the robot control area 20 also includes a communication interface 76 and an input / output interface 78 and can through these interfaces with the other two of the control areas 20A . 20B and 20C and with the image processing area 18 and peripheral devices such as the photocell sensor 56 , communicate.
8th shows an example of the hardware configuration of the image processing area 18 , The image processing area 18 is with a CPU 80 equipped with a microprocessor. A ROM 82 , a ram 84 , a screen interface 88 to connect to an external screen 86 and a camera interface for connection to each of the first to third cameras 16A . 16B and 16C or the image pickup area 16 ( 1 ) are via a bus 92 with the CPU 80 connected. From each of the first to third camera 16A . 16B and 16C or the image pickup area 16 taken pictures are in the RAM 84 via the camera interface 90 saved. The camera interface 90 can any of the cameras 16A . 16B and 16C control independently of the others, allowing an image capture process always with a specific timing regardless of the shooting conditions of the cameras 16A . 16B and 16C can be executed. The ones in the RAM 84 stored data are from the CPU 80 analyzed and from the image processing area 18 as the information representing the position and orientation of each article M (information about the starting position D1, information about the displaced position D2 and additional information about the displaced position D3). The image processing area 18 may also contain information about the appearance properties, such as the shape and color of the article M, of which in the RAM 84 get stored data. The ROM 82 stores an analysis program and various setting information for the image processing area 18 , The CPU 80 supports so-called multi-core, multi-threaded operations and can analyze the first to third camera 16A . 16B and 16C run recorded image data simultaneously. The image processing area 18 also includes a communication interface 94 and an input / output interface 96 and can have these interfaces with each of the control areas 20A . 20B and 20C and peripheral devices.
As in 9 shown, each of the tax areas 20A . 20B and 20C or the robot control area 20 and the image processing area 18 in a single common control device 98 be integrated. The CPU 80 in the image processing area 18 is with the CPU 60 in each of the tax areas 20A . 20B and 20C or the robot control area 20 over a bus 100 in the control device 98 connected. The CPU 80 in the image processing area 18 can on the SRAM 66 access to various setting information too save or to the DSP datastore 68 access to information of the encoder 52 via the CPU 60 in each of the tax areas 20A . 20B and 20C or the robot control area 20 to read. Besides, the CPU can 80 in the image processing area 18 as required with the communication interface 76 and the input / output interface 78 via the CPU 60 in each of the tax areas 20A . 20B and 20C or the robot control area 20 get connected. In this case, the communication interface 94 and the input / output interface 96 be omitted.
In the control device 98 can the CPU 60 , the ROM 62 and the RAM 64 in each of the tax areas 20A . 20B and 20C or the robot control area 20 as a replacement for CPU 80 , ROME 82 and RAM 84 in the image processing area 18 be used. 10 shows the control device 98 with such a simplified hardware configuration.
11 shows in functional block form the different areas that are in the image processing area 18 and the first to third control areas 20A . 20B and 20C the article transfer device 40 are integrated to handle information regarding the articles M and information concerning the work of taking up and transmitting for the articles M. In the drawing, arrows show the flow of the information regarding the articles M and the flow of information concerning the work of picking up and transferring the articles M.
As in 11 shown, include the first to third control area 20A . 20B and 20C Arbeitsinhalteinstellbereiche 102A . 102B and 102C , The work content setting areas 102A . 102B and 102C provide the work content regarding the work of taking up and transferring articles, such as the work-sharing ratio between the first to third mechanical areas 14A . 14B and 14C ( 5 ), the layout pattern 58 the article M in the shell 54 and the workrooms 20A . 22B and 22C ( 5 ) on the conveyor 12 , The by one of the work content adjustment areas 102A . 102B and 102C set work content can be sent over any communication network to any of the work content setting areas 102A . 102B and 102C in the tax areas 20A . 20B . 20C be transferred so that the work content between the Arbeitsinhalteinstellbereichen 102A . 102B and 102C in the tax areas 20A . 20B and 20C can be shared.
The work division ratio between the first to third control areas 14A . 14B and 14C will be described below. By way of example, consider a situation in which the items M1, M2, M3, M4 are selected in this order from the top as in 12 are shown on the conveyor 12 be transported and the work areas 22A . 22B and 22C ( 5 ) and from the first to the third mechanical area 14A . 14B and 14C ( 5 ) and to the shell 54 on the discharge conveyor 42 are transmitted, so that the articles M1, M2, M3, M4 in the arrangement pattern described above 58 are arranged. An example of the corresponding mechanical areas 14A . 14B and 14C The work of picking up and transferring which follows is described below for the case where the work division ratio between the first to third mechanical portions 14A . 14B and 14C is set to 1: 1: 1 in the situation described above.
First, it attacks from the first to the third mechanical area 14A . 14B and 14C ( 5 ) of the first mechanical portion located at the most upstream position 14A the first supplied article M1 from the conveyor 12 and places it according to the arrangement pattern 58 on the pattern element P1 in the shell 54 , Then let the first mechanical area 14A The articles M2 and M3 pass, leaving the second and third mechanical area 14B and 14C downstream, they can handle so that the fixed work division ratio R = 1: 1: 1 is satisfied, and then accesses the last article M4 and places it according to the arrangement pattern 58 on the pattern element P2 in the shell 54 , The second mechanical area 14B engages the article M2, the first of the two articles, the first mechanical section 14A lets happen from the conveyor 12 and places it according to the arrangement pattern 58 on the pattern element P3 in the shell 54 , Then the third mechanical area engages 14C the remaining article M3 from the conveyor 12 and places it according to the arrangement pattern 58 on the pattern element P4 in the shell 54 , At this time, the division ratio of the work of grasping and transferring is between the first to third mechanical ranges 14A . 14B and 14C 2: 1: 1.
When a plurality of articles M (not shown) are fed following articles M1 to M4, the first mechanical region leaves 14A passing the first two articles M to meet the specified work-sharing ratio R = 1: 1: 1, and then picking up the third article M and transferring it to the next tray 54 , Then grab the second and the third mechanical area 14B and 14C each one of the articles on which the first mechanical area 14A let pass, and transfer him to the next bowl 54 , At this time, the split ratio is the work of picking up and transferring between the first to third mechanical areas 14A . 14B and 14C 3: 2: 2nd By repeating this process, the first lead up to third mechanical area 14A . 14B and 14C the work of taking up and transmitting on all on the conveyor 12 conveyed articles M until finally the work-sharing ratio R = 1: 1: 1 is met.
In the example of the work described above, the first to third mechanical areas are engaged 14A . 14B and 14C the plurality of articles M from the conveyor 12 in the order in which the articles M are supplied, and transfer them to the shell 54 in the order of the pattern elements P1 to P4 in the arrangement pattern 58 , In addition, to achieve the division ratio closest to the work division ratio R, the first to third mechanical ranges are determined 14A . 14B and 14C in this order, whether or not the corresponding articles M should be picked up. A desired operation rule may be made in advance by the user of the article transfer device 40 can be determined and made in advance in the SRAM 66 in the first to third control areas 20A . 20B and 20C get saved.
Again with reference to 11 includes the first tax area 20A a shell detection area 104 , an information management area 106A and a work execution area 108A , The second tax area 20B includes an information management area 106B and a work execution area 108B , The third tax area 20C includes an information management area 106C and a work execution area 108C ,
The shell detection area 104 detects the shell 54 ( 5 ) based on an input of a switch detection signal from the photocell sensor 56 ( 5 ), reads the detection value from the encoder 52 ( 5 ) of the discharge conveyor 42 ( 5 ) at the time of detection and transmits the detection value to the information management area 106A , When the position of the photocell sensor 56 and the direction of movement of the discharge conveyor 42 in advance of the work content setting area 102A a robot coordinate system of the first mechanical area 14A ( 5 ) (in the embodiment according to FIG 1 the conveyor coordinate system 30 ), the information management area 106A the position of the detected shell 54 and the coordinate values and relative rotation angles of the pattern elements P1 to P4 of the arrangement pattern 58 ( 12 ) in the robot coordinate system using the detection value of the encoder 52 obtained on the detection of the shell of the Schalendetektierbereich 104 is transmitted. Information regarding the coordinate values and relative rotation angles of the pattern elements P1 to P4 temporarily becomes in the information management area 106A but information regarding any of the pattern elements P1 to P4 on which the first mechanical area is stored 14A has not placed the item M is referred to as the information of the packet γ to the information management area 106B in the second control area 20B transmitted over the communication network. Similarly, in the information management area 106B stored information of the pattern elements P1 to P4 information regarding any of the pattern elements P1 to P4 on which the second mechanical area 14B has not placed the item M as the information of the packet γ to the information management area 106C in the third tax area 20C transmitted over the communication network.
As in 11 shown includes the image processing area 18 in addition to the article detection area 32 , the article tracking area 34 and the auxiliary tracking area 50 a condition setting area 110 and an article management area 112 , The condition setting area 110 sets various conditions for detecting or tracking each article M, such as a teaching model image for the article M to be detected or tracked, parameters used for article detection (coordinate values, rotation angles, dimensions, etc.) and calibration data for the first to third cameras 16A . 16B . 16C ( 5 ). The condition setting area 110 can provide information regarding the work spaces 22A . 22B and 22C the first to third mechanical range 14A . 14B and 14C ( 5 ) by communicating with the work content setting area 102A in the first control area 20A gain.
The article detection area 32 begins the initial detection of the article M using the first camera 16A when the conveyor 12 has moved for a predetermined period from the beginning of the work. Alternatively, the initial detection of the article M may be started when an external sensor, such as a photocell sensor (not shown), controls the movement of the conveyor 12 detected for a predetermined period of time. The article detection area 32 can the information about the initial position D1 ( 5 ) of the article M using any of a variety of known detection algorithms, such as a method that retrieves the article M from the first camera by searching for an image that matches a pre-registered model image 16A recorded images (for example, a method of normalized correlation), a method that extracts the contour of the article M from the pre-registered model image and based on the information of the detected contour, the position information of the article M in the from the first camera 16A captured image ( for example, a generalized Hough transform method), or a method that detects the position information of the article M by detection from that of the first camera 16A acquired image of a region having an area size that falls within a pre-registered area (for example, a blob detection method).
Upon successful initial detection of the article M, the article detection area is set 32 a mark (hereinafter referred to as a new detection mark) indicating that the detected article M is newly detected, and transmits the new detection mark to the article management area 112 together with such information as the information about the starting position D1 of the detected article M, the camera ID of the first camera 16A used to take the image of the article M, the image taking time of the article M and the new article ID of the article M. The article management area 112 transmits them from the article detection area 32 received information to the article tracking area 34 while also providing the information as the information of the packet α to the information management area 106A in the first control area 20A is transmitted.
The article tracking area 34 begins tracking the article M using the second camera 16B if the article M, that of the article management area 112 received information about the initial position D1, in the field of view 24 ( 5 ) of the second camera 16B occurred. The article tracking area 34 follows the article M by the article management section 112 received information about the starting position D1 of the article M as the initial value. The unique article identifier associated with article M at the time of initial detection by the article detect area 32 will be throughout the article tracking area 34 maintained process carried out. Tracking the article M through the article tracking area 34 is sequentially performed by comparing the various information (position information and information on the external appearance characteristics) obtained at the time of the current image pickup with the different information obtained at the time of previous or previous image pickup, based on the image data the second camera 16B by repeatedly recording the image of the article M with the extremely short second period D2. As the detection algorithm for performing this type of sequential tracking, for example, particulate filtering (also referred to as a particulate filter or sequential Monte Carlo method) that can estimate a nonlinear non-Gaussian system can be used (for example, for particulate filtering see " Journal of the Robotics Society of Japan, Vol. 29, No. 5, pp. 427-430, 2011 ").
The article tracking area 34 transmits information such as the most recent information about the displaced position D2 obtained during the tracking (FIG. 5 ) of the article M, the camera identifier of the second camera used to capture the image of the article M. 16B , the image taking time of the article M and the article identifier of the article M successively to the article management area 112 , The article management area 112 transfers these from the article tracking area 34 received information to the auxiliary tracking area 50 while also providing the information as the information of the packet β to the information management areas 106A . 106B and 106C in the first to third control areas 20A . 20B and 20C transfers.
The auxiliary tracking area 50 begins tracking the article M using the third camera 16C if the article M, that of the article management area 112 received information about the displaced position D2 corresponds to the field of view 46 ( 5 ) of the third camera 16C occurred.
The auxiliary tracking area 50 tracks the article M by taking as its initial value that of the article management area 112 received information about the displaced position D2 takes when the article M in the field of view 46 ( 5 ) of the third camera 16C occurred. The one from the auxiliary tracking area 50 The executed tracking process is substantially the same as that of the article tracking section described above 34 executed persecution process, except that the third camera 16C is used to capture the image of the article M. The auxiliary tracking area 50 transmits information such as the most recent additional information about the displaced position D3 obtained during the tracking (FIG. 5 ) of the article M, the camera identifier of the third camera used to capture the image of the article M. 16C , the image taking time of the article M and the article identifier of the article M, successively to the article management area 112 , The article management area 112 , that of the auxiliary tracking area 50 receives received information, transmits the information as the information of the packet β to the information management areas 106A . 106B and 106C in the first to third control areas 20A . 20B and 20C ,
Instead of the particulate filter method described above, the article tracking range 34 and the auxiliary tracking area 50 as the article tracking detection algorithm use any of various known pattern recognition algorithms, such as the normalized correlation method, the generalized Hough transform method, or the blob detection method, as in the article detect area 32 , The normalized correlation method, the generalized Hough transform method, the blob detection method, etc., usually provide a higher detection accuracy than the particulate filter method, but the processing speed is slower. Using the normalized correlation method, the generalized Hough transform method, or the blob detection method or the like as the article tracking method, lengthening the time required for tracking by narrowing the range of the detection parameter (coordination values, rotation angle, dimensions, area size, etc.) or the detection search area in the one of the second or third camera 16B or 16C taken two-dimensional image compared to that of the article detection area 32 to prevent the area used for the initial detection. For example, when the rotation angle is used as the detection parameter, the article detection area is obtained 32 the information about the initial position D1 of the article M by performing the initial detection, for example, within the range of ± 180 degrees, because at this time the position and orientation of the article M are unknown; if, on the other hand, the article tracking area 34 and the auxiliary tracking area 50 Using the method of normalized correlation, generalized Hough transform or blob detection methods, or the like, the article tracking range may be used 34 and the auxiliary tracking area 50 perform the article detection (tracking) by narrowing the range to within ± 5 degrees, assuming that the position and orientation do not change significantly from the result of the initial detection or previous tracking.
Next is an example of the article tracking area 34 ( 11 ) process for tracking the articles M in the image processing area 18 with reference to 13 described. The one from the auxiliary tracking area 50 executed tracking process ( 11 ) in the image processing area 18 is substantially the same as the article tracking section described below 34 executed tracking process.
As in 13 is shown, a plurality of arranged in any manner items M on the conveyor 12 transported; stops here if any of the items M are within the workspace 22A located, the first mechanical area 14A the article M and attacks him and if another of the article M is within the working space 22B is the second mechanical area stops 14B the article M and attack him. On the other hand, the second camera takes 16B the pictures of everyone in the field of vision 24 is repeatedly on the second period T2. The computational burden of the item tracking area executing the tracking process 34 is mainly due to the size of the second camera 16B recorded image data, the type of algorithm used for the article detection, the number of articles M to be detected and the area of the article detection parameter. Assuming that the number of articles M to be detected and the algorithm used are the same, the computational load of the article tracking area 34 by reducing the resolution of the image data or by narrowing the range of the detection parameter.
The resolution, ie the size of the second camera 16B captured image data affects the accuracy with which the article tracking area 34 each article M detected, which in turn affects the accuracy of the first and second mechanical area 14A and 14B affected item retention process. In reality, within the work spaces 22A and 22B the first and the second mechanical area 14A and 14B high detection accuracy and high operational accuracy required, whereas outside the work spaces 22A and 20B the detection accuracy is not relevant to that of the first and the second mechanical area 14A and 14B has executed holding operation. Considering this fact, when the resolution of the outside of the workrooms 20A and 22B used image data compared to those within the workspaces 22 at 22B used image data in the field of view 24 the second camera 16B is reduced, the computational burden of the article tracking area 34 be reduced.
As in 11 shown, the condition setting area communicates 110 in the image processing area 18 with the work content setting area 102A in the first control area 20A and obtains information about the locations on the conveyor 12 the workrooms 22A and 22B the first and the second mechanical area for 14A and 14B as position information in the conveyor coordinate system 30 , The article tracking area 34 can be configured to work within the workspace 22A . 22B Article M using image data with a first resolution, which corresponds to a required detection accuracy to detect and one outside the work space 22A . 22B using the image data with a second resolution lower than the first one Resolution, based on the condition setting range 110 obtained information of the workrooms 22A and 22B to detect. This configuration serves to calculate the computational burden of the item tracking area 34 compared to reduce the configuration, in which all in the field of view 24 the second camera 16B be detected using the image data with the first resolution. In the auxiliary tracking area 50 For example, the computational burden of the tracking process can also be reduced using a similar technique.
In the above-described configuration, the article tracking area is 34 configured to capture the image data at the first resolution (or high resolution) by capturing the interior of the workspaces 22A . 22B the first and the second mechanical area 14A and 14B ( 13 ), and image data at the second resolution (or low resolution) obtained by capturing the exterior of the workspaces 22A and 22B to be able to gain. Similarly, the auxiliary tracking area is 50 configured to capture the image data with the first resolution obtained by capturing the interior of the workspace 22C of the third mechanical area 14C ( 5 ), and the second resolution image data obtained by capturing the exterior of the workspace 22C to be able to gain. To obtain the data with different resolutions, the second or third camera can 16B or 16C with a switching mechanism 114 ( 4 ) configured to change a resolution between the first resolution and the second resolution. In this configuration take the second or third camera 16B or 16C Pictures of the entire field of vision 24 or 46 with the first resolution as well as the second resolution on, so that the article tracking area 34 or the auxiliary tracking area 50 located within the workrooms 22A and 22B or the workroom 22C M using the image data with the first resolution can detect and located outside the work spaces 22A and 22B or the workroom 22C can detect M using the image data with the second resolution. If the second or third camera 16B or 16C not with the switching mechanism 114 equipped, takes the second or third camera 16B or 16C a picture of the field of vision 24 or 46 with a fixed resolution; in this case, the article tracking area 34 or the auxiliary tracking area 50 obtain the image data at the second resolution in addition to the image data at the first resolution by reducing the resolution of the image data using a known image processing technique.
Instead of or in addition to the selectable resolution configuration described above, the article tracking area may 34 be configured to select and use one or the other of two detection algorithms, of which the first detection algorithm ensures a first processing speed and first accuracy, and the second detection algorithm provides a second processing speed higher than the first processing speed, and / or a second one Accuracy that is less than the first accuracy, guaranteed. In this case, the article tracking area 34 be configured to work inside the workspaces 22A and 22B the first and the second mechanical area 14A and 14B Detecting article M using the first detection algorithm and an outside of the workspaces 22A and 22B to detect the article M using the second detection algorithm. The normalized correlation method, generalized Hough transform method or blob detection method or the like may be used as the first detection algorithm. On the other hand, the particulate filter method can be used as the second algorithm. The pursuit area 50 can be configured in a similar way.
For example, the article tracking area detects 34 from the image data, the second camera 16B by taking the entire field of view 24 at the second resolution (or low resolution), the articles M in the entire field of view 24 using the second detection algorithm (or the particulate filter method). In the article tracking section 34 After the tracking process is executed, this article detection is executed as a preparatory step for the step of acquiring the accurate information about the displaced position D2 of each article M. Next, the article tracking area detects 34 from the image data, the second camera 16B by taking the entire field of view 24 has generated at the first resolution (or high resolution), or more precisely from the image data of a region containing the first and second working spaces 22A and 22B contains, and is slightly larger than the work spaces 22A and 22B , any item M, of which at least a part within one or the other of the work spaces 22A and 22B is located, using of the first detection algorithm (or the normalized correlation method, generalized Hough-transformation method or blob-detection method). In the article tracking section 34 In accordance with the executed tracking process, this article detection is performed as a supplementary step to the step of obtaining the accurate information about the displaced position D2 of each article M. Using the first detection algorithm (or the normalized correlation method, generalized Hough-transformation or blob-detection method), since the region containing the articles M may already be to some extent within the range of the detection parameter (coordinate values, rotation angles, dimensions, Area size, etc.) is narrowed down by the preliminary detection performed using the second detection algorithm (or the particulate filtering method), an extension of the time required for the tracking can be prevented by corresponding the area of the detection parameter (coordinate values, rotation angles, dimensions, area size, etc.) is diminished.
For anything within the common work spaces 22A and 22B Article M may be the article tracking area 34 execute the tracking process using detection algorithms that achieve different processing speeds and different accuracies. For example, the detection using the first (high resolution) detection algorithm may be performed only on a fixed number of articles M as counted from the downstream side in the conveying direction, which is likely to be from the first or the second mechanical region 14A or 14B be taken up. For example, if the hand of the first or second mechanical area 14A or 14B configured to hold two items M together, the item tracking area 34 be configured to detect from the downstream side only two articles with high accuracy in each cycle of the article tracking process that is repeatedly executed. According to this configuration, the computational load of the article tracking area 34 by reducing the number of articles M to be detected.
The article transfer device 10 according to 1 may be configured in a similar manner as described above. In the article transfer device 10 can the article tracking area 34 be configured to image data at a first resolution by capturing the interior of the workspace 22 be included where the robot 14 performs the article retrieval work, and image data with a second resolution obtained by capturing the exterior of the workspace 22 to obtain, wherein the second resolution of the image data with the second resolution is less than the first resolution of the image data with the first resolution; and may also be configured to be located within the workspace 22 Detecting article M using the image data with the first resolution and an outside of the workspace 22 M to be detected using the image data with the second resolution.
In addition, in the article transfer device 10 the article tracking area 34 be configured to select either a first detection algorithm that ensures a first processing speed and a first accuracy, or a second detection algorithm that ensures a second processing speed and a second accuracy, wherein the second processing speed is higher than the first processing speed and the second accuracy is less than the first precision, and to use the selected one of the detection algorithms; and may also be configured to be located within the workspace 22 of the robot 14 Detect the article M using the first detection algorithm and an outside of the workspace 22 to detect the article M using the second detection algorithm.
Next is another example of the article tracking area 34 ( 11 ) in the image processing area 18 executed process of tracking the article M with reference to 14 described. The one from the auxiliary tracking area 50 ( 11 ) executed tracking process in the image processing area 18 is substantially the same as the above-described article tracking section 34 executed tracking process.
As in 14 shown, a situation is considered in which two articles M1 and M2, one of which on the conveyor 12 overlaps the workspace 22B of the second mechanical area 14B be supplied. In this situation, it is assumed that the article M2, the main part of which is obscured by the article M1, is initially from the article detection area 32 ( 11 ) could not be detected. For the article M2, which could not be detected initially, the information of the packet α does not reach the first control area 20A ( 11 ) and thus the information of the package γ is not created; accordingly recognizes the second control area 20B ( 11 ) the presence of the article M2 not. For the article M1, the beginning of the article detection area 32 was detected, recognizes the second control area 20B on the other hand, the presence of the information of the packet γ.
In the situation described above, the second control section hits 20B a decision as to whether the article M1 is in accordance with the work content setting range 102B ( 11 ) adjusted employment division ratio is to be taken or not. If it is decided that article M1 pick up, and the second mechanical area 14B the article M1 according to an instruction from the work execution area 108A from the conveyor 12 The article M2 is exposed and viewed by the second camera 16B taken a picture of it. The article tracking area 34 in the image processing area 18 who does not realize that the second mechanical area 14B the article M1 from the conveyor 12 has tried to continue to pursue Article M1. That means the article tracking area 34 Further, the information about the displaced position D2 is repeatedly obtained until a predetermined time elapses from the moment when the robot 14 (the second mechanical area 14B ) has taken up Article M1. The article M2, whose look is similar to that of the article M1, is taken from the article tracking area 34 detected in a similar manner as the detection of the article M1.
If the value of the detection parameter (coordinate values, rotation angle, dimensions, area size, etc.) for the article M2 on the conveyor 12 just as for the article M1, the article tracking area would 34 Continue to apply Article M2 as M1. If the value of the detection parameter (coordinate values, rotation angle, dimensions, area size, etc.) for the article M2 on the conveyor 12 on the other hand, differs from that for the article M1, as shown here, pursues the article tracking area 34 the article M2 as a newly detected article M. If the article tracking area 34 the article M1 in the workspace 22B when the preliminary detection of the article M1 is first performed using the second (low resolution) detection algorithm (or the particulate filtering method) and then the supplemental detection of the article M1 using the first (high resolution) detection algorithm (or the normalized correlation method Method of generalized Hough transform or blob detection method) by narrowing the range of the detection parameter (coordinate values, rotation angles, dimensions, area size, etc.) as described above, it may become difficult to detect the article M2 after that the detection parameter (coordinate values, rotation angles, dimensions, area size, etc.) differs from that of the first article M1. Thus, the article tracking area increases 34 if he could not detect the article M1 in the additional detection although he detected the presence of an article which looked like the article M1 in the preliminary detection, the range of the detection parameter (coordinate values, rotation angle, dimensions, area size, etc.) by a predetermined appropriate amount and repeatedly attempts the supplemental detection of the article M1 using the first detection algorithm (or the normalized correlation method, generalized Hough-transformation method, or blob-detection method). In the example according to 14 Articles M1 and M2 are approximately in the same position; the only difference is their orientation. For this reason, the article tracking area can 34 For example, if the range of the rotation angle as the detection parameter narrowed to ± 5 degrees is widened to ± 180 degrees, for example, the article M1 (actually, article M2) is detected.
The article tracking area 34 which has detected the article M2 as the article M1, takes into consideration that the orientation of the article M1 has changed during its tracing, and treats the article M1 whose orientation has changed (actually, article M2) as a newly detected article M. The position information of the article tracking area 34 newly detected article M is processed in the same way as the information about the initial position D1. The article tracking area 34 sets a new detection mark for the newly detected article M (the article M2) and transmits this new detection mark to the article management area 112 together with information such as the position information (corresponding to the initial position information D1) of the detected article M, the camera ID of the camera used to acquire the image of the article M. 16B , the image taking time of the article M and the new article ID of the article M. In general, the article management area transfers 112 that the information from the article tracking area 34 has received the information as the packet β to the first to third control areas 20A . 20B . 20C as described above, but if that of the article tracking area 34 When the received information carries the re-detection mark, the article management area transmits 112 this information as the information of the packet α only to the information distribution area 106A in the first control area 20A ,
In the example off 14 created because the article M2 the work space 22A of the first mechanical area 14A ( 5 ) has already happened and in the workspace 22B of the second mechanical area 14B occurred, the information management area 106A the information of the packet γ based on the information of the article tracking area 34 about the article management area 112 received packet α and transmits the information of the packet γ directly to the information management area 106B in the tax area 20B , The information management area 106B that recognizes that the second mechanical area 14B the article M1 from the conveyor 12 has deleted the information of the package γ concerning the article M1 and stores the information of the package γ concerning the new article M (the article M2) and makes a decision as to whether the new article M (the article M2) is in accordance with the work content setting range 102B adjusted division of labor should be taken or not.
Even if the position of the item M on the conveyor 12 The article tracking area can be relocated 34 After the article detection area 32 ( 11 ) has obtained the information about the starting position D1 of the article M, the article M is tracked by performing substantially the same tracking process as the above-described tracking process performed when the articles overlap each other. For example, in the in 14 when the part of the article M2 obscured by the article M1 is small, the article detecting area 32 initially both articles M1 and M2. The article tracking area 34 pursues both Articles M1 and M2 and if Articles M1 and M2 in the work space 22B of the second mechanical area 14B enter, controls the second control area 20B ( 11 ) the second mechanical area 14B such that it picks up the article M2 in front of the covered article M1, since the article M2 is located downstream of the article M1. When the article M2 is picked up first, there is a high probability that the position of the article M1, which is no longer supported, on the conveyor 12 is relocated.
If the position of the item M1 on the conveyor 12 is relocated, the article tracking area leads 34 the prosecution process by considering article M1 as a new article. At this time, the approximate position of the article M1 after the positional shift can be determined by performing the preliminary detection using the second algorithm (or the particulate filtering method). After the preliminary detection, the detection range is narrowed to the range containing the approximate position determined thereby, and by performing the supplementary detection using the first detection algorithm (or the normalized correlation method, generalized Hough-transformation method, or blob detection method), the accurate information about the displaced position D2 of the article M1 after the positional displacement are obtained. When the amount of positional displacement of the article M1 on the conveyor 12 is so large that the article M1 can not be detected by the supplementary detection, the supplementary detection should be repeatedly tried by broadening the range of the detection parameter (coordinate values, rotation angle, dimensions, area size, etc.) by a predetermined suitable amount as in FIG The tracking process described above is performed when the articles overlap each other.
Next, referring to the flowcharts of FIG 15 to 22 those of the in 11 shown image processing area 18 and the robot control area 20 executed processing sequences (the first to third control area 20A . 20B . 20C ) described in detail.
15 shows the process in which the information management area 112 in the image processing area 18 Article information from the article detection area 32 , the article tracking area 34 and the auxiliary tracking area 50 receives and these as the information of the packet α or the information of the packet β to the information management areas 106A . 106B and 106C by doing first to third tax area 20A . 20B and 20C transfers. First, in step S10, the article management area receives 112 from the article detection area 32 , the article tracking area 34 or the auxiliary tracking area 50 skillful information. The article information is sent for each detected article M. In step S11, the article management area checks 112 whether the received article information is information that is not suitable to be sent to the first to third control areas (ie, whether to add a transmission deactivation flag (described later) to the article information). If such a transmission disable flag is appended, the process returns to step S10 and waits until the next item information is received. If no such transfer deactivation flag is appended, the article management area moves 112 proceeds to step S12 and checks whether the detected article is a newly detected article (whether a re-detection mark of the article information is attached or not). If the re-detection mark is attached, the article management area creates 112 a packet α in step S13 and sends the packet α to the first control section 20A , Since the information about the starting position D1 of the article M contained in the packet α is information necessary for the article tracking area 34 begins with the prosecution of the article M, the article management section sends 112 in step S15, the information carried in the packet α (hereinafter referred to as new article information) to the article tracking area 34 ,
If the re-detection mark is not attached to the article information in step S12, the article management area creates 112 in step S16, a packet β and sends the packet β in steps S17, S18 and S19 to the information management areas 106A . 106B and 106C in the first to third control areas 20A . 20B and 20C , Next, in step S20, the article management area checks 112 Whether the article information received in step S10 is that of the second camera 16B is generated. If the information is information not from the second camera 16B is generated, the process returns to step S10 and waits until the next article information is received. If the information is the information from the second camera 16B is generated, the information about the displaced position D2 of the article M contained in the packet β may be information necessary for the auxiliary tracking region 50 begins with the prosecution of the article M, so the article management area 112 in step 21, the information carried in the packet β to the auxiliary tracking area 50 sends. When the process described above is completed, the process returns to step S10 and waits until the next article information is received.
16 shows the flow of the article tracking area 34 in the image processing area 18 executed prosecution process. The article tracking area 34 repeatedly executes the tracking process of the article M, driving a timer at time intervals corresponding to the second period T, and appropriately processes the information about a new article every time it comes from the article management area 112 Will be received. First, in step S30, the article tracking area receives 34 a timer interrupt signal that occurs at predetermined time intervals, or receives information about a new article from the article management area 112 and checks in step S31 whether the interrupt signal is received or not. If the received signal is not the interrupt signal (ie, if information about a new article is being received), the article tracking area is drawing 34 Thus, in step 37, the article to be re-indexed in a pre-stored article tracking list updates the article tracking list and then returns to step S30 to wait for the next reception. On the other hand, when the interrupt signal is received, the article tracking area starts 34 the tracking process for the current cycle. The article tracking area 34 acquires two-dimensional image data obtained by taking the field of view in step S32 24 the second camera 16B ( 5 ) are generated at the first resolution (or high resolution), and at step S33 obtains two-dimensional image data obtained by capturing the field of view 24 be generated at the second resolution (or low resolution).
The first resolution image data acquired in step S32 is used for "additional detection" (FIG. 18 ) are used in step S35, and the second resolution image data acquired in step S33 are used for "preliminary detection" (FIG. 17 ) is used in step S34. Information regarding the article M detected in step S35 is determined by the "article information transmission process" (step S36) (FIG. 19 ) to the article management area 112 cleverly. After sending the article M information to the article management area 112 updates the article tracking area 34 the recording time and moving picture information D2 of the contained article M carried in the article tracking list so that the information updated in the next cycle of the tracking process to be executed on the same article M can be used in the "preliminary detection" and the "supplementary detection"; if any item M is rediscovered in the supplemental detection, the newly discovered item M is added to the item tracking list. After updating the article tracking list, the process returns to step S30 and waits to receive the next interrupt signal or information about a new article.
17 shows the detailed flow of the in the flowchart according to 16 implemented preliminary detection step S34. The preliminary detection is performed by using the image data of the second resolution (or the low resolution). In step S40, the article tracking area reads 34 the acquired image with the second resolution (or low resolution). Next, in step S41, the article tracking area reads the article tracking list containing the information of the article M to be tracked in the current process. The tracking process is executed for each article M in the thus-read article tracking list. In step S42, the article tracking area initializes 34 1, a counter variable "i" indicating that the item M to be tracked in the current process is the "ith" item in the item tracking list. In step S43, the counter variable i is compared with the number of articles contained in the article tracking list. If the counter variable i is greater than the number of articles, the preliminary detection is completed by determining that the tracking of all the articles M contained in the article tracking list is completed. If the counter variable i is not larger than the number of articles M, the article tracking area travels 34 to step S44 to execute the preliminary detection of the third article M in the article tracking list. In this process, the particulate filter method is selected as the preliminary detection algorithm; however, another suitable detection algorithm can also be selected.
Next, in step S45, the article tracking area is determined 34 whether the approximate Position information of the article M to be tracked was obtained as the result of the preliminary detection (that is, whether the preliminary detection was successfully performed); if the preliminary detection has been successfully completed, the article tracking area moves 34 proceeds to step S46 and updates the information carried in the article tracking list, such as the position information, image recording time, etc. of the article M, with the information acquired in the current process. If the preliminary detection has not been successfully performed, this suggests that the item M to be tracked physically from the conveyor 12 disappeared because, for example, the first or the second mechanical area 14A or 14B have taken up the article M; therefore, in step S47, the information of the article M is deleted from the article tracking list, so that the article M is no longer pursued. In the flowchart according to 17 if the detection fails on the first attempt, the article information is deleted, but alternatively, if the detection fails, the preliminary detection may be repeatedly tried, and if the preliminary detection fails three times in succession, the article information may be deleted; In this way, the condition for deleting the article information can be changed as desired. During the period in which the preparatory detection of the article M to be tracked is tried repeatedly, the conveying speed of the conveyor device can 12 can be calculated from the result of tracking another article, and using the thus calculated conveying speed and the time interval between the article image taking times in the preceding and present preparatory detection, the current position and orientation of the article M to be tracked can be predicted, and the position information can be updated accordingly , When the updating or deletion of the article information is completed, the article tracking area travels 34 to step 48 to increment the counter variable by one, and then returns to step 43 to evaluate the value of the counter variable i.
18 shows the detailed flow of the in the flowchart according to 16 implemented step S35 of the supplementary detection. The supplemental detection is performed using the image data of the first resolution (or high resolution) obtained by taking pictures of the work spaces 22A . 22B and 22C the first to third mechanical range 14A . 14B and 14C is produced. In step S50, the article tracking area initializes 34 to 1, the counter variable i indicating that the corresponding one of the first to third mechanical ranges 14A . 14B and 14C the i-th mechanical region is from the upstream side. Next, in step S51, the counter variable i becomes the total number of the first to third mechanical ranges 14A . 14B and 14C (three in the present embodiment) and if the counter variable i is greater than the total number, the additional detection in the current tracking process is terminated. If the counter variable i is not larger than the total number, the article tracking area moves 34 to step S52 to check if the working space of the ith mechanical area is within the field of view 24 the second camera 16B located. When the working area of the ith mechanical area is within the field of view 24 is located, the image data having the first resolution (or the high resolution) generated by taking the working space of the ith mechanical area is read in step S53. The check of whether the work space is in the visual field can be performed by converting the coordinate values of the work space in the robot coordinate system into the coordinate values in the camera coordinate system or by the user setting the work space in advance as the tracking space. In the execution according to 5 will, since the work space 22A of the first mechanical area 14A and the workspace 22B of the second mechanical area 14B within the field of view 24 Step S53 is executed when i = 1 or 2. If the working space of the i-th mechanical area is not within the field of view 24 is the process proceeds to step S66 to increase the counter variable i by one, and then returns to step S51.
Next, in step S54, the article tracking area is initialized 34 1, a counter variable "j" indicating that the article to be tracked in the current process is the "jth" article from the downstream side in the conveying direction in the working space of the ith mechanical section and initializes to 0 a counter variable " k "indicating the number of articles on which the supplementary detection was successfully performed in the current tracking process. Since the supplementary detection is carried out to allow the mechanical area to physically hold and pick up the article M in its working space, the supplementary detection need not be performed on all the articles located in the work space, but only needs to be executed in that the predetermined number of articles to be picked up from the mechanical area in a single operation can be detected from the downstream side in the conveying direction (ie the necessary number of articles to be detected determined by the actual application). For this, the article tracking area compares in step S55 34 the counter variable k with the necessary number of articles to be detected and if this is greater than or equal to the necessary Is number, the supplementary detection in the current work space is ended, and the process proceeds to step S66. If the counter variable k is smaller than the necessary number, the article tracking area moves 34 proceeds to step S56 and checks whether the currently traced jth article M from the downstream side is in the conveying direction in the working space of the ith mechanical area.
If the jth currently tracked article M is not in the working space of the i-th mechanical area, the supplementary detection in that work space is ended and the process proceeds to step S66. If the jth item M is located in the working space of the ith mechanical portion, then in step S57, the detection range is narrowed based on the information about the approximate position of the item M used in the preliminary detection (FIG. 17 ) (for example, the detection range is narrowed to a range twice as large as the dimensions measured in the X-axis and Y-axis directions of the article M in the two-dimensional image), while at the same time Range of the detection parameter is set to a relatively narrow range (for example, in the case of the rotation angle to a range of about ± 5 degrees of the previous detection result) and the supplementary detection is performed using this first detection parameter. The normalized correlation method, the generalized Hough transform method, the blob detection method, etc. may be used here as the detection algorithm.
In step S58, the article tracking area checks 34 whether or not the supplementary detection of the article M executed using the first detection parameter has been successfully completed. When the supplemental detection has been successfully completed, the article tracking area travels 34 to step S63 to increment the counter variable k by one and then to step S64 to reproduce the item M information obtained in the supplementary detection in the item tracking list. If the supplementary detection has not been successfully completed, there is a high possibility that the condition of the article M is different from that detected in the preceding tracking process, for example, the position of the article M has been on the conveyor 12 or because the article M covering another article behind it has already been picked up by the mechanical area and the concealed article has been newly detected, as described above with reference to FIG 14 described. For this reason, the article tracking area results in step S59 34 the supplementary detection of the article M using a second detection parameter whose range is set wider than the range of the first detection parameter (for example, in the case of the rotation angle to a range of about ± 180 degrees of the previous detection result).
In step S60, the article tracking area checks 34 whether or not the supplementary detection of the article M performed using the second detection parameter has been successfully completed. If the supplementary detection has been successfully completed, the article tracking area will stop 34 in step S61, the re-detection mark so that the article M is treated as a new article that has not existed in the previous tracing process, and proceeds to step S63 to increase the counter variable k by 1, and then to step S64 to reproduce the information of the item M obtained in the supplementary detection in the item tracking list. Although the tracking process continues the foregoing process, the information of the above article M is input to the article tracking list as the information of a newly detected article; thus, as described above, handles the article management area 112 this information as the information of the packet α, not the information of the packet β.
If the supplemental detection has not been successfully completed, the situation may be such that, although the preliminary detection of the article M has been successfully completed, the article M is not in a state where it can be picked up, for example, the position or Alignment of the article M has changed beyond the expected range of that in the initial detection or the previous supplementary detection. Thus, the article tracking area 34 in step S62, a transmission deactivation flag and updates the article information in step S64 by appending the transmission deactivation flag to the information of the article M contained in the article tracking list. Thereafter, the article tracking area increases 34 the counter variable j in step S65 by 1 and then returns to step S55 to repeat the supplementary detection.
19 shows the detailed flow of the in the flowchart according to 60 converted article information transfer step S36. First, in step S70, the article tracking area initializes 34 1, the counter variable i, which indicates the number of articles, their article information to the article management area 112 was transferred. Next, in step S71, the counter variable i is compared with the total number of articles that belong to the article management area 112 already to be transmitted as a result of preparatory detection and supplementary detection, and if the counter variable i is greater than the total number of articles, the article information transmission is ended. If the counter variable i is not larger than the total number of articles, then in step S72, transfers the article tracking area 34 the article information to the article management area 112 , After that, the article tracking area increases 34 the counter variable i by 1 in step S73 and then returns to step S71.
20 shows that of the information management area 106A ( 11 ) in the first control area 20A executed processing. The information management area 106A the processing leads to updating the article information or the arrangement pattern information 58 in the bowl 54 ( 6A ) in response to the timer control, the processing of the switch detection area 104 received peel detection information and the processing of the article management area 112 received article information. First, in step S80, the information management area receives 106A a timer interrupt signal occurring at predetermined time intervals, or receives the article information (packet α or packet β) from the article management area 112 or the shell detection information from the shell detection area 104 and checks in step S81 if the interrupt signal is received or not. When the interrupt signal is received, the information management area leads 106A the following processing which is executed at predetermined time intervals. That is, in step S87, the information of any article for which information update by the packet β for a predetermined period of time is not from the article management area 112 was received. Then, in step S88, the item information becomes the work execution area as needed 108A or the information management area 106B in the second control area 20B based on the current position of the article (see 21 for details). In addition, in step S89, the arrangement pattern information (described later) is applied to the work execution area as needed 108A or to the information management area 106B in the second control area 20B based on the current position information of each pattern element in the arrangement pattern 58 in the bowl 54 transferred (see 22 for details). After executing the above-described processing, the information management area returns 106A back to step S80 and waiting for the next reception.
If the received signal is not the interrupt signal, the information management area moves 106A proceeds to step S82 and checks whether the received information is the Schalendetektionsinformation or not. When the received information is the peel detection information, the information management area moves 106A proceeds to step S85 and generates the arrangement pattern information including the position information of each pattern element from the information of the arrangement form 59 ( 6B ), the arrangement pattern 58 Are defined. The arrangement pattern information also includes the detection value of the encoder 52 at the time of detection of the shell 54 is read, and the position information of each pattern element can be read whenever necessary by reading the present detection value of the encoder 52 to be updated.
If the received information is not the scarf detection information, it means that the article information is from the article management area 112 is received, and drives the information management area 106A proceeds to step S83 and checks whether or not the received article information is the information of the packet α. If it is the information of the packet α, it means that the received information is the information about the new article and travels the information management area 106A proceeds to step S86 and generates the information of the packet γ between the first to third control areas 20A . 20B and 20C to manage. If it is not the information of the packet α, it means that the received article information is the information of the packet β (ie, the information for updating the existing article information), and drives the information management area 106A proceeds to step S84 and updates the information of the packet γ stored in the first control area 20 is managed by the packet β.
21 shows the detailed flow of the in the flowchart according to 20 converted article information transfer step S88. In step S90, the information management area checks 106A whether the article M the work space 22A of the first mechanical area 14A already happened. If the article M the working space 22A has already passed, drives the information management area 106A proceeds to step S94 and sends the information of the package γ of the article M to the information management area 106B in the second control area 20B so that the second or third mechanical area 14B or 14C on the downstream side can do the work, and the article information transmission is terminated thereafter. If the article M the working space 22A has not yet happened, drives the information management area 106A proceeds to step S91 and checks according to the pre-determined work division ratio, whether the article M of the first mechanical range 14A is to be transmitted article.
For example, if the work division ratio is between the first to third mechanical ranges 14A . 14B and 14C set to 1: 1: 1, the first mechanical range 14A Repeats the work of picking up an article while letting two other articles pass. When the hand of the first mechanical area 14A configured to hold two items together, can be the first mechanical section 14A do the work of picking up two articles while passing four other articles. If it is determined that the article M is not that of the first mechanical section 14A is to be transferred, taking into account that of the first mechanical area 14A At this time, the division of labor dividing ratio terminates the article information transmission. If it is determined that the item M is the first of the mechanical range 14A is the information management area 106A proceeds to step S92 and checks whether the article M within the workspace 22A of the first mechanical area 14A located. If this is within the workspace 22A is the information of the package γ of the article M in step S93 to the work execution area 108A sent, after which the article information transfer is terminated.
22 shows the detailed flow of the in the flowchart according to 20 converted layout pattern information transfer step S89. In step S100, the information management area is updated 106A the position information of each pattern element using the present value of the encoder 56 , Next, in step S101, it is checked if the pattern element already has the working space 22A of the first mechanical area 14A happened. If the pattern element is the work space 22A has already passed, drives the information management area 106A proceeds to step S105 and sends the arrangement pattern information to the information management area 106B in the second control area 20B so that the second or third mechanical area 14B or 14C on the downstream side can perform the work, and thereafter, the arrangement pattern information transmission is terminated. If the pattern element is the work space 22A has not yet happened, drives the information management area 106A proceeds to step S102 and checks whether the pattern element is the pattern element on which the article from the first mechanical area 14A is to be placed according to the predetermined work division ratio. When it is determined that the pattern element is not the pattern element on which the article from the first mechanical area 14A taking into account that of the first mechanical area 14A is to be placed at this time to divide work division ratio, the arrangement pattern information transfer is terminated.
When it is determined that it is the pattern element on which the article from the first mechanical area 14A is to place, drives the information management area 106A proceeds to step S103 and checks if the pattern element is within the working space 22A of the first mechanical area 14A located. If it is within the workspace 22A is the arrangement pattern information in step S104 to the work execution area 108A sent, whereupon the arrangement pattern information transfer is terminated. On the basis of the information of the packet γ and that of the information management area 106A received arrangement pattern information leads the work execution area 108A the article transferring work for picking up the article M from the conveyer 12 and placing them in the shell 54 on the discharge conveyor 42 according to the predetermined arrangement pattern.
20 to 22 each have the information management area 106A in the first control area 20A but it will be appreciated that the information management area 106B in the second control area 20B and the information management area 106C in the third control area 20C can perform substantially the same processing as that of the information management area 106A except that neither the processing of the shell detection information nor the processing of the packet α is carried out.
Although the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes or modifications may be made to the invention without departing from the scope of the following claims.
JP 08-063214A [0003]
Journal of the Robotics Society of Japan, Vol. 29, No. 5, pp. 427-430, 2011 [0094]
Article transfer device ( 10 ; 40 ), comprising: a conveyor device ( 12 ) configured to convey an article (M); a robot ( 14 ) configured to hold and transmit the article (M); an image pickup area ( 16 ) configured to capture an image of the article (M); an image processing area ( 18 ), which is configured to control the image pickup area ( 16 ) and to control the article (M) on the basis of data from that of the image pickup area ( 16A ) detected image; and a robot control area ( 20 ), which is configured to control the robot ( 14 ) using information from the image processing area ( 18 ) detected article (M) to control; the image processing area ( 18 ) comprises: an article detection area ( 32 ) configured to perform image pick-up and detection of a plurality of articles (M) that move in accordance with a conveying movement of the conveying device ( 12 ), wherein a first period (T1) allows all of the plurality of articles (M) to be picked up and detected, and to obtain information about the initial position (D1) of each of all articles (M); and an article tracking area ( 34 ) configured to perform image pick-up and detection of the plurality of articles (M) that conform to the conveying movement of the conveyor (FIG. 12 ), wherein a second period (T2) is shorter than the first period (T1), and information about the displaced position (D2) of each of the plurality of articles (M) repeatedly obtained with the second period (T2), wherein the Information about the displaced position (D2) based on the information on the initial position (D1); and wherein the robot control area ( 20 ) is configured to control the robot ( 14 ) using the information about the displaced position (D2) to cause the robot ( 14 ) holds and transmits each of the plurality of articles (M) while controlling the conveying movement of the conveyor device ( 12 ) tracked.
An article transfer device according to claim 1, wherein the article tracking area ( 34 ) is configured by taking an image of an interior of a workspace ( 22 ) of the robot ( 14 ) recorded image data with a first resolution and by taking an image of an exterior of the work space ( 22 ) with a second resolution, wherein a second resolution of the second resolution image data is less than a first resolution of the first resolution image data; and wherein the article tracking area ( 34 ) is configured to be located within the workspace ( 22 ) (M) using the image data with the first resolution and an outside the work space ( 22 ) using the image data at the second resolution.
Article transfer device according to claim 2, wherein said image pickup area ( 16 ) comprises a switching mechanism configured to change a resolution between the first resolution and the second resolution.
An article transfer device according to any one of claims 1 to 3, wherein said article tracking area ( 34 ) is configured to select either a first detection algorithm providing a first processing speed and a first accuracy or a second detection algorithm providing a second processing speed or a second accuracy, the second processing speed being higher than the first operating speed second accuracy is less than the first accuracy, and to use a selected detection algorithm; and wherein the article tracking area ( 34 ) is configured to be located within a workspace ( 22 ) of the robot ( 14 ) (M) using the first detection algorithm and one outside the workspace ( 22 ) ( 14 ) using the second detection algorithm.
Article transfer device according to one of claims 1 to 4, wherein the article tracking area ( 34 ) is further configured to repeatedly obtain the information about the displaced position (D2) until a predetermined time has passed after the robot ( 14 ) has kept the article (M).
An article transfer device according to any one of claims 1 to 5, wherein the image pickup area (16) 16 ) comprises a single camera; and wherein the article detection area ( 32 ) and the article tracking area ( 34 ) are configured to cause the camera to capture the image of the article (M) and to detect the article (M) based on the data of the image taken by the camera.
An article transfer device according to any one of claims 1 to 5, wherein the image pickup area (16) 16 ) a first camera ( 16A ) and a second camera ( 16B ) configured to operate independently of each other; the article detection area ( 32 ) is configured to cause the first camera ( 16A ) picks up the picture of the article (M), and the article (M) based on the data of the first one Camera ( 16A ) detected image; and wherein the article tracking area ( 34 ) is configured to cause the second camera ( 16B ) receives the image of the article (M), and the article (M) based on the data of the second camera ( 16B ) detected image.
An article transfer device according to claim 7, wherein the image pickup area ( 16 ) a third camera ( 16C ) configured to operate independently of the first and second cameras ( 16A . 16B ) to work; the image processing area ( 18 ) an auxiliary tracking area ( 50 ) configured to connect the third camera ( 16C ) and image acquisition and detection of a plurality of articles (M) to be carried out according to the conveying movement of the conveyor ( 12 ), wherein a third period (T3) is shorter than the first period (T1); the auxiliary tracking area ( 50 ) is configured to acquire additional information about the displaced position (D3) of each of the plurality of articles (M) repeatedly with the third period (T3), the additional information on the displaced position (D3) being on the of the article tracking area (D3). 34 ) obtained information about the displaced position (D2) based; and wherein the robot control area ( 20 ) is configured to control the robot ( 14 ) using the additional information about the displaced position (D3) to cause the robot ( 14 ) holds and transmits each of the plurality of articles (M) while controlling the conveying movement of the conveyor device ( 12 ) tracked.
Article transfer device according to one of claims 1 to 8, wherein the robot ( 14 ) a first mechanical area ( 14A ) and a second mechanical area ( 14B ) configured to operate independently of each other; and wherein the robot control area ( 20 ) a first control area ( 20A ) configured to define the first mechanical area ( 14A ) and a second control area ( 20B ) configured to define the second mechanical area ( 14B ), the first and second control areas ( 20A . 20B ) are configured to cause the first and second mechanical regions ( 14A . 14B ) Hold items (M) whose corresponding numbers are defined according to a predetermined operating ratio.
Article transmission method configured to cause a robot ( 14 ) a plurality of from a conveyor device ( 12 ) carries and transmits conveyed articles (M), the method comprising: performing image acquisition and detection of the plurality of articles (M), which according to a conveying movement of the conveying device ( 12 ), wherein a first period (T1) allows all of the plurality of articles (M) to be picked up and detected, and obtaining information about the initial position (D1) of each of all the articles (M); and performing image acquisition and detection of the plurality of articles (M) that are in accordance with the conveying movement of the conveyor ( 12 ), wherein a second period (T2) is shorter than the first period (T1), and repeatedly obtaining information about the displaced position (D2) of each of the plurality of articles (M) with the second period (T2), wherein the Information about the displaced position (D2) based on the information on the initial position (D1); and controlling the robot ( 14 ) using the information about the displaced position (D2) to cause the robot ( 14 ) holds and transmits each of the plurality of articles (M) while tracking the conveying movement of the conveyor.
An article transfer method according to claim 10, wherein acquiring said relocated position information (D2) includes continuing repeatedly acquiring said relocated position information (D2) until a predetermined time passes after said robot ( 14 ) has kept the article (M).
An article transfer method according to claim 10 or 11, further comprising performing image pick-up and detection of the plurality of articles (M) that conform to the conveyance movement of the conveyance device (Fig. 12 ), wherein a third period (T3) is shorter than the first period (T1), and repeatedly obtaining additional information about the displaced position (D3) of each of the plurality of articles (M) with the third period (T3), wherein the additional information about the displaced position (D3) on the article tracking area (D3) 34 ) previously obtained information about the displaced position (D2), and controlling the robot ( 14 ) using the additional information about the displaced position (D3) to cause the robot ( 14 ) holds and transmits each of the plurality of articles (M) while controlling the conveying movement of the conveyor device ( 12 ) tracked.
DE102015015638.1A 2014-12-02 2015-12-02 Apparatus and method for transferring articles using a robot Active DE102015015638B4 (en)
JP2014-244004 2014-12-02
JP2014244004A JP6042860B2 (en) 2014-12-02 2014-12-02 Article transferring apparatus and article transferring method for transferring article using robot
DE102015015638A1 true DE102015015638A1 (en) 2016-10-13
DE102015015638B4 DE102015015638B4 (en) 2018-11-29
ID=56078595
DE102015015638.1A Active DE102015015638B4 (en) 2014-12-02 2015-12-02 Apparatus and method for transferring articles using a robot
US (1) US9604365B2 (en)
JP (1) JP6042860B2 (en)
CN (1) CN105645010B (en)
DE (1) DE102015015638B4 (en)
JP6581049B2 (en) * 2016-08-15 2019-09-25 ファナック株式会社 Robot system
CN106552772B (en) * 2016-10-25 2019-01-22 北京京东尚科信息技术有限公司 Visual identifying system and the classification sorting system for using the visual identifying system
JP6589847B2 (en) * 2016-12-26 2019-10-16 株式会社ダイフク Material loading equipment
JP2018104179A (en) * 2016-12-28 2018-07-05 株式会社イシダ Article transshipment device
JP6496333B2 (en) 2017-01-30 2019-04-03 ファナック株式会社 Article conveying apparatus using at least one sensor
JP2019038073A (en) * 2017-08-25 2019-03-14 川崎重工業株式会社 Robot system and method for performing work on conveyed workpiece
JPH0863214A (en) 1994-08-25 1996-03-08 Fanuc Ltd Visual tracking method
CH629613A5 (en) 1978-05-26 1982-04-30 Bbc Brown Boveri & Cie Method for image data processing for surveying objects within a material flow
JPS60217085A (en) * 1984-04-12 1985-10-30 Fuji Electric Co Ltd Handling system of moving body
JPH10105217A (en) * 1996-09-27 1998-04-24 Mitsubishi Electric Corp Tracking controlling method of robot and robot control system
JP4265088B2 (en) * 2000-07-10 2009-05-20 株式会社豊田中央研究所 Robot apparatus and control method thereof
JP2002116810A (en) * 2000-10-06 2002-04-19 Seiko Epson Corp Tracking system
JP4174342B2 (en) * 2003-02-19 2008-10-29 ファナック株式会社 Work transfer device
JP2007030087A (en) * 2005-07-26 2007-02-08 Fanuc Ltd Physical distribution tracking device
2014-12-02 JP JP2014244004A patent/JP6042860B2/en active Active
2015-12-01 US US14/955,608 patent/US9604365B2/en active Active
2015-12-02 CN CN201510873842.2A patent/CN105645010B/en active IP Right Grant
2015-12-02 DE DE102015015638.1A patent/DE102015015638B4/en active Active
Journal of the Robotics Society of Japan, Vol. 29, Nr. 5, S. 427–430, 2011
US9604365B2 (en) 2017-03-28
US20160151916A1 (en) 2016-06-02
JP2016107349A (en) 2016-06-20
CN105645010A (en) 2016-06-08
JP6042860B2 (en) 2016-12-14
CN105645010B (en) 2017-11-14
DE102015015638B4 (en) 2018-11-29
EP0291965A1 (en) 1988-11-23 Method and system for controlling robot for constructing products
US20040117066A1 (en) 2004-06-17 Article pickup device
DE112009001414T5 (en) 2011-04-21 Processing system, control device, control method and program
DE102006031178B4 (en) 2008-08-28 Gripper guide system, processing system and program
JPH058186A (en) 1993-01-19 Automatic calibration method
2018-08-22 R018 Grant decision by examination section/examining division