Apparatus and method for picking up article randomly piled using robot

An article pickup apparatus configured so as to set a grip unit model including a substantial region of the grip unit in an opened state and a grip region inside the substantial region, set position posture candidates of the grip unit, calculate a grip success possibility of any of the articles by the grip unit in each of the grip position posture candidates based on the position information acquired by a three-dimensional measurement instrument and the grip unit model, select position posture candidates from the position posture candidates based on the grip success possibility and setting the selected position posture candidates as grip unit position postures, and control the robot so as to move the grip unit to the grip unit position postures to pick up any of the articles.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an article pickup apparatus and an article pickup method for picking up an article randomly piled in a three-dimensional space using a robot including a grip unit.

2. Description of the Related Art

As an apparatus of this type, conventionally, there is known an apparatus configured to recognize a position of an article by applying three-dimensional matching processing to a three-dimensional point set obtained by measuring articles randomly piled using a three-dimensional measurement instrument. This apparatus is described, for example, in Japanese Laid-open Patent Publication No. 2011-179909 (JP2011-179909A). Further, an apparatus configured to measure articles randomly piled using a three-dimensional measurement instrument and then extract a region able to be gripped by a grip unit is also known. This apparatus is described, for example, in Japanese Laid-open Patent Publication No. 2011-093058 (JP2011-093058A).

In the apparatus described in JP2011-179909A, while a three-dimensional model pattern of an article is previously acquired from a CAD model or the like, surfaces of articles in a three-dimensional space are measured using a three-dimensional measurement instrument and a three-dimensional point set (an distance image) is acquired, and then the three-dimensional point set is divided into partial regions surrounded by an edge extracted from the three-dimensional point set. Then, initially, one of the partial regions is set as an article region, and both matching processing of the three-dimensional model pattern for the article region and update processing for adding another partial region to the article region are repeated to measure positions and postures of the articles.

In the apparatus described in JP2011-093058A, a grip region including a grip mechanism region determined by a grip mechanism and a grip portion region determined by a grip portion of a target gripped by the grip mechanism is previously stored, and a region equal in size to the grip region and a region where an article is present in the entire region equal in size to the grip portion region but the article is not present in a region equal in size to the grip mechanism region is extracted as a grippable region.

However, in the apparatus described in JP2011-179909A, it is necessary to previously prepare a three-dimensional model pattern for each type of article, and therefore, time and effort are needed. In particular, regarding a large number of types of articles, it is necessary to prepare model patterns for the number of types and therefore, much time and effort are needed. Further, for an indefinitely shaped article, it is inherently difficult to prepare a model pattern, resulting in difficulty in an application thereof. Further, when a grip unit is moved to a pickup position determined by a position posture of an article, a collision between another article and the grip unit may occur.

Further, in the apparatus described in JP2011-093058A, it is necessary to previously set a grip portion region of an article. In addition, there occurs a case where the grip portion region is not exposed on a three-dimensional measurement instrument side in some positions and postures of the article and in this case, it is difficult to pick up the article.

SUMMARY OF THE INVENTION

An article pickup apparatus according to an aspect of the present invention includes: a robot including a grip unit provided in an openable and closable manner; a three-dimensional measurement instrument measuring surface positions of a plurality of articles randomly piled on a three-dimensional space to acquire position information of a plurality of three-dimensional points; a grip unit model setting unit setting a grip unit model including a substantial region and a grip region inside the substantial region, the substantial region being a region of a substantial portion of the grip unit in an opened state; a position posture candidate setting unit setting one or more position posture candidates as a candidate of a position and a posture of the grip unit; a gripability calculation unit calculating a grip success possibility of any of the articles by the grip unit in each of the grip position posture candidates, assuming that the grip unit is placed at the grip position posture candidates set by the position posture candidate setting unit, based on the position information acquired by the three-dimensional measurement instrument and the grip unit model set by the grip unit model setting unit; a position posture setting unit selecting one or more position posture candidates from the position posture candidates set by the position posture candidate setting unit based on the grip success possibility calculated by the gripability calculation unit, and setting as a grip unit position posture; and a robot control unit controlling the robot so as to move the grip unit to the grip unit position posture set by the position posture setting unit to pick up any of the articles.

Another aspect of the present invention is an article pickup method for picking up any of articles randomly piled on a three-dimensional space using a robot including a grip unit provided in an openable and closable manner, the method includes: measuring surface positions of a plurality of the articles to acquire position information of a plurality of three-dimensional points; setting a grip unit model including a substantial region and a grip region inside the substantial region, the substantial region being a region of a substantial portion of the grip unit in an opened state; setting one or more position posture candidates as a candidate of a position and a posture of the grip unit; calculating a grip success possibility of any of the articles by the grip unit in each of the grip position posture candidates, assuming that the grip unit is placed at the grip position posture candidates, based on the position information acquired by the three-dimensional measurement instrument and the grip unit model; selecting one or more position posture candidates from the position posture candidates based on the grip success possibility, and setting as a grip unit position posture; and controlling the robot so as to move the grip unit to the grip unit position posture to pick up any of the articles.

DETAILED DESCRIPTION

Hereinafter, with reference toFIG. 1toFIG. 18, an article pickup apparatus according to the embodiment of the present invention will be described.FIG. 1is a view illustrating a schematic configuration of an article pickup apparatus10according to one embodiment of the present invention. The article pickup apparatus10includes a three-dimensional measurement instrument11, a robot12, and a robot control device13for controlling the three-dimensional measurement instrument11and the robot12by being connected to the three-dimensional measurement instrument11and the robot12. The robot12includes a gripper14mounted on a tip of an arm12a. A container16is disposed sideward of the robot12. Together therewith,FIG. 1illustrates an orthogonal three-axis coordinate system of X, Y, and Z. The Z-direction is a vertical direction, and the X-direction and the Y-direction are horizontal directions. The container16is illustrated on an XZ plane.

In the container16opened upward, a plurality of articles17are randomly piled. The article pickup apparatus10of the present embodiment determines a position and a posture (a position posture) of the gripper14capable of gripping the article17and controls the robot12to move the gripper14to the determined position posture. Further, with this position posture, the gripper14grips the article17and then picks up the article17from the container16by an operation of the robot12to convey the article17to a predetermined position outside the container16.FIG. 1illustrates a plurality of articles17as the same shape as each other, but indefinitely shaped articles and a plurality of types of articles are employable.

The three-dimensional measurement instrument11is disposed above a center portion of the container16and measures a surface of an exposed article17among articles17randomly piled in the container16to acquire position information (three-dimensional information) of a plurality of three-dimensional points. A measurement range of the three-dimensional measurement instrument11needs to include the container16but an excessively large measurement range decreases measurement resolution. Therefore, preferably, the measurement range is equivalent to an occupied range of the container16and, for example, accords with the occupied range of the container16. InFIG. 1, the three-dimensional measurement instrument11is fixed to a dedicated cradle15but may be mounted on a tip of the robot12. The three-dimensional measurement instrument11and the robot control device13are connected to each other via a communication unit such as a communication cable so as to be communicable with each other.

As the three-dimensional measurement instrument11, various non-contact types are can be used. There are cited, for example, a stereoscopic type using two cameras, a scanning type using laser slit light, a scanning type using laser spot light, a type of projecting pattern light on an article using a device such as a projector, and a type of utilizing a flight time from emission of light from a projector to incidence to a light receiver via reflection on an article surface.

The three-dimensional measurement instrument11expresses the acquired three-dimensional information as the format of a distance image or a three-dimensional map. The distance image is an image where three-dimensional information is expressed as an image format, and expresses a height of a position on an image or a distance from the three-dimensional measurement instrument11using brightness or a color of each pixel of the image. On the other hand, the three-dimensional map is a map where three-dimensional information is expressed as a set of measured three-dimensional coordinate values (x, y, z). In the present embodiment, each pixel in a distance image or a point having three-dimensional coordinate values in a three-dimensional map is referred to as a three-dimensional point, and a set including a plurality of three-dimensional points is referred to as a three-dimensional point set. The three-dimensional point set is a set of all the three-dimensional points measured using the three-dimensional measurement instrument11and can be acquired using the three-dimensional measurement instrument11.

FIG. 2is a perspective view illustrating a schematic configuration of the gripper14. The gripper14includes a shaft unit141fixed to a tip of the robot arm12a, a plate unit142disposed in a lower end portion of the shaft unit141, and a pair of grip nails143disposed on both end portions of the plate unit142. The shaft unit141extends along an axis line L1passing through the center thereof, and the plate unit142extends along an axis line L2vertical to the axis line L1. The paired grip nails143each have a length L parallel to the axis line L1, a thickness T parallel to the axis line L2, and a width W parallel to an axis line line L3vertical to each of the axis line L1and the axis line L2; and are symmetrically shaped with respect to the axis line L1. The grip nail143is movable parallel to the axis line L2along a lower surface of the plate unit142and thereby, a distance D between one nail and the other nail of the paired grip nails143is changed. A length from a tip of the grip nail143necessary for the pair of grip nails143to grip the article17where the length is parallel to the axis line L1is designated as a grip depth La.

In the figure, when a point located downward from a point where the axis lines L1, L2, and line L3intersect by a predetermined distance, for example, a point P0where the axis line L1and a bottom of the grip nail143intersect is designated as a reference point of the gripper14, a position of the gripper14is determined by position coordinates (x, y, z) of the reference point P0. When a posture where the axis line L1and the Z-axis, the axis line L2and the X-axis, and the axis line L3and the Y-axis (FIG. 1) are parallel to each other, respectively, is designated as a reference posture of the gripper14, the gripper14is rotatable from the reference posture by a predetermined angle ψ (for example, −90°≦ψ≦90°) around the axis line L1, is rotatable by a predetermined angle φ (for example, −30°≦φ≦30°) around the axis line L2, and is rotatable by a predetermined angle θ (for example, −30°≦θ≦30°) around the axis line L3. Therefore, the angles φ, θ, and ψ determine a posture of the gripper14. In other words, a position posture of the gripper14in a three-dimensional space is determined by six degrees of freedom of six directions (x, y, z, φ, θ, and ψ) including translation three directions (x, y, z) and rotation three directions (φ, θ, ψ).

FIG. 3AandFIG. 3Beach are a view schematically illustrating an operation of the grip nail143of the gripper14. The article17includes, as one example, a disc portion171and a cylindrical portion172vertically disposed from a center portion of the disc portion171. When gripping the article17, for example, as illustrated inFIG. 3A, while the grip nail143is opened, a pair of grip nails143is disposed so that respective grip nails are located on both sides of the article (cylindrical portion172). From this state, as illustrated inFIG. 3B, the paired grip nails143are caused to be close to each other to grip the article17using the paired grip nails143. When the article17is disposed upside down, the grip nail143can grip the disc portion171.

In the present embodiment, the grip nail143being opened as illustrated inFIG. 3Ais previously set in the robot control device13as a gripper model20. The gripper model20includes a substantial region SP1which is a region of a substantial portion where a pair of grip nails143exists and a grip region SP2which is located inside the substantial region SP1. When a position posture of the gripper14being opened is controlled so that the article17is disposed in the grip region SP2, the article17becomes able to be gripped.

FIG. 4is a flowchart illustrating processing executed in the robot control device13, and specifically illustrating one example of processing for article pickup. An operation of the article pickup apparatus10will be described with reference to the flowchart ofFIG. 4and the drawings associated therewith.

Processing ofFIG. 4is started when, for example, a pickup start command of an article17is input by operating an operation switch not illustrated.

Initially, step S1measures surfaces of a plurality of articles17randomly piled in a three-dimensional space using the three-dimensional measurement instrument11and then acquires a three-dimensional point set30.FIG. 5is a view illustrating one example of the three-dimensional point set30acquired using the three-dimensional measurement instrument11and three-dimensional points31configuring the three-dimensional point set30. In the figure, the three-dimensional points31are illustrated with black circles and the three-dimensional point set30is illustrated as a region surrounded by a dotted line including all the black circles.

Then, step S2sets at least one grip position posture candidate which is a candidate of a position posture of the gripper14in a three-dimensional space. The grip position posture candidate is expressed by six degrees of freedom, i.e., position coordinates (x, y, z) and angles (φ, θ, ψ), indicating a position posture of the gripper14, and of six degrees of freedom (x, y, z, φ, θ, ψ), at least one degree of freedom is set as a parameter.

When, for example, x is set as a parameter (x1, x2, . . . ) and y, z, φ, θ, and ψ are set as fixed values (y0, z0, φ0, θ0, and ψ0), initially, a range (search range) Δx able to be provided for x is set. The search range Δx can be provided using a minimum value xmin and a maximum value xmax of x-coordinates of three-dimensional points31belonging to the three-dimensional point set30(xmin≦x≦xmax). The search range Δx may be previously set in the robot control device13. Then, a pitch (search interval) px of the x-direction of a grip position posture candidate is set. The search interval px may be previously set or may be determined by dividing the range Δx by a predetermined number. Thereby, grip position posture candidates (x1, y0, z0, φ0, θ0, ψ0), (x2, y0, z0, φ0, θ0, ψ0), . . . are set in the x-direction at equal intervals.

When any of y, z, φ, θ, and ψ is set as a parameter, the same manner as described above is employed. However, search ranges of φ, θ, and ψ may be set as ranges able to be provided for the gripper itself, i.e., −30°≦φ≦30°, −30°θ≦30°, and −90°≦ψ≦90°, respectively. When of six degrees of freedom, a plurality of degrees of freedom (for example, x and y) are set as parameters, grip position posture candidates are provided by combining grip position posture candidates where x is set as a parameter and grip position posture candidates where y is set as a parameter. When, for example, there are five grip position posture candidates where x is set as a parameter and four grip position posture candidates where y is set as a parameter, grip position posture candidates total 5×4=20. When x and y are set as parameters and z is set as a fixed value, z satisfying a predetermined condition may be selected from measured three-dimensional points31. For example, among three-dimensional points31present within a predetermined distance from (x, y), a z-coordinate of a three-dimensional point31located at the highest position may be designated as a z-coordinate of a grip position posture candidate.

Step S3calculates a possibility of succeeding in gripping the article17, i.e., a grip success possibility E, on an assumption that the gripper14is placed in each grip position posture candidate set in step S2. The grip success possibility E is calculated, for example, in a range of 0 to 1.0 based on position information of three-dimensional points31and a preset gripper model20, and a larger numerical value means a higher possibility capable of gripping the article17. The grip success possibility E is calculable using the following equation (I), based on a gravity center position of all the three-dimensional points31present in the grip region SP2, for example.

A vector p of the above equation (I) is a vector up to a center position40of an upper surface of the grip region SP2, and a vector g is a vector up to a gravity center position41of three-dimensional points31present in the grip region SP2, as illustrated inFIG. 6. D1is a distance from the center position40of the upper surface of the grip region SP2to a most distant point42in the grip region SP2. According to the above equation (I), as the gravity center position41of three-dimensional points31is close to the center position40of the upper surface of the grip region SP2, the grip success possibility E increases. In other words, in this case, a length of a portion for gripping the article17increases and also a pair of grip nails143can grip the article17evenly from both sides thereof and therefore, the grip success possibility E increases.

When the grip success possibility E is calculated, it is possible to determine a distance D1between the gravity center position41of three-dimensional points31present in the grip region SP2and the upper surface of the grip region SP2ith respect to each of the X-, Y-, and Z-axes to calculate E. The grip success possibility E is also calculable using a Manhattan distance instead of a Euclidean distance. The grip success possibility E is also calculable using the following equation (II) based on a distribution of three-dimensional points31present in the grip region SP2.

In the above equation (II), N represents the number of three-dimensional points31present in the grip region SP2, z0represents a z-coordinate of a gravity center of all the three-dimensional points31in the grip region SP2, z1represents a z-coordinate of an ith three-dimensional point31present in the grip region SP2, and D2represents a height (a length of the Z-direction) of the grip region SP2. The above equation (II) takes it into consideration that when an upper surface of the article17present in the grip region SP2is flat, the article17is easily gripped, and according to the above equation (II), when the upper surface of the article17is flatter, the grip success possibility E increases. For example, the state ofFIG. 7Ais higher than the state ofFIG. 7Bin the grip success possibility E. In the above equation (II), a variance value of z-coordinates is also usable, and the grip success possibility E may be calculated based on a distribution other than z-coordinates (for example, x-coordinates or y-coordinates).

From a sum (E1+E2) of a grip success possibility (expressed by E1) determined by the above equation (I) and a grip success possibility (expressed by E2) determined by the above equation (II), the grip success possibility E may also be calculated, or from a sum obtained using a plurality of other evaluation equations, the grip success possibility E may also be calculated. In this case, it is possible that weighting coefficients are previously set and then each evaluation equation is multiplied by a corresponding predetermined weighting coefficient to calculate the grip success possibility E.

When calculating the grip success possibility E, it is possible to judge whether three-dimensional points31are present in the substantial region SP1nd then to set the grip success possibility E to be 0 when any one of the three-dimensional points31exists. Thereby, when the gripper14grips the article17, the gripper (the grip nail143) can be prevented from colliding with another article17. Therefore, a possibility of failing to grip the article17is reduced and breakage of the article17and the gripper14becomes preventable.

When a model (a container model) of the container16is previously set in the robot control device13and the grip success possibility E is calculated, it is possible to judge the presence or absence of a collision between the gripper14and the container16using the container model.FIG. 8is a view illustrating one example of a container model161. As illustrated inFIG. 8, the container model161includes a substantial region SP3which is a region of a substantial portion where the container16exists. It is possible to set the grip success possibility E to be 0 when the substantial region SP3of the container16is present in the substantial region SP1of the gripper14with respect to each position posture candidate. Thereby, when the gripper14grips the article17, the gripper14can be prevented from colliding with the container16and breakage of the gripper14and the container16becomes preventable.

Step S4selects at least one grip position posture candidate from the grip position posture candidates set in step S2based on the grip success possibility E calculated in step S3and sets the selected candidate as a grip position posture (a gripper position posture) of the gripper14. For example, a grip position posture candidate where the grip success possibility E is maximized is selected and set it as the gripper position posture. It is possible that in a three-dimensional space or a predetermined two-dimensional plane, a grip position posture candidate where the grip success possibility E is locally maximized is selected and set it as the gripper position posture.

Step S5numbers respective gripper position postures as P1, P2, . . . , Pn. N represents the number of gripper position postures.FIG. 9is a view illustrating numbered gripper position postures, and numbering is performed in descending order of a coordinate value with respect to a predetermined coordinate axis35, i.e., in order from one located in a higher position (a larger z-coordinate). InFIG. 9, n=3. Numbering may be performed in order from one having a larger grip success possibility E, instead of descending order of a coordinate value.

FIG. 9illustrates gripper position postures P1, P2, and P3using a pair of arrows A1and A2for each; and an intersection of the pair of arrows A1and A2indicates a position of a gripper position posture and directions of the arrows A1and A2indicate a posture of the gripper position posture. The position of the gripper position posture is, for example, a point (a reference point P0) where the axis line L1of the gripper14ofFIG. 2and the bottom of the grip nail143intersect. Further, the posture (the directions of the arrows A1and A2) of the gripper position posture indicates directions parallel to the axis lines L1and line L2of the gripper14ofFIG. 2. The gripper position posture is expressed in a three-dimensional space and therefore, a direction of the axis line L3is also defined in the gripper position posture.

In step S6, an initial value is provided for a variable k having a natural number value. In other words, processing for k←1 is executed. The variable k is used for specifying the number of a gripper position posture Pk.

Step S7outputs a control signal to a robot driving actuator (an electric motor) and moves the gripper14to the gripper position posture Pk (for example, P1) as illustrated inFIG. 10. Thereby, a pair of grip nails143is disposed on both sides of an article17to be picked up so as to nip the article17.

Step S8outputs a control signal for gripping the article17to a gripper driving actuator. Thereby, as illustrated inFIG. 11, the grip nail143grips the article17. The gripper position posture Pk is obtained via numbering in descending order with respect to the predetermined coordinate axis35(step S5) and therefore, the article17to be picked up becomes an article17located in the highest position in the container16, whereby a possibility that the gripper14collides with the article17during movement of the gripper14can be reduced. When numbering is performed in order from one having a larger grip success possibility E, a pickup is performed from an article17having a higher possibility of being gripped and therefore, a change in a loading state of the article17due to load shifting or the like of the article17can be inhibited.

Then, step S9outputs a control signal to the robot driving actuator to raise the gripper14, while gripping the article17, to a predetermined direction, for example, in a direction of the predetermined coordinate axis35(FIG. 9) by a predetermined amount, as illustrated inFIG. 12.

Step S10judges whether the gripper14has succeeded in gripping the article17in the raised position of the article17. When, for example, the gripper14includes a weight detector for detecting weight and a detected value is at least a predetermined value, it is judged that a grip has been successfully performed. It is possible that a proximity sensor judges whether the article17exists to judge whether the grip has been successfully performed. It is possible that a switch is disposed on a tip of the gripper14to judge whether the grip has been successfully performed by ON and OFF of the switch. When it is judged that the grip has been successfully performed, the processing moves to step S11, but when it is judged that the grip has not been successfully performed, the processing passes step S11and moves to step S12.

Step S11outputs a control signal to the robot driving actuator and conveys the article17to a predetermined position by an operation of the robot12to remove the article17from the gripper14.

Step S12adds 1 to k for processing for k←k+1 and further step S13judges whether k<n is satisfied. This judgment is a judgment whether any gripper position posture where the gripper14has not reached yet exists among n (3inFIG. 9) gripper position postures Pk. A judgment of k<n indicates that the gripper14has not reached yet the gripper position posture Pk and therefore, processing returns to step S7. Then, the gripper14is moved to the next gripper position posture Pk (for example, P2) to grip the article17. A judgment of k≧n in step S13indicates that the gripper14has reached all n gripper position postures Pk and therefore, the processing is ended.

In the above processings, step S2sets grip position posture candidates and step S3calculates the grip success possibility E in each grip position posture candidate. However, these processings are executable, for example, as follows.FIG. 13is a flowchart illustrating a modified example of step S2and step S3ofFIG. 4.

Initially, step S21projects (orthogonally projects) three-dimensional points31acquired using the three-dimensional measurement instrument11in a predetermined direction and generates a projected plane (referred to as an orthogonal projection image).FIG. 14is a view illustrating one example of an orthogonal projection image50. The orthogonal projection direction refers to an approach direction (for example, −Z-direction) of the gripper14when gripping the article17using the gripper14. InFIG. 14, three-dimensional points31a,31b, and31ceach are projected on the orthogonal projection image50parallel to an X-Y plane.

Pixels corresponding to the respective three-dimensional points31a,31b, and31cof the orthogonal projection image50have pixel values equivalent to z-coordinates of the three-dimensional points31a,31b, and31c(“1”, “3”, and “9” inFIG. 14), respectively. A size (lengths of the x- and y-directions) of a pixel of the orthogonal projection image50is previously set. It is also possible to set a size of a pixel so that the number of pixels of the image50becomes a predetermined value. Regarding a posture of a grip position posture candidate, a rotation angle φ around the x-axis and a rotation angle θ around the y-axis are set as a fixed value (for example, 0) and then a rotation angle ψ around the z-axis is set as a parameter. Therefore, when the rotation angle ψ is changed at a predetermined number of times in a predetermined angle pitch in each pixel, a total number of grip position posture candidates becomes (the number of pixels of the orthogonal projection image50)×(the number of changes of ψ).

Step S22projects (orthogonally projects) the gripper model20(FIG. 3A) corresponding to a posture of each grip position posture candidate on the orthogonal projection image50to generate a filtering image60. In the present modified example, according to the changes of the rotation angle ψ, the same number of filtering images60as the number of changes of the rotation angle ψ (the number of candidates of ψ) are generated.FIG. 15is a view illustrating one example of the filtering image60. In the figure, D is equivalent to a distance between a pair of grip nails143, and T and W are equivalent to a thickness and a width of the grip nail143, respectively (refer toFIG. 2). The filtering image60includes a substantial image61corresponding to the substantial region SP1of the gripper model20and a grip image62corresponding to the grip region SP2.

Step S23calculates a height za of each grip position posture candidate using the filtering image60. For example, when the filtering image is overlaid on the orthogonal projection image50aligning the center of the filtering image60and the center of pixels on the orthogonal projection image50corresponding to a grip position posture candidate at the same position, a value (Zb−La) can be obtained as a height za of the grip position posture candidate by subtracting the grip depth La (FIG. 2) from a maximum pixel value Zb in the orthogonal projection image50included in the grip image62at that time.

Step S24calculates the grip success possibility E corresponding to each grip position posture candidate in the same manner as step S3. In this case, it is judged whether a maximum pixel value in the orthogonal projection image50included in the substantial image61among the filtering image60is larger than the height za of the grip position posture candidate. When the maximum pixel value is larger than za, a lower end of the grip nail143collides with the article17and therefore, the grip success possibility E is set to be 0. Thereby, the grip success possibility E can be quickly calculated and the presence or absence of interference between the gripper14and the article17can be determined in a short period of time.

In step S2ofFIG. 4, position posture candidates of the gripper14are set using six degrees of freedom of (x, y, z, φ, θ, and ψ), but when an opening amount d of the gripper14, i.e., a distance D between a pair of grip nails143is adjustable, grip position posture candidates can also be set using the opening amount d as an additional parameter. In this case, a position posture of the gripper14has seven degrees of freedom (x, y, z, φ, θ, ψ, and d).

FIG. 16is a plan view illustrating one example of a grip position posture candidate where the opening amount d is set as a parameter.FIG. 16illustrates three articles17A,17B, and17C with postures different from each other. In other words, the cylindrical portion172of the article17A is directed downward, the cylindrical portion172of the article17B is directed upward, and the cylindrical portion172of the article17C is directed laterally. Regarding the article17A, when the opening amount d of the gripper14is increased, the article17A can be stably gripped. Further, regarding the articles17B and17C, when the opening amount d is decreased, the articles17B and17C can be gripped without interference of the grip nail143with another article17. When the opening amount d is set as a parameter, a large number of grip position posture candidates are generated and therefore, it is preferable to set grip position posture candidates using the orthogonal projection image50as illustrated inFIG. 14. Thereby, the grip success possibility E can be quickly calculated. In this case, the filtering image60is generated by increasing or decreasing a size of the grip image62according to the opening amount d. At that time, the number of generated filtering images60becomes a candidate number of ψ×a candidate number of the opening amount d.

The present embodiment makes it possible to achieve the following operations and effects.

(1) Surface positions of a plurality of articles17randomly piled in a three-dimensional space are measured using the three-dimensional measurement instrument11and position information of a plurality of three-dimensional points31are acquired (step S1); the gripper model20including the substantial region SP1and the grip region SP2of the gripper14in an opened state is set and at least one position posture candidate is set as a candidate of a position and a posture of the gripper14(step S2); based on the position information acquired using the three-dimensional measurement instrument11and the gripper model20, the grip success possibility E of the article17in each position posture candidate is calculated, assuming that the gripper14is placed at each position posture candidate (step S3); at least one position posture candidate is selected from the position posture candidates based on the grip success possibility E and set as a gripper position posture (step S4); and the robot12is controlled to pick up the article17by moving the gripper14to this gripper position posture (step S7to step S13). Thereby, the article17randomly piled can be picked up by being gripped by the gripper without previously inputting information of the article17. Since information of the article17need not be input, the article17can be automatically picked up even regarding a large number of types of articles17or an indefinitely shaped article17.

In contrast, for example, in a method (a method according to a first comparative example) for recognizing a position of an article via three-dimensional pattern matching using a three-dimensional model pattern for the article, the three-dimensional model pattern needs to be prepared and therefore, time and effort are needed. Especially in the case of a large number of types of articles, model patterns for the number of types need to be prepared and therefore, much time and effort are needed. Further, in the method according to the first comparative example, the following problems are produced compared with the present embodiment. It is difficult to prepare a three-dimensional model pattern for an indefinitely shaped article, resulting in difficulty in recognizing a position thereof. In the article17randomly piled, it is difficult to acquire three-dimensional points31on a side which does not face the three-dimensional measurement instrument11, and also a large inclination and an obstacle by an adjacent article cause poor photograph conditions. Therefore, it is difficult to obtain three-dimensional point sets sufficient in quality and amount to the extent that a three-dimensional posture of an article can be determined via three-dimensional pattern matching, resulting in possibilities that a recognition failure and a recognition error of a position posture of an article occur, a position of an article to be picked up located upward fails to be recognized, and a position of an article located downward is recognized first. When a position posture of the gripper14of the robot12is controlled toward an article position posture erroneously recognized or an article position located downward, there are produced possibilities that missing a pickup of the article17causes a decrease in operation efficiency of the apparatus and also a collision between the gripper14and the article17causes damage thereto. When the damage is intended to be avoided, a moving velocity of the robot12is forced to decrease, resulting in poor operation efficiency.

Further, for example, in a method (a method according to a second comparative example) for recognizing a position posture of an article using a grip portion region which is a partial region where the article is gripped to set a grip position posture, the user needs to teach a grip portion region of the article previously and therefore, time and effort are needed. Further, the grip portion region is not exposed occasionally on the three-dimensional measurement instrument side depending on a shape and a posture of the article, and in this case, it is difficult to recognize the article, resulting in difficulty in picking up the article. When an article is intended to be recognized regardless of a position posture of the article, a plurality of grip portion regions need to be taught and therefore, much time and effort are needed. Further, in the method according to the second comparative example, it is difficult to teach a grip portion region regarding an indefinitely shaped article, resulting in difficulty in picking up the indefinitely shaped article.

(2) When the grip success possibility E is calculated based on a distance between the center position40of the upper surface of the grip region SP2and the gravity center position41of three-dimensional points31present in the grip region SP2(above equation (I)), it is possible to set a position posture capable of deeply gripping the article17using a pair of grip nails143and also capable of performing a grip in a center portion of the pair of grip nails143as a gripper position posture. Therefore, the article17can be stably gripped. When the grip success possibility E is calculated based on a distribution, for example, a flatness of three-dimensional points31present in the grip region SP2(above equation (II)), it is possible to grip a portion where the article17is more easily gripped.

(3) When the presence or absence of an interference between the substantial region SP1of the gripper14in a grip position posture candidate and the article17or the container16is judged, and then the grip success possibility E is set to be 0 when the interference occurs, a collision between the gripper14and the article17or the container16becomes avoidable. Therefore, the article17can be stably picked up and also damage caused between the gripper14and the article17or the container16becomes preventable.

In contrast, when, for example, a position posture of an article is recognized and a gripper position posture is determined only from the position posture (for example, the method according to the first comparative example), another article interferes with the gripper when moving the gripper to the gripper position posture, resulting in a possibility of failing to grip an article to be gripped. Further, a collision between the gripper and the article or the container may cause breakage of the gripper, the article, and the container.

(4) When a grip position posture candidate of the gripper14is set and the grip success possibility E is calculated on an image50where three-dimensional points31and a gripper model20are projected (FIG. 13), a gripper position posture can be quickly set. In this case, processing is executed using the image50where three-dimensional points31are orthogonally projected and the filtering image60where the gripper model20each are orthogonally projected and therefore, the following advantages are also created. On a distance image having a common viewpoint, a distance between pixels next to each other on a real space is not constant. In other words, as a target on the real space is distant from the view point, the target appears on the pixel by being reduced. Therefore, it is difficult to deal with a size of the gripper14on the real space based on a distance between pixels and therefore, it is necessary to express a gripper model using a size on a distance image. However, when a size of the gripper14is expressed on the distance image, poor accuracy is obtained, resulting in difficulty in setting a gripper position posture. In contrast, when the orthogonal projection image50is used, a distance between pixels next to each other becomes, as such, a distance on the real space and then a unit of the real space is directly usable for setting the gripper model20and therefore, the gripper position posture is easily set.

(5) If an opening amount of the gripper14is adjustable, a grip position posture candidate is set using an opening amount d of the gripper14when causing the gripper14to approach the article17as a parameter, and therefore a gripper position posture where the opening amount d of the gripper14is optimized according to a loading state of the article17can be determined. Thereby, the article17can be assuredly picked up without interference between the gripper14and an article17other than an article17to be picked up (targeted article).

An article pickup method for picking up an article randomly piled in a three-dimensional space using the robot12including the gripper14provided in an openable and closable manner may be configured in any manner, as long as the method includes: measuring surface positions of a plurality of articles17using the three-dimensional measurement instrument11to acquire position information of a plurality of three-dimensional points31; setting a gripper model20including the substantial region SP1which is a region of a substantial portion of the gripper14in an opened state and the grip region SP2inside the substantial region SP1; setting at least one position posture candidate as a candidate of a position and a posture of the gripper14; calculating the grip success possibility E of the article17by the gripper14in each position posture candidate, assuming that the gripper14is placed at the position posture candidates, based on the position information acquired by the three-dimensional measurement instrument11and the gripper model20; selecting at least one position posture candidate selected from the position posture candidates based on the grip success possibility E and set the selected position posture candidate as a gripper position posture; and controlling the robot12so as to move the gripper14to the set gripper position posture to pick up any of the articles17.

FIG. 17is a block diagram illustrating an internal configuration of the robot control device13ofFIG. 1. The robot control device13includes a grip unit model setting unit131, a position posture candidate setting unit132, a gripability calculation unit133, a position posture setting unit134, and a robot control unit135.

In the above embodiment, the gripper14including a pair of grip nails (two fingers) grips the article17, but the gripper14may include at least three fingers and a configuration of the grip unit provide in an openable and closable manner for gripping the article17is not limited to the configuration described above. Therefore, a configuration of the grip unit model setting unit131for setting a grip unit model including a substantial region which is a region of a substantial portion of the grip unit in an opened state and a grip region inside the substantial region is not limited to the configuration described above.

In the embodiment, a position posture candidate of the gripper14is set (step S2) by using at least one degree of freedom among degrees of freedom in six directions (x, y, z, φ, θ, and ψ), as a parameter or an opening amount d of the gripper14as a parameter, but a configuration of the position posture candidate setting unit132for setting at least one position posture candidate as a candidate of a position and a posture of the grip unit is not limited to the configuration described above.

For example, it is possible that a connected set calculation unit included in the robot control device13determines at least one connected set made by connecting three-dimensional points31present in the vicinity of each other from a plurality of three-dimensional points31measured using the three-dimensional measurement instrument11to set a position posture candidate of the gripper14based on each connected set. When, for example, a distance between a first three-dimensional point31and a second three-dimensional point31next to each other falls within a predetermined value, the connected set is configured by connecting the first three-dimensional point31and the second three-dimensional point31with each other.

FIG. 18is a view illustrating one example of a connected set32. When as illustrated inFIG. 18, a plurality of three-dimensional points31(expressed by311to317) are measured using the three-dimensional measurement instrument11and of these,311and312,312and313,313and314, and315and316are present within a predetermined distance, respectively, these points are connected with each other. In this case,311and314are also connected via312and313and therefore,311to314configure the same connected set321. On the other hand,315and316are not connected with any one of311to314and therefore, configure another connected set322. Since371is not connected to any one of311to316,317singly configures a connected set323.

When surfaces of articles17randomly piled are measured using the three-dimensional measurement instrument11, three-dimensional points31(for example,313and314ofFIG. 18) next to each other on the same article17are located close to each other. In contrast, in a boundary of articles17, positions of three-dimensional points (for example,314and315ofFIG. 18) next to each other are largely changed. Therefore, while the three-dimensional points313and314belong to the same connected set32, the three-dimensional points314and315belong to connected sets32different from each other. Therefore, when a maximum distance between three-dimensional points configuring a connected set32, a minimum number of points and a maximum number of points in the connected set32, and others are appropriately set, the connected set32can be considered a surface shape of a single article17.

When a grip position posture candidate of the gripper14is set based on the connected set32, for example, a gravity center position of three-dimensional points31configuring each connected set32is set as a grip position candidate of the grip position posture candidate. Then, in this grip position candidate, a posture is changed or in the grip position candidate, predetermined postures are combined to set the grip position posture candidate. It is possible to set a grip position posture candidate in a predetermined range (search range) around a gravity center position of the connected set32or to set a grip position posture candidate using a region where three-dimensional points31configuring the connected set32exist as the search range.

In the above-described embodiment, the grip success possibility E of the article17in each grip position posture candidate when the gripper14is placed in the grip position posture candidates is calculated, using the predetermined calculation equations (equation (I) and equation (II)), based on position information of three-dimensional points31acquired using the three-dimensional measurement instrument11and a gripper model20(a grip unit model), but the configuration of the gripability calculation unit133is not limited thereto. It is judged whether the article17is present in the substantial region SP1of the gripper model20and then the grip success possibility E in a grip position posture candidate where the article17is judged to exist is set to be 0, but the grip success possibility E may be decreased to the extent of being unequal to 0.

In the above-described embodiment, it is judged whether the substantial region SP3of the container model161is present in the substantial region SP1of the gripper14, and when the substantial region SP3is present, the grip success possibility E is set to be 0, but the grip success possibility E may be decreased to the extent of being unequal to 0. Various shapes are employable for the container16(storage unit), and the robot control device13functioning as a storage unit model setting unit for setting a storage unit model which is a model of the storage unit may be configured in any manner.

In the above-described embodiment (FIG. 13), an orthogonal projection image50obtained by orthogonally projecting a plurality of three-dimensional points31measured using the three-dimensional measurement instrument11is generated, a position posture candidate is set on the orthogonal projection image50, and the grip success possibility E is calculated based on this position posture candidate and a filtering image60orthogonally projected on the orthogonal projection image50. However, it is possible to project three-dimensional points31on an image using a method other than orthogonal projection. In other words, as long as a position posture candidate is set based on projected points obtained by projecting three-dimensional points31on a plane and then the grip success possibility E is calculated based on a projection model such as the filtering image60obtained by projecting a grip unit model such as the gripper model20on the plane and the projected points, various modifications may be made for this embodiment.

In the above-described embodiment, a grip position posture candidate where the grip success possibility E calculated in the gripability calculation unit133is maximized or locally maximized is set as a gripper position posture. However, as long as at least one position posture candidate is selected from position posture candidates set by the position posture candidate setting unit132based on the grip success possibility E calculated in the gripability calculation unit133and then the selected position posture candidate is set as a grip unit position posture, the position posture setting unit134may be configured in any manner.

In the above-described embodiment, the gripper position posture is expressed using the arrows A1and A2(FIG. 9) and the gripper14is moved to this gripper position posture. However, as long as the robot12is controlled so as to move the gripper14to a gripper position posture (a grip unit position posture) set by the position posture setting unit134to pick up the article17, the robot control unit135may be configured in any manner.

It is possible to arbitrarily combine the embodiment with one modified example or a plurality of modified examples.

According to the present invention, position posture candidates of the grip unit are set, grip success possibilities of an article in the position posture candidates are calculated, and one or more position posture candidates are selected from the position posture candidates based on the grip success possibilities and are set as a grip unit position posture. Therefore, information of an article need not be input and the article can be easily gripped regardless of a shape and a posture of the article.

The present invention has been described in association with the preferred embodiment, but it should be understood by those skilled in the art that various modifications and conversions may be made without departing from the disclosed scope of the claims to be described later.