Image information processing device, gripping system, and image information processing method

An object of the present invention is to ascertain parts of an object that are approachable by finger portions of a hand mechanism. In an image information processing device, a housing container is retrieved from an image by using a first retrieval frame, and an object is retrieved from the image by using a second retrieval frame. A plurality of determination points are set in advance on the second retrieval frame, and a predetermined approach direction is set for each determination point. A determination is then made as to whether or not the finger portions of the hand mechanism can be caused to approach parts of one object, existing within the second retrieval frame, that correspond respectively to the determination points set on the second retrieval frame from the predetermined approach directions set in relation to the determination points.

TECHNICAL FIELD

The present invention relates to a gripping system for gripping an object by using a hand mechanism having a plurality of finger portions, as well as an image information processing device and an image information processing method for processing image information including the object in the gripping system.

BACKGROUND ART

A gripping system for gripping an object by using a hand mechanism having a plurality of finger portions has been developed in the prior art. A system having an imaging device for capturing an image that includes the object is also known as a gripping system. In this type of a gripping system having an imaging device, the object is recognized on the basis of image information acquired by the imaging device. Also, the hand mechanism is controlled on the basis of the image information so as to grip the recognized object.

PTL 1, for example, discloses a picking system (a gripping system) having a monocular camera that acquires an overall image by photographing a plurality of box-shaped workpieces from above. In addition to the monocular camera, the picking system described in PTL 1 includes a distance sensor for measuring the overall three-dimensional shape of the plurality of box-shaped workpieces. Edge parts of the box-shaped workpieces are detected from the overall image acquired by the monocular camera, and a point group of points that are measured by the distance sensor and correspond to the detected edge parts is extracted using the edge parts as boundaries. Further, on the basis of the extracted point group, the position and attitude of each of the box-shaped workpieces are recognized from the three-dimensional shape measured by the distance sensor. The position and attitude of a box-shaped workpiece to be subjected to picking is then output.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Technical Problem

When the hand mechanism attempts to grip an object, first it is necessary to cause the finger portions of the hand mechanism to approach the object and then bring the finger portions into contact with the object. Depending on the manner in which the object is placed, however, parts of the object may be difficult for the finger portions of the hand mechanism to approach. More specifically, when the object is placed such that a certain part of the object is in contact with another object or a housing container housing the object, it is difficult to cause the finger portions of the hand mechanism to approach this contacting part. Moreover, even if a certain part of the object is not in contact with another object or the housing container, when the distance (the interval) between the part and the other object or the housing container is too small, it is likewise difficult to cause the finger portions of the hand mechanism to approach the part. Hence, when an attempt is made to grip an object by using the hand mechanism, it is necessary to ascertain the parts of the object that can be approached by the finger portions of the hand mechanism.

The present invention has been designed in consideration of this problem, and an object thereof is to provide a technique with which it is possible to ascertain parts of an object to be gripped by a hand mechanism, which are approachable by finger portions of the hand mechanism.

Solution to Problem

An image information processing device according to the present invention is a device, which is applied to a gripping system for gripping an object housed in a housing container by using a hand mechanism having a plurality of finger portions, the gripping system including an imaging device for capturing an image including the housing container and at least one object housed in the housing container, and which processes image information acquired by the imaging device, the image information processing device including a retrieval unit which, on the basis of shape information about the housing container and shape information about the object, with the shape information being stored in advance, retrieves the housing container and the object from the image captured by the imaging device, the housing container being retrieved using a first retrieval frame, which is a retrieval frame for retrieving the housing container, and the object being retrieved using a second retrieval frame, which is a retrieval frame for retrieving the object, and a detection unit that detects a part of one object, housed in the housing container, that can be approached by the finger portions of the hand mechanism on the basis of contact states of the one object relative to other objects and the housing container, the contact states being acquired from retrieval results acquired by the retrieval unit, wherein a plurality of determination points are set in advance in predetermined positions on the second retrieval frame, and a predetermined approach direction is set for each of the determination points, and the detection unit determines whether or not the finger portions of the hand mechanism can be caused to approach parts of one object, existing within the second retrieval frame, that correspond respectively to the determination points set on the second retrieval frame from the predetermined approach directions set in relation to the determination points.

Advantageous Effects of Invention

According to the present invention, it is possible to ascertain parts of an object to be gripped by a hand mechanism, which are approachable by finger portions of the hand mechanism.

MODE FOR CARRYING OUT THE INVENTION

A gripping system according to the present invention is a system for gripping an object housed in a housing container using a hand mechanism having a plurality of finger portions. The gripping system includes an imaging device that captures an image including the housing container and at least one object housed in the housing container. In an image information processing device according to the present invention, a retrieval unit retrieves the housing container and the object from the image captured by the imaging device on the basis of shape information about the housing container and shape information about the object. Further, in the image information processing device, a detection unit detects a part of one object housed in the housing container that can be approached by the finger portions of the hand mechanism on the basis of contact states of the one object relative to other objects and the housing container.

More specifically, the shape information of the housing container and the shape information of the object are stored in the image information processing device in advance. Note that the shape information of the respective components includes not only information relating to the shape of the housing container or the object, but also information relating to the dimensions thereof. The retrieval unit retrieves the housing container and the object from the image captured by the imaging device on the basis of the shape information thereof. At this time, the retrieval unit retrieves the housing container using a first retrieval frame and retrieves the object using a second retrieval frame. The first retrieval frame is a retrieval frame for retrieving the housing container, and the second retrieval frame is a retrieval frame for retrieving the object. More specifically, when the retrieval unit searches the image using the first retrieval frame and detects a shape matching the shape information of the housing container within the first retrieval frame, the retrieval unit recognizes that the housing container exists within the first retrieval frame. Note that the first retrieval frame may be a retrieval frame for retrieving the part of the housing container that houses the objects. Further, when the retrieval unit searches the image using the second retrieval frame and detects a shape matching the shape information of the object within the second retrieval frame, the retrieval unit recognizes that the object exists within the second retrieval frame.

Furthermore, a plurality of determination points are set in advance in predetermined positions on the second retrieval frame. Moreover, a predetermined approach direction is set for each determination point. Here, each determination point indicates a position on the one object existing within the second retrieval frame (in other words, the object whose existence on the image was recognized using the second retrieval frame) in relation to which it is to be determined whether or not that position can be approached by the finger portions of the hand mechanism. Further, the predetermined approach directions indicate approach directions used when the finger portions of the hand mechanism are caused to approach parts corresponding to the respective determination points on the one object existing within the second retrieval frame.

The detection unit then determines, in relation to the one object existing within the second retrieval frame, whether or not the finger portions of the hand mechanism can be caused to approach the parts corresponding to the respective determination points set on the second retrieval frame from the predetermined approach directions set in relation to the determination points. By determining whether or not the finger portions of the hand mechanism can be caused to approach the parts corresponding to the respective determination points on the one object existing within the second retrieval frame in relation to each of the plurality of determination points set on the second retrieval frame in this manner, it is possible to detect the parts of the one object that can be approached by the finger portions of the hand mechanism.

Thus, the form of the hand mechanism when the hand mechanism is caused to approach the one object and the gripping style to be used to grip the one object using the hand mechanism can be determined on the basis of detection results indicating the parts of the one object that can be approached by the finger portions of the hand mechanism. As a result, improvements in stability and tact time can be achieved when an object is gripped using the hand mechanism.

Specific embodiments of the present invention will be described below on the basis of the figures. Unless specified otherwise, the technical scope of the invention is not limited only to the dimensions, materials, shapes, relative arrangements, and so on of constituent components described in the embodiments.

First Embodiment

Here, a case in which the present invention is applied to a gripping system having a robot arm will be described.FIG.1is a schematic view showing a configuration of the gripping system according to this embodiment. A gripping system100includes a robot arm1and an imaging device8. The imaging device8is a camera, for example. The imaging device8captures an image including a housing container11and at least one object10housed in the housing container11. At this time, the imaging device8captures an image including the housing container11and the object10housed in the housing container11from a predetermined direction. Note that the imaging device8does not necessarily have to be provided separately to the robot arm1, as shown inFIG.1. In other words, the imaging device8may be attached to the robot arm1. Further, the robot arm1includes a hand mechanism2for gripping the object10, an arm mechanism3, and a support portion4. The configurations of the arm mechanism3, hand mechanism2, and support portion4of the robot arm1will now be described in detail.

FIG.2is a schematic view showing the configuration of the robot arm1. The hand mechanism2is attached to one end of the arm mechanism3. Further, the support portion4is attached to the other end of the arm mechanism3. The arm mechanism3includes a first arm link portion31, a second arm link portion32, a third arm link portion33, a fourth arm link portion34, a fifth arm link portion35, and a connecting member36. A base portion20of the hand mechanism2is connected to a first joint portion30aformed on one end side of the first arm link portion31of the arm mechanism3. A motor (not shown) for rotating the hand mechanism2relative to the first arm link portion31about the first arm link portion31is provided in the first joint portion30a. The other end side of the first arm link portion31is connected to one end side of the second arm link portion32by a second joint portion30b. The first arm link portion31and the second arm link portion32are connected so that respective central axes thereof intersect perpendicularly. A motor (not shown) for rotating the first arm link portion31relative to the second arm link portion32axially about the other end side thereof is provided in the second joint portion30b. Further, the other end side of the second arm link portion32is connected to one end side of the third arm link portion33by a third joint portion30c. A motor (not shown) for rotating the second arm link portion32relative to the third arm link portion33is provided in the third joint portion30c.

Similarly, the other end side of the third arm link portion33is connected to one end side of the fourth arm link portion34by a fourth joint portion30d. Furthermore, the other end side of the fourth arm link portion34is connected to the fifth arm link portion35by a fifth joint portion30e. A motor (not shown) for rotating the third arm link portion33relative to the fourth arm link portion34is provided in the fourth joint portion30d. Moreover, a motor (not shown) for rotating the fourth arm link portion34relative to the fifth arm link portion35is provided in the fifth joint portion30e. Further, the fifth arm link portion35is connected to the connecting member36, which is disposed to extend vertically from the support portion4, by a sixth joint portion30f. The fifth arm link portion35and the connecting member36are connected so that respective central axes thereof are coaxial. A motor (not shown) for rotating the fifth arm link portion35about the fifth arm link portion35and the connecting member36is provided in the sixth joint portion30f. By configuring the arm mechanism3in this manner, the arm mechanism3can be realized as a mechanism having six degrees of freedom, for example.

Next, the configuration of the hand mechanism2will be described on the basis ofFIGS.3to6.FIG.3is a perspective view of the hand mechanism2.FIG.4is a top view of the hand mechanism2. Note that inFIG.4, arrows represent rotation ranges of respective finger portions21. The hand mechanism2includes a base portion20connected to the arm mechanism3, and four finger portions21provided on the base portion20. Further, as shown inFIGS.3and4, in the hand mechanism2, the four finger portions21are disposed on the base portion20at equal angular intervals (more specifically, at90deg intervals) on a circumference centering on a longitudinal direction (a perpendicular direction to the paper surface inFIG.4) axis of the hand mechanism2. Furthermore, the four finger portions21all have identical structures and identical lengths. Note, however, that the operations of the respective finger portions21are controlled independently.

As shown inFIG.3, each finger portion21includes a first finger link portion211, a second finger link portion212, and a base end portion213. The base end portion213of the finger portion21is connected to the base portion20. Here, as indicated by arrows inFIG.4, the base end portion213is connected to the base portion20so as to be capable of rotating relative to the base portion20about a longitudinal direction (a perpendicular direction to the paper surface inFIG.4) axis of the finger portion21. Further, in the finger portion21, one end of the second finger link portion212is connected to the base end portion213. A second joint portion23is formed in a connecting portion between the second finger link portion212and the base end portion213. A motor for driving the base end portion213to rotate and a motor for driving the second finger link portion212to rotate relative to the base end portion213are provided in the interior of the base portion20. According to this configuration, the base end portion213is driven to rotate, and in accordance therewith, the entire finger portion21is driven to rotate within a range indicated by an arrow inFIG.4. Furthermore, as shown inFIG.5, the second joint portion23is formed to be capable of bending and extending.

Further, as shown inFIGS.3,5, and6, in the finger portion21, one end of the first finger link portion211is connected to the other end of the second finger link portion212. A first joint portion22is formed in a connecting portion between the first finger link portion211and the second finger link portion212. A motor for driving the first finger link portion211to rotate relative to the second finger link portion212is provided in the interior of the second finger link portion212. According to this configuration, as shown inFIG.6, the first joint portion22is formed to be capable of bending and extending.

Moreover, as shown inFIG.3, in this embodiment, a pressure sensor70is provided on the tip end side of the first finger link portion211of the finger portion21. The pressure sensor70is a sensor for detecting external force (pressure) acting on the tip end portion of the first finger link portion211.

Next, configurations of an arm control device42, a hand control device43, and an image information processing device44, which are built into the support portion4, will be described on the basis ofFIG.7. The arm control device42is a control device for controlling the arm mechanism3of the robot arm1. The hand control device43is a control device for controlling the hand mechanism2of the robot arm1. The image information processing device44is a processing device for processing image information acquired when the imaging device8captures an image including the housing container11and the object10.FIG.7is a block diagram showing respective function units included in the arm control device42, the hand control device43, and the image information processing device44.

The arm control device42is configured to include a plurality of drivers that generate drive signals for driving the motors provided in the respective joint portions of the arm mechanism3, the drive signal from each driver being supplied to the corresponding motor. The arm control device42also includes a computer having a calculation processing device and a memory. The arm control device42includes an arm control unit420and a motor state quantity acquisition unit421as function units. These function units are formed by executing a predetermined control program on the computer included in the arm control device42.

The arm control unit420controls the motors provided respectively in the joint portions30a,30b,30c,30d,30e,30fof the arm mechanism3by supplying the drive signals from the respective drivers thereto on the basis of object information acquired by an object information acquisition unit430, to be described below, which serves as a function unit of the hand control device43. The arm control unit420moves the arm mechanism3by controlling the respective motors, whereby the position of the hand mechanism2is moved to a desired position (a position in which the object10can be gripped by the hand mechanism2). Further, each of the motors provided in the joint portions30a,30b,30c,30d,30e,30fof the arm mechanism3is provided with an encoder (not shown) for detecting state quantities (a rotation position and a rotation speed of a rotary shaft of the motor, and so on) relating to the rotation state of the corresponding motor. The state quantities of the motors, detected by the encoders of the respective motors, are input into the motor state quantity acquisition unit421of the arm control device42. Then, on the basis of the state quantities of the respective motors, input into the motor state quantity acquisition unit421, the arm control unit420servo-controls the respective motors so that the hand mechanism2moves to the desired position.

Furthermore, the hand control device43is configured to include a plurality of drivers that generate drive signals for driving the motors provided in the hand mechanism2, the drive signal from each driver being supplied to the corresponding motor. The hand control device43also includes a computer having a calculation processing device and a memory. The hand control device43includes the object information acquisition unit430, a hand control unit431, a motor state quantity acquisition unit432, and a sensor information acquisition unit433as function units. These function units are formed by executing a predetermined control program on the computer included in the hand control device43.

The object information acquisition unit430acquires object information, which is information relating to the object to be gripped by the hand mechanism2, from the image information processing device44. Further, the hand control unit431controls the motors for driving the respective finger portions21of the hand mechanism2by supplying drive signals from the drivers thereto on the basis of the object information acquired by the object information acquisition unit430. As a result, the object10is gripped by the plurality of finger portions21of the hand mechanism2. Moreover, each motor of the hand mechanism2is provided with an encoder (not shown) for detecting state quantities (the rotation position and rotation speed of the rotary shaft of the motor, and so on) relating to the rotation state of the corresponding motor. The state quantities of the motors, detected by the encoders of the motors, are input into the motor state quantity acquisition unit432of the hand control device43. Then, on the basis of the state quantities of the motors, input into the motor state quantity acquisition unit432, the hand control unit431servo-controls the motors of the respective finger portions21so that the plurality of finger portions21grip the object.

The hand control device43also includes a sensor information acquisition unit (not shown). Detection values from the pressure sensors70provided in the first finger link portions211of the respective finger portions21of the hand mechanism2are input into the sensor information acquisition unit. Then, when the pressure sensors70detect contact with the object by the finger portions21, the hand control unit431can control the motors of the respective finger portions21on the basis of detection signals therefrom.

The image information processing device44includes a computer having a calculation processing device and a memory. Further, the image information processing device44includes an image information acquisition unit440, a retrieval unit441, and a detection unit442as function units. These function units are formed by executing a predetermined control program on the computer included in the image information processing device44. The image information processing device44also includes a storage unit443. The storage unit443is means for storing shape information about the housing container11, shape information about the object10, and so on. The shape information of the housing container11and the shape information of the object10, stored by the storage unit443, are registered in advance by a user. Note that the shape information of the respective components includes not only information relating to the shape of the housing container11or the object10, but also information relating to the dimensions thereof. Further, image information processing results generated by the image information processing device44are input into the object information acquisition unit430of the hand control unit431. Note that functions realized by the respective function units of the image information processing device44will be described below.

Note that inFIG.7, the arm control device42and the hand control device43are shown separately as control devices included in the gripping system, but as a different method, a configuration in which the respective function units are formed in a single control device integrating the two devices may be employed. Further, likewise in a case where the control devices included in the gripping system are separated into the arm control device42and the hand control device43, the function units shown inFIG.7may essentially be formed in either of the control devices, as long as no technical inconsistencies occur as a result, and information may be exchanged appropriately as required between the arm control device42and the hand control device43. A configuration in which some of the function units of either the arm control device42or the hand control device43are formed in a separate control device to the arm control device42and the hand control device43may also be employed.

Here, specific examples of gripping styles realized when the object10is gripped by the hand mechanism2of the robot arm1in the gripping system100according to this embodiment will be described on the basis ofFIGS.8to13.FIG.8is a view illustrating direct gripping, which is a gripping style in which the object10is gripped by the hand mechanism2in the state in which the object10is placed. Further,FIGS.9to11are views illustrating tilt gripping, which is a gripping style in which the object10is tilted before being gripped by the hand mechanism2.FIGS.12and13are views illustrating shift gripping, which is a gripping style in which the position of the object10is shifted before the object10is gripped by the hand mechanism2. Note that here, the finger portions21of the hand mechanism2will be referred to respectively as a first finger portion21A, a second finger portion21B, a third finger portion21C, and a fourth finger portion21D.

FIG.8shows the states of the hand mechanism2and the object10when the object10is gripped by direct gripping. In direct gripping, as shown inFIG.8, the placed object10is sandwiched between and gripped by the finger portions21of the hand mechanism2as is. Note that the object10does not necessarily have to be sandwiched between all four of the finger portions21of the hand mechanism2, and as long as the object10can be sandwiched between at least two of the finger portions21, direct gripping is established. Gripping of the object10by direct gripping can be implemented in any situation where the finger portions21of the hand mechanism2can be brought into contact with each of two mutually opposing parts of the object10.

Further,FIGS.9and10show an operation for tilting the object10using the first finger portion21A of the hand mechanism2in order to grip the object10by tilt gripping. Furthermore,FIG.11shows a state in which the object10has been gripped by the second finger portion21B, the third finger portion21C, and the fourth finger portion21D of the hand mechanism2.

More specifically, inFIG.9, the object10is placed side by side and in contact with another, adjacent object10′. Even in this case, the finger portions21of the hand mechanism2can contact an upper face S1of the object10. Therefore, first, as shown inFIG.9, the first finger link portion211A of the first finger portion21A is brought into contact with the upper face S1of the object10. Note that at this time, in a state where the first finger portion21A is in contact with the upper face S1of the object10, the other finger portions21B,21C,21D of the hand mechanism2do not contact the object10. Next, as shown inFIG.10, the object10is tilted forward by the first finger portion21A while the first finger link portion211A contacts the upper face S1of the object10. By modifying the attitude of the object10using the first finger portion21A in this manner, the interval between the object10and the adjacent object10′ can be increased. As a result, a situation in which the finger portions21of the hand mechanism2can be brought into contact with a side face of the object10that could not be contacted by the finger portions21of the hand mechanism2while the object10was placed in contact with the adjacent object10′ is established. Next, as shown inFIG.11, with the object10having been tilted by the first finger portion21A, the object10is sandwiched between and gripped by the second finger portion21B, the third finger portion21C, and the fourth finger portion21D. Gripping of the object10by tilt gripping can be implemented in any situation where the object10can be tilted by the hand mechanism2, even in a state where the finger portions21of the hand mechanism2cannot be brought into contact with mutually opposing parts of the object10.

Further,FIGS.12and13are views showing an operation for shifting the position of the object10using the first finger portion21A of the hand mechanism2in order to grip the object10by shift gripping.

More specifically, inFIG.12, the object10is placed side by side and in contact with another, adjacent object10′. Even in this case, the finger portions21of the hand mechanism2can contact a side face S2of the object10, which opposes a side face S3that is in contact with the other object10′. Therefore, first, as shown inFIG.12, the first finger link portion211A of the first finger portion21A is brought into contact with the side face S2of the object10. The object10is then pressed against the other object10′ as is by pressing the side face S2of the object10using the first finger portion21A, and while the object10is being pressed, the position of the object10is shifted by the first finger portion21A in a direction indicated by a black-outlined arrow.

Thus, as shown inFIG.13, the position of the object10moves upward, and as a result, the object10projects upward beyond the height of the upper face S1of the other object10′. In other words, on the object10, the upper part of the side face S3contacting the other object10′ is exposed. Hence, the finger portions21(the finger portions21other than the first finger portion21A) of the hand mechanism2can contact not only the side face S2of the object10but also the side face S3opposing the side face S2. The finger portions21of the hand mechanism2are then brought into contact with the side face S2and the side face S3of the object10in the state shown inFIG.16such that the object10is sandwiched between the finger portions21and gripped thereby. Gripping of the object10by tilt gripping can be implemented in any situation where it is possible to shift the position of the object10using the hand mechanism2, even in a state where the finger portions21of the hand mechanism2cannot be brought into contact with mutually opposing parts of the object10.

In the gripping system100according to this embodiment, as described above, the object10can be gripped using various gripping styles, including direct gripping, tilt griping, and shift gripping, which are realized by the hand mechanism2. Note that inFIGS.8to13, the object10has a rectangular parallelepiped shape, but even when the object10has a different shape, tilt gripping and shift gripping can be implemented in addition to direct gripping.

In order to grip the object10using the various gripping styles described above, it is necessary in all cases to bring the finger portions21of the hand mechanism2into contact with the object10. For this purpose, it is necessary to ascertain the parts (also referred to hereafter as “approachable parts”) of the object10housed in the housing container11that can be approached by the finger portions21of the hand mechanism2. Hence, in the gripping system100according to this embodiment, the approachable parts of the object10housed in the housing container11are detected by processing, in the image information processing device44, the image information acquired when the imaging device8captures an image including the housing container11and the object10.

Specific examples of the image information processing performed in the image information processing device44will be described below. First, a first specific example will be described on the basis ofFIGS.14to18.FIG.14is a view showing an image acquired when the imaging device8photographs the housing container11and a plurality of objects10housed in the housing container11according to the first specific example. In this specific example, as shown inFIG.14, the housing container11takes the shape of a box in which a front part (a front side face) and an upper part (a top face) are open. Further, the objects10have a rectangular parallelepiped shape. Twelve objects10are housed in the housing container11in rows without gaps. As noted above, the shape information of the housing container11and the shape information of the object10are stored in the storage unit443of the image information processing device44in advance.

In the gripping system100, the imaging device8captures an image including the housing container11and the objects10in the state described above from a direction in which the front faces thereof are visible.FIG.15is a view showing an image acquired by projecting the image captured by the imaging device8from a viewpoint in front of the housing container11(in other words, a projected image taken from the direction of a black-outlined arrow inFIG.14). As shown inFIG.15, the projected image includes the housing container11and three objects10ato10cdisposed in the frontmost part of the housing container11.

In the image information processing device44, an image such as that shown inFIG.14is acquired by the image information acquisition unit440as image information. Then, in the image information processing device44, the retrieval unit441retrieves the housing container11and the objects10from a projected image such as that shown inFIG.15, which is acquired on the basis of the image captured by the imaging device8.FIG.16is a view showing the manner in which the housing container11and the objects10are retrieved from the projected image shown inFIG.15.

In the image information processing device44, the retrieval unit441retrieves the housing container11using a first retrieval frame, which is a retrieval frame for retrieving the housing container, and retrieves the object10using a second retrieval frame, which is a retrieval frame for retrieving the object. InFIG.16, a frame f1indicated by dot-dot-dash lines denotes the first retrieval frame, and a frame f2indicated by dot-dash lines denotes the second retrieval frame. Here, the first retrieval frame f1is set on the basis of the shape information of the housing container11, which is registered in the storage unit443. As shown inFIG.16, the first retrieval frame f1is set as a retrieval frame for retrieving the part of the housing container11that houses the objects10. Further, the second retrieval frame f2is set on the basis of the shape information of the object10, which is registered in the storage unit443. As shown inFIG.16, the second retrieval frame f2is set in accordance with the size of the object10on the image (more specifically, the size of an outer periphery of the object10when the object10is seen from the front). Furthermore, the first retrieval frame f1and the second retrieval frame f2are both rectangular. Note, however, that the first retrieval frame f1and the second retrieval frame f2do not necessarily have to be rectangular and may have a different polygonal shape.

As shown inFIG.16, when the retrieval unit441searches the image using the first retrieval frame f1and detects, within the first retrieval frame f1, a shape matching the shape information relating to the part of the housing container11that houses the objects10, the retrieval unit441recognizes that the part of the housing container11that houses the objects10exists within the first retrieval frame f1. Further, when the retrieval unit441searches the image using the second retrieval frame f2and detects a shape matching the shape information of the object10within the second retrieval frame f2, the retrieval unit441recognizes that the object10exists within the second retrieval frame f2. As a result of the retrieval performed by the retrieval unit441in this manner, it is recognized in the image information processing device44that three objects10ato10care arranged in the housing container11in a row without gaps.

Furthermore, a plurality of determination points are set in advance in predetermined positions on the second retrieval frame f2. Moreover, a predetermined approach direction is set for each determination point.FIG.17is a view showing the plurality of determination points set on the second retrieval frame f2and the predetermined approach direction set for each determination point.

Here, each determination point indicates a position, on the one object10existing within the second retrieval frame f2, in relation to which it is to be determined whether or not the finger portions21of the hand mechanism2can approach that position. InFIG.17, black circles on the second retrieval frame f2indicate the determination points. As shown inFIG.17, determination points are set in a total of twelve locations on the second retrieval frame f2, namely one at each vertex and in two locations on each side. Further, the predetermined approach directions indicate approach directions used when the finger portions21of the hand mechanism2are caused to approach parts corresponding to the respective determination points on the one object10existing within the second retrieval frame f2. InFIG.17, arrows set respectively in relation to the black circles denote the predetermined approach directions. As shown inFIG.17, directions that are orthogonal to the respective sides of the second retrieval frame f2are set as the predetermined approach directions in relation to the determination points set on the respective sides. Further, two directions that are respectively orthogonal to the two sides forming each of the vertices of the second retrieval frame f2are set as the predetermined approach directions in relation to the determination points set on the respective vertices.

In the image information processing device44, the detection unit442then detects the approachable parts of each of the objects10whose existence was recognized on the image, as shown inFIG.16, as a result of the retrieval performed by the retrieval unit441. At this time, the detection unit442detects the approachable parts of one object10on the basis of contact states of the one object10relative to the other objects10and the housing container11, the contact states being acquired from the retrieval results acquired by the retrieval unit441.

More specifically, the detection unit442determines whether or not the finger portions21of the hand mechanism2can be caused to approach the parts of the one object10existing within the second retrieval frame f2that correspond respectively to the determination points set on the second retrieval frame f2from the predetermined approach directions set in relation to the determination points. Hereafter, this determination processing performed by the detection unit442may be referred to as “approach determination processing”. Further, to facilitate description of the approach determination processing, a case in which it is determined during the approach determination processing that the finger portions21of the hand mechanism2can be caused to approach a part corresponding to a determination point on the object10from the predetermined approach direction set in relation to the determination point may be referred to as “an approach vector is established”. Furthermore, a case in which it is determined during the approach determination processing that the finger portions21of the hand mechanism2cannot be caused to approach a part corresponding to a determination point on the object10from the predetermined approach direction set in relation to the determination point may be referred to as “an approach vector is not established”.

FIG.18is a view showing some results of the approach determination processing performed by the detection unit442.FIG.18shows results of the approach determination processing performed in relation to the objects10b,10cshown inFIG.16. InFIG.18, a circle on an arrow indicating a predetermined approach direction indicates that an approach vector is established. Further, inFIG.18, a cross on an arrow indicating a predetermined approach direction indicates that an approach vector is not established. Note that a solid-line cross indicates that an approach vector is not established due to the one object10in the retrieval frame f2being in contact with the housing container11. Further, a dot-dash-line cross indicates that an approach vector is not established due to the one object10in the retrieval frame f2being in contact with another object10. In this embodiment, as described above, the retrieval unit441retrieves the housing container11and the objects10. The detection unit442then executes the approach determination processing on each object10on the basis of the retrieval results acquired by the retrieval unit441. As a result, it is possible to determine whether the contact partner of the one object10existing within the retrieval frame f2is the housing container11or another object10.

As described above, the three objects10ato10care disposed in the housing container11in a row without gaps. Hence, as shown inFIG.18, on both of the objects10b,10c, an approach vector is not established in relation to the determination points set in positions corresponding to the lower face and the side faces of each of the objects10b,10c. In other words, during the approach determination processing, when another object10or the housing container11is in contact with a part of the one object10existing within the second retrieval frame f2that corresponds to a determination point set on the second retrieval frame f2from the predetermined approach direction set in relation to the determination point, the detection unit442determines that the finger portions21of the hand mechanism2cannot be caused to approach the part of the one object10that corresponds to the determination point from the predetermined approach direction.

On the other hand, on both of the objects10b,10c, an approach vector is basically established in relation to the predetermined approach directions indicating approach from above on the determination points set in positions corresponding to the respective upper faces of the objects10b,10c. In other words, on each of the objects10b,10c, approach vectors are established in relation to the determination points in the two locations set on the upper side of the retrieval frame f2.

Further, in the object10b, approach vectors are established in relation to the predetermined approach directions indicating approach from above on the determination points set on the two vertices forming the two ends of the upper side of the retrieval frame f2. Also, in the object10c, an approach vector is established in relation to the predetermined approach direction indicating approach from above on the determination point set on the vertex on the object10bside (the left side inFIG.18), of the two vertices forming the two ends of the upper side of the retrieval frame f2. Note, however, that in the object10c, an approach vector is not established in relation to the predetermined approach direction indicating approach from above on the determination point set on the vertex on the side of a side wall of the housing container11(the right side inFIG.18), of the two vertices forming the two ends of the upper side of the retrieval frame f2. The reason for this, as shown inFIGS.14and15, is that the height of the side wall of the housing container11that is contacted by the object10cis higher than the height of the object10c. In this case, even though the top of the housing container11is open, the side wall of the housing container11obstructs the end portion of the upper face of the object10con the side of the side wall of the housing container11, making it impossible to bring the finger portions21of the hand mechanism2into contact therewith. Hence, as described above, in the object10c, an approach vector is not established in relation to the predetermined approach direction indicating approach from above on the determination point set on the vertex on the side of the side wall of the housing container11, of the two vertices forming the two ends of the upper side of the retrieval frame f2.

In other words, in the approach determination processing, the detection unit442determines that the finger portions21of the hand mechanism2cannot be caused to approach a vertex formed by a side that contacts the housing container11on the outer periphery of the image of the one object10and another side from either of the two predetermined approach directions set in relation to the determination point set on the vertex of the second retrieval frame f2that corresponds to this vertex.

Next, a second specific example will be described on the basis ofFIGS.19to22.FIG.19is a view showing an image acquired when the imaging device8photographs the housing container11and the plurality of objects10housed in the housing container11according to the second specific example. In this specific example, as shown inFIG.19, the housing container11takes the shape of a box in which only the upper part (the top face) is open. Further, similarly to the first specific example, the objects10have a rectangular parallelepiped shape. Nine objects10are housed in the housing container11in rows. Here, the nine objects10are housed in the housing container11so as to packed into the front side. Therefore, in the part of the housing container11housing the nine objects10, no gaps are formed between the objects10and between the objects10and the housing container11. However, a space where no objects10are placed is formed in the rear part of the housing container11. As noted above, likewise in this specific example, the shape information of the housing container11and the shape information of the object10are stored in the storage unit443of the image information processing device44in advance.

In the gripping system100, the imaging device8captures an image including the housing container11and the objects10in the state described above from a direction in which the upper faces thereof are visible.FIG.20is a view showing an image acquired by projecting the image captured by the imaging device8from a viewpoint above the housing container11(in other words, a projected image taken from the direction of a black-outlined arrow inFIG.19). As shown inFIG.20, the projected image includes the housing container11and nine objects10ato10ihoused in the housing container11.

In the image information processing device44, an image such as that shown inFIG.19is acquired by the image information acquisition unit440as image information. Then, in the image information processing device44, the retrieval unit441retrieves the housing container11and the objects10from a projected image such as that shown inFIG.20, which is acquired on the basis of the image captured by the imaging device8.FIG.21is a view showing the manner in which the housing container11and the objects10are retrieved from the projected image shown inFIG.20.

Likewise in this specific example, similarly to the first specific example, the retrieval unit441retrieves the housing container11using the first retrieval frame f1, which is a retrieval frame for retrieving the housing container, and retrieves the object10using the second retrieval frame f2, which is a retrieval frame for retrieving the object. Further, as shown inFIG.21, the first retrieval frame f1is set as a retrieval frame for retrieving the part of the housing container11housing the objects10. Furthermore, as shown inFIG.21, the second retrieval frame f2is set in accordance with the size of the object10on the image (in other words, the size of the outer periphery of the object10when the object10is seen from above). As a result of the retrieval performed by the retrieval unit441, it is recognized in the image information processing device44that the nine objects10ato10iare arranged in the front part of the housing container11in rows without gaps.

Furthermore, likewise in this specific example, similarly to the first specific example, a plurality of determination points are set on the second retrieval frame f2, and similarly to the first specific example, a predetermined approach direction is set for each determination point. Moreover, likewise in this specific example, the approach determination processing is performed by the detection unit442on the basis of the contact states of the one object10relative to the other objects10and the housing container11, the contact states being acquired from the retrieval results acquired by the retrieval unit441.

FIG.22is a view showing some results of the approach determination processing performed by the detection unit442.FIG.22shows results of the approach determination processing performed in relation to the objects10b,10cshown inFIG.21. Note that similarly toFIG.18, inFIG.22, black circles on the second retrieval frame f2indicate the determination points. Further, inFIG.22, arrows set in relation to the black circles indicate the predetermined approach directions. Furthermore, inFIG.22, a black circle on an arrow indicating a predetermined approach direction indicates that an approach vector is established. Moreover, inFIG.22, a cross on an arrow indicating a predetermined approach direction indicates that an approach vector is not established. Note that a solid-line cross indicates that an approach vector is not established due to the one object10in the retrieval frame f2being in contact with the housing container11. Further, a dot-dash-line cross indicates that an approach vector is not established due to the one object10in the retrieval frame f2being in contact with another object10.

As noted above, the nine objects10ato10iare arranged in the front part of the housing container11in rows without gaps. Therefore, as shown inFIG.22, on both of the objects10b,10c, an approach vector is not established in relation to any of the determination points set in positions corresponding to the front side faces, right-side side faces, and left-side side faces of the objects10b,10c.

However, as shown inFIG.21, a space is formed in the rear part of the housing container11behind the objects10ato10c. Therefore, on both of the objects10b,10c, an approach vector is basically established in relation to the predetermined approach directions indicating approach from the rear on the determination points set in positions corresponding to the rear side faces of the objects10b,10c. In other words, on both of the objects10b,10c, approach vectors are established for the determination points in the two locations set on the rear side of the retrieval frame f2.

Further, in the object10b, approach vectors are also established in relation to the predetermined approach directions indicating approach from the rear on the determination points set on the two vertices forming the two ends of the rear side of the retrieval frame f2. Also, in the object10c, an approach vector is established in relation to the predetermined approach direction indicating approach from the rear on the determination point set on the vertex on the object10bside (the left side inFIG.22), of the two vertices forming the two ends of the rear side of the retrieval frame f2. Note, however, that in the object10c, an approach vector is not established in relation to the predetermined approach direction indicating approach from the rear on the determination point set on the vertex on the side of the side wall of the housing container11(the right side inFIG.22), of the two vertices forming the two ends of the rear side of the retrieval frame f2. The reason for this, as shown inFIG.21, is that even though a space is formed in the rear part of the housing container11behind the objects10ato10c, the side wall of the housing container11obstructs the end portion of the rear side face of the object10con the side of the side wall of the housing container11, making it impossible to bring the finger portions21of the hand mechanism2into contact therewith.

Note that in the first and second specific examples described above, when another object10or the housing container11is in contact with the one object10existing within the second retrieval frame f2, it is determined that an approach vector is not established. However, even when the one object10is not in contact with another object10or the housing container11, the finger portions21of the hand mechanism2cannot be caused to approach the one object10in a part where the gap between the two components is too small. Therefore, in the approach determination processing, when a distance (also referred to hereafter as a “determination distance”) between a part of the one object10in the second retrieval frame f2that corresponds to a determination point set on the second retrieval frame f2and another object10or the housing container11in the predetermined approach direction set in relation to the determination point is equal to or shorter than a predetermined threshold, it may be determined that the finger portions21of the hand mechanism2cannot be caused to approach the part of the one object10corresponding to the determination point from the predetermined approach direction.

Next, a third specific example will be described on the basis ofFIGS.23to28.FIG.23is a view showing an image acquired when the imaging device8photographs the housing container11and the plurality of objects10housed in the housing container11according to the third specific example. In this specific example, as shown inFIG.23, the housing container11takes the shape of a box in which only the upper part (the top face) is open. Further, the objects10have a columnar shape. Nine objects10are housed in the housing container11in rows so that adjacent objects10contact each other. Further, in the housing container11, the objects10disposed in positions adjacent to the side walls of the housing container11also contact the housing container11. As noted above, likewise in this specific example, the shape information of the housing container11and the shape information of the object10are stored in the storage unit443of the image information processing device44in advance.

In the gripping system100, the imaging device8captures an image including the housing container11and the objects10in the state described above from a direction in which the upper faces thereof are visible.FIG.24is a view showing an image acquired by projecting the image captured by the imaging device8from a viewpoint above the housing container11(in other words, a projected image taken from the direction of a black-outlined arrow inFIG.23). As shown inFIG.24, the projected image includes the housing container11and the nine objects10ato10ihoused in the housing container11.

In the image information processing device44, an image such as that shown inFIG.23is acquired by the image information acquisition unit440as image information. Then, in the image information processing device44, the retrieval unit441retrieves the housing container11and the objects10from a projected image such as that shown inFIG.24, which is acquired on the basis of the image captured by the imaging device8.FIG.25is a view showing the manner in which the housing container11and the objects10are retrieved from the projected image shown inFIG.24.

Likewise in this specific example, similarly to the first specific example, the retrieval unit441retrieves the housing container11using the first retrieval frame f1, which is a retrieval frame for retrieving the housing container, and retrieves the object10using the second retrieval frame f2, which is a retrieval frame for retrieving the object. Further, as shown inFIG.25, the first retrieval frame f1is set as a retrieval frame for retrieving the part of the housing container11housing the objects10. Furthermore, as shown inFIG.25, the second retrieval frame f2is set in accordance with the size of the object10on the image (in other words, the size of the outer periphery of the object10when the object10is seen from above). Note that in this specific example, the objects10have a columnar shape, and therefore, as shown inFIG.25, the shape of the outer periphery of the object10when the object10is seen from above is circular. Note, however, that likewise in this case, the second retrieval frame f2is rectangular. Also note that likewise in this case, the second retrieval frame f2may have a polygonal shape other than a rectangle.

As a result of the retrieval performed by the retrieval unit441, it is recognized in the image information processing device44that the nine objects10ato10iare housed in the housing container11in rows so that adjacent objects10contact each other, and the objects10ato10dand10fto10idisposed in positions adjacent to the side walls of the housing container11also contact the housing container11.

Furthermore, likewise in this specific example, similarly to the first specific example, a plurality of determination points are set on the second retrieval frame f2. Also, a predetermined approach direction is set for each determination point. Note, however, that in this specific example, since the objects10have a columnar shape, the method of setting predetermined approach directions in relation to the determination points partially differs from the first specific example. More specifically, the predetermined approach directions set in relation to the determination points set on the vertices of the second retrieval frame f2differ from the first specific example.

FIG.26is a view showing the plurality of determination points set on the second retrieval frame f2and the predetermined approach directions set in relation to the determination points. Similarly toFIG.17, inFIG.26, black circles on the second retrieval frame f2indicate the determination points. Further, inFIG.26, arrows set in relation to the black circles indicate the predetermined approach directions. As shown inFIG.26, likewise in this specific example, similarly to the first specific example, determination points are set in a total of twelve locations on the second retrieval frame f2, namely one at each vertex and in two locations on each side. Furthermore, as shown inFIG.26, directions that are orthogonal to the respective sides of the second retrieval frame f2are set as the predetermined approach directions in relation to the determination points set on the respective sides. With respect to the determination points set on the vertices of the second retrieval frame f2, meanwhile, directions of diagonal lines extending from the respective vertices are set as the predetermined approach directions.

Likewise in this specific example, the approach determination processing is performed by the detection unit442on the basis of the contact states of the one object10relative to the other objects10and the housing container11, the contact states being acquired from the retrieval results acquired by the retrieval unit441. Note, however, that in this specific example, since the objects10have a columnar shape, the parts of the one object10that correspond to the determination points do not contact another object10or the housing container11. Nevertheless, as noted above, even when the one object10does not contact another object10or the housing container11, the finger portions of the hand mechanism2cannot be caused to approach the one object10in parts where the interval between the two components is too small. In this specific example, therefore, the approach determination processing is performed on the basis of the determination distance relating to each determination point.

FIG.27is a view illustrating specific examples of determination distances. Here, the determination distance will be described using the object10fshown inFIG.25as an example.FIG.27is a partially enlarged view of the second retrieval frame f2in which the object10fexists. Determination points P1, P2, P3are included in the enlarged part. As described above, the determination distance is the distance between a part of the one object10in the second retrieval frame f2that corresponds to a determination point set on the second retrieval frame f2and another object10or the housing container11in the predetermined approach direction set in relation to the determination point. Hence, the determination distance for the determination point P1shown inFIG.27is d1, which is the distance between the object10fand the side wall of the housing container11in the predetermined approach direction. Further, the determination distance for the determination point P2shown inFIG.27is d2, which is the distance between the object10fand the side wall of the housing container11in the predetermined approach direction. The determination distance for the determination point P3shown inFIG.27is d3, which is the distance between the object10fand the adjacent object10iin the predetermined approach direction. The determination distance d1for the determination point P1, which is set on a vertex of the second retrieval frame f2, is greater than the determination distances d2, d3for the determination points P2, P3, which are set on respective sides of the second retrieval frame f2. Accordingly, the determination distance d1exceeds a predetermined threshold. Moreover, the determination distance d2and the determination distance d3are equal to or smaller than the predetermined threshold. In the approach determination processing, therefore, it is determined that an approach vector is established in relation to the determination point P1but approach vectors are not established in relation to the determination points P2and P3.

FIG.28is a view showing some results of the approach determination processing performed by the detection unit442.FIG.28shows results of the approach determination processing performed in relation to the object10fshown inFIG.25. Note that similarly toFIG.18, inFIG.28, a black circle on an arrow indicating a predetermined approach direction indicates that an approach vector is established. Further, inFIG.28, a cross on an arrow indicating a predetermined approach direction indicates that an approach vector is not established. Furthermore, a solid-line cross indicates that an approach vector is not established due to the fact that the determination distance between the object10fin the retrieval frame f2and the housing container11is equal to or smaller than the predetermined threshold. Moreover, a dot-dash-line cross indicates that an approach vector is not established due to the fact that the determination distance between the object10fin the retrieval frame f2and another object10is equal to or smaller than the predetermined threshold.

As described above, the nine objects10ato10iare housed in the housing container11in rows so that adjacent objects10contact each other, and the objects10ato10dand10fto10idisposed in positions adjacent to the side walls of the housing container11also contact the housing container11. Hence, the only determination distances that exceed the predetermined threshold are the determination distances relating to the determination points set on the vertices of the second retrieval frame f2. Therefore, on the object10f, as shown inFIG.28, approach vectors are established only in relation to the determination points set on the vertices of the second retrieval frame f2. Meanwhile, approach vectors are not established in relation to the determination points set on the sides of the second retrieval frame f2.

By determining whether or not the finger portions21of the hand mechanism2can be caused to approach the parts corresponding to the respective determination points on the one object10existing within the second retrieval frame f2in relation to each of the plurality of determination points set on the second retrieval frame f2, as in the first to third specific examples described above, it is possible to detect the parts of the one object10that can be approached by the finger portions21of the hand mechanism2.

(Flow of Image Information Processing)

Next, a flow of the image information processing performed in the image information processing device44will be described on the basis of a flowchart shown inFIG.29. The flow of the image information processing is realized by executing a predetermined processing program in the image information processing device44. In this flow, first, in S101, image information is acquired from the imaging device8by the image information acquisition unit440.

Next, in S102, the retrieval unit441retrieves the housing container11and the object10from the image captured by the imaging device8. Here, as described above, the housing container11and the object10are retrieved on the basis of the shape information of the housing container11and the shape information of the object10, which are stored in the storage unit443. Further, when the image captured by the imaging device8includes a plurality of objects10, the retrieval unit441recognizes the plurality of objects10individually.

Next, in S103, the detection unit442executes the approach determination processing described above on each of the objects10retrieved by the retrieval unit441. As a result, an approach determination processing result is derived for each object10, as shown inFIGS.18,22, and28. The approachable parts of each object10can thus be detected.

Next, a flow of the hand control performed by the hand control device43when the object10is gripped by the hand mechanism2will be described on the basis of a flowchart shown inFIG.30. The flow of the hand control is realized by executing a predetermined processing program in the hand control device43.

In this flow, first, in S201, object information, which is information relating to the object10, is acquired from the image information processing device44. The object information acquired at this time includes not only the shape information of the object10but also the detection results indicating the approachable parts of the respective objects10housed in the housing container11(in other words, the results of the approach determination processing derived for each of the objects10in the image information processing described above), which are acquired by the detection unit442of the image information processing device44.

Next, in S202, the gripping style to be used by the hand mechanism2to grip one object10housed in the housing container11is determined on the basis of the object information acquired from the image information processing device44. Further, in S203, the parts of the one object10housed in the housing container11with which the finger portions21of the hand mechanism2are to be brought into contact when the one object10is gripped by the hand mechanism2are determined on the basis of the object information acquired from the image information processing device44.

For example, when the detection results indicating the approachable parts, acquired by the detection unit442of the image information processing device44, are the results illustrated in the first specific example of the image information processing, described above, it can be determined that the finger portions21of the hand mechanism2can be caused to approach the respective upper faces of the objects10b,10cincluded in the image shown inFIG.15. It can also be determined that by tilting the object10bor the object10cforward with the finger portions21of the hand mechanism2in contact with the upper face of the object10bor the object10c, tilt gripping can be performed thereon. In this case, therefore, tilt gripping is determined during the hand control as the gripping style to be used to grip the object10bor the object10cusing the hand mechanism2. Furthermore, the upper face of the object10bor the object10cis determined as the part of the object10bor the object10cto be contacted by the finger portions21of the hand mechanism2when tilt gripping is performed thereon.

Further, when the detection results indicating the approachable parts, acquired by the detection unit442of the image information processing device44, are the results illustrated in the second specific example of the image information processing, described above, for example, it can be determined that the finger portions21of the hand mechanism2can be caused to approach the respective rear side faces of the objects10b,10cincluded in the image shown inFIG.20. It can also be determined that by pressing the object10bor the object10cagainst another object10(i.e., the object10eor the object10f) placed in front thereof with the finger portions21of the hand mechanism2in contact with the rear side face of the object10bor the object10c, and while pressing the object10bor the object10c, shifting the position thereof upward, shift gripping can be performed thereon. In this case, therefore, shift gripping is determined during the hand control as the gripping style to be used to grip the object10bor the object10cusing the hand mechanism2. Furthermore, the rear side face of the object10bor the object10cis determined as the part of the object10bor the object10cto be contacted by the finger portions21of the hand mechanism2when shift gripping is performed thereon.

Furthermore, when the detection results indicating the approachable parts, acquired by the detection unit442of the image information processing device44, are the results illustrated in the third specific example of the image information processing, described above, for example, it can be determined that the finger portions21of the hand mechanism2can be caused to approach mutually opposing parts on the side faces of the object10fincluded in the image shown inFIG.24. Accordingly, it can be determined that direct gripping can be performed on the object10f. In this case, therefore, direct gripping is determined during the hand control as the gripping style to be used to grip the object10fusing the hand mechanism2. Furthermore, mutually opposing parts on the side faces of the object10fare determined as the parts of the object10fto be contacted by the finger portions21of the hand mechanism2when direct gripping is performed thereon.

Thus, by acquiring the detection results indicating the approachable parts, which are acquired by the detection unit442of the image information processing device44, during the hand control, it is possible to determine the gripping style and the part of the one object10housed in the housing container11to be contacted by the finger portions21of the hand mechanism2when the one object10is gripped by the hand mechanism2.

MODIFIED EXAMPLE

Next, a modified example of the hand control performed by the hand control device according to this embodiment will be described on the basis ofFIGS.31to33.FIG.31is a view showing the housing container11and two objects10a,10bhoused in the housing container11according to this modified example. In this modified example, the housing container11takes the shape of a box in which only the upper part is open. Further, the objects10a,10bhave a rectangular parallelepiped shape. Note thatFIG.31shows a case in which the housing container11and the two objects10a,10bhoused in the housing container11are seen from above.

Further,FIG.32shows the form of the hand mechanism2when the finger portions21of the hand mechanism2are caused to approach the object10ashown inFIG.31in order to grip the object10a. Hereafter, the form of the hand mechanism2shown inFIG.32may be referred to as a “first approach form”. Furthermore,FIG.33shows the form of the hand mechanism2when the finger portions21of the hand mechanism2are caused to approach the object10bshown inFIG.31in order to grip the object10b. Hereafter, the form of the hand mechanism2shown inFIG.33may be referred to as a “second approach form”. Note that inFIGS.32and33, for convenience, only the first finger portion21A and the third finger portion21C are shown, while the second finger portion21B and the fourth finger portion21D are omitted.

Furthermore, the distance between a tip end portion of the third finger portion31C (the tip end portion of the first finger link portion211C) and an outside portion of the first joint portion22C thereof when the hand mechanism2is set in the first approach form, shown inFIG.32, or the second approach form, shown inFIG.33, may be referred to hereafter as the “finger width”. Note that inFIG.32, the finger width in the first approach form is indicated by dfl. Further, inFIG.33, the finger width in the first approach form is indicated by df2.

In this modified example, as shown inFIG.31, the two objects10a,10bare both placed in the housing container11such that the left-side side faces S2and right-side side faces S3thereof are separated from the side walls of the housing container11. Therefore, in the image information processing performed in the image information processing device44, the left-side side faces S2and right-side side faces S3of the objects10a,10bare detected as approachable parts. As a result, in the hand control performed by the hand control device43, direct gripping is determined as the gripping style to be used by the hand mechanism2to grip the objects10a,10b. Furthermore, when direct gripping is performed on the objects10a,10b, the left-side side faces S2and right-side side faces S3thereof are determined as the parts thereof to be contacted by the finger portions21of the hand mechanism2.

Here, as shown inFIG.31, the object10ais placed in a substantially central part of the housing container11in a lateral direction of the housing container11. More specifically, a distance dLl from the left-side side face S2of the object10ato the side wall of the housing container11is substantially equal to a distance dR1from the right-side side face S3of the object10ato the side wall of the housing container11. Hence, a sufficient distance from the side wall of the housing container11is secured in relation to both the left-side side face S2and the right-side side face S3of the object10a. Therefore, the hand control device43determines the first approach form shown inFIG.32as the form to be taken by the hand mechanism2when the finger portions21of the hand mechanism2are caused to approach the object10a. When the object10ais gripped in the first approach form, as shown inFIG.32, the first finger portion21A, which is brought into contact with the left-side side face S2of the object10a, and the third finger portion21C, which is brought into contact with the right-side side face S3of the object10a, are opened into a substantially left-right symmetrical shape.

Note that when the first finger portion21A and the third finger portion21C of the hand mechanism2are caused to approach the object10a(or the object10b) so that the object10a(or the object10b) is sandwiched between and gripped by the first finger portion21A and the third finger portion21C of the hand mechanism2, an interval W2between the respective tip end portions of the first finger portion21A and the third finger portion21C (the tip end portion of the first finger link portion211A of the first finger portion21A and the tip end portion of the first finger link portion211C of the third finger portion21C) is set to be much wider than a width W1between the left-side side face S2and the right-side side face S3of the object10a(or the object10b).

The object10b, meanwhile, as shown inFIG.31, is placed in the housing container11in a position close to the right-side side wall of the housing container11. Accordingly, a distance dL2from the left-side side face S2of the object10bto the side wall of the housing container11is considerably greater than the distance dL1from the left-side side face S2of the object10ato the side wall of the housing container11. Moreover, a distance dR2from the right-side side face S3of the object10bto the side wall of the housing container11is considerably smaller than the distance dR1from the right-side side face S3of the object10ato the side wall of the housing container11.

Hence, the distance dR2from the right-side side face S3of the object10bto the side wall of the housing container11is smaller than the finger width df1in the first approach form, shown inFIG.32. Therefore, when the form of the hand mechanism2is set in the first approach form shown inFIG.32, the finger portions21of the hand mechanism2cannot be caused to approach the object10b.

Accordingly, the hand control device43determines the second approach form shown inFIG.33as the form to be taken by the hand mechanism2when the finger portions21of the hand mechanism2are caused to approach the object10b. When the object10bis gripped in the second approach form, as shown inFIG.33, the first finger portion21A, which is brought into contact with the left-side side face S2of the object10b, and the third finger portion21C, which is brought into contact with the right-side side face S3of the object10b, are opened into a left-right non-symmetrical shape. Further, in the second approach form, the finger width df2is smaller than the distance dR2from the right-side side face S3of the object10bto the side wall of the housing container11. Note, however, that likewise in the second approach form, the interval W2between the respective tip end portions of the first finger portion21A and the third finger portion21C is kept equal to the interval W2in the first approach form shown inFIG.32. By setting the form of the hand mechanism2in the second approach form in this manner, the finger portions21of the hand mechanism2can be caused to approach the object10b.

In this modified example, as described above, the hand control device43determines the form of the hand mechanism2to be taken when the finger portions21of the hand mechanism2are caused to approach the object10on the basis of the distance between the part of the gripping target object10to be contacted by the finger portions21of the hand mechanism2and the housing container11(or another object10). As a result, the finger portions21of the hand mechanism2can be caused to approach the object10in a form corresponding to the placement condition of the gripping target object10within the housing container11.

Note that the form of the hand mechanism2when the finger portions21of the hand mechanism2are caused to approach is not limited to the first approach form shown inFIG.32and the second approach form shown inFIG.33, and any forms that can be taken by the hand mechanism2may be used.

REFERENCE SIGNS LIST