Grasping device, control method, and program

A grasping device includes: a grasping part module including a first surface and a second surface and configured to grasp an object between the first surface and the second surface; an arm part configured to change a position of the grasping part module; an imaging unit provided at a position that moves together with the grasping part module and configured to capture an image of at least a part of the object; and a control unit configured to control, based on specified amount information indicating a contact state in a case where a specified amount of the object and the first surface are in contact with each other, and information indicating a contact state captured by the imaging unit, at least one of the grasping part module and the arm part such that an amount of the object that is grasped approaches the specified amount.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is the U.S. National Stage entry of International Application No. PCT/JP2019/049141 filed under the Patent Cooperation Treaty on Dec. 16, 2019, which is incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to a grasping device, a control method, and a program.

BACKGROUND ART

In the related art, there is known a technique for grasping a grasping object by controlling a robot hand based on image information captured by an imaging unit provided on an upper part of the grasping object (see, for example, PTL 1).

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Technical Problem

Here, there has been a request to grasp a specified amount of a flexible grasping object such as a prepared food by a robot hand and move the grasping object to a target location. However, when the robot hand according to the related art is used to grasp the specified amount of the flexible grasping object, there is a problem that an internal state of the flexible grasping object cannot be recognized from information captured by the imaging unit provided on the upper part of the grasping object.

That is, according to a method in the related art, there is a problem that it is difficult to estimate the specified amount of the flexible grasping object.

The invention has been made in view of such circumstances, and an object of the invention is to provide a grasping device capable of grasping a specified amount of a flexible grasping object, a control method, and a program.

Solution to Problem

A grasping device according to one aspect of the invention includes: a grasping part module including a first surface and a second surface and configured to grasp an object between the first surface and the second surface; an arm part configured to change a position of the grasping part module; an imaging unit provided at a position that moves together with the grasping part module and configured to capture an image of at least a part of the object; and a control unit configured to control, based on specified amount information indicating a contact state in a case where a specified amount of the object and the first surface are in contact with each other, and information indicating a contact state captured by the imaging unit when the grasping part module grasps at least a part of the object, at least one of the grasping part module and the arm part such that an amount of the object that is grasped approaches the specified amount.

In the grasping device according to one aspect of the invention, the imaging unit is provided on a back surface side of the first surface and at a position where an image of the object is able to be captured.

In the grasping device according to one aspect of the invention, the control unit is configured to control at least one of the grasping part module and the arm part based on the specified amount information and information indicating a contact state captured by the imaging unit when the grasping part module grasps at least a part of the object, the specified amount information being information indicating a ratio between a portion indicating a background and a portion indicating the object.

In the grasping device according to one aspect of the invention, the control unit is configured to control at least one of the grasping part module and the arm part based on the specified amount information and information indicating a contact state captured by the imaging unit when the grasping part module grasps at least a part of the object, the specified amount information being information indicating a ratio of a portion where luminance change of the object is large.

In the grasping device according to one aspect of the invention, the control unit is configured to control at least one of the grasping part module and the arm part based on the specified amount information and information indicating a contact state captured by the imaging unit when the grasping part module grasps at least a part of the object, the specified amount information being information indicating a color of the object.

A control method for a grasping device according to one aspect of the invention, the grasping device including: a grasping part module including a first surface and a second surface and configured to grasp an object between the first surface and the second surface; an arm part configured to change a position of the grasping part module; and an imaging unit provided at a position that moves together with the grasping part module and configured to capture an image of at least a part of the object, the control method includes: a control step of controlling, based on specified amount information indicating a contact state in a case where a specified amount of the object and the first surface are in contact with each other, and information indicating a contact state captured by the imaging unit when the grasping part module grasps at least a part of the object, at least one of the grasping part module and the arm part such that an amount of the object that is grasped approaches the specified amount.

A program according to an aspect of the invention is configured to cause a computer connected to a grasping device to execute a control step, in which the grasping device includes: a grasping part module including a first surface and a second surface and configured to grasp an object between the first surface and the second surface; an arm part configured to change a position of the grasping part module; and an imaging unit provided at a position that moves together with the grasping part module and configured to capture an image of at least a part of the object, and the control step is to control, based on specified amount information indicating a contact state in a case where a specified amount of the object and the first surface are in contact with each other, and information indicating a contact state captured by the imaging unit when the grasping part module grasps at least a part of the object, at least one of the grasping part module and the arm part such that an amount of the object that is grasped approaches the specified amount.

Advantageous Effect

According to the invention, it is possible to provide a grasping device capable of grasping a specified amount of a flexible grasping object, a control method, and a program.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of the invention will be described with reference to drawings.

FIG.1is a diagram illustrating an example of a configuration of a grasping device100according to an embodiment. A posture of the grasping device100is shown by a three-dimensional orthogonal coordinate system of an x axis, a y axis, and a z axis. The grasping device100according to the present embodiment detects a grasping state by coming into contact with a grasping object, and grasps a specified amount of the grasping object.

In the present embodiment, the grasping device100includes a grasping part module10, a grasping device control unit90(or a control unit, the same applies to the following), a distal end part110, an upper arm part120, a joint part130, a lower arm part140, a main horizontal shaft part150, a main vertical shaft part160, and a base part170. When the distal end part110, the upper arm part120, the joint part130, the lower arm part140, the main horizontal shaft part150, and the main vertical shaft part160are not distinguished from one another, each of these parts is referred to as an arm part180.

The base part170is a part connected to the main vertical shaft part160.

The main vertical shaft part160is a part connecting the main horizontal shaft part150and the base part170. The main vertical shaft part160is controlled by the grasping device control unit90, and displaces the main horizontal shaft part150about a shaft of the main vertical shaft part160.

The main horizontal shaft part150is a part connecting the lower arm part140and the main vertical shaft part160. The main horizontal shaft part150is controlled by the grasping device control unit90, and displaces the lower arm part140about a shaft of the main horizontal shaft part150.

The lower arm part140is a part connecting the joint part130and the main horizontal shaft part150.

The joint part130is a part connecting the upper arm part120and the lower arm part140. The joint part130is controlled by the grasping device control unit90, and displaces the upper arm part120about a shaft of the joint part130.

The upper arm part120is a part connecting the distal end part110and the joint part130.

The distal end part110is connected to the grasping part module10. A posture (for example, a position and a direction) of the distal end part110is controlled by the grasping device control unit90. A posture of the grasping part module10changes as the posture of the distal end part110changes.

The arm part180is not limited to this example as long as a position of the grasping part module10can be changed on any one of the x axis, the y axis, and the z axis. In this example, the arm part180has at least 6 degrees of freedom, and further has 7 degrees of freedom, including 1 degree of freedom, which is an interval between a grasping part1and a support part2included in the grasping part module10.

The grasping part module10detects a contact state of the grasping object, and outputs information indicating the detected contact state of the grasping object to the grasping device control unit90.

The grasping device control unit90acquires information output by the grasping part module10. The grasping device control unit90drives the arm part180to change the position of the grasping part module10. The grasping device control unit90performs control to change the position of the grasping part module10based on the information acquired from the grasping part module10.

FIG.2is a diagram illustrating an example of a case where the grasping device100according to the embodiment grasps a flexible grasping object21. (A) inFIG.2illustrates an example of a case where the grasping device100comes into contact with the flexible grasping object21and acquires information indicating a contact state, and (B) inFIG.2illustrates an example of a case where the grasping device100grasps a specified amount22that is a part of the flexible grasping object21. The posture of the grasping device100is shown by the three-dimensional orthogonal coordinate system of the x axis, the y axis, and the z axis.

In this example, the flexible grasping object21is accommodated in a container20.

The example of the case where the grasping device100comes into contact with the flexible grasping object21and acquires the information indicating the contact state will be described with reference to (A) inFIG.2.

The grasping device control unit90drives the arm part180to change the position of the grasping part module10with respect to the grasping object21. In the same drawing, the grasping part module10is in contact with the grasping object21. The grasping part module10acquires the information indicating the contact state between the grasping part module10and the grasping object21, and provides the information to the grasping device control unit90. The grasping device control unit90controls the arm part180based on the acquired information indicating the contact state between the grasping part module10and the grasping object21.

The example of the case where the grasping device100grasps the specified amount22, which is the part of the flexible grasping object21, will be described with reference to (B) inFIG.2.

The grasping device control unit90determines whether the grasping part module10grasps the specified amount22based on the information indicating the contact state between the grasping part module10and the grasping object21. The information indicating the specified amount22may be stored in the grasping device control unit90or may be stored in an external storage unit84.

The grasping device control unit90compares the information indicating the specified amount22with the information indicating the contact state between the grasping part module10and the grasping object21, and when the specified amount22is smaller, the grasping device control unit90controls at least one of the arm part180and the grasping part module10so as to perform an operation for grasping a smaller amount. Such a control method is not particularly limited. For example, an amount of the grasping object may be reduced by operating the grasping part module10and changing an opening and closing amount between the grasping part1and the support part2, or the amount of the grasping object may be reduced by raising the arm part180, and further the amount of the grasping object may be reduced by appropriately combining the operation of the grasping part module10and the operation of the arm part180. By performing such operation control, the information indicating the contact state between the grasping part module10and the grasping object21approaches the information indicating the specified amount22, so that the amount of the grasping object grasped by the grasping part module10is controlled to approach the specified amount22.

The grasping device control unit90compares the information indicating the specified amount22with the information indicating the contact state between the grasping part module10and the grasping object21, and when the specified amount22is larger, the grasping device control unit90controls the at least one of the arm part180and the grasping part module10so as to perform an operation for grasping a larger amount. Such a control method is not particularly limited. For example, the amount of the grasping object may be increased by operating the grasping part module10and changing the opening and closing amount between the grasping part1and the support part2, the amount of the grasping object may be increased by lowering the arm part180, or the amount of the grasping object may be increased by appropriately combining the operation of the grasping part module10and the operation of the arm part180. By performing such operation control, the information indicating the contact state between the grasping part module10and the grasping object21approaches the information indicating the specified amount22, so that the amount of the grasping object grasped by the grasping part module10is controlled to approach the specified amount22.

The grasping device control unit90compares the information indicating the specified amount22with the information indicating the contact state between the grasping part module10and the grasping object21, and when the information indicating the contact state between the grasping part module10and the grasping object21is within a range of the information indicating the specified amount22, the grasping device control unit90moves the specified amount22from the grasping object21to a target location.

For example, the grasping device100moves the specified amount22of the grasping object21accommodated in the container20to the target location located outside the container20.

FIG.3is a diagram illustrating an example of the grasping part module10according to the embodiment. A configuration of the grasping part module10will be described with reference to the same drawing. The posture of the grasping part module10is shown by the three-dimensional orthogonal coordinate system of the x axis, the y axis, and the z axis.

The grasping part module10includes a connection part11, the grasping part1, and the support part2.

The connection part11is a part connecting the distal end part110, the grasping part1, and the support part2.

The grasping part1is connected to the connection part11. The grasping part1includes a contact surface41that comes into contact with at least a part of the grasping object21. Hereinafter, the contact surface41is also referred to as a first surface.

The support part2is connected to the connection part11. The support part2includes a support surface42that comes into contact with at least a part of the grasping object21. Hereinafter, the support surface42is also referred to as a second surface.

In this example, the grasping part1and the support part2are disposed in positions where the contact surface41and the support surface42face each other. The connection part11includes a driving device (not shown), and displaces at least the grasping part1in a y axis direction based on a grasping indication from the grasping device control unit90. The grasping part module10grasps the grasping object21between the grasping part1and the support part2by driving at least the grasping part1in the y axis direction.

In this example, the connection part11is described as displacing the grasping part1in the y axis direction, but the support part2may be similarly configured to be displaced in the y axis direction.

In this example, the grasping part module10is described as grasping the grasping object21by the grasping part1and the support part2, but the configuration of the grasping part module10is not limited to this example. For example, the grasping part module10may be implemented by a plurality of grasping parts1, a plurality of support parts2, or a combination of the plurality of grasping parts1and the plurality of support parts2. In this case, the support surface42(second surface) is provided at a position where normal lines of the support surface42and the contact surface41(first surface) intersect with each other. For example, the support surface42(second surface) is provided at a position facing the contact surface41(first surface).

The grasping part1grasps at least a part of a predetermined amount of the grasping object21by the contact surface41(first surface) and the support surface42(second surface) of the support part2.

FIG.4is a diagram illustrating examples of a cross-sectional view of the grasping part1in the embodiment.

FIG.4shows cross-sectional views on an x-y plane of the grasping part1inFIG.3. The direction of the grasping part module10is indicated by the three-dimensional orthogonal coordinate system of the x axis, the y axis, and the z axis.

(A) inFIG.4is a diagram illustrating an example of a cross-sectional view of a grasping part1A in the embodiment. (B) inFIG.4is a diagram illustrating an example of a cross-sectional view of a grasping part1B which is a modification of the grasping part1in the embodiment.

In a case where the support part2is configured to be drivable, the support part2may be configured to have a configuration the same as that of the grasping part1.

The example of the configuration of the grasping part1A will be described with reference to (A) inFIG.4. In this example, the grasping part1A detects the contact state between the grasping object21and the contact surface41by capturing an image of the grasping object21.

The grasping part1A in this example is an example of the grasping part1. The grasping part1includes an imaging unit30, a contact state detection unit43, a hard layer70, and a frame50.

The frame50holds the imaging unit30, the contact state detection unit43, and the hard layer70.

The contact state detection unit43includes the contact surface41. The contact surface41is a surface that comes into contact with the grasping object21. The contact state detection unit43detects the contact state between the grasping object21in contact with the contact surface41and the contact surface41.

For example, when the contact state between the grasping object21in contact with the contact surface41and the contact surface41is detected by the image, the contact state detection unit43may be formed of a transparent material that transmits light.

The hard layer70is provided at a position in contact with the contact state detection unit43. For example, the hard layer70may be made of a transparent and hard material such as acrylic.

Although the contact state detection unit43and the hard layer70are described as separate constituent elements in the present embodiment, the hard layer70may not be provided by implementing the contact state detection unit43and the hard layer70with the same member.

The imaging unit30is disposed on a back side of the contact surface41that can be in contact with the grasping object21, of two surfaces of front and back surfaces of the contact state detection unit43. Here, regarding the contact state detection unit, the contact surface41is a front surface, a side of the contact surface41is a front surface side, an opposite surface of the contact surface41is a back surface, and an opposite side of the contact surface41is a back surface side. More specifically, the imaging unit30is provided at a position where an image of the grasping object21can be taken from the back surface of the contact surface41(first surface) in contact with the grasping object21.

In this example, the imaging unit30will be described as being provided inside the grasping part1, but the position where the imaging unit30is provided is not limited to this example. The imaging unit30may be provided at a position that moves together with the grasping part module10, which is at a distal end of the arm part180. The imaging unit30may be provided at a position where an image indicating the contact state between at least a part of the grasping object21and the contact surface41(first surface) is captured.

The imaging unit30captures an image of the grasping object21in contact with the contact surface41, and outputs information of the captured image to the grasping device control unit90.

The example of the configuration of the grasping part1B will be described with reference to (B) inFIG.4. In this example, the grasping part1B detects the contact state between the grasping object21and a contact surface41B by displacement of a marker45.

The grasping part1B in this example is an example of the grasping part1. The same configurations as those of the grasping part1A are designated by the same reference numerals as those in (A) inFIG.4, and the description thereof will be omitted.

The grasping part1B is different from the grasping part1A in that the marker45is provided in a contact state detection unit43B. The contact state detection unit43B includes the contact surface41B. The contact state detection unit43B is a modification of the contact state detection unit43, and the contact surface41B is a modification of the contact surface41.

A plurality of markers45are disposed at predetermined positions of the contact state detection unit43B. In this example, the marker45is an opaque member disposed at positions of grid points separated at equal intervals inside the contact state detection unit43B. The marker45is disposed inside the contact state detection unit43B, but the invention is not limited thereto, and the marker45may be provided on the contact surface41B or may be provided on the back surface of the contact surface41B. In addition, the marker45is described as being discretely disposed at the positions of the grid points, but the invention is not limited thereto. The marker45may be a grid pattern or other continuous patterns.

In this example, the contact state detection unit43B is at least partially deformed along a shape of the grasping object21in contact with a transmission part contact surface40that is the contact surface. For example, when the contact state detection unit43B is made of a transparent material, a specific material for the contact state detection unit43B is a silicone material having a thickness of 2 millimeters and a transmittance of about 94 percent.

The position of the marker45is also displaced by the contact state detection unit43B being at least partially deformed along the shape of the grasping object21. The grasping device control unit90detects the contact state between the grasping object21and the contact surface41B from the displacement of the marker45captured by the imaging unit30.

The pattern of the marker45may be irregular patterns such that the contact state between the grasping object21and the contact surface41B can be easily detected. The marker45is described as the opaque members, but the invention is not limited thereto, and the marker45may be a translucent member or a transparent member as long as the displacement when the grasping object and the contact surface are in contact with each other can be optically recognized.

The contact state detection unit43is not limited to the example illustrated in the drawing. The contact state detection unit43only needs to be able to detect the contact state of the grasping object21, and for example, may be configured to detect the deformation of the contact state detection unit43by refraction of light. In this case, the contact state detection unit43may be configured not to transmit light from the contact surface41to the back surface of the contact surface41.

FIG.5is a diagram illustrating an example of a functional configuration of the grasping device100according to the embodiment.

The grasping device100includes, as functional configurations, the grasping device control unit90, an input unit92, an output unit93, the imaging unit30, the arm part180, and the grasping part1.

The grasping device control unit90includes a microcomputer (not shown), a memory such as a random access memory (RAM) and a read only memory (ROM), a communication unit that communicates with an external device, and the like.

The input unit92acquires information from sensors such as a pressure sensor, a position sensor, a temperature sensor, and an acceleration sensor, a camera, and a microphone (all not shown). When the information that is output by the input unit92and acquired by the grasping device control unit90is not distinguished, the information is described as input information IIN.

The output unit93outputs a drive signal to a motor (not shown) or the like for driving a robot. When the information that is output to the output unit93by the grasping device control unit90is not distinguished, the information is described as output information IOUT.

The grasping device control unit90outputs the output information IOUTto the output unit93based on the input information IN acquired from the input unit92. Hereinafter, the grasping device control unit90will also be referred to as a drive control unit.

The imaging unit30outputs the information of the captured image as image information IIMGto the grasping device control unit90.

The grasping device control unit90outputs drive information IDRVincluding information for driving the arm part180and the grasping part1connected to the distal end of the arm part180based on the acquired image information IIMG. Specifically, the grasping device control unit90drives, based on specified amount information ITGTindicating the contact state in the case where the specified amount of the grasping object21and the contact surface41(the first surface) are in contact with each other and the information indicating the contact state captured by the imaging unit30when the grasping part module10grasps at least a part of the grasping object21, at least one of the grasping part module10and the arm part180such that the amount of the grasping object21that is grasped approaches the information indicated in the specified amount information ITGT. More specifically, the grasping device control unit90compares the information indicated in the specified amount information ITGTwith the information indicated in the image information IIMG, outputs the drive information IDRVbased on a comparison result, and drives at least one of the grasping part module10and the arm part180.

Here, the specified amount information ITGTmay be stored in the storage unit84. Specifically, the specified amount information ITGTis the information indicating the contact state in the case where the specified amount of the grasping object21and the contact surface41(first surface) are in contact with each other. The specified amount is an amount to be grasped by the grasping device100. The specified amount may be a weight or a volume.

The storage unit84may be provided in the grasping device100.

When the grasping device100learns that there is an error between the specified amount information ITGTand an actual specified amount as a result of a plurality of times of grasping, the grasping device100may be configured to update the specified amount information ITGTstored in the storage unit84by outputting specified amount update information IFBto the storage unit84.

FIG.6is a diagram illustrating operations of the grasping part1with respect to the grasping object21according to the embodiment.

(A) inFIG.6is a cross-sectional view of the grasping device100in an x-z plane in the case where the grasping part1and the grasping object21are in contact with each other. (B) inFIG.6is a diagram illustrating an example of an image I10captured by the imaging unit30provided in the grasping part1. (C) inFIG.6is a diagram illustrating an example of an image I20captured by the imaging unit30of the support part2in a case where the support part2includes the imaging unit30.

The operations of the grasping part1with respect to the grasping object21will be described with reference to (A) inFIG.6.

The grasping device control unit90controls displacement of the arm part180in an x axis direction, the y axis direction, and a z axis direction, and rotatable operation about respective axes. In addition, the grasping device control unit90controls a distance L1 between the contact surface41of the grasping part1and the support surface42of the support part2. That is, the grasping part1has seven degrees of freedom in which one degree of freedom is added to six degrees of freedom.

The grasping device control unit90adjusts a grasping amount to be the specified amount by changing the position of the grasping part1with respect to the grasping object21.

The imaging unit30provided in the grasping part1captures the image of the contact surface41.

(B) inFIG.6is the example of the image I10of the grasping object21captured by the imaging unit30provided in the grasping part1. The image I10is represented by a two-dimensional orthogonal coordinate system of a ycaxis and a zcaxis. A yc-zcplane indicates an image plane on which an image of a y-z plane in (A) inFIG.6is captured.

In the image I10, a portion occupied by the grasping object21is referred to as an object image I11. In the image I10, a portion occupied by a background is referred to as a background image112. The background is a portion where the grasping object21is not captured.

The support part2may have a configuration the same as the grasping part1. In this example, the case where the support part2has the configuration the same as the grasping part1will be described. When the support part2has the configuration the same as the grasping part1, the support part2includes the imaging unit30.

(C) inFIG.6illustrates the image120of the grasping object21captured by the imaging unit30provided in the support part2. The image120is represented by the two-dimensional orthogonal coordinate system of the ycaxis and the zcaxis. The yc-zcplane indicates the image plane on which the image of the y-z plane in (A) inFIG.6is captured.

In the image120, a portion occupied by the grasping object21is referred to as an object image121. In the image120, a portion occupied by the background is referred to as a background image122. The background is the portion where the grasping object21is not captured.

FIG.7is a diagram illustrating an example of a series of operations of the grasping device100according to the embodiment. The series of operations of the grasping device100will be described with reference to the drawing.

(Step S10) The grasping device control unit90acquires, from the storage unit84, the specified amount information ITGTthat is the information indicating the specified amount to be grasped. The storage unit84in which the specified amount information ITGTis stored may be provided in the grasping device100or may not be provided in the grasping device100. When the storage unit84is not provided in the grasping device100, the grasping device control unit90communicates with the storage unit84using a communication unit (not shown) to acquire the specified amount information ITGT. After acquiring the specified amount information ITGT, the grasping device control unit90advances the processing to step S11.

(Step S11) The grasping device control unit90drives the arm part180provided in the grasping part1to bring the grasping part1close to the grasping object21. The grasping device control unit90advances the processing to step312.

(Step S12) The imaging unit30provided in the grasping part1captures an image of the grasping object21. The grasping device control unit90acquires the image information IIMGfrom the imaging unit30. The grasping device control unit90advances the processing to step313.

(Step S13) The grasping device control unit90compares the specified amount information ITGTacquired from the storage unit84with the information indicating the contact state indicated by the image information IIMGacquired from the grasping part1. Here, the information indicating the contact state may be, for example, a ratio between the image of the object and the image of the background indicated by the image information IIMG. When the image information IIMGis the image I10, the information indicating the contact state may be a ratio of an area of the object image I11to an area of the background image112.

When the information indicating the contact state indicated by the image information IIMGis within a range indicated by the specified amount information ITGT(that is, step S13; YES), the processing proceeds to step S19. When the information indicating the contact state indicated by the image information IIMG; is not within the range indicated by the specified amount information ITGT(that is, step S13; NO), the processing proceeds to step S14.

(Step S14) The grasping device control unit90extracts a difference between the information indicating the contact state indicated by the image information IIMGacquired from the grasping part1and the information indicated in the specified amount information ITGT. When the information indicating the contact state indicated by the image information IIMGacquired from the grasping part1is smaller than the information indicated in the specified amount information ITGT(that is, step S14; YES), the processing proceeds to step S16. When the information indicating the contact state indicated by the image information IIMGacquired from the grasping part1is larger than the information indicated in the specified amount information ITGT(that is, step S14; NO), the processing proceeds to step318.

(Step S16) The grasping device control unit90drives and controls at least one of the arm part180and the grasping part module10to further drive the at least one of the arm part180and the grasping part module10at a position where the difference is further acquired. That is, an operation of picking up (grasping) a larger amount of the grasping object is performed. Specifically, for the arm part180, for example, when the arm part180is shifted downward, a contact area between the grasping part module10and the grasping object increases, and a grasping amount is increased. For the grasping part module10, for example, when an interval between the grasping part1and the support part2is narrower than that in a state in which the grasping part module10is inserted into a mountain of the grasping object, the grasping amount is increased. Further, both the arm part180and the grasping part module10may perform the respective operations described above to grasp a larger amount of the grasping object. The grasping device control unit90drives and controls the arm part180and the grasping part module10as described above, and then advances the processing to step S12.

(Step S18) The grasping device control unit90drives and controls at least one of the arm part180and the grasping part module10to drive the at least one of the arm part180and the grasping part module10at a position where the difference is discarded. That is, an operation of picking up (grasping) a smaller amount of the grasping object is performed. Specifically, for the arm part180, for example, when the arm part180is shifted upward, the contact area between the grasping part module10and the grasping object decreases, and the grasping amount is reduced. For the grasping part module10, for example, when the interval between the grasping part1and the support part2is widened to return to the state in which the grasping part module10is inserted into the mountain of the grasping object, the grasping amount is reduced. Further, both the arm part180and the grasping part module10may perform the respective operations described above to grasp a smaller amount of the grasping object. The grasping device control unit90drives and controls the arm part180and the grasping part module10as described above, and then advances the processing to step S12.

(Step S19) The grasping device control unit90moves the grasped grasping object21to a predetermined position while maintaining the grasping state of the grasping part1. The maintaining of the grasping state means, for example, maintaining the distance L1 between the contact surface41of the grasping part1and the support surface42of the support part2.

Example of Control Based on Ratio Between Object and Background

FIG.8is a diagram illustrating an example of control based on the ratio between the object and the background according to the embodiment.

Here, as an example of a comparison between the specified amount information ITGTand the information indicating the contact state indicated by the image information IIMG, the grasping device control unit90may perform the control based on the ratio between the object and the background. The example of the control based on the ratio between the object and the background will be described with reference to the drawing.

In the same drawing, a horizontal axis represents the “ratio between the object and the background”, a vertical axis represents the “operation” of the arm part180, and a relationship is represented by a relational expression W1.

The grasping device control unit90calculates the ratio between the object and the background indicated in the acquired image. In this example, the “ratio between the object and the background” will be described as the ratio of the object to the ratio of the background. When a calculated value is between a reference sign P13and a reference sign P14, the grasping device control unit90determines the operation. When the calculated value is smaller than the reference sign P13, the grasping device control unit90drives the arm part180to perform the operation of picking up a larger amount of the grasping object21. When the calculated value is larger than the reference sign P14, the grasping device control unit90drives the arm part180to perform the operation of discarding the grasping object21.

That is, in this example, the grasping device control unit90controls the arm part180based on the information indicating the ratio between the portion indicating the background and the portion indicating the grasping object, which is the information indicated in the specified amount information, and the image indicating the contact state captured by the imaging unit30.

In this example, the relational expression W1is described as a linear function, but the relational expression W1is not limited to this example. The relational expression W1may be obtained by a regression model such as a neural network or a support vector machine.

Example of Control Based on Ratio of Edge

FIG.9is a diagram illustrating an example of control based on a ratio of an edge in the embodiment.

Here, as the example of the comparison between the specified amount information ITGTand the information indicating the contact state indicated by the image information IIMG, the grasping device control unit90may perform the control based on the ratio of the edge indicated by the image information IIMG.

Specifically, a weight indicated by the specified amount information ITGTand the ratio of the edge indicated by the image information IIMGmay have a positive correlation. For example, in a case where a focal position of the imaging unit30is adjusted to the contact surface41, the image of the grasping object21in the image information IIMGis more clearly captured as the grasping object21approaches (comes into close contact with) the contact surface41, and therefore, the ratio of the edge indicated by the image information IIMGis increased. On the other hand, as the grasping object21moves away from the contact surface41(a degree of close contact decreases), an amount of blur increases, and therefore, the ratio of the edge indicated by the image information IIMG, decreases. That is, there is a correlation between the ratio of the edge indicated by the image information IIMGand the distance between the grasping object21and the contact surface41. Since the weight is larger as the larger amount of the grasping object21is in close contact with the contact surface41, the weight indicated in the specified amount information ITGTand the ratio of the edge indicated in the image information IIMGmay have a positive correlation.

The example of the control based on the ratio of the edge indicated by the image information IIMGwill be described with reference to the drawing.

In the same drawing, the horizontal axis represents the “ratio of the edge”, the vertical axis represents the “operation” of the arm part180, and a relationship is represented by a relational expression W2.

The grasping device control unit90calculates the edge of the acquired image. For example, the grasping device control unit90calculates the edge of the image indicated by the image information IIMGby an image processing such as applying a differential filter. In this example, the edge means a portion where luminance change of the image indicated by the image information IIMGis large. When the calculated value is between a reference sign P23and a reference sign P24, the grasping device control unit90determines the operation. When the calculated value is smaller than the reference sign P23, the grasping device control unit90drives the arm part180to perform the operation of picking up a larger amount of the grasping object21. When the calculated value is larger than the reference sign P24, the grasping device control unit90drives the arm part180to perform the operation of discarding the grasping object21.

That is, in this example, the grasping device control unit90controls the arm part180based on the information indicating the ratio of the portion where the luminance change of the grasping object21is large, which is the information indicated in the specified amount information, and the image indicating the contact state captured by the imaging unit30.

In this example, the relational expression W2is described as a linear function, but the relational expression W2is not limited to this example. The relational expression W2may be obtained by a regression model such as a neural network or a support vector machine.

Example of Control Based on Color of Image

FIG.10is a diagram illustrating an example of control based on a color of the image according to the embodiment.

Here, as the example of the comparison between the specified amount information ITGTand the information indicating the contact state indicated by the image information IIMG, the grasping device control unit90may perform the control based on the color of the image indicated by the image information IIMG. The example of the control based on the color of the image indicated by the image information IIMGwill be described with reference to the drawing.

In the same drawing, the horizontal axis represents “R”, the vertical axis represents the “operation” of the arm part180, and a relationship is represented by a relational expression W3. In this example, a case where the color is R (red) will be described, but a relational expression based on three axes of RGB may be used.

The grasping device control unit90calculates the color of the acquired image. For example, the grasping device control unit90calculates an average value of colors of the image indicated by the image information IIMG. In this example, the color refers to an average color of the image indicated by the image information IIMG, but the color may be the color of a portion occupied by the object, and the embodiment may be configured to perform the operation based on the color of each point of the image information IIMG.

When the calculated value is between a reference sign P33and a reference sign P34, the grasping device control unit90determines the operation. When the calculated value is smaller than the reference sign P33, the grasping device control unit90drives the arm part180to perform the operation of picking up a larger amount of the grasping object21. When the calculated value is larger than the reference sign P34, the grasping device control unit90drives the arm part180to perform the operation of discarding the grasping object21.

That is, in this example, the grasping device control unit90controls the arm part180based on the information indicating the color of the grasping object21, which is the information indicated by the specified amount information, and the image indicating the contact state captured by the imaging unit30.

In this example, the relational expression W3is described as a linear function, but the relational expression W3is not limited to this example. The relational expression W3may be obtained by a regression model such as a neural network or a support vector machine.

Summary of Effects of Embodiment

As described above, the grasping device100of the present embodiment includes the grasping part1and the arm part180, so as to grasp the specified amount of the grasping object21. The grasping device control unit90controls the arm part180based on the image acquired by the imaging unit30provided in the grasping part1. The grasping device control unit90compares the information acquired from the storage unit84with the information indicated in the image acquired by the imaging unit30, and controls the arm part180.

Therefore, the grasping device100can grasp the specified amount by controlling the arm part180based on the image acquired by the imaging unit30provided in the grasping part1.

Further, the grasping device100of the present embodiment includes the imaging unit30at a position where the image of the grasping object21can be captured from the back surface of the contact surface41of the grasping part1.

For example, compared to a case where a camera is provided on a ceiling as in the related art, the grasping device100of the present embodiment can acquire more information on the grasping object21.

Therefore, according to the present embodiment, the grasping device100can more accurately grasp the specified amount from the grasping object21.

In addition, the grasping device100of the present embodiment controls the arm part180based on the information indicating the ratio between the portion indicating the background and the portion indicating the grasping object.

Therefore, the grasping device control unit90can estimate a amount grasped by the grasping part1, and can grasp the specified amount.

In addition, the grasping device100of the present embodiment controls the arm part180based on the ratio of the portion where the luminance change of the grasping object21is large.

Here, the grasping object21grasped by the grasping device100of the present embodiment may be soft. Weights may be different between a case where the grasping object21is soft and a case where the grasping object21is hard.

Therefore, according to the grasping device100of the present embodiment, the specified amount can be estimated according to the state of the grasping object21(for example, softness of the grasping object21) by controlling the arm part180based on the ratio of the portion where the luminance change of the grasping object21is large.

In addition, the grasping device100of the present embodiment controls the arm part180based on the color of the grasping object21.

Here, the weight of the grasping object21grasped by the grasping device100of the present embodiment may be different according to the color. For example, a cabbage of yellow-green may be light, and a cabbage of deep green may be heavy.

Therefore, according to the grasping device100of the present embodiment, the specified amount can be more accurately estimated by controlling the arm part180based on the color of the grasping object21.

Some or all of the functions of the respective units provided in the grasping device100in the above-described embodiment may be implemented by recording a program for implementing these functions in a computer-readable recording medium and causing a computer system to read and execute the program recorded in the recording medium. Here, the “computer system” includes hardware such as an OS or a peripheral device.

Further, “computer-readable recording medium” refers to a storage unit such as a portable medium such as a magneto-optical disk, a ROM, and a CD-ROM, and a hard disk built in the computer system. Further, the “computer-readable recording medium” may also include a recording medium that retains a program dynamically in a short time, such as a communication line for transmitting a program via a network such as the Internet, or a recording medium that retains a program for a predetermined time, such as a volatile memory in the computer system serving as a server or a client in this case. Further, the program may be a program for implementing some of the functions described above, or may be a program capable of implementing the above-described functions in combination with the program already recorded in the computer system.

Although the embodiments for carrying out the invention have been described above using the embodiments, the invention is not limited to these embodiments, and various modifications and substitutions can be made without departing from the gist of the invention.

REFERENCE SIGN LIST