ROBOT SYSTEM, ROBOT ARM, END EFFECTOR, AND ADAPTER

A robot system including a robot arm with a movable portion includes a first imaging device and a second imaging device attached to the robot arm, a control unit that controls the robot system, a distance information acquisition unit that acquires information on a distance to a target object, the control unit is capable of changing a baseline length that is a distance between the first imaging device and the second imaging device, and the distance information acquisition unit acquires the information on the distance to the target object on the basis of the baseline length.

TECHNICAL FIELD

The present invention relates to a robot system, robot arm, end effector, and adapter.

BACKGROUND ART

A robot system including an imaging device is known. For example, Patent Literature 1 describes a configuration in which an imaging device is attached to a robot arm.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: a first imaging device and a second imaging device attached to the robot arm; a control unit configured to control the robot system; and a distance information acquisition unit configured to acquire information on a distance to a target object, wherein the control unit is capable of changing a baseline length, the baseline length being a distance between the first imaging device and the second imaging device, and the distance information acquisition unit acquires the information on the distance to the target object on the basis of the baseline length.

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: a first imaging device and a second imaging device attached to the robot arm, wherein at least one of the first imaging device and the second imaging device is movable with respect to the robot arm. An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: an end effector attached to the robot arm; and a first imaging device and a second imaging device attached to the end effector, wherein at least one of the first imaging device and the second imaging device is movable with respect to the end effector.

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: an adapter for attaching an end effector to the robot arm; and a first imaging device and a second imaging device attached to the adapter, wherein at least one of the first imaging device and the second imaging device is movable with respect to the adapter.

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: a first imaging device and a second imaging device attached to the robot arm, wherein relative positions of the first imaging device and the second imaging device are variable.

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: an end effector attached to the robot arm; and a first imaging device and a second imaging device attached to the end effector, wherein relative positions of the first imaging device and the second imaging device are variable.

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: an adapter for attaching an end effector to the robot arm; and a first imaging device and a second imaging device attached to the adapter, wherein relative positions of the first imaging device and the second imaging device are variable.

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: a first imaging device configured to be movable with respect to the robot arm; and a distance information acquisition unit configured to acquire information on a distance to a target object on the basis of an image captured by the first imaging device, wherein the first imaging device captures a first image of the target object at a first position, and captures a second image of the target object at a second position different from the first position, and the distance information acquisition unit acquires the information on the distance to the target object on the basis of the first image and the second image.

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: an end effector attached to the robot arm; a first imaging device configured to be movable with respect to the end effector; and a distance information acquisition unit configured to acquire information on a distance to a target object on the basis of an image captured by the first imaging device, wherein the first imaging device captures a first image of the target object at a first position, and captures a second image of the target object at a second position different from the first position, and the distance information acquisition unit acquires the information on the distance to the target object on the basis of the first image and the second image.

An aspect of a robot system of the present invention is a robot system including a robot arm with a movable portion, the robot system including: an adapter for attaching the end effector to the robot arm; and a first imaging device configured to be movable with respect to the adapter; and a distance information acquisition unit configured to acquire information on a distance to a target object on the basis of an image captured by the first imaging device, wherein the first imaging device captures a first image of the target object at a first position, and captures a second image of the target object at a second position different from the first position, and the distance information acquisition unit acquires the information on the distance to the target object on the basis of the first image and the second image.

An aspect of a robot system of the present invention includes a robot arm; three or more imaging devices configured to image a target object; and a control unit configured to acquire information on a distance to the target object on the basis of information of images of the target object acquired by two of the three or more imaging devices.

An aspect of a robot system of the present invention includes a robot arm; three or more imaging devices configured to image a target object; and a control unit configured to control at least one of the robot arm and an end effector connected to the robot arm on the basis of information on images acquired by two of the three or more imaging devices.

An aspect of a robot system of the present invention includes a robot arm; and three or more imaging devices, wherein the three or more imaging devices are disposed around any one of the robot arm, an end effector connected to the robot arm, and an adapter for attaching the end effector.

An aspect of a robot arm of the present invention includes a first holding portion configured to hold a first imaging device; and a second holding portion configured to hold a second imaging device, wherein the first imaging device is held to be movable by the first holding portion, or the second imaging device is held to be movable by the second holding portion.

An aspect of an end effector of the present invention is an end effector attached to a robot arm, the end effector including: a first holding portion configured to hold a first imaging device; and a second holding portion configured to hold a second imaging device, wherein the first imaging device is held to be movable by the first holding portion, or the second imaging device is held to be movable by the second holding portion.

An aspect of an adapter of the present invention is an adapter for attaching an end effector to a robot arm, the adapter including: a first holding portion configured to hold a first imaging device; and a second holding portion configured to hold a second imaging device, wherein the first imaging device is held to be movable by the first holding portion, or the second imaging device is held to be movable by the second holding portion.

An aspect of a robot arm of the present invention includes: a first holding portion configured to hold a first imaging device; and a second holding portion configured to hold a second imaging device, wherein relative positions of the first imaging device and the second imaging device are variable.

An aspect of an end effector of the present invention is an end effector attached to a robot arm, the end effector including: a first holding portion configured to hold a first imaging device; and a second holding portion configured to hold a second imaging device, wherein relative positions of the first imaging device and the second imaging device are variable.

An aspect of an adapter of the present invention is an adapter for attaching an end effector to a robot arm, the adapter including: a first holding portion configured to hold a first imaging device; and a second holding portion configured to hold a second imaging device, wherein relative positions of the first imaging device and the second imaging device are variable.

An aspect of a robot arm of the present invention includes: a holding portion configured to hold three or more imaging devices configured to image a target object; and a control unit configured to acquire information on a distance of the target object on the basis of information of images of the target object acquired by two of the three or more imaging devices.

DESCRIPTION OF EMBODIMENTS

Hereinafter, a robot system, a robot arm, an end effector, and an adapter according to embodiments of the present invention will be described with reference to the drawings. A scope of the present invention is not limited to the following embodiments, and can be arbitrarily changed within the scope of the technical idea of the present invention. Further, in following the drawings, a scale, numbers, and the like of each structure may be different from a scale, numbers, and the like of an actual structure in order to make each configuration easier to understand.

First Embodiment

FIG.1is a perspective view illustrating a robot system10of the present embodiment.FIG.2is a block diagram illustrating a portion of a configuration of the robot system10of the present embodiment.

As illustrated inFIG.1, the robot system10includes a robot20and an imaging device30, a control unit40, and a display unit50. The robot20performs, for example, work on a target object W on a workbench WB.

The robot20includes a robot arm21, an end effector22, and an adapter23. The robot arm21includes an arm portion24as a movable portion. In the present embodiment, a plurality of arm portions24are provided. The robot arm21is, for example, a multi-joint arm configured by connecting the plurality of arm portions24. The arm portion24includes, for example, five arm portions including a first arm portion24a, a second arm portion24b, a third arm portion24c, a fourth arm portion24d, and a fifth arm portion24e. The first arm portion24a, the second arm portion24b, the third arm portion24c, the fourth arm portion24d, and the fifth arm portion24eare connected in this order from an installation surface of the robot arm21.

As illustrated inFIG.2, the robot arm21includes an arm drive unit25and an arm position acquisition unit26. The arm drive unit25is, for example, a servomotor. The arm drive unit25is provided to each arm portion24, for example. That is, five arm drive units25are provided, for example.

The arm drive unit25provided on the first arm portion24adisplaces the first arm portion24awith an installation surface of the robot20as a reference. The arm drive unit25provided on the second arm portion24bdisplaces the second arm portion24bwith the first arm portion24aas a reference. The arm drive unit25provided on the third arm portion24cdisplaces the third arm portion24cwith the second arm portion24bas a reference. The arm drive unit25provided on the fourth arm portion24ddisplaces the fourth arm portion24dwith the third arm portion24cas a reference. The arm drive unit25provided in the fifth arm portion24edisplaces the fifth arm portion24ewith the fourth arm portion24das a reference. Each arm drive unit25rotates one arm portion24, for example.

The arm position acquisition unit26includes, for example, a rotary encoder (not illustrated). The arm position acquisition unit26is provided to each arm portion24, for example. That is, five arm position acquisition units26are provided, for example. The arm position acquisition unit26provided in the first arm portion24acan detect an amount of displacement of the first arm portion24awith the installation surface of the robot20as a reference. The arm position acquisition unit26provided in the second arm portion24bcan detect an amount of displacement of the second arm portion24bwith the first arm portion24aas a reference. The arm position acquisition unit26provided in the third arm portion24ccan detect an amount of displacement of the third arm portion24cwith the second arm portion24bas a reference. The arm position acquisition unit26provided in the fourth arm portion24dcan detect an amount of displacement of the fourth arm portion24dwith the third arm portion24cas a reference. The arm position acquisition unit26provided in the fifth arm portion24ecan detect an amount of displacement of the fifth arm portion24ewith the fourth arm portion24das a reference. The amount of displacement of each arm portion24that can be detected by each arm position acquisition unit26includes, for example, a rotation angle of each arm portion24detected by the rotary encoder (not illustrated).

FIG.3is a perspective view illustrating a portion of the robot arm21, the end effector22, the adapter23, and the imaging device30of the present embodiment.FIG.4is a plan view illustrating the portion of the robot arm21, the end effector22, the adapter23, and the imaging device30of the present embodiment.

The end effector22is attached to the robot arm21as illustrated inFIGS.3and4. In the present embodiment, the end effector22is attached to a distal end portion of the fifth arm portion24evia the adapter23. The end effector22is detachably attached to the robot arm21, for example. The end effector22is interchangeable with another end effector.

As the end effector22attached to the robot arm21, an end effector having various shapes, structures, and functions can be appropriately adopted according to work performed by the robot20. Examples of the end effector22attached to the robot arm21may include a robot hand capable of gripping the target object W, a processing head that performs laser processing or ultrasonic processing, a camera, an injector that injects molten metal or resin, or particles for blast processing, or the like, a manipulator, and an air blower.

In the present embodiment, the end effector22is a multi-fingered robot hand capable of gripping the target object Won the workbench WB. As illustrated inFIG.3, the end effector22includes a base portion22a, a plurality of finger portions22b, an end effector drive unit28, and an end effector position acquisition unit29. The base portion22ais connected to the fifth arm portion24evia the adapter23, for example. The base portion22ahas, for example, a cylindrical shape around a central axis CL illustrated in each figure. The central axis CL appropriately illustrated in each figure is a central axis of the end effector22, the adapter23, and the fifth arm portion24e.

In the following description, a direction parallel to the central axis CL is referred to as a “central axis direction” and is indicated as a Z axis in each figure. A positive side (+Z side) of the Z axis in a central axis direction is referred to as a “distal end side”, and a negative side (−Z side) in the central axis direction is referred to as a “proximal end side”. Further, unless otherwise specified, a radial direction centered on the central axis CL is simply referred to as a “radial direction”, and a circumferential direction around the central axis CL is simply referred to as a “circumferential direction”.

In the present embodiment, the base portion22ais provided with a guide rail portion22e. The guide rail portion22eis, for example, an annular groove surrounding the base portion22ain the circumferential direction. The plurality of finger portions22bprotrude from the base portion22atoward a distal end side (+Z side) in the central axis direction. In the present embodiment, the end effector22can grip the target object W with the plurality of finger portions22b. The number of finger portions22bis not particularly limited.

The end effector drive unit28can drive the end effector22. The end effector drive unit28includes a rotation drive unit22c. The rotation drive unit22cis provided inside the base portion22a, for example. The rotation drive unit22cis, for example, a servomotor capable of rotating the base portion22aaround the central axis CL. Although illustration is omitted, the end effector drive unit28has a finger drive unit that drives the plurality of finger portions22b. The finger drive unit is provided, for example, for the plurality of finger portions22b. One finger drive unit may be provided for each finger portion22b, or a plurality of finger drive units may be provided for each finger portion22b. The finger drive unit, for example, displaces an angle of the finger portion22bwith respect to the base portion22a. The finger drive unit is, for example, a servomotor.

The end effector position acquisition unit29can acquire a relative position of the end effector22with respect to the adapter23. The end effector position acquisition unit29has a rotational position acquisition unit22d. The rotational position acquisition unit22dis provided inside the base portion22a, for example. The rotational position acquisition unit22dcan detect a rotational position of the end effector22. The rotational position acquisition unit22dcan detect, for example, a rotational angle of the base portion22aaround the central axis CL. The rotational position acquisition unit22dis, for example, a rotary encoder. The end effector position acquisition unit29may include, for example, a sensor capable of detecting the position and angle of the finger portion22bwith respect to the base portion22a.

A camera unit60is attached to the end effector22in the present embodiment. The camera unit60is fixed to the base portion22a. The camera unit60includes a support portion61, a first camera62and a second camera63. The support portion61protrudes from the base portion22atoward the distal end side (+Z side) in the central axis direction. An end portion on the proximal end side (−Z side) of the support portion61is connected to an outer peripheral surface at an end portion on the distal end side of the base portion22a. The first camera62and the second camera63are fixed to an end portion on the distal end side of the support portion61. The first camera62and the second camera63can image the plurality of finger portions22band the target object W gripped by the finger portions22b. A stereo camera is configured of the first camera62and the second camera63. InFIG.4, illustration of the camera unit60is omitted.

The camera unit60has an image sensor64, a memory65, and a digital signal processing unit66, as illustrated inFIG.2. The image sensor64is, for example, a CCD image sensor or a CMOS image sensor. The image sensor64is provided for each of the first camera62and the second camera63. Each image sensor64converts an optical signal incident on a camera with the image sensor64into an analog electrical signal, converts the converted analog electrical signal into a digital image signal, and outputs the digital image signal.

The digital signal processing unit66performs image processing such as digital amplification, color interpolation processing, and white balance processing on the digital image signal output from the image sensor64. The digital image signal processed by the digital signal processing unit66may be temporarily stored in the memory65or may be output to the control unit40without being stored in the memory65. The digital image signal output from the digital signal processing unit66to the control unit40is output to a distance information acquisition unit44, which will be described below.

The memory65can store the digital image signal output from the image sensor64and the digital image signal output from the digital signal processing unit66. The memory65is, for example, a volatile memory. The memory65may be a non-volatile memory. The digital image signal output from the image sensor64is, for example, stored in the memory65, and then is sent from the memory65to the digital signal processing unit66and subjected to image processing in the digital signal processing unit66.

One memory65and one digital signal processing unit66may be provided inside the camera unit60and used for both the image sensor64of the first camera62and the image sensor64of the second camera63or may be provided for each of the image sensor64of the first camera62and the image sensor64of the second camera63. Further, some or both of the memory65and the digital signal processing unit66may be provided outside the camera unit60, such as the control unit40. Further, the camera unit60may be configured to have only one camera.

The adapter23is a member for attaching the end effector22to the robot arm21. As illustrated inFIG.4, the adapter23includes a support portion23a, a pedestal portion23b, a connection portion23c, and a pedestal drive unit23d. The support portion23ais a portion connected to the robot arm21. The support portion23ais detachably connected to, for example, the distal end portion of the fifth arm portion24e. The support portion23aincludes a concave portion23gthat is recessed from the distal end side (+Z side) to the proximal end side (−Z side) in the central axis direction. The support portion23amay be non-detachably fixed to the robot arm21.

The pedestal portion23bis disposed on a distal end side (+Z side) of the support portion23a. The pedestal portion23bis a portion to which the end effector22is connected. In the present embodiment, the base portion22aof the end effector22is detachably connected to an end portion on the distal end side (+Z side) of the pedestal portion23b. The end effector22may be non-detachably fixed to the pedestal portion23b. A portion on the proximal end side (−Z side) of the pedestal portion23bis inserted, for example, into the concave portion23g. For example, a gap is provided between the support portion23aand the pedestal portion23b, and the support portion23aand the pedestal portion23bdo not come into direct contact with each other.

The connection portion23cis provided inside the concave portion23g. The connection portion23cis provided between the support portion23aand the pedestal portion23b. The connection portion23cconnects the support portion23ato the pedestal portion23b. That is, in the present embodiment, the support portion23aand the pedestal portion23bare indirectly connected to each other via the connection portion23cin a state in which the support portion23aand the pedestal portion23bdo not come into direct contact with each other. The connection portion23csupports a mass of the pedestal portion23band a mass of the end effector22.

The connection portion23cincludes a damper element23eand a spring element23fThe spring element23fmay be, for example, an elastic member whose elastic force is adjustable. In this case, the robot system10may include an adjustment unit capable of adjusting the elastic force of the spring element23fThe spring element23fmay be, for example, an air spring. In this case, the elastic force of the spring element23fmay be adjusted by adjusting air pressure of an air spring with the adjustment unit. For example, a plurality of connection portions23care provided. The plurality of connection portions23cincludes, for example, a connection portion23cincluding a damper element23eand a spring element23fthat receive force and are displaced in the central axis direction, and a connection portion23cincluding a damper element23eand a spring element23fthat receive force and are displaced in a direction perpendicular to the central axis direction.

The connection portion23ccan reduce, for example, a vibration of the end effector22and a vibration given from the outside. The connection portion23csuppresses a displacement of the end effector22and a displacement of the pedestal portion23bcaused by a weight of the end effector22and a weight of the pedestal portion23b. The connection portion23ccan support the end effector22in a direction of gravity regardless of a posture of the end effector22. In addition to the damper element23eand the spring element23f, the connection portion23cmay include other elements capable of reducing the vibration of the end effector22, for example. The other elements include, for example, a piezoelectric element (piezo element).

The pedestal drive unit23dis provided, for example, between the support portion23aand the pedestal portion23binside the concave portion23g. The pedestal drive unit23dcan displace a position of the pedestal portion23bwith respect to the support portion23a. The pedestal drive unit23dcan move the end effector22connected to the pedestal portion23bby moving the pedestal portion23b.

The pedestal drive unit23dincludes, for example, a plurality of linear motors27. The linear motor27is, for example, a voice coil motor. The linear motor27includes a magnetic field generation unit27aand a magnet portion27b. One of the magnetic field generation unit27aand the magnet portion27bis attached to the support portion23a, and the other is attached to the pedestal portion23b. InFIG.4, for example, the magnetic field generation unit27ais attached to the support portion23a, and the magnet portion27bis attached to the pedestal portion23b. The magnetic field generation unit27amay be attached to the pedestal portion23b, and the magnet portion27bmay be attached to the support portion23a.

The magnetic field generation unit27ais, for example, a coil. A current is supplied to the magnetic field generation unit27aso that at magnetic field is generated. A repulsive force or an attractive force is generated between the magnetic field generation unit27aand the magnet portion27bby a magnetic field generated by the magnetic field generation unit27aand a magnetic field generated by the magnet portion27b. Due to this repulsive force or attractive force, the magnet portion27bis displaced with respect to the magnetic field generation unit27a. Accordingly, the linear motor27displaces the pedestal portion23bto which the magnet portion27bis attached, with respect to the support portion23ato which the magnetic field generation unit27ais attached. Thus, the pedestal drive unit23dcan drive the pedestal portion23bin a non-contact state without bringing the support portion23aand the pedestal portion23binto direct contact with each other.

The plurality of linear motors27include, for example, a linear motor27that can displace the pedestal portion23bwith respect to the support portion23ain the central axis direction, and a linear motor27that can displace the pedestal portion23bwith respect to the support portion23ain a direction perpendicular to the central axis direction.

The adapter23may have any configuration as long as the adapter23can attach the end effector22to the robot arm21. As a configuration of the adapter23, for example, a configuration of an adapter described in International Application No. PCT/JP2019/016043 may be adopted.

As illustrated inFIG.3, a plurality of imaging devices30are provided in the present embodiment. For the imaging device30, for example, two imaging devices including a first imaging device31and a second imaging device32are provided. The first imaging device31and the second imaging device32may be, for example, RGB cameras or infrared cameras. A stereo camera is configured of the first imaging device31and the second imaging device32. In the present embodiment, the first imaging device31and the second imaging device32are attached to the end effector22. The first imaging device31and the second imaging device32are disposed around the end effector22.

The first imaging device31and the second imaging device32are located, for example, radially outward of the base portion22aand disposed in the circumferential direction.

In the present embodiment, an optical axis AX1of the first imaging device31and an optical axis AX2of the second imaging device32are parallel to each other. The optical axes AX1and AX2are parallel to the central axis CL, for example. In the present specification, “the optical axes of the plurality of imaging devices are parallel to each other” includes a case in which the optical axes of the plurality of imaging devices are substantially parallel to each other, in addition to a case in which the optical axes of the plurality of imaging devices are strictly parallel to each other. The case in which the optical axes of the plurality of imaging devices are substantially parallel to each other includes, for example, a case in which the optical axes of the plurality of imaging devices are tilted with respect to each other within 5°.

FIG.5is a view of a portion of the end effector22, the first imaging device31, and the second imaging device32viewed from the distal end side (+Z side) in the central axis direction.FIG.6is a view of a portion of the end effector22, the first imaging device31, and the second imaging device32viewed from the distal end side (+Z side) in the central axis direction, and is a view which illustrates a case in which the first imaging device31and the second imaging device32are located in predetermined initial positions. InFIGS.5and6, illustration of the finger portion22bof the end effector22and the camera unit60is omitted.

At least one of the first imaging device31and the second imaging device32is movable with respect to the end effector22, as illustrated inFIGS.5and6. In the present embodiment, both the first imaging device31and the second imaging device32are movable with respect to the end effector22. Relative positions of the first imaging device31and the second imaging device32are variable. In the present embodiment, at least one of the first imaging device31and the second imaging device32is movable in a predetermined circumferential direction around the end effector22. In the present embodiment, the “predetermined circumferential direction” is the circumferential direction around the central axis CL around the base portion22a.

In the present embodiment, both the first imaging device31and the second imaging device32are movable in the predetermined circumferential direction around the end effector22. That is, in the present embodiment, one of the first imaging device31and the second imaging device32is movable in the predetermined circumferential direction around the end effector22, and the other is also movable in the predetermined circumferential direction around the end effector22. As illustrated inFIG.6, in the present embodiment, the first imaging device31and the second imaging device32can comes into contact with each other in the circumferential direction. In the present embodiment, the first imaging device31and the second imaging device32come in contact with each other in the circumferential direction when the first imaging device31and the second imaging device32are located at the initial positions illustrated inFIG.6.

As illustrated inFIG.3, the first imaging device31includes a housing31a, a first drive unit31b, a first position acquisition unit31c, a lens31e, and an image sensor31fThe housing31ahas, for example, a cylindrical shape that has an opening on a distal end side (+Z side) and extends in the central axis direction. The central axis of the housing31amatches the optical axis AX1of the first imaging device31, for example. The housing31ais attached to the base portion22aof the end effector22via a slider31d.

The slider31dis fixed, for example, to a radially inner side portion in a portion on the proximal end side (−Z side) of the housing31a. The slider31dconnects the housing31ato the base portion22aof the end effector22. That is, in the present embodiment, the first imaging device31is connected to the end effector22via the slider31d. The slider31dis connected to the guide rail portion22eof the end effector22. The slider31dcan move in the circumferential direction along the guide rail portion22e. This makes it possible for the first imaging device31to be movable in the circumferential direction along the guide rail portion22e.

The lens31eis fitted into the opening on the distal end side (+Z side) of the housing31a. The lens31eis, for example, a circular lens when viewed in the central axis direction. The optical axis AX1of the first imaging device31passes through a center of the lens31e.

The image sensor31fis disposed inside the housing31a. The image sensor31fis, for example, a CCD image sensor or a CMOS image sensor. Light incident on the inside of the housing31ais incident on the image sensor31fthrough the lens31e. The image sensor31fconverts the incident optical signal into an analog electrical signal, converts the converted analog electrical signal into a digital image signal, and outputs the digital image signal.

As illustrated inFIG.5, the image sensor31fhas a rectangular shape when viewed in the central axis direction. When viewed in the central axis direction, a long side of the image sensor31fis perpendicular to a direction passing through the optical axis AX1of the first imaging device31in a radial direction. In the present embodiment, the first imaging device31is movable in the circumferential direction so that a state in which the long side of the image sensor31fis perpendicular to the radial direction passing through the optical axis AX1of the first imaging device31when viewed in the central axis direction is maintained.

As illustrated inFIG.3, the first drive unit31bis disposed inside the housing31a, for example. The first drive unit31bis, for example, a servomotor. The first drive unit31bmoves the first imaging device31in the circumferential direction around the end effector22. In the present embodiment, the entire first imaging device31, including the first drive unit31b, moves in the circumferential direction together with the slider31d.

The first position acquisition unit31cis disposed inside the housing31a, for example. The first position acquisition unit31cis, for example, a rotary encoder. The first position acquisition unit31ccan detect rotation of the first drive unit31bto acquire position information in the circumferential direction of the first imaging device31. The first position acquisition unit31cdetects a rotation speed of the first drive unit31b, for example, with the rotation speed of the first drive unit31bwhen the first imaging device31is located at the initial position illustrated inFIG.6set to zero, to detect a position in the circumferential direction of the first imaging device31.

The second imaging device32includes a housing32a, a second drive unit32b, a second position acquisition unit32c, a lens32e, and an image sensor32fThe housing32aincludes, for example, a cylindrical shape that has the opening on a distal end side (+Z side) and extends in the central axis direction. A central axis of the housing32amatches the optical axis AX2of the second imaging device32, for example. The housing32ais attached to the base portion22aof the end effector22via a slider32d. The slider32dis fixed, for example, to a radially inner portion on the proximal end side (−Z side) of the housing32a. The slider32dconnects the housing32aand the base portion22aof the end effector22. That is, in the present embodiment, the second imaging device32is connected to the end effector22via the slider32d. The slider32dis connected to the guide rail portion22eof the end effector22. The slider32dcan move in the circumferential direction along the guide rail portion22e. This makes it possible for the second imaging device32to move in the circumferential direction along the guide rail portion22e.

Thus, in the present embodiment, the guide rail portion22ecorresponds to a first holding portion that holds the first imaging device31and corresponds to a second holding portion that holds the second imaging device32. That is, in the present embodiment, the end effector22includes the guide rail portion22eas a first holding portion holding the first imaging device31and a second holding portion holding the second imaging device32. In the present embodiment, the first imaging device31is held to be movable by the guide rail portion22eserving as the first holding portion, and the second imaging device32is held to be movable by the guide rail portion22eserving as the second holding portion.

The lens32eis fitted into an opening on the distal end side (+Z side) of the housing32a. The lens32eis, for example, a circular lens when viewed in the central axis direction. The optical axis AX2of the second imaging device32passes through a center of the lens32e.

The image sensor32fis disposed inside the housing32a. The image sensor32fis, for example, a CCD image sensor or a CMOS image sensor. Light incident on the inside of the housing32ais incident on the image sensor32fthrough the lens32e. The image sensor32fconverts an incident optical signal into an analog electrical signal, converts the converted analog electrical signal into a digital image signal, and outputs the digital image signal.

As illustrated inFIG.5, the image sensor32fhas a rectangular shape when viewed in a central axis direction. When viewed in the central axis direction, a long side of the image sensor32fis perpendicular to a direction passing through the optical axis AX2of the second imaging device32in a radial direction. In the present embodiment, the second imaging device32is movable in a circumferential direction so that a state in which the long side of the image sensor32fis perpendicular to the radial direction passing through the optical axis AX2of the second imaging device32when viewed in the central axis direction is maintained. The image sensor32fhas the same shape and size as the image sensor31fof the first imaging device31, for example.

In the present specification, a “long side of the image sensor” is a long side in a rectangular area of the image sensor on which light is incident. For the image sensors31fand32fillustrated in each figure, only a body portion having the rectangular area on which the light is incident is illustrated. The image sensors31fand32fmay include portions other than the body portion, such as a frame portion that holds the body portion on which the light is incident. In this case, even when the image sensors31fand32fhave an external shape other than a rectangle when viewed in a direction of the optical axes AX1and AX2, the long sides of the image sensors31fand32fare the long sides of the rectangular area on which the light is incident in the image sensors31fand32f.

As illustrated inFIG.3, the second drive unit32bis disposed inside the housing32a, for example. The second drive unit32bis, for example, a servomotor. The second drive unit32bmoves the second imaging device32in the circumferential direction around the end effector22. In the present embodiment, the entire second imaging device32, including the second drive unit32b, moves in the circumferential direction together with the slider32d.

In the present embodiment, a drive unit33that drives the imaging device30is configured of the first drive unit31band the second drive unit32b. The drive unit33can move at least one of the first imaging device31and the second imaging device32with respect to the end effector22. In the present embodiment, the drive unit33can move both the first imaging device31and the second imaging device32with respect to the end effector22using respective drive unit provided in the respective imaging devices30.

The second position acquisition unit32cis disposed inside the housing32a, for example. The second position acquisition unit32cis, for example, a rotary encoder. The second position acquisition unit32ccan detect rotation of the second drive unit32bto acquire position information in the circumferential direction of the second imaging device32. The second position acquisition unit32cdetects a rotation speed of the second drive unit32b, for example, with the rotation speed of the second drive unit32bwhen the second imaging device32is located at the initial position illustrated inFIG.6set to zero, to detect the position in the circumferential direction of the second imaging device32.

In the present embodiment, a position acquisition unit34that acquires at least position information of the first imaging device31is configured of the first position acquisition unit31cand the second position acquisition unit32c. In the present embodiment, the position acquisition unit34can acquire both the position information of the first imaging device31and position information of the second imaging device32using each position acquisition unit provided in each imaging device30. The position acquisition unit34can acquire, for example, the position in the circumferential direction of the first imaging device31and the position in the circumferential direction of the second imaging device32.

Each imaging device30includes a memory35and a digital signal processing unit36, as illustrated inFIG.2. The digital signal processing unit36performs image processing such as digital amplification, color interpolation processing, and white balance processing on the digital image signal output from the image sensor of each imaging device30. The digital image signal processed by the digital signal processing unit36may be temporarily stored in the memory35or may be output to the control unit without being stored in the memory35. The digital image signal output from the digital signal processing unit36to the control unit40is output to the distance information acquisition unit44, which will be described below.

The memory35can store the digital image signal output from the image sensor of each imaging device30and the digital image signal output from the digital signal processing unit36. The memory35is, for example, a volatile memory. The memory may be a non-volatile memory. The digital image signal output from the image sensor of each imaging device30is, for example, stored in the memory35, sent from the memory35to the digital signal processing unit36, and subjected to image processing in the digital signal processing unit36.

In the above description, one memory35and one digital signal processing unit36are provided inside each imaging device30, but the present invention is not limited thereto. One memory35and one digital signal processing unit36may be provided for each of the two imaging devices30, and used for both the image sensor31fof the first imaging device31and the image sensor32fof the second imaging device32. Further, some or both of the memory35and the digital signal processing unit36may be provided outside the imaging device30, such as the control unit40.

The control unit40controls the robot system10. As illustrated inFIG.2, in the present embodiment, the control unit40includes an arm control unit41, an end effector control unit42, an imaging device control unit43and the distance information acquisition unit44. Each of the arm control unit41, the end effector control unit42, an imaging device control unit43, and the distance information acquisition unit44may be realized by dedicated hardware, or may be realized by a memory and a microprocessor.

The arm control unit41controls the arm drive unit25. In the present embodiment, the arm control unit41receives information on position and posture of the arm portion24from the arm position acquisition unit26and also receives the information on the distance to the target object W from the distance information acquisition unit44. In the present embodiment, the arm control unit41controls the arm drive unit25on the basis of information on the position and posture of the arm portion24and the information on the distance to the target object W. More specifically, the arm control unit41, for example, calculates target values of the position and posture of the arm portion24on the basis of the information on the distance to the target object W, and controls the arm drive unit25so that the position and posture of the arm portion24become the target values through feedback control using the information from the arm position acquisition unit26. Thus, the control unit40controls the arm drive unit25with the arm control unit41to control at least one of the position and the posture of the robot arm21. The target values of the position and posture of the arm portion24may be input to the arm control unit41from the outside.

The end effector control unit42controls the end effector drive unit28. In the present embodiment, the end effector control unit42receives information on position and posture of the end effector22from the end effector position acquisition unit29, and also receives the information on the distance to the target object W from the distance information acquisition unit44. In the present embodiment, the end effector control unit42controls the end effector drive unit28on the basis of the information on the position and posture of the end effector22and the information on the distance to the target object W. More specifically, the end effector control unit42calculates target values of the position and posture of the end effector22on the basis of the information on the distance to the target object W, and controls the end effector drive unit28so that the position and posture of the end effector22become the target values through feedback control using information from the end effector position acquisition unit29. Thus, the control unit40controls the end effector drive unit28using the end effector control unit42to control at least one of the position and posture of the end effector22. The target values of the position and posture of the end effector22may be input to the end effector control unit42from the outside.

The imaging device control unit43controls the drive unit33of the imaging device30. In the present embodiment, information on the position of the imaging device30is input to the imaging device control unit43from the position acquisition unit34of the imaging device30. More specifically, the imaging device control unit43receives the position information of the first imaging device31in the circumferential direction from the first position acquisition unit31c, and also receives the position information of the second imaging device32in the circumferential direction from the second position acquisition unit32c. Further, the information on the distance to the target object W is input from the distance information acquisition unit44to the imaging device control unit43. The imaging device control unit43, for example, controls the first drive unit31band the second drive unit32bon the basis of the position information of the imaging device30input from the position acquisition unit34and the information on the distance to the target object W input from the distance information acquisition unit44. Accordingly, the control unit40controls the drive unit33using the imaging device control unit43to control the position of the imaging device30.

The imaging device control unit43can change a baseline length L that is a distance between the first imaging device31and the second imaging device32. As illustrated inFIG.5, the baseline length L is a distance between the optical axis AX1of the first imaging device31and the optical axis AX2of the second imaging device32. For example, when the first imaging device31and the second imaging device32are moved to be away from each other in the circumferential direction from a position indicated by a two-dot chain line to a position indicated by a solid line inFIG.5, the baseline length L can be changed from a baseline length L1to a baseline length L2larger than the baseline length L1. Thus, the baseline length L between the first imaging device31and the second imaging device32can be increased. On the other hand, when the first imaging device31and the second imaging device32are moved to approach each other in the circumferential direction, the baseline length L between the first imaging device31and the second imaging device32can be reduced. Thus, in the present embodiment, the control unit40controls the drive unit33of the imaging device using the imaging device control unit43to be able to change the baseline length L, which is the distance between the first imaging device31and the second imaging device32.

The imaging device control unit43calculates a target value of the baseline length L to be changed, on the basis of the information on the distance to the target object W, for example. When the distance from the imaging device30to the target object W is relatively large, the imaging device control unit43makes the baseline length L relatively large. On the other hand, when the distance from the imaging device30to the target object W is relatively small, the imaging device control unit43makes the baseline length L relatively small. The target value of the baseline length L may be input to the imaging device control unit43from the outside. The target value of the baseline length L may be input from the distance information acquisition unit44to the imaging device control unit43.

In the present embodiment, the control unit40changes the baseline length L according to work content of the robot system10using the imaging device control unit43. For example, when the target object W on the workbench WB is searched for, the control unit40makes the baseline length L relatively large. On the other hand, for example, when the end effector22is brought closer to the target object W on which work is performed after the target object W is found, the control unit40makes the baseline length L relatively small. In this case, the control unit40may reduce the baseline length L as the end effector22approaches the target object W.

In the present embodiment, the imaging device control unit43controls the drive unit33to move the imaging device30to the predetermined initial position after the robot system10is powered on. For example, after the robot system10is powered on, the imaging device control unit43moves the first imaging device31to the predetermined initial position illustrated inFIG.6and moves the second imaging device32to the predetermined initial position illustrated inFIG.6. The movement of the first imaging device31and the second imaging device32to respective initial positions is performed, for example, after the robot system10is powered on and before the first imaging device31and the second imaging device32are used. The movement of the first imaging device31and the second imaging device32to respective initial positions may be performed, for example, immediately after the robot system10is powered on.

The imaging device control unit43, for example, brings the first imaging device31and the second imaging device32into contact with each other in the circumferential direction and preferably and easily moves each of the first imaging device31and the second imaging device32to the initial position. The robot system10may include a sensor capable of detecting contact between the first imaging device31and the second imaging device32in the circumferential direction.

Thus, in the present embodiment, at least the first imaging device31moves to the predetermined initial position after the robot system10is powered on. More specifically, both the first imaging device31and the second imaging device32move to the predetermined initial positions after the robot system10is powered on. Only the first imaging device31between the first imaging device31and the second imaging device32may move to the predetermined initial position after the robot system10is powered on, and only the second imaging device32between the first imaging device31and the second imaging device32may move to the predetermined initial position after the robot system10is powered on.

In the present embodiment, when the control unit40moves the first imaging device31and the second imaging device32using the imaging device control unit43, the control unit40stops the member to which the imaging device30is attached, that is, the end effector22in the present embodiment. That is, in the present embodiment, the movement of the first imaging device31and the movement of the second imaging device32are performed in a state in which the members to which the first imaging device31and the second imaging device32are attached are stationary. In the present embodiment, the movement of the first imaging device31to the predetermined initial position and the movement of the second imaging device32to the predetermined initial position are also performed in a state in which the member (the end effector22) to which the first imaging device31and the second imaging device32are attached is stationary.

When at least one of the first imaging device31and the second imaging device32cannot image the target object W, the control unit40may move the at least one of the first imaging device31and the second imaging device32so that both the first imaging device31and the second imaging device32are located to be able to image the target object W. A case in which the target object W cannot be imaged by the imaging device30is, for example, a case in which an obstacle is disposed between the imaging device30and the target object W and the target object W is not captured by the imaging device30. The control unit40may move the first imaging device31and the second imaging device32to positions at which the work of the end effector22are not hindered, depending on the work of the target object W by the end effector22.

The distance information acquisition unit44acquires the information on the distance to the target object W. The information on the distance to the target object W includes, for example, the distance from the imaging device30to the target object W, a distance from the end effector22to the target object W, a distance from the robot arm21to the target object W, distances between a plurality of target objects W, 3D point cloud data for the target object W, and the like.

The distance information acquisition unit44receives information of images captured by the image sensors31fand32f.

The information on the position of the imaging device30is input to the distance information acquisition unit44from the position acquisition unit34of the imaging device30. The distance information acquisition unit44acquires the baseline length L on the basis of the position information of the first imaging device31acquired by the position acquisition unit34. In the present embodiment, the distance information acquisition unit44acquires the baseline length L on the basis of the position information of the first imaging device31acquired by the first position acquisition unit31cand the position information of the second imaging device32acquired by the second position acquisition unit32c. Specifically, the distance information acquisition unit44calculates a distance between the optical axis AX1of the first imaging device31and the optical axis AX2of the second imaging device32from the position in the circumferential direction of the first imaging device31and the position in the circumferential direction of the second imaging device32and acquires the baseline length L. The distance information acquisition unit44may acquire the baseline length L from another portion such as the imaging device control unit43, for example.

The distance information acquisition unit44acquires the information on the distance to the target object W on the basis of the acquired baseline length L, the first image acquired by the first imaging device31, and the second image acquired by the second imaging device32. Here, in the present embodiment, the posture of the image sensor31fof the first imaging device31and the posture of the image sensor32fof the second imaging device32are different from each other when viewed in the central axis direction. Therefore, the distance information acquisition unit44rotates at least one of the first image acquired by the first imaging device31and the second image acquired by the second imaging device32to align the direction (orientation) of the first image with the direction (orientation) of the second image.

Thus, in the present embodiment, the distance information acquisition unit44rotates the at least one of the first image acquired by the first imaging device31and the second image acquired by the second imaging device32to adjust the direction of the acquired image. The distance information acquisition unit44may rotate only the first image acquired by the first imaging device31to align the direction of the first image with the direction of the second image, may rotate only the second image acquired by the second imaging device32to align the direction of the second image with the direction of the first image, or may rotate both the first image acquired by the first imaging device31and the second image acquired by the second imaging device32to align the direction of the first image with the direction of the second image. In the present embodiment, the distance information acquisition unit44measures the distance from the imaging device to the target object W using the baseline length L and the first image and the second image whose directions are aligned through the rotation. The control unit40controls at least one of the robot arm21and the end effector22on the basis of the information on the distance to the target object W that has been acquired in this way.

The display unit50displays information based on the information on the distance. The information on the distance includes, for example, the information on the distance to the target object W, and information on the baseline length L, which is the distance between the first imaging device31and the second imaging device32. The information based on the information on the distance may be the information on the distance itself or may be information obtained from the information on the distance. For example, a current distance to the target object W and the current baseline length L may be displayed on the display unit50. For example, the display unit50may display changes in the distance to the target object W and the baseline length L in a graph form. The display unit50may have any structure as long as the display unit50can display the information based on the information on the distance. The display unit50may be provided separately from the robot20or may be provided on the robot arm21, for example. The display unit50is controlled by the control unit40.

According to the present embodiment, at least one of the first imaging device31and the second imaging device32attached to the end effector22is movable with respect to the end effector22. Therefore, at least one of the first imaging device31and the second imaging device32is moved so that the distance between the first imaging device31and the second imaging device32can be changed. Accordingly, the baseline length L, which is the distance between the first imaging device31and the second imaging device32, can be changed.

Here, when the baseline length L is relatively large, a resolution for the target object W relatively far from the imaging device30can be made relatively high, and the distance to the target object W relatively far from the imaging device30can be accurately detected. However, in this case, since the target object W relatively close to the imaging device30is not captured by the imaging device30, a distance to the target object W relatively close to the imaging device30cannot be detected. On the other hand, when the baseline length L is relatively small, the target object W relatively close to the imaging device30can be imaged, but it is difficult to focus on the target object W relatively far from the imaging device30, and to accurately detect the distance to the target object W relatively far from the imaging device30.

Thus, a position of the target object W to which the distance can be preferably detected differs depending on a magnitude of the baseline length L. Therefore, for example, in an imaging device whose magnitude of the baseline length L is fixed, a position of the target object W with respect to the imaging device at which the information on the distance can be preferably acquired is limited. Accordingly, when only the imaging device is used, work content of the robot system may be limited.

On the other hand, according to the present embodiment, the baseline length L between the first imaging device31and the second imaging device32attached to the end effector22can be changed as described above. Therefore, when a distance between the end effector22and the target object W is relatively large, the baseline length L is made relatively large, and when the distance between the end effector22and the target object W is relatively small, the baseline length L is made to relatively small, making it possible to accurately measure the distance to the target object W with only the imaging device30attached to the end effector22even when the distance between the end effector22and the target object W changes greatly to some extent. Further, when the distance between the end effector22and the target object W is relatively large, the baseline length L can be made relatively large, making it possible to detect the distance to the target object W more accurately through stereo matching. Further, when the distance between the end effector22and the target object W is relatively small, the baseline length L can be made relatively small, making it possible to increase a degree of overlapping (an overlapping portion) between the image captured by the first imaging device31and the image captured by the second imaging device32, and to perform stereo matching within a relatively wide range in the image captured by each imaging device30to measure the distance to the target object W. This makes it possible to perform work on the target object W with the robot system10regardless of the distance between the end effector22and the target object W. Therefore, it is possible to suppress work content of the robot system10being restricted. This makes it possible to improve workability for the target object W.

Specifically, for example, it is possible to perform a work of searching the workbench WB from a relatively long distance to find the target object W, a work of bringing the end effector22closer to the target object W that has been searched for, a work of gripping the target object W with the end effector22, a work of moving the gripped target object W to another place, and the like by using only the first imaging device31and the second imaging device32attached to the end effector22while preferably acquiring the distance to the target object W.

Further, for example, when a facility capable of measuring the distance to the target object W on which the robot system performs work is provided on a ceiling of a place at which the robot system is disposed, or the like, a baseline length L in the facility can be made different from the baseline length L of the imaging device attached to the end effector, making it possible to perform the work using the robot system while ascertaining the distance to the target object even when the distance between the end effector and the target object changes in a somewhat wide range. However, in this case, there is a problem that a cost for providing the facility is required. Further, there is a problem that the robot system can only be used at the place at which the facility is provided.

On the other hand, according to the present embodiment, since the baseline length L between the first imaging device31and the second imaging device32attached to the end effector22can be changed, the robot system10can perform work on the target object W without providing the facility provided on the ceiling or the like described above even when the distance between the end effector22and the target object W changes in the somewhat wide range. This eliminates the cost of providing the facility. Further, the robot system10can be used even in places at which the above facility is not provided. Therefore, a degree of freedom of a place at which the robot system10can be used can be improved.

Further, when either the first imaging device31or the second imaging device32is at a position at which the target object W cannot be imaged, the first imaging device31or the second imaging device32is moved with respect to the end effector22, making it easy to image the target object W using both the first imaging device31and the second imaging device32without moving the end effector22. This makes it possible to preferably acquire the information on the distance to the target object W regardless of for example, the position and posture of the end effector22.

Further, it is possible to move the position of at least one of the first imaging device31and the second imaging device32to a preferred position depending on a path along which the robot arm21and the end effector22move, a surrounding environment in which the robot arm21and the end effector22are disposed, or the like. For example, when the robot arm21and the end effector22are moved with respect to the target object W, it is possible to move the first imaging device31and the second imaging device32so that the first imaging device31and the second imaging device32do not come into contact with other objects or the like. Therefore, a freedom of movement of the robot arm21and the end effector22can be improved.

Further, for example, it is possible to move at least one of the first imaging device31and the second imaging device32to optimize inertia when the robot20moves. Specifically, for example, when the end effector22does not grip the target object W, the first imaging device31and the second imaging device32are disposed on opposite sides with the central axis CL therebetween, making it easy to minimize overall inertia of the end effector22, the first imaging device31, and the second imaging device32. This makes it easy to preferably move the end effector22to which the first imaging device31and the second imaging device32are attached. Further, for example, when the end effector22is gripping the target object W, it is possible to move at least one of the first imaging device31and the second imaging device32to a position at which the first imaging device31and the second imaging device32function as counterweights for the gripped target object W. This makes it easy to minimize the overall inertia of the end effector22, the first imaging device31, the second imaging device32, and the target object W. Therefore, it is possible to make it easy to preferably move the end effector22in a state in which the end effector22grips the target object W.

Further, when the distance between the target object W and the first imaging device31and the second imaging device32changes within a range in which the target object W can be imaged by the first imaging device31and the second imaging device32, the baseline length L may be increased as the first imaging device31and the second imaging device32approach the target object W. Here, the detection accuracy of the distance to the target object W can be improved as the baseline length L increases. Therefore, when the target object W is within the range in which the target object W can be imaged, the baseline length L is increased as the imaging device approaches the target object W, making it possible to acquire the distance to the target object W more accurately and easy to perform precision work on the target object W.

For example, when the end effector22is a robot hand that grips the target object W or when the end effector22is a tool that perform work on the target object W, the distance to target object W can be accurately acquired as the end effector22is closer to the target object W, making it easy for the end effector22to perform precise work on the target object W. Further, when the end effector22is relatively far from the target object W, the baseline length L is relatively small, and thus, an overlapping area (the overlapping portion) between the image captured by the first imaging device31and the image captured by the second imaging device32is increased, and the distance can be measured within a relatively wide range including the target object W.

Further, according to the present embodiment, the optical axis AX1of the first imaging device31and the optical axis AX2of the second imaging device32are parallel to each other. Therefore, it is easy to preferably acquire the distance to the target object W on the basis of the first image acquired by the first imaging device31and the second image acquired by the second imaging device32.

Further, according to the present embodiment, the control unit40that controls the robot system10can change the baseline length L, which is the distance between the first imaging device31and the second imaging device32, and the distance information acquisition unit44can acquire the information on the distance to the target object W on the basis of the baseline length L. Therefore, it is possible to easily change the baseline length L depending on work content of the robot system10. Further, the information on the distance to the target object W can be easily acquired by the distance information acquisition unit44.

Further, according to the present embodiment, the position acquisition unit34that acquires the position information of at least the first imaging device31is provided, and the distance information acquisition unit44acquires the baseline length L on the basis of the position information of the first imaging device31acquired by the position acquisition unit34. Therefore, the distance information acquisition unit44can preferably acquire the information on the distance to the target object W on the basis of the acquired baseline length L while preferably acquiring the baseline length L.

Further, according to the present embodiment, the control unit40changes the baseline length L depending on work content of the robot system10. Therefore, it is possible to preferably change the baseline length L between the first imaging device31and the second imaging device32depending on work content of the robot system10. This makes it possible to preferably perform the work with the robot system10. Specifically, for example, when work for searching for the target object W from a relatively long distance is performed, the baseline length L is made relatively large, making it possible to accurately acquire the distance to the target object W, which is relatively far away, and easy to search for the target object W. Further, for example, when the end effector22performs work for gripping the target object W, the baseline length L is made relatively small, making it possible to acquire the distance to the target object W, which is relatively small, and easy to preferably grip the target object W with the end effector22.

Further, according to the present embodiment, the distance information acquisition unit44rotates the at least one of the first image acquired by the first imaging device31and the second image acquired by the second imaging device32to adjust the direction of the acquired image. Therefore, even when the image sensor31fof the first imaging device31and the image sensor32fof the second imaging device32are disposed in different postures, it is possible to align the direction of the first image acquired by the first imaging device31with the direction of the second image acquired by the second imaging device32. This makes it possible to preferably acquire the information on the distance to the target object W using the distance information acquisition unit44on the basis of the images acquired by each imaging device30regardless of a relative position and relative posture of the first imaging device31and the second imaging device32. Therefore, even when the at least one of the first imaging device31and the second imaging device32is moved so that the first imaging device31and the second imaging device32are at arbitrary positions and postures, it is possible to preferably acquire the information on the distance to the target object W.

Further, according to the present embodiment, the first imaging device31and the second imaging device32are disposed around the end effector22. Therefore, it is easy to measure the distance between the end effector22and the target object W from the images captured by the first imaging device31and the second imaging device32. Further, for example, the first imaging device31and the second imaging device32are disposed radially outward of the base portion22aas in the present embodiment, making it difficult for the first imaging device31and the second imaging device32to hinder work of gripping the target object W by the end effector22.

Further, according to the present embodiment, at least one of the first imaging device31and the second imaging device32is movable in the predetermined circumferential direction around the end effector22. Therefore, at least one of the first imaging device31and the second imaging device32is moved so that a distance in the circumferential direction between the first imaging device31and the second imaging device32can be changed and the baseline length L between the first imaging device31and the second imaging device32can be easily changed.

Further, according to the present embodiment, at least the first imaging device31moves to the predetermined initial position after the robot system10is powered on. This makes it possible to perform calibration of the first position acquisition unit31cin the first imaging device31before the first imaging device31is used. Accordingly, even when the first imaging device31is moved, it is possible to accurately detect a position of the first imaging device31with the predetermined initial position as a reference. In the present embodiment, both the first imaging device31and the second imaging device32move to the predetermined initial positions after the robot system10is powered on. Therefore, it is possible to calibrate each position detection unit of each imaging device30, and to accurately detect the position of each imaging device30. This makes it possible to accurately change the baseline length L between the first imaging device31and the second imaging device32, and to acquire the information on the distance to the target object W more preferably on the basis of the baseline length L. Further, in the present embodiment, the first imaging device31and the second imaging device32come into contact with each other in the circumferential direction, making it possible to easily move the first imaging device31and the second imaging device32to the initial positions.

Further, according to the present embodiment, the movement of the first imaging device31to the predetermined initial position is performed in a state in which the member to which the first imaging device31is attached, that is, the end effector22in the present embodiment is stationary. Therefore, it is easier to move the first imaging device31as compared with a case in which the first imaging device31is moved to the initial position while the end effector22is moving. Further, since it is possible to suppress the movement of the first imaging device31while the end effector22is moving, it is possible to suppress the complication of the calculation of the movement of the end effector22and the calculation of the movement of the robot arm21. In the present embodiment, the movement of the second imaging device32to the predetermined initial position is also performed in a state in which the end effector22is stationary. This makes it easy to move the second imaging device32to the initial position. Further, since it is possible to suppress the movement of the second imaging device32while the end effector22is moving, it is possible to further suppress the complication of the calculation of the movement of the end effector22and the calculation of the movement of the robot arm21.

In the present embodiment, the movement of the first imaging device31with respect to the end effector22and the movement of the second imaging device32with respect to the end effector22are both performed in a state in which the member to which the first imaging device31and the second imaging device32are attached, that is, the end effector22in the present embodiment is stationary. Therefore, the first imaging device31and the second imaging device32do not move relative to the end effector22while the end effector22is moving. This makes it possible to further suppress the complication of the calculation of the movement of the end effector22and the calculation of the movement of the robot arm21.

Further, according to the present embodiment, the display unit50that displays the information based on the information on the distance is provided. Therefore, an operator or the like of the robot system10can easily acquire the information on the distance by viewing the display unit50.

In the above-described description, a method of rotating the at least one of the first image acquired by the first imaging device31and the second image acquired by the second imaging device32to adjust the direction of the acquired image has been adopted, but the present invention is not limited thereto. In the present embodiment, the control unit40may rotate at least one of the image sensor31fof the first imaging device31and the image sensor32fof the second imaging device32to adjust the direction of the acquired image. In this case, the control unit40rotates at least one of the image sensor31fof the first imaging device31and the image sensor32fof the second imaging device32so that the long side of the image sensor31fand the long side of the image sensor32fare parallel, for example. This makes it possible to align the directions of the images acquired by the respective imaging devices30without performing processing such as rotation on the captured image. Therefore, the load of image processing in the control unit40can be reduced as compared with a case in which the processing such as rotation is performed on the acquired image.

When the control unit40rotates the image sensor31fof the first imaging device31, the control unit40may rotate the entire first imaging device31for each the image sensor31f, or may rotate only the image sensor31fof the first imaging device31. When the control unit40rotates the image sensor31fof the first imaging device31, the control unit40rotates the image sensor31faround the optical axis AX1. In this case, the first imaging device31is attached to the end effector22to be rotatable around the optical axis AX1.

When the control unit40rotates the image sensor32fof the second imaging device32, the control unit40may rotate the entire second imaging device32for each the image sensor32f, or may rotate only the image sensor32fin the second imaging device32. When the control unit40rotates the image sensor32fof the second imaging device32, the control unit40rotates the image sensor32faround the optical axis AX2. In this case, the second imaging device32is attached to the end effector22to be rotatable around the optical axis AX2.

Further, in the present embodiment, the end effector22may include the first holding portion that holds the first imaging device31not to be movable. In this case, the end effector22may include a guide rail portion22eas a second holding portion that movably holds the second imaging device32. Further, the end effector22may include the second holding portion that holds the second imaging device32not to be movable. In this case, the end effector22may include a guide rail portion22eas a first holding portion that movably holds the first imaging device31.

Further, in the present embodiment, at least the first imaging device31may move to a predetermined end position before the robot system10is powered off. In this case, for example, after the control unit40receives a command to stop the robot system10, the control unit40causes the first imaging device31and the second imaging device32to come into contact with each other in the circumferential direction to move both the first imaging device31and the second imaging device32to the predetermined end position. The predetermined end position may be the same as the predetermined initial position, or may be different from the predetermined initial position.

For example, when the predetermined end position is the same as the predetermined initial position, the first imaging device31and the second imaging device32are located at the predetermined initial position at a point in time when the robot system10is powered on again. Therefore, it is not necessary to provide a process of moving the first imaging device31and the second imaging device32to the predetermined initial positions after the robot system10is powered on. This makes it possible to preferably acquire the position of each imaging device30using the position acquisition unit34even when the first imaging device31and the second imaging device32are moved immediately after the robot system10is powered on.

The movement of the first imaging device31to the predetermined end position is performed, for example, in a state in which the member to which the first imaging device31is attached is stationary. Therefore, the first imaging device31can be easily moved to the end position. Further, since it is possible to suppress the movement of the first imaging device31while the end effector22is moving, it is possible to further suppress complication of the calculation of the movement of the end effector22and the calculation of the movement of the robot arm21.

The movement of the second imaging device32to the predetermined end position is performed, for example, in a state in which the member to which the second imaging device32is attached is stationary. Therefore, the second imaging device32can be easily moved to the end position. Further, since it is possible to suppress the movement of the second imaging device32while the end effector22is moving, it is possible to further suppress complication of the calculation of the movement of the end effector22and the calculation of the movement of the robot arm21.

Further, for the overlapping portion (a portion commonly appearing in the two images) between the image captured by the first imaging device31and the image captured by the second imaging device32, it is possible to measure a distance to a feature portion such as the target object W appearing in the overlapping portion over the entire overlapping portion. Therefore, the control unit40may measure the distance over the entire overlapping portion between the image captured by the first imaging device31and the image captured by the second imaging device32, and create a depth map of the overlapping portion. The depth map is, for example, an image showing distance information with different colors, shades of colors, and the like.

Further, the control unit40may measure the distance to the target object W using the images captured by the first imaging device31and the second imaging device32and the image captured by the camera unit60. For example, when the distance to the target object W measured from the images captured by the first imaging device31and the second imaging device32becomes equal to or smaller than a predetermined distance, the control unit40may power on the camera unit60and measure the distance to the target object W using each of the images captured by the first imaging device31and the second imaging device32and the image captured by the camera unit60.

Further, the control unit40may switch between a first imaging mode in which the distance to the target object W is measured using images captured by the first imaging device31and the second imaging device32, and a second imaging mode in which the distance to the target object W is measured using the image captured by the camera unit60depending on the distance to the target object W. In this case, for example, the control unit40may measure the distance to the target object W in the first imaging mode described above when the distance to the target object W is larger than the predetermined distance, and measure the distance to the target object W in the second imaging mode described above when the distance to the target object W is equal to or smaller than the predetermined distance.

Second Embodiment

FIG.7is a perspective view illustrating a portion of a robot system110of the present embodiment. InFIG.7, illustration of a finger portion22bof an end effector122and a camera unit60is omitted. The same configurations as those in the above-described embodiment are appropriately denoted by the reference signs, for example, and description thereof will be omitted.

As illustrated inFIG.7, in the robot system110of the present embodiment, the second imaging device132is fixed with respect to the end effector122. That is, the second imaging device132does not move relative to the end effector122in the present embodiment. Therefore, in the present embodiment, only the first imaging device31is movable in the predetermined circumferential direction around the end effector122. Thus, in the present embodiment, one of the first imaging device31and the second imaging device132is movable in the predetermined circumferential direction around the end effector122, and the other is fixed to a predetermined portion of the end effector122.

The second imaging device132is fixed to a base portion122a, for example. The base portion122aincludes a hole portion122frecessed toward the proximal end side (−Z side) from a distal end side (+Z side) surface of the base portion122a. The hole portion122fis, for example, a circular hole centered on the central axis CL.

A portion on the proximal end side (−Z side) of the second imaging device132is fitted and held in the hole portion122fIn the present embodiment, the hole portion122fcorresponds to the second holding portion that holds the second imaging device132. That is, in the present embodiment, the end effector122has the hole portion122fas the second holding portion. In the present embodiment, the second imaging device132is held not to be movable by the hole portion122fserving as the second holding portion. A portion on the distal end side (+Z side) of the second imaging device132protrudes to the distal end side from a center of a surface on the distal end side in the base portion122a. The base portion122ahas the same configuration as the base portion22aof the first embodiment described above except that the hole portion122fis provided.

An optical axis AX2aof the second imaging device132, for example, is parallel to the optical axis AX1of the first imaging device31and matches the central axis CL. The second imaging device132includes a cylindrical housing132a, a lens132efitted in an opening on a distal end side (+Z side) of the housing132a, and an image sensor132fdisposed inside the housing132a. Each unit of the second imaging device132can be the same as each unit of the second imaging device32of the first embodiment described above. The second imaging device132does not includes the second drive unit32band the second position acquisition unit32c, unlike the second imaging device32of the first embodiment.

In the present embodiment, the baseline length L between the first imaging device31and the second imaging device132is fixed. In the present embodiment, the distance information acquisition unit44acquires the information on the distance to the target object W on the basis of the baseline length L and the two images acquired from the first imaging device31and the second imaging device132, similarly to the first embodiment described above.

Further, in the present embodiment, the distance information acquisition unit44can acquire the information on the distance to the target object W on the basis of two images captured by the first imaging device31. Specifically, for example, when the first imaging device31is located at a first position P1indicated by a solid line inFIG.7, a first image is acquired by the first imaging device31, and when the first imaging device31is located at a second position P2indicated by a two-dot chain line inFIG.7, a second image is acquired by the first imaging device31. In the present embodiment, the distance information acquisition unit44can also acquire the information on the distance to the target object W on the basis of the first image and the second image that have been obtained in this way. More specifically, the distance information acquisition unit44can acquires the information on the distance to the target object W on the basis of the first image and the second image acquired by the first imaging device31, and a baseline length La, which is a distance between the first imaging device31at the first position P1and the first imaging device31at the second position P2.

The baseline length La is a distance between the optical axis AX1aof the first imaging device31at the first position P1and the optical axis AX1bof the first imaging device31at the second position P2. The baseline length La is determined by the first position P1and the second position P2. In the present embodiment, the control unit40can change the baseline length La by changing the first position P1and the second position P2at which the first imaging device31acquires an image. The control unit40changes the baseline length La, for example, depending on work content of the robot system110.

In the present embodiment, the position acquisition unit134acquires position information on the first position P1and the second position P2. The position information on the first position P1and the second position P2includes, for example, information on the first position P1, information on the second position P2, and information indicating a relative positional relationship between the first position P1and the second position P2. In the present embodiment, the position acquisition unit134is configured of only the first position acquisition unit31cof the first imaging device31. The position acquisition unit134acquires the position information of the first position P1and the second position P2on the basis of the rotational position of the drive unit133, for example. In the present embodiment, the drive unit133is configured of only the first drive unit31bof the first imaging device31. The distance information acquisition unit44acquires the baseline length La on the basis of the position information on the first position P1and the second position P2acquired by the position acquisition unit134.

The distance information acquisition unit44, for example, can selectively acquire the information on the distance to the target object W using images acquired by the first imaging device31and the second imaging device132or acquire the information on the distance to the target object W using two images acquired at the first position P1and the second position P2different from each other by the first imaging device31, depending on the work content of the robot system110.

When the information on the distance to the target object W is acquired using the images acquired by the first imaging device31and the second imaging device132, the distance information acquisition unit44rotates the at least one of the image acquired by the first imaging device31and the image acquired by the second imaging device132to adjust the direction of the acquired image as in the first embodiment.

On the other hand, when the information on the distance to the target object W is acquired by using the two images acquired at the first position P1and the second position P2different from each other by the first imaging device31, the distance information acquisition unit44rotates the at least one of the first image acquired at the first position P1by the first imaging device31and the second image acquired at the second position P2by the first imaging device31to adjust the direction of the acquired image. The distance information acquisition unit44may rotate only the first image acquired at the first position P1to align the direction of the first image with the direction of the second image, may rotate only the second image acquired at the second position P2to align the direction of the second image with the direction of the second image, or may rotate both the first image acquired at the first position P1and the second image acquired at the second position P2to adjust the direction of the first image and the direction of the second image. Other configurations of the robot system110of the present embodiment can be the same as those of the robot systems of the above-described embodiments.

According to the present embodiment, the first imaging device31is movable in the predetermined circumferential direction around the end effector122, and the second imaging device132is fixed to a predetermined portion of the end effector122. Therefore, it is easy to simplify a structure of the robot system110as compared with a case in which both the first imaging device31and the second imaging device132are provided to be movable together.

Further, according to the present embodiment, the first imaging device31captures the first image of the target object W at the first position P1and the second image of the target object W at the second position P2different from the first position P1. The distance information acquisition unit44acquires the information on the distance to the target object W on the basis of the first image and the second image. Therefore, the information on the distance to the target object W can be acquired using only the image acquired by one first imaging device31. Accordingly, even when the second imaging device132is not provided, the information on the distance to the target object W can be acquired only by the one first imaging device31. Further, it is possible to change the baseline length La by changing the relative position between the first position P1and the second position P2.

Further, according to the present embodiment, the distance information acquisition unit44rotates the at least one of the first image acquired at the first position P1by the first imaging device31and the second image acquired at the second position P2by the first imaging device31to adjust the direction of the acquired image. Therefore, even when the image sensor31fof the first imaging device31at the first position P1and the image sensor132fof the second imaging device132at the second position P2are disposed in different postures, it is possible to align the direction of the first image acquired at the first position P1with the direction of the second image acquired at the second position P2. This makes it possible to preferably acquire the information on the distance to the target object W on the basis of the image acquired by the first imaging device31using the distance information acquisition unit44regardless of the first position P1and the second position P2.

Further, according to the present embodiment, the control unit40can change the baseline length La, which is the distance between the first imaging device31at the first position P1and the first imaging device31at the second position P2, and the distance information acquisition unit44acquires the information on the distance to the target object W on the basis of the baseline length La. Therefore, using only the first imaging device31, it is possible to acquire the information on the distance to the target object W at different baseline lengths La.

Further, according to the present embodiment, the position acquisition unit134acquires the position information on the first position P1and the second position P2, and the distance information acquisition unit44acquires the baseline length La on the basis of the position information on the first position P1and the second position P2acquired by the position acquisition unit134. Therefore, the distance information acquisition unit44can preferably acquire the information on the distance to the target object W on the basis of the acquired baseline length La while preferably acquiring the baseline length La.

Further, according to the present embodiment, the control unit40changes the baseline length La depending on work content of the robot system110. Therefore, it is possible to preferably change the baseline length La, which is the distance between the first imaging device31at the first position P1and the first imaging device31at the second position P2, depending on work content of the robot system110. This makes it possible to preferably perform each work with the robot system110.

Further, according to the present embodiment, the distance information acquisition unit44rotates the at least one of the first image acquired by the first imaging device31and the second image acquired by the second imaging device32to adjust the direction of the acquired image. Therefore, even when the image sensor31fof the first imaging device31and the image sensor32fof the second imaging device32are disposed in different postures, it is possible to align the direction of the first image acquired by the first imaging device31with the direction of the second image acquired by the second imaging device32. This makes it possible to preferably acquire the information on the distance to the target object W using the distance information acquisition unit44on the basis of the images acquired by each imaging device30regardless of a relative position and relative posture of the first imaging device31and the second imaging device32. Therefore, even when the at least one of the first imaging device31and the second imaging device32is moved so that the first imaging device31and the second imaging device32are at arbitrary positions and postures, it is possible to preferably acquire the information on the distance to the target object W.

In the present embodiment, the first imaging device31may be movable in the radial direction. In this case, a position in the radial direction of the first imaging device31is changed, making it possible to change the baseline length, which is the distance between the first imaging device31and the second imaging device132. Further, in the present embodiment, the second imaging device132may not be provided. In this case, the information on the distance to the target object W can be acquired by using only the first imaging device31, as described above.

Further, in the present embodiment, the first imaging device31may operate so that the long sides of the image sensor31fof the first imaging device31at the first position P1and the image sensor31fof the first imaging device31at the second position P2are parallel to each other. This makes it possible to align the directions of the images captured at the first position P1and the second position P2by the first imaging device31without performing processing such as rotation on the captured images. Therefore, the load of image processing in the control unit40can be reduced as compared with a case in which the processing such as rotation is performed on the acquired image. In this case, the first imaging device31is attached to be rotatable around the optical axis AX1, for example. For example, the control unit40rotates the first imaging device31around the optical axis AX1depending on the position in the circumferential direction of the first imaging device31, to perform adjustment so that the long sides of the image sensor31fare always in the same direction. Thus, in the present embodiment, the control unit40may rotate the image sensor31fof the first imaging device31to adjust a direction of the image acquired by the first imaging device31.

Further, when only one first imaging device31is movable relative to the member to which the first imaging device31is attached as in the present embodiment, the first imaging device31can be moved relative to the robot arm21or may be movable with respect to the adapter23. When the first imaging device31is movable with respect to the robot arm21, the first imaging device31may be movable in a predetermined circumferential direction around the robot arm21. When the first imaging device31is movable with respect to the adapter23, the first imaging device31may be movable in a predetermined circumferential direction around the adapter23. In these cases, the distance information acquisition unit44can acquire the information on the distance to the target object W on the basis of the first image and the second image captured at the first and second different positions P1and P2by the first imaging device31.

Third Embodiment

FIG.8is a view of a portion of the robot system210of the present embodiment viewed from the distal end side (+Z side) in the central axis direction. InFIG.8, illustration of the finger portion22bof the end effector22and the camera unit60is omitted. The same configurations as those in the above-described embodiment are appropriately denoted by the reference signs, for example, and description thereof will be omitted.

As illustrated inFIG.8, the robot system210of the present embodiment includes three or more imaging devices230that images a target object W. For the imaging devices230, for example, three imaging devices including an imaging device230a, an imaging device230b, and an imaging device230care provided. In the present embodiment, the three imaging devices230a,230b, and230care disposed side by side on a predetermined axis VA. The axis VA is, for example, an imaginary axis extending in a direction (a horizontal direction inFIG.8) perpendicular to both a central axis direction and a radial direction. The imaging device230a, the imaging device230b, and the imaging device230care disposed at equal intervals in an axial direction of the axis VA, for example. In the axial direction of axis VA, the imaging device230bis located between the imaging device230aand the imaging device230c. An optical axis AX3aof the imaging device230a, an optical axis AX3bof the imaging device230b, and an optical axis AX3cof the imaging device230c, for example, extend in the central axis direction and are parallel to each other.

An image sensor235aof the imaging device230a, an image sensor235bof the imaging device230b, and an image sensor235cof the imaging device230chave a rectangular shape when viewed in the central axis direction. In the present embodiment, the image sensor235a, the image sensor235b, and the image sensor235care disposed in the same posture. In the present embodiment, the three imaging devices230a,230b, and230care disposed such that long sides of the image sensors235a,235b, and235cin the three imaging devices230a,230b, and230care parallel to each other.

The robot system210includes a holding member230dthat holds the three imaging devices230a,230b, and230c, and a slider230ethat connects the holding member230dto a base portion22aof the end effector22. The three imaging devices230a,230b, and230care fixed to the holding member230dnot to move relative to each other. The slider230eis connected to the guide rail portion22eprovided on the base portion22aof the end effector22, like the sliders31dand32dof the first embodiment. The slider230eis movable in the circumferential direction around the base portion22aalong the guide rail portion22e.

Although not illustrated, the robot system210includes a driving unit that moves the slider230ein the circumferential direction. The slider230eis moved in the circumferential direction along the guide rail portion22eby the drive unit, such that the holding member230dand the three imaging devices230a,230b, and230cheld by the holding member230dmove in the circumferential direction. In the present embodiment, the three imaging devices230a,230b, and230cmove in the circumferential direction while the long sides of the image sensors235a,235b, and235cremain parallel to each other.

In the present embodiment, the control unit40acquires information on a distance to the target object W on the basis of information of images of the target object W acquired by the two imaging devices230among the three imaging devices230a,230b, and230c. The control unit40selects, for example, the two imaging devices230from among the three imaging devices230a,230b, and230c, and acquires the information on the distance to the target object W on the basis of the information on the images acquired by the two selected imaging devices230. As the two imaging devices230to be selected, there are three patterns of the imaging device230aand the imaging device230b, the imaging device230band the imaging device230c, and the imaging device230aand the imaging device230c.

A baseline length L3that is a distance between the imaging device230aand the imaging device230b, and a baseline length L4that is a distance between the imaging device230aand the imaging device230care different from each other. That is, the baseline lengths differ between a case in which the imaging device230aand the imaging device230bare selected as the two imaging devices230and a case in which the imaging device230aand the imaging device230care selected. Accordingly, in the present embodiment, the control unit40can change the two imaging devices230to be selected, to change the baseline length when acquiring the information on the distance to the target object W. The baseline length L3is, for example, smaller than the baseline length L4. The baseline length L3is a distance between the optical axis AX3aof the imaging device230aand the optical axis AX3bof the imaging device230b. The baseline length L4is a distance between the optical axis AX3aof the imaging device230aand the optical axis AX3cof the imaging device230c.

The baseline length that is a distance between the imaging device230band the imaging device230cis, for example, the same as the baseline length L3that is the distance between the imaging device230aand the imaging device230b. The baseline length that is the distance between the imaging device230band the imaging device230cis a distance between the optical axis AX3bof the imaging device230band the optical axis AX3cof the imaging device230c.

The control unit40, for example, changes the two imaging devices230to be selected, according to work content of the robot system210or the like to change the baseline length. The control unit40acquires the information on the distance to the target object W on the basis of the images acquired by the two selected imaging devices230and the baseline length between the two imaging devices230using the distance information acquisition unit44. The control unit40controls at least one of the robot arm21and the end effector22on the basis of the information on the distance to the target object W that has been acquired in this way.

In the present embodiment, the control unit40controls at least one of the robot arm21and the end effector22connected to the robot arm21on the basis of the information on the images acquired by the two imaging devices230among the three imaging devices230a,230b, and230c. In the present embodiment, the control unit40selects two imaging devices230from among the three imaging devices230a,230b, and230c, and controls at least one of the robot arm21and the end effector22on the basis of the information on the images acquired by the two selected imaging devices230.

Specifically, for example, when the target object W does not appear in at least one of the images captured by the two selected imaging devices230, the control unit40moves at least one of the robot arm21and the end effector22so that the target object W can be imaged by the two selected imaging devices230.

For example, when the target object W does not appear in at least one of the images captured by the two selected imaging devices230, the control unit40may move the three imaging devices230in the circumferential direction so that the target object W can be imaged by the two selected imaging devices230.

In the present embodiment, the control unit40performs control so that imaging of at least two imaging devices230among the three imaging devices230a,230b, and230cis synchronized. The control unit40performs control, for example, so that imaging of the at least two selected imaging devices230described above is synchronized.

According to the present embodiment, the control unit40acquires the information on the distance to the target object W on the basis of the information on the images of the target object W acquired by the two imaging devices230among the three imaging devices230a,230b, and230c. Therefore, images acquired by the two imaging devices230to be used is changed depending on the work content of the robot system210, the target object W, or the like, so that the information on the distance to the target object W can be preferably acquired. In the present embodiment, for example, the baseline length can be changed depending on whether images acquired by the two imaging devices230aand230bare used or whether images acquired by the two imaging devices230aand230care used. Therefore, the two imaging devices230to be appropriately selected are changed depending on the distance to the target object W or the like, so that the information on the distance to the target object W can be preferably acquired.

Further, according to the present embodiment, the control unit40selects two imaging devices230from the three imaging devices230a,230b, and230c, and acquires the information on the distance to the target object W on the basis of the information on the images acquired by the two selected imaging devices230. Therefore, when the information on the distance to the target object W is acquired, imaging may be performed by the two imaging devices230among the three imaging devices230a,230b, and230c, and imaging may not be performed by one remaining imaging device230. Therefore, a load on the control unit40at the time of acquisition of the information on the distance to the target object W can be reduced.

Further, according to the present embodiment, the control unit40controls at least one of the robot arm21and the end effector22connected to the robot arm21on the basis of the information on the images acquired by the two imaging devices230among the three imaging devices230a,230b, and230c. Therefore, it is possible to acquire information such as the position of the target object W and an environment in which the robot system210is disposed, from the images acquired by the two imaging devices230, and to preferably move the robot arm21and the end effector22depending on, for example, the work content of the robot system210.

Further, according to the present embodiment, the control unit40selects two imaging devices230from among the three imaging devices230a,230b, and230c, and controls at least one of the robot arm21and the end effector22connected to the robot arm21on the basis of the information on the images acquired by the two selected imaging devices230. Therefore, when at least one of the robot arm21and the end effector22is controlled on the basis of the information on the images acquired by the two imaging devices230, imaging may not be performed by the other imaging device230. This makes it possible to reduce a load on the control unit40when controlling the robot arm21and the end effector22.

Further, according to the present embodiment, the control unit40performs control to synchronize imaging in at least two imaging devices230among the three imaging devices230a,230b, and230c. Therefore, it is possible to preferably image the target object W at the same timing with the at least two imaging devices230. This makes it possible to preferably acquire the information on the distance to the target object W on the basis of the images obtained by the at least two imaging devices230.

Further, according to the present embodiment, the three imaging devices230a,230b, and230care disposed side by side on the predetermined axis VA. Therefore, as in the present embodiment, the imaging devices230a,230b, and230ccan be disposed side by side in a state in which postures of the image sensors235a,235b, and235care aligned. This makes it easy to acquire the information on the distance to the target object W, for example, without rotating the image acquired by the imaging device230even when the information on the distance to the target object W is acquired using the images acquired by any two imaging devices230among the three imaging devices230a,230b, and230c. Therefore, a load on the control unit40at the time of acquisition of the information on the distance to the target object W can be reduced.

Further, according to the present embodiment, the optical axes AX3a, AX3b, and AX3cof the three imaging devices230a,230b, and230care parallel to one another. Therefore, even when images acquired by any two imaging devices230among the three imaging devices230a,230b, and230care used, the information on the distance to the target object W can be preferably acquired from the two images.

Further, according to the present embodiment, the three imaging devices230a,230b, and230care disposed so that the long sides of the image sensors235a,235b, and235cof the three imaging devices230a,230b, and230care parallel to one another. Therefore, it is easy to acquire the information on the distance to the target object W from the image acquired by the imaging device230, for example, without rotating the acquired image. Therefore, a load on the control unit40at the time of acquisition of the information on the distance to the target object W can be reduced.

In the present embodiment, four or more imaging devices230may be disposed side by side on the predetermined axis VA. In three or more imaging devices230disposed side by side on the axis VA, a distance between two adjacent imaging devices230may be different.

Fourth Embodiment

FIG.9is a view of a portion of a robot system310of the present embodiment viewed from the distal end side (+Z side) in the central axis direction. InFIG.9, illustration of the finger portion22bof the end effector22and the camera unit60is omitted. The same configurations as those in the above-described embodiment are appropriately denoted by the reference signs, for example, and description thereof will be omitted.

As illustrated inFIG.9, the robot system310of the present embodiment includes a connection member336that connects the first imaging device331to the second imaging device332. The connection member336is, for example, a guide rail. The connection member336has a linearly extending groove336a. The groove336ais, for example, a groove recessed from a distal end side (+Z side) to a proximal end side (−Z side). The groove336ais open at both end portions in the direction in which the groove336aextends, for example.

The robot system310includes a first slider331gthat attaches the first imaging device331to the connection member336, and a second slider332gthat attaches the second imaging device332to the connection member336.

The first slider331gis fixed to the first imaging device331. The second slider332gis fixed to the second imaging device332. The first slider331gand the second slider332gare fitted in the groove336ato be movable in the direction in which the groove336aextends, for example. Thus, in the present embodiment, the first imaging device331and the second imaging device332are connected by the connection member336via the first slider331gand the second slider332g.

The first slider331gand the second slider332gare restricted from moving and rotating relative to the connection member336in directions other than a direction in which the groove336aextends. The first slider331gis attached to be movable in the circumferential direction and rotatable around the optical axis AX1cof the first imaging device331. The second slider332gis attached to be movable in the circumferential direction and rotatable around the optical axis AX2cof the second imaging device332.

The first imaging device331and the second imaging device332are movable in the circumferential direction around the base portion22aof the end effector22. In the present embodiment, the first imaging device331is rotatable around the optical axis AX1cof the first imaging device331together with the first slider331g. In the present embodiment, the second imaging device332is rotatable around the optical axis AX2cof the second imaging device332together with the second slider332g. A long side of the image sensor331fof the first imaging device331and a long side of the image sensor332fof the second imaging device332are, for example, disposed parallel to each other, and parallel to the direction in which the groove336aextends.

For example, when the first imaging device331and the second imaging device332move in the circumferential direction from a position indicated by a two-dot chain line to a position indicated by a solid line inFIG.9. The connection member336connecting the first imaging device331to the second imaging device332also moves depending on a position of the first imaging device331and a position of the second imaging device332. InFIG.9, for example, the connection member336is moving upward while rotating about an axis in the central axis direction.

The first imaging device331and the second imaging device332move relative to the connection member336in the direction in which the groove336aextends according to the position in the circumferential direction. A relative change in the position of the first imaging device331and the position of the second imaging device332in the direction in which the groove336aextends causes a change in a baseline length that is a distance between the first imaging device331and the second imaging device332. The baseline length between the first imaging device331and the second imaging device332is a distance between the optical axis AX1cof the first imaging device331and the optical axis AX2cof the second imaging device332. For example, when the first imaging device331and the second imaging device332move in the circumferential direction from the position indicated by a two-dot chain line to the position indicated by a solid line inFIG.9, the first imaging device331and the second imaging device332move in a direction in which the first imaging device331and the second imaging device332approach each other, and the baseline length becomes smaller.

Here, the first slider331gand the second slider332gare restricted from moving and rotating relative to the connection member336in directions other than the direction in which the groove336aextends. Therefore, the first slider331gand the second slider332grotate around the optical axes AX1cand AX2cof the respective imaging devices so that relative postures with respect to the connection member336is maintained, depending on the change in at least one of a position and a posture of the connection member336according to movement of the first imaging device331and the second imaging device332. Accordingly, the first imaging device331to which the first slider331gis fixed and the second imaging device332to which the second slider332gis fixed also rotate around the respective optical axes AX1cand AX2cso that relative postures with respect to the connection member336are maintained. Therefore, even when the first imaging device331and the second imaging device332move in the circumferential direction, relative postures of the image sensor331fof the first imaging device331and the image sensor332fof the second imaging device332can be maintained. That is, the long side of the image sensor331fof the first imaging device331and the long side of the image sensor332fof the second imaging device332are kept parallel regardless of the position of the first imaging device331and the position of the second imaging device332. Thus, in the present embodiment, the connection member336can hold the first imaging device331and the second imaging device332in a state in which the relative postures of the first imaging device331and the second imaging device332are maintained in predetermined postures. Other configurations of the robot system310can be the same as those of the robot system of each embodiment described above.

According to the present embodiment, as described above, at least one of the first imaging device331and the second imaging device332moves such that the long sides of the image sensor331fof the first imaging device331and the image sensor332fof the second imaging device332are parallel to each other. Therefore, it is possible to preferably acquire the information on the distance to the target object W on the basis of the image acquired by the first imaging device331and the image acquired by the second imaging device332without, for example, rotating the images acquired by the respective imaging devices.

Further, according to the present embodiment, the connection member336connecting the first imaging device331to the second imaging device332is provided, and the connection member336can hold the first imaging device331and the second imaging device332in a state in which the relative postures of the first imaging device331and the second imaging device332are maintained in the predetermined postures. Therefore, for example, even when a drive unit that rotates the first imaging device331and the second imaging device332around the respective optical axes AX1cand AX2cis not provided, the relative postures of the image sensor331fof the first imaging device331and the image sensor332fof the second imaging device332can be easily maintained in postures in which the long sides are parallel to each other by the connection member336. Accordingly, even when at least one of the first imaging device331and the second imaging device332is moved to change the baseline length, the information on the distance to the target object W can be acquired easily preferably on the basis of the images acquired by the first imaging device331and the second imaging device332.

In the present embodiment, for example, only the image sensor331fof the first imaging device331may be rotatable around the optical axis AX1c, or only the image sensor332fof the second imaging device332may be rotatable around the optical axis AX2c. In this case, at least one of the image sensor331fof the first imaging device331and the image sensor332fof the second imaging device332may be movable so that the long sides of the image sensor331fof the first imaging device331and the image sensor332fof the second imaging device332are parallel to each other.

Fifth Embodiment

FIG.10is a view of a portion of the robot system410of the present embodiment viewed from the distal end side (+Z side) in the central axis direction. InFIG.10, illustration of the finger portion22bof the end effector22and the camera unit is omitted.FIG.11is a diagram illustrating a portion of a procedure when the robot system410of the present embodiment acquires the information on the distance to the target object W. The same configurations as those in the above-described embodiment are appropriately denoted by the reference signs, for example, and description thereof will be omitted.

In the present embodiment, the three or more imaging devices430are provided. For example, 24 imaging devices430are provided. A plurality of imaging devices430are disposed side by side in the circumferential direction around the end effector22. That is, in the present embodiment, three or more imaging devices430are disposed side by side on a predetermined circumference. In the present embodiment, the predetermined circumference is a circumference around the central axis CL. The plurality of imaging devices430, for example, are disposed at regular intervals over a circumference in the circumferential direction. That is, in the present embodiment, the three or more imaging devices430are disposed at equal intervals on a predetermined circumference. The optical axes AX4of the three or more imaging devices430are parallel to each other.

A long side of the image sensor435of each imaging device430is perpendicular to the radial direction passing through the optical axis AX4of each imaging device430when viewed in the central axis direction. When the number of imaging devices430is N, the N imaging devices430are disposed in N-fold symmetry around the central axis CL. That is, in the present embodiment, the 24 imaging devices430are disposed in 24-fold symmetry around the central axis CL. The imaging device430is fixed to the end effector22, for example. More specifically, the imaging device430is fixed to, for example, the outer peripheral surface of the base portion22a. That is, the end effector22includes, for example, a holding portion that holds the three or more imaging devices430on the outer peripheral surface of the base portion22a.

In the present embodiment, the control unit40selects two images from three or more images acquired by the three or more imaging devices430, and acquire the information on the distance to the target object W on the basis of information on the two selected images. The control unit40, for example, selects two images from among 24 images acquired by the 24 imaging devices430on the basis of information on occlusion of the target object W. The information on occlusion of the target object W includes, for example, information on whether or not the target object W appears in the image, information on a shielding state of the target object W, and information on a proportion of a portion of the target object W appearing in the image. The control unit40, for example, selects two images in which the target object W most preferably appears from among the 24 acquired images. The control unit40acquires the information on the distance to the target object W on the basis of the two selected images.

Here, a case in which the control unit40selects an image F1acquired by the imaging device431and an image F2acquired by the imaging device432among the plurality of imaging devices430will be described as an example. The image sensor435aof the imaging device431and the image sensor435bof the imaging device432are disposed in different postures. In this case, the control unit40cuts out a portion of the two acquired images F1and F2along a rectangular frame Fs, as illustrated inFIG.11. Long sides of the rectangular frame Fs are parallel to a virtual line IL connecting the optical axis of the imaging device431to the optical axis of the imaging device432. The control unit40acquires the information on the distance to the target object W on the basis of a cut-out portion of the two images F1and F2.

In the present embodiment, the control unit40selects the two images from the three or more images acquired by the three or more imaging devices430, and controls at least one of the robot arm21and the end effector22connected to the robot arm21on the basis of the information on the two selected images. In the present embodiment, the control unit40performs control to synchronize imaging in the three or more imaging devices430. Other configurations of the robot system410can be the same as those of the robot system of each embodiment described above.

According to the present embodiment, the control unit40selects the two images from among the three or more images acquired by the three or more imaging devices430, and acquire the information on the distance to the target object W on the basis of the information on the two selected images. Therefore, two images that are most suitable for acquisition of the information on the distance to the target object W are selected from among the three or more images acquired by the three or more imaging devices430, and the information on the distance to the target object W can be acquired. This makes it possible to preferably acquire the information on the distance to the target object W depending on work content of the robot system410, an environment in which the robot system410is disposed, a position and posture of the robot arm21, and the like.

Further, according to the present embodiment, the control unit40selects two images from among three or more images acquired by the plurality of imaging devices430on the basis of the information on occlusion of the target object W. Therefore, even when at least a portion of the target object W cannot be imaged by some of the imaging devices430due to a shielding object or the like, two images in which the target object W is preferably imaged can be selected. Accordingly, even when the target object W is partially blocked by the shielding object or the like, it is easy to preferably acquire the information on the distance to the target object W.

Further, according to the present embodiment, the control unit40selects the two images from among the three or more images acquired by the three or more imaging devices430, and controls at least one of the robot arm21and the end effector22connected to the robot arm21on the basis of the information on the two selected images. Therefore, two images including optimal information are selected from among the plurality of images acquired by the respective imaging devices430to move the robot arm21and the end effector22, making it possible to preferably move the robot arm21and the end effector22.

Further, according to the present embodiment, the three or more imaging devices430are disposed side by side on the predetermined circumference. Therefore, it is possible to dispose a relatively large number of imaging devices430side by side around the base portion22aof the end effector22, for example, as in the present embodiment. This makes it difficult for the imaging devices430to protrude from the robot20as compared with a case in which the same number of imaging devices430are disposed in a straight line form. Therefore, even when a relatively large number of imaging devices430are attached to the robot20, the robot20can be easily moved. Further, the plurality of imaging devices430are disposed side by side along the circumference, making it easy to image the target object W from various angles using the plurality of imaging devices430. Therefore, it is easy to more preferably acquire the information on the distance to the target object W using the plurality of imaging devices430.

Further, according to the present embodiment, the three or more imaging devices430are disposed at regular intervals on the predetermined circumference. Therefore, it is difficult for, for example, the number of the imaging devices430capable of imaging the target object W to be different depending on a position and posture of the member (for example, the end effector22) to which the imaging devices430are attached, as compared with a case in which the imaging devices430are disposed at non-equidistant intervals. This makes it easy to acquire the information on the distance to the target object W using the imaging device430regardless of the position and posture of the member to which the imaging device430is attached.

In the present embodiment, the control unit40may select two images from the three or more images acquired by the plurality of imaging devices430on the basis of at least one of distance information related to the target object W obtained in advance, the information on occlusion of the target object W, focal length information of the imaging devices430, and information on shape change of the images obtained by the three or more imaging devices430.

The distance information related to the target object W obtained in advance includes, for example, a distance from the robot20to the target object W when the robot and the target object W are disposed at the initial position with the initial posture, a distance to a shielding object disposed near the target object W, and distances between a plurality of target objects W when the plurality of target objects W are disposed at the initial positions in the initial postures. The distance from the robot20to the target object W includes, for example, a distance from a certain portion of the robot arm21to the target object W, a distance from a certain portion of the end effector22to the target object W, and a distance from a certain portion of the adapter23to the target object W. The control unit40can select two images on the basis of the distance information related to the target object W obtained in advance, to select two images having a preferred baseline length depending on the position of the target object W and preferably acquire the information on the distance to the target object W.

The control unit40can select two images on the basis of the focal length information of the imaging device430to select two images having a preferred baseline length according to the focal length of the imaging devices430. Here, for example, when the zoom magnification of the imaging device430is relatively large and two images with a relatively large baseline length are selected, an overlapping portion of the two images (a range of the image of a feature portion that overlaps and appears) becomes smaller. Therefore, for example, when the zoom magnification of the imaging device430is relatively large, two images with a relatively small baseline length are selected so that the overlapping portion of the two images can be increased. This makes it possible to acquire the distance to the target object W more preferably on the basis of the two images.

FIGS.12A to12Care diagrams illustrating change in the overlapping portion of two images depending on the baseline length and the zoom magnification.FIG.12Ais a diagram illustrating an example of a case in which two images F1aand F2awith a relatively large baseline length are selected when the zoom magnification of the imaging device430is relatively small.FIG.12Bis a diagram illustrating an example of a case in which two images F1band F2bwith a relatively large baseline length are selected when the zoom magnification of the imaging device430is relatively large.FIG.12Cis a diagram illustrating an example of a case in which two images F1cand F2cwith a relatively small baseline length are selected when the zoom magnification of the imaging device430is relatively large.FIGS.12A to12Cillustrate a case in which the target object W is a tree T and a car V.

In the case illustrated inFIG.12A, that is, a case in which the zoom magnification of the imaging device430is relatively small, it is assumed that the entire tree T and the entire car V appear in the two images F1aand F2awhen the two images Fla and F2awith a relatively large baseline length are selected. In this case, an overlapping portion (a range of the image of a feature portion that overlaps and appears) OPa of the two images F1aand F2aincludes the entire tree T and the entire car V. Therefore, the distance to each target object W can be acquired in the entire tree T and the entire car V.

On the other hand, as illustrated inFIG.12B, when the two images F1band F2bare selected so that the zoom magnification is larger than that illustrated inFIG.12Aand the baseline length is the same as that illustrated inFIG.12A, a range reflected in the images F1band F2bbecomes narrower than the range reflected in each of the images F1aand F2ainFIG.12A, whereas since the baseline length remains relatively large, a deviation of a position at which the target object W appears in each of the images F1band F2bremains relatively large. Therefore, in each of the images F1band F2b, a portion of the target object W may be cut off, and an overlapping portion between the images F1band F2bmay become smaller, as illustrated inFIG.12B.

In the example ofFIG.12B, most of the image F1bincluding a left portion of the tree T is cut off, and a right portion of the car V is cut off in the image F2b. In this case, an overlapping portion OPb includes only the portion of the tree T and the portion of the car V. Therefore, even when the entire tree T or the entire car V appears in one of the images, the distance to the target object W cannot be acquired for portions not included in the overlapping portion OPb.

On the other hand, even when the zoom magnification is the same as that illustrated inFIG.12B, the two images F1cand F2cwith the baseline length smaller than that shown inFIG.12Bare selected as shown inFIG.12C, making it possible to reduce a deviation of a position at which the target object W appears in each of the images F1cand F2c. Thus, it is possible to increase a range in which the target object W appears in each of the images F1cand F2c, and to increase the overlapping portion OPc between the images F1cand F2c. In the example ofFIG.12C, the entire tree T and the entire car V appear in each of the images F1cand F2c. That is, the overlapping portion OPc includes the entire tree T and the entire car V, as inFIG.12A. Therefore, the distance to each target object W can be acquired in the entire tree T and the entire car V.

As described above, when the zoom magnification of the imaging device430is relatively large as illustrated inFIGS.12B and12C, the two images F1cand F2cwith a relatively small baseline length are selected as illustrated inFIG.12C, so that the overlapping portion OPc between the two images F1cand F2ccan be increased. This makes it possible to acquire the distance to the target object W more preferably on the basis of the two images F1cand F2c.

The information on the shape change of the images obtained by the three or more imaging devices430includes, for example, information on a difference in appearance of the target object W appearing in any two images of the three or more imaging devices430. A difference in appearance of the target object W in the images captured by the different imaging devices430varies in size depending on a shape of the target object W, a shadow caused by an illumination with which the target object W is irradiated, and the like. The information on the shape change of the images obtained by the three or more imaging devices430includes, for example, a matching degree of appearance of the target object W in the two images, shape information of the target object W, and information on the illumination with which the target object W is irradiated. The control unit40, for example, may perform matching on all combinations in which two images are selected from a plurality of images acquired by the respective imaging devices430, and acquire the matching degree of the appearance of the target object W appearing in the respective images as a parameter. Further, the information on the shape change of the image may be input to the control unit40in advance.

Here, when the baseline length between the two imaging devices430is larger, it is easy for the appearance of the target object W to be greatly different in the images acquired by the two imaging devices430. When the appearance of the target object W differs to some extent, it becomes difficult to perform matching between the two images. Therefore, in such a case, the baseline length is reduced so that it is possible to suppress a large difference in the appearance of the target object W and it is easy to perform matching between the two images. Therefore, the information on the distance to the target object W can be preferably acquired on the basis of the two images.

Therefore, the control unit40can select two images on the basis of the information on the shape change of the images obtained by the three or more imaging devices430to select two images that have a preferred baseline length based on the difference in appearance of the target object W for each image. That is, for example, when the target object W is a target object whose appearance is greatly different in the plurality of imaging devices430, two images with a smaller baseline length are selected so that the information on the distance to the target object W can be preferably acquired on the basis of the two selected images.

Further, in the present embodiment, the control unit40may select two imaging devices430from among the three or more imaging devices430on the basis of at least one of the distance information related to the target object W obtained in advance, the information on occlusion of the target object W, the focal length information of the imaging devices430, and information on shape change of the image obtained by the three imaging devices430. In this case, imaging can be performed only by the two selected imaging devices430and the information on the distance to the target object W can be selected. Therefore, the load on the control unit40can be reduced as compared with a case in which two images are selected from the images acquired by performing imaging using all the imaging devices430.

Further, in the present embodiment, when information such as a difference in appearance of the shape of the target object W and luminance of the target object W is known in advance, the control unit40may determine the imaging device430to be selected, on the basis of such information. The difference in appearance of the shape of the target object W and the luminance of the target object W, and the like may be acquired from the distance information related to the target object W obtained in advance.

Further, in the present embodiment, the three or more imaging devices430may be disposed at non-equidistant intervals on a predetermined circumference. Further, the three or more imaging devices430may be disposed side by side on a predetermined axis, like the imaging devices230a,230b, and230cof the third embodiment described above. In this case, the three or more imaging devices430may be disposed such that the long sides of the image sensors435of the three or more imaging devices430are parallel. The three or more imaging devices430may be disposed in a matrix form. Further, the three or more imaging devices430may be time of flight cameras (TOF cameras).

Further, in the above-described embodiment, the three or more imaging devices430are disposed around the end effector22, but the present invention is not limited thereto. The three or more imaging devices430may be disposed around any of the robot arm21, the end effector22connected to the robot arm21, and the adapter23for attaching the end effector22to the robot arm21. Even when the three or more imaging devices430are disposed around the robot arm21or around the adapter23, it is possible to obtain the same effects as those obtained when the three or more imaging devices430are disposed around the end effector22described above.

When the three or more imaging devices430are disposed around the robot arm21, the robot arm21may include a holding portion that holds the three or more imaging devices430that image the target object W. Further, when the three or more imaging devices430are disposed around the adapter23, the adapter23may include a holding portion that holds the three or more imaging devices430that image the target object W. Also in these cases, the control unit40may acquire the information on the distance to the target object W on the basis of information of images of the target object W acquired by the two imaging devices430of the three or more imaging devices430held by the robot arm21or the adapter23.

Further, the control unit40may perform, a plurality of times, work for acquiring the information on the distance to the target object W based on the images acquired by the two imaging devices430among the three or more imaging devices430, using two imaging devices430in different combination. In this case, the control unit40can collate the pieces of information on the distance to the target object W acquired through a plurality of works to acquire the information on the distance to the target object W more accurately.

In particular, when at least a portion of the target object W is blocked by a shielding object when viewed from at least some of the imaging devices430, a plurality of pieces of information obtained using the two imaging devices430in a plurality of sets of combinations can be used to acquire the information on the distance to the target object W while minimizing an influence of shielding by the shielding object.

Further, after the target object W is imaged by all the imaging devices430, the control unit40may move the end effector22by moving the robot arm21or the like, and image the target object W again using all the imaging devices430from other places. This makes it possible for the control unit40to acquire the image obtained by imaging the target object W from multiple angles. In this case, when the number of imaging devices430is relatively large, many images obtained by imaging the target object W from different angles can be acquired even when the end effector22is moved and the number of times of imaging is small. In this case, each acquired image may be associated with information such as the position and posture of the end effector22when the image has been captured, and the position and posture of the imaging device430that performs imaging.

As described above, when imaging of the target object W a plurality of times from different positions, the control unit40may control the robot system410through visual servo using the acquired images. In this case, the control unit40performs control, for example, to move the imaging device430to a position at which a target image of the target object W can be captured by the imaging device430. Here, when the image of the target object W captured by the imaging device430at the initial position is greatly different from the target image of the target object W, it may be difficult to correspond the images to each other, and to move the imaging device430to the position at which the target image of the target object W can be captured.

On the other hand, imaging of the target object W are performed from different positions a plurality of times to acquire images of the target object W imaged from different angles, making it easy to bring the image captured by the imaging device430closer to the target image with the images captured from the different angles as intermediate images. That is, it is easy to preferably move the imaging device430to the position at which the target image of the target object W can be captured. Further, when the intermediate image is associated with distance information from the target object W when the intermediate image has been captured, the control unit40may arrange the intermediate images in an order of imaging at positions far from the target object W, and bring the image captured by the imaging device430closer to the target image while passing the plurality of intermediate images in that order.

Further, in the present embodiment, the robot system410may include, for example, a plurality of general-purpose cameras capable of simply capturing an image of the target object W, in addition to the plurality of imaging devices430described above. In this case, the control unit40may construct a 3D model of the target object W using a plurality of images captured by the plurality of general-purpose cameras. By using the 3D model, for example, it is possible to further improve acquisition accuracy of the information on the distance to the target object W using the plurality of imaging devices430. In this case, as an example, 12 imaging devices430may be provided side by side at equal intervals in the circumferential direction, and two general-purpose cameras may be provided side by side in the circumferential direction between adjacent imaging devices430. In this case, for example, 24 general-purpose cameras are provided. As the general-purpose camera, for example, a relatively inexpensive camera that is used in smartphones can be used.

Further, in the present embodiment, the control unit40may perform simultaneous localization and mapping (SLMA). That is, the control unit40may simultaneously perform self-position estimation of the robot system410and creation of a map of an environment in which the robot system410is disposed. In this case, when a relatively large number of imaging devices430are provided, it is possible to easily acquire a relatively large amount of 3D point cloud data for the environment using the plurality of imaging devices430. Therefore, it is easy to create the map of the environment in which the robot system410is disposed.

Sixth Embodiment

FIG.13is a perspective view illustrating a robot system510of the present embodiment. The same configurations as those in the above-described embodiment are appropriately denoted by the reference signs, for example, and description thereof will be omitted.

As illustrated inFIG.13, in the robot system510of the present embodiment, the imaging device530includes a first imaging device531attached to the robot arm521and a second imaging device32attached to the end effector22. The first imaging device531is disposed, for example, around a fifth arm portion524e. The first imaging device531, for example, is connected to a guide rail portion521aprovided on the fifth arm portion524eaccording to the same structure as a structure in which the first imaging device31is connected to the guide rail portion22ein the first embodiment.

The guide rail portion521ahas an annular shape surrounding the fifth arm portion524e. The first imaging device531is movable in a predetermined circumferential direction around the robot arm521along the guide rail portion521a. The robot arm521has the same configuration as the robot arm21of the first embodiment except that the guide rail portion521ais provided. Other configurations of the robot system510of the present embodiment can be the same as those of the robot system of each embodiment described above.

According to the present embodiment, the first imaging device531and the second imaging device32are attached to different members and are movable relative to respective members to which the first imaging device531and the second imaging device32are attached. Therefore, it is possible to suppress the movement of the first imaging device531and the movement of the second imaging device32being hindered by the other imaging device, as compared with a case in which the two imaging devices are attached to the same member. This makes it possible to preferably move each of the first imaging device531and the second imaging device32relative to each of the members to which the first imaging device531and the second imaging device32are attached.

Seventh Embodiment

FIG.14is a perspective view illustrating a robot system610of the present embodiment. The same configurations as those in the above-described embodiment are appropriately denoted by the reference signs, for example, and description thereof will be omitted.

As illustrated inFIG.14, the robot system610of the present embodiment includes a projection device670that projects light SL. In the present embodiment, the projection device670is disposed around the robot arm621. The projection device670is disposed, for example, around a fifth arm portion624e. The projection device670is connected to a guide rail portion621aof the fifth arm portion624e. The guide rail portion621ahas the same configuration as the guide rail portion521aof the sixth embodiment, except that a projection device670is connected instead of the imaging device. In the present embodiment, the projection device670is movable in a predetermined circumferential direction around the robot arm621along the guide rail portion621a. The projection device670projects the light SL in a grid pattern onto the target object W, for example. A structure of the projection device670is not particularly limited as long as the projection device670can project the light SL.

In the present embodiment, the first imaging device31and the second imaging device32executes imaging to acquire images in a state in which the light SL is projected by the projection device670. In the present embodiment, the control unit40controls the projection device670, the first imaging device31, and the second imaging device32so that the light SL projected onto the target object W by the projection device670can be imaged by the first imaging device31and the second imaging device32. The robot arm621has the same configuration as the robot arm21of the first embodiment except that the guide rail portion621ais provided. Other configurations of the robot system610can be the same as those of the robot system of each embodiment described above.

According to the present embodiment, the first imaging device31and the second imaging device32executes imaging to acquire images in a state in which the light SL is projected by the projection device670. Therefore, the target object W onto which the light SL is projected can be imaged by the first imaging device31and the second imaging device32. This makes it possible to acquire the image of the target object W more preferably using the first imaging device31and the second imaging device32. Further, the light SL projected from the projection device670is light with a pattern such as a grid pattern, making it possible to also measure, for example, a three-dimensional shape of the target object W on the basis of the pattern appearing in the images acquired by the first imaging device31and the second imaging device32.

In the present embodiment, only the first imaging device31between the first imaging device31and the second imaging device32may execute imaging to acquire an image in a state in which the light SL is projected by the projection device670. Further, only the second imaging device32between the first imaging device31and the second imaging device32may execute imaging to acquire an image in a state in which the light SL is projected by the projection device670. Further, in the present embodiment, the robot system610may include only the first imaging device31between the first imaging device31and the second imaging device32. In this case, the first imaging device31may execute imaging to acquire an image in a state in which the light SL is projected by the projection device670, and acquire the information on the distance to the target object W using the same method as a method of acquiring the information on the distance to the target object W using one first imaging device31described in the second embodiment.

Further, in the present embodiment, the projection device670may be disposed around the end effector22or may be disposed around the adapter23. Which portion among the robot arm621, the end effector22, and the adapter23around which the projection device670is disposed can be determined appropriately depends on positions at which the first imaging device31and the second imaging device32are attached, the work content of the robot system610, and the like. The projection device670is disposed around the portion of the robot arm621, the end effector22, or the adapter23, making it difficult for the light SL projected from the projection device670to be blocked by the portion of the robot system610, and easy for the light SL projected from the projection device670to be preferably projected onto the target object W.

Further, the projection device670may be fixed not to move relative to a member to which the projection device670is attached. Further, a plurality of projection devices670may be provided. In this case, the plurality of projection devices670may be attached to different members.

Eighth Embodiment

FIG.15is a perspective view illustrating a robot system710of the present embodiment. The same configurations as those in the above-described embodiment are appropriately denoted by the reference signs, for example, and description thereof will be omitted.

As illustrated inFIG.15, in the present embodiment, the adapter723includes an annular guide rail portion723hin the circumferential direction. The adapter723has the same configuration as the adapter23of the first embodiment except that the adapter723includes the guide rail portion723h. In the present embodiment, the end effector722has the same configuration as the end effector22of the first embodiment except that the guide rail portion22eis not included, and the first imaging device731and the second imaging device732are not attached.

The first imaging device731and the second imaging device732are attached to the adapter723in the present embodiment. The first imaging device731and the second imaging device732are disposed around the adapter723. The first imaging device731is connected to the guide rail portion723hvia the slider731d. The second imaging device732is connected to the guide rail portion723hvia the slider732d. That is, in the present embodiment, the adapter723includes the guide rail portion723hserving as a first holding portion that holds the first imaging device731and a second holding portion that holds the second imaging device732.

In the present embodiment, the sliders731dand732dextend radially outward from the guide rail portion723hand protrude radially outward relative to the end effector722. Accordingly, the first imaging device731and the second imaging device732provided at the radially outer end portions of the sliders731dand732dare located radially outward relative to the end effector722.

In the present embodiment, at least one of the first imaging device731and the second imaging device732is movable with respect to the adapter723. The at least one of the first imaging device731and the second imaging device732is movable in a predetermined circumferential direction around the adapter723. In the present embodiment, both the first imaging device731and the second imaging device732are movable with respect to the adapter723and movable in the predetermined circumferential direction around the adapter723. That is, in the present embodiment, the first imaging device731and the second imaging device732are held to be movable by the guide rail portion723hserving as the first holding portion and the second holding portion. The relative positions of the first imaging device731and the second imaging device732are variable. Other configurations of the robot system710can be the same as those of the robot system of each embodiment described above.

According to the present embodiment, with the first imaging device731and the second imaging device732attached to the adapter723, it is possible to obtain the same effects as those obtained by the first imaging device31and the second imaging device32attached to the end effector22in the first embodiment.

In the present embodiment, the adapter723may have the first holding portion that holds the first imaging device731not to be movable. In this case, the adapter723may include the guide rail portion723hserving as the second holding portion that holds the second imaging device732to be movable. Further, the adapter723may include the second holding portion that holds the second imaging device732not to be movable. In this case, the adapter723may include the guide rail portion723hserving as the first holding portion that holds the first imaging device731to be movable. Thus, in the present embodiment, one of the first imaging device731and the second imaging device732is movable in the predetermined circumferential direction around the adapter723, and the other may be fixed to a predetermined portion of the adapter723.

Ninth Embodiment

FIG.16is a perspective view illustrating a robot system810of the present embodiment. The same configurations as those in the above-described embodiment are appropriately denoted by the reference signs, for example, and description thereof will be omitted.

As illustrated inFIG.16, the first imaging device831and the second imaging device832are attached to the robot arm521in the present embodiment. The first imaging device831and the second imaging device832are disposed around the robot arm521. More specifically, the first imaging device831and the second imaging device832are disposed around the fifth arm portion524e. The first imaging device831and the second imaging device832, for example, are connected to the guide rail portion521aaccording to the same structure as the structure in which the first imaging device31and the second imaging device32are connected in the first embodiment. That is, in the present embodiment, the robot arm521includes the guide rail portion521aserving as a first holding portion that holds the first imaging device831and a second holding portion that holds the second imaging device832.

In the present embodiment, at least one of the first imaging device831and the second imaging device832is movable with respect to the robot arm521. The at least one of the first imaging device831and the second imaging device832is movable in a predetermined circumferential direction around the robot arm521. In the present embodiment, both the first imaging device831and the second imaging device832are movable with respect to the robot arm521and is movable in the predetermined circumferential direction around the robot arm521. That is, the first imaging device831and the second imaging device832are held to be movable by the guide rail portion521aas the first holding portion and the second holding portion. The relative positions of the first imaging device831and the second imaging device832are variable. Other configurations of the robot system810can be the same as those of the robot system of each embodiment described above.

According to the present embodiment, with the first imaging device831and the second imaging device832attached to the robot arm521, it is possible to obtain the same effects as those obtained by the first imaging device31and the second imaging device32attached to the end effector22in the first embodiment.

In the present embodiment, the robot arm521may include a first holding portion that holds the first imaging device831not to be movable. In this case, the robot arm521may include the guide rail portion521aas a second holding portion that movably holds the second imaging device832.

Further, the robot arm521may have the second holding portion that holds the second imaging device832not to be movable. In this case, the robot arm521may include the guide rail portion521aas a first holding portion that movably holds the first imaging device831. Thus, in the present embodiment, one of the first imaging device831and the second imaging device832is movable in a predetermined circumferential direction around the robot arm521, and the other may be fixed to a predetermined portion of the robot arm521.

Although the embodiments of the present invention have been described above in detail with reference to the drawings, a specific configuration is not limited to these embodiments, and changes can be made appropriately without departing from the scope of the present invention.

When the imaging device can move relative to the member to which the imaging device is attached, any method may be used to calibrate the position of the imaging device. For example, a panel or the like with a specific mark is disposed at a specific distance with respect to the imaging device and the mark is imaged so that the position of the imaging device may be calibrated. Further, When the imaging device can move relative to the member to which the imaging device is attached, the imaging device may be movable only between a plurality of predetermined locations. In this case, positions to which the imaging device can move may be structurally determined. In this case, it is possible to structurally ascertain the position of the imaging device.

When the imaging device can move relative to the member to which the imaging device is attached, the imaging device may move relative to the member in any way. The imaging device may move linearly with respect to the member, or may move in a curved line other than an arc. When the plurality of imaging devices can move relative to the member to which the plurality of imaging device are attached, each imaging device may move along different movement paths. There is no particular limitation on a structure of the drive unit that relatively moves the imaging device with respect to the member to which the imaging device is attached.

When a plurality of imaging devices are provided, the plurality of imaging devices may include different types of imaging devices. The plurality of imaging devices may include, for example, an infrared camera and an RGB camera. In this case, the target object may be imaged from the same position by the infrared camera and the RGB camera and images may be acquired.

The robot system may include an external sensor capable of detecting at least one of a position, posture, shape, and the like of the robot. The external sensor may be disposed on a ceiling of a place at which the robot is disposed, or may be disposed on a floor of the place at which the robot is disposed. The external sensor may be, for example, a sensor capable of detecting the position and posture of the robot arm, may be a sensor capable of detecting the position and posture of the end effector, or may be a sensor capable of detecting the position and posture of the adapter.

The external sensor may be, for example, a laser tracker. In this case, the external sensor may detect, for example, position information of each portion on the basis of a distance measurement result using a time-of-flight (TOF) method based on a difference between an irradiation timing at which light is radiated and a light reception timing at which reflected light is received. Further, the external sensor may detect the position information of each portion by obtaining a geometrical positional relationship through triangulation on the basis of a measurement result of a reflection position of reflected light generated due to radiation of light on a plurality of optical paths. In this case, the external sensor may include a variable focus lens (for example, a zoom lens) in an optical system of a light reception unit that receives reflected light, in order to improve measurement accuracy of a reflection position of the reflected light. Further, for position detection of each portion in the external sensor, a distance measurement scheme using an optical comb based on extremely short-time light pulses may be used.

The external sensor may be able to detect the position and posture of the imaging device. In this case, when the imaging device is movable relative to the member to which the imaging device is attached, the control unit may move the imaging device on the basis of information on the imaging device obtained by the external sensor. Further, in this case, the imaging device may be provided with a marker that can be detected by the external sensor.

The external sensor may be an imaging device with a variable baseline length. In this case, the control unit may change the baseline length of the external sensor, for example, depending on a distance between the external sensor and the target object on which work is performed by the end effector. As an example, the control unit may reduce the baseline length of the external sensor when the target object gripped by the end effector is brought closer to the external sensor by moving the robot arm. The baseline length of the external sensor can be changed, for example, by using the same method as a method of changing the baseline length appropriately described in each of the above-described embodiments.

Use of the robot system described above is not particularly limited. The respective configurations and the respective methods described above can be appropriately combined unless these are mutually inconsistent.

REFERENCE SIGNS LIST