Patent ID: 12233556

MODE FOR CARRYING OUT THE INVENTION

Modes for carrying out the present technology will be described below. The description will be given in the following order.1. Failure state transition2. Example configuration of robot3. Operation of robot4. Examples of updating failure state transition5. Examples of recovery6. Modified examples
<Failure State Transition>

FIG.1is a diagram showing an example of an outline of a robot according to an embodiment of the present technology.

A robot1shown in A ofFIG.1is an arm-type robot. The robot1includes a base portion11, an arm portion12extending from the base portion11, and a hand portion13attached to an end of the arm portion12. The robot1is provided in a factory or the like and performs a predetermined task such as grasping and moving an object.

Instead of the robot1, a robot2that is a human-type robot capable of bipedal walking as shown in B ofFIG.1may be used. The robot2also includes an arm portion and the like and can grasp and move an object.

The robot1and the robot2shown inFIG.1take autonomous actions by executing a predetermined program by a built-in computer and driving individual parts.

Processes performed in the robot1, which is an arm-type robot, will be described below as appropriate. Processes similar to the processes performed in the robot1are performed by the robot2or a robot having another shape such as a robot capable of quadrupedal walking.

The robot1monitors its own state at each timing in a case of performing tasks such as grasping and moving an object. In a case where the transition of the state of the robot1follows the same transition as a failure state transition, which is a state transition that results in failure of a task, the robot1performs a predetermined action such as alerting a user before reaching the failure state.

The robot1has failure state transition information, which is information indicating a failure state transition for each task. The failure state transition information is generated in advance at a predetermined timing and prepared for the robot1.

Here, states include operations and events. An operation indicates a matter that is actively performed by the robot1. Operations include matters such as “moving an arm” and “recognizing”. Furthermore, an event indicates a passive matter detected in the robot1. Events include matters such as “losing sight”. Herein, it is determined whether or not the state of the robot1corresponds to each state, instead of representing each state by a value.

Note that states include recognition results such as a result of recognizing an image, a result of detecting an unusual noise, and a result of detecting a smell. Furthermore, states also include recognition results such as the shape, slipperiness, and softness of an object identified by grasping the object.

FIG.2is a diagram showing an example of a failure state transition.

The failure state transition shown inFIG.2represents a state transition that results in failure of a task of “grasping an object”. The task of “grasping an object” is, for example, a task of recognizing, grasping, and lifting an object placed near the robot1.

In performing the task of “grasping an object”, the state of the robot1first becomes a state #1, which is a state where the arm portion12is raised, and then becomes a state #2, which is a state where the object to be grasped is recognized. For example, the object to be grasped is recognized on the basis of an image captured by a camera or sensor data detected by a sensor.

After the state #2, which is a state where the object to be grasped is recognized, is reached, it is determined whether or not to move the arm portion12.

In a case where it is determined to move the arm portion12, the state of the robot1becomes a state #3, which is a state where the arm portion12is moved. In the case where the state #3 is reached, the robot1moves the arm portion12by driving individual parts. In a case where the arm portion12is not moved, it is regarded that the task will not result in failure.

After the state #3, which is a state where the arm portion12is moved, is reached, it is determined whether or not the object to be grasped is lost sight of.

In a case where it is determined that the object to be grasped is lost sight of, the state of the robot1becomes a state #4, which is a state where the object to be grasped is lost sight of.

In the case where the state #4 is reached, an action of alert is performed as indicated by an arrow A1at its head. The robot1drives a speaker or the like to alert the user that the object to be grasped is lost sight of by means of sound. In a case where the object to be grasped is not lost sight of, it is regarded that the task will not result in failure.

After the state #4, which is a state where the object to be grasped is lost sight of, is reached and the action of alert is performed, it is determined whether or not to further move the arm portion12.

In a case where it is determined to further move the arm portion12, the state of the robot1becomes a state #5, which is a state where the arm portion12is moved.

In the case where the state #5 is reached, it is determined that a pre-failure state, which is a state before the task results in failure, is reached, and an action of emergency stop is performed as indicated by an arrow A2at its head. The robot1stops driving individual parts and does not move the arm portion12. As the arm portion12is not moved, the task will not result in failure.

In this manner, the state transition including the states #1 to #5 is set as the failure state transition for the case of performing the task of “grasping an object”. In a case where the state transition of the robot1follows the same transition as the failure state transition, the action of alert or emergency stop is performed.

In a case where the failure state transition is followed and the arm portion12is to be moved even after the object to be grasped is lost sight of and the alert is performed, by becoming the pre-failure state and causing an emergency stop of the operation, the robot1can prevent the failure of the task and prevent the arm portion12or the like from hitting surrounding objects.

FIG.3is a diagram showing another example of a failure state transition.

The failure state transition shown inFIG.3represents a state transition that results in failure of a task of “passing an object to a person”. The task of “passing an object to a person” is, for example, a task of grasping an object placed near the robot1and presenting it in front of a nearby person and releasing the grasp to pass it to the person.

In performing the task of “passing an object to a person”, the state of the robot1first becomes a state #11, which is a state where the object is held by the hand portion13, and then becomes a state #12, which is a state where the hand portion13is placed in front of the person. The operation of holding the object is realized by recognizing the target object on the basis of an output of a sensor such as a camera and driving the arm portion12and the hand portion13.

After the state #12, which is a state where the hand portion13grasping the object is placed in front of the person, it is determined whether the person's hand is not touching the object. Touching the object by the person's hand is recognized on the basis of an image captured by a camera or sensor data detected by a sensor, for example.

In a case where it is determined that the person's hand is not touching the object, the state of the robot1becomes a state #13, which is a state where the person's hand is not touching the object.

In the case where the state #13 is reached, an action of alert is performed as indicated by an arrow A11at its head. The robot1drives a speaker or the like to alert the user that the hand is not touching the object by means of sound. In a case where the person's hand is touching the object, it is regarded that the task will not result in failure.

After the state #13, which is a state where the person's hand is not touching the object, is reached and the action of alert is performed, it is determined whether the person is not looking at the object presented by the hand portion13. For example, whether or not the object is being looked at is determined on the basis of the line-of-sight direction of the person identified by analyzing an image captured by a camera.

In a case where it is determined that the person is not looking at the object, the state of the robot1becomes a state #14, which is a state where the person is not looking at the object.

In the case where the state #14 is reached, an action of alert is performed as indicated by an arrow A12at its head. The robot1drives a speaker or the like to alert the user to concentrate on looking at the object by means of sound. In a case where the person is looking at the object, it is regarded that the task will not result in failure.

After the state #14, which is a state where the person is not looking at the object, is reached and the action of alert is performed, it is determined whether or not to reduce the force of the hand portion13grasping the object.

In a case where it is determined to reduce the force of the hand portion13grasping the object, the state of the robot1becomes the state #15, which is a state where the force of the hand portion13grasping the object is reduced. In a case where the force of the hand portion13grasping the object is not reduced, it is regarded that the task will not result in failure.

In the case where the state #15 is reached, it is determined that the pre-failure state is reached, and an action of grasping again is performed as indicated by an arrow A13at its head. The robot1drives individual parts to increase the force of the hand portion13to grasp the object again.

In this manner, the state transition including the states #11 to #15 is set as the failure state transition for the case of performing the task of “passing an object to a person”. In a case where the state transition of the robot1follows the same transition as the failure state transition, the action of alert or emergency stop is performed.

In a case where the failure state transition is followed and the force of the hand portion13is to be reduced even after the alert to look at the object is performed, by becoming the pre-failure state and performing an operation for grasping the object again, the robot1can prevent the failure of the task and prevent the lifted object from being dropped.

In this manner, a failure state transition is set for each task, and information indicating such a transition is prepared for the robot1.

The robot1determines that the task will fail in a case where its own state transition follows the same transition as the failure state transition, and performs optimum operation according to the task before actually resulting in failure, so that the failure can be prevented.

Processes of the robot1for preventing the failure as described above will be described later with reference to flow charts.

<Example Configuration of Robot>

FIG.4is a block diagram showing an example hardware configuration of the robot1.

As shown inFIG.4, the robot1is constituted by connecting an input/output section32, a drive section33, a wireless communication section34, and a power supply section35to a control section31.

The control section31includes a computer including a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a flash memory, and the like. The control section31executes a predetermined program by the CPU to control the overall operation of the robot1. The computer constituting the control section31functions as a control device that controls the operation of the robot1.

For example, the control section31monitors its own state on the basis of information supplied from the input/output section32and information supplied from each drive unit of the drive section33.

Furthermore, in a case of performing a predetermined task, the control section31determines whether or not its own state transition follows the same transition as a failure state transition on the basis of failure state transition information for the task. In a case where it is determined that its own state transition follows the same transition as the failure state transition, the control section31performs a predetermined action before actually resulting in failure.

The input/output section32includes a camera41, a mic (microphone)42, a speaker43, a touch sensor44, and a light emitting diode (LED)45.

The camera41sequentially captures images of the surrounding environment. The camera41outputs data of captured images, which are still or moving images obtained by the image capture, to the control section31.

The mic42detects environmental sounds. The mic42outputs data of the environmental sounds to the control section31.

The speaker43outputs predetermined sounds such as speech sounds and BGM.

The touch sensor44is provided at a predetermined portion such as the base portion11. The touch sensor44detects a touch by the user and outputs information indicating the content of an operation by the user to the control section31.

The LED45emits light according to control by the control section31to present information to the user. Instead of the LED45, a small display such as an LCD or an organic EL display may be provided.

The input/output section32is provided with various modules such as a distance measurement sensor that measures the distance to a surrounding object and a position measurement sensor such as a global positioning system (GPS).

The drive section33is driven according to control by the control section31to realize actions of the robot1. The drive section33includes a plurality of drive units provided for respective joint axes such as roll, pitch, and yaw.

Each drive unit is provided to a respective joint of the robot1, for example. Each drive unit includes a combination of a motor that rotates about an axis, an encoder that detects the rotational position of the motor, and a driver that adaptively controls the rotational position and rotational speed of the motor on the basis of an output of the encoder. The hardware configuration of the robot1is determined by the number of drive units, the positions of the drive units, and the like.

In the example ofFIG.4, drive units51-1to51-nare provided as the drive units. For example, the drive unit51-1includes a motor61-1, an encoder62-1, and a driver63-1. The drive units51-2to51-nhave configurations similar to that of the drive unit51-1.

The wireless communication section34is a wireless communication module such as a wireless LAN module, a mobile communication module capable of Long Term Evolution (LTE). The wireless communication section34communicates with an external device such as a server on the Internet. The wireless communication section34sends data supplied from the control section31to the external device and receives data sent from the external device.

The power supply section35supplies electric power to individual parts in the robot1. The power supply section35includes a charge battery71and a charge/discharge control section72that manages the charge/discharge state of the charge battery71.

FIG.5is a block diagram showing an example functional configuration of the control section31.

As shown inFIG.5, the control section31includes an operation output section101, an operation acquisition section102, a failure state transition determination section103, a failure state transition storage section104, a failure state transition generation section105, and a drive control section106. At least part of the functional sections shown inFIG.5is realized by the execution of a predetermined program by the CPU constituting the control section31.

The operation output section101receives information supplied from each drive unit of the drive section33and outputs it to the operation acquisition section102.

The operation acquisition section102detects the content of an operation of the robot1on the basis of the information supplied from the operation output section101. The operation acquisition section102outputs information indicating the content of the operation to the failure state transition determination section103.

In a case of performing a predetermined task, the failure state transition determination section103reads and acquires information indicating the failure state transition for the task to be performed from the failure state transition storage section104.

The failure state transition determination section103identifies the state of the robot1on the basis of the operation indicated by the information supplied from the operation acquisition section102, and determines whether or not its own state transition follows the same transition as the failure state transition.

The failure state transition determination section103controls the drive control section106on the basis of a result of the determination on the state transition. For example, in a case of determining that its own state transition does not follow the same transition as the failure state transition, the failure state transition determination section103causes each operation leading to the success of the task to be performed. Furthermore, in a case of determining that its own state transition follows the same transition as the failure state transition, the failure state transition determination section103causes a preset action to be performed.

The failure state transition storage section104stores the failure state transition information of each task.

The failure state transition generation section105sets the failure state transition of each task and generates the failure state transition information. The failure state transition generation section105outputs the failure state transition information to the failure state transition storage section104for storage.

The drive control section106controls each drive unit of the drive section33to perform a predetermined operation on the basis of the information supplied from the failure state transition determination section103.

<Operation of Robot>

Here, the operation of the robot1having the above configuration will be described.

First, a failure state transition generation process of the robot1will be described with reference to the flow chart ofFIG.6.

In step S1, the failure state transition generation section105sets a failure state transition for a predetermined task. The setting of the failure state transition may also be performed according to an operation of a manager of the robot1, for example.

In step S2, the failure state transition generation section105outputs failure state transition information to the failure state transition storage section104for storage.

The above process is performed for each task. The failure state transition storage section104stores the failure state transition information for each task.

Next, a control process of the robot1will be described with reference to the flow chart ofFIG.7.

In step S11, the failure state transition determination section103identifies the state of the robot1on the basis of an operation indicated by information supplied from the operation acquisition section102. Information indicating how the robot1moves (such as moving the arm portion12upward or moving forward), what information the robot1obtains (such as results of image/sound recognition and any other sensor information), and the like is supplied from the operation acquisition section102as information indicating the content of the operation.

In step S12, the failure state transition determination section103compares the state of the robot1to the failure state transition.

In step S13, the failure state transition determination section103determines whether or not the state transition of the robot1follows the same transition as the failure state transition and a transition for which an action is to be performed has occurred.

In a case where it is determined that a transition for which an action is to be performed has not occurred in step S13, the failure state transition determination section103returns to step S11and repeats the above-described process.

In a case where it is determined that a transition for which an action is to be performed has occurred in step S13, in step S14, the failure state transition determination section103controls the drive control section106to perform a predetermined action. As described above, an action such as an alert to the user, an emergency stop of the operation, or grasping again is performed according to the state of the robot1.

After the action is performed, the process ends. The above process is repeatedly performed during the task.

As described above, the robot1performs an optimum operation before a task fails and can thereby prevent the failure of the task.

<Examples of Updating Failure State Transition>

The failure state transition may be automatically updated by the robot1. For example, the update of the failure state transition is performed in a case where the pre-failure state is reached a large number of times.

Update Example 1

FIG.8is a diagram showing a first example of updating a failure state transition for a task of “grasping an object”.

States #21 to #23 inFIG.8are similar to the states #1 to #3 inFIG.2. After a state #23, which is a state where the arm portion12is moved, is reached, it is determined whether or not the object to be grasped is lost sight of.

In a case where it is determined that the object to be grasped is lost sight of, the state of the robot1becomes a state #24, which is a state where the object to be grasped is lost sight of.

In the case where the state #24 is reached, an action of emergency stop is performed as indicated by an arrow A21at its head. The robot1stops driving individual parts and does not move the arm portion12.

That is, in the failure state transition ofFIG.8, as shown in the region enclosed by a broken line, in a case where the state #24, which is a state where the object to be grasped is lost sight of, is reached, it is determined that the pre-failure state is reached and the action of emergency stop is performed without alerting the user. In other words, the failure state transition is updated such that the pre-failure state is more likely to be reached.

As the failure state transition is updated such that it is determined that the pre-failure state is reached early, it is possible to more reliably prevent the failure of the task.

Update Example 2

FIG.9is a diagram showing a second example of updating a failure state transition for a task of “grasping an object”.

States #31 to #33 inFIG.9are similar to the states #1 to #3 inFIG.2. After a state #33, which is a state where the arm portion12is moved, is reached, it is determined whether or not half or more of the object to be grasped is out of the angle of view.

In a case where it is determined that half or more of the object to be grasped is out of the angle of view, the state of the robot1becomes a state #34, which is a state where half or more of the object to be grasped is out of the angle of view.

In the case where the state #34 is reached, an action of alert is performed as indicated by an arrow A31at its head. The robot1drives a speaker or the like to alert the user that half or more of the object to be grasped is out of the angle of view by means of sound.

That is, in the failure state transition ofFIG.9, as shown in the region enclosed by a broken line, the action of alert is performed on the basis of a criterion that is stricter than a criterion in the transition ofFIG.2.

In the failure state transition ofFIG.2, the action of alert is performed in a case where the object to be grasped, for example the entire portion thereof, is out of the angle of view, and it is determined that the object is lost sight of. Since the state where half of the object to be grasped is out of the angle of view is a state that is more likely to occur than the state where the entire object is out of the angle of view, the criterion for performing the action of alert is stricter in the failure state transition ofFIG.9. In other words, the failure state transition is updated such that the failure state transition is more likely to be followed.

In the failure state transition ofFIG.9, the transition after the action of alert is performed is similar to the transition described with reference toFIG.2.

As the failure state transition is updated such that the criterion for performing an action is stricter, it is possible to more reliably prevent the failure of the task.

Update Example 3

FIG.10is a diagram showing a first example of updating a failure state transition for a task of “passing an object to a person”.

States #41 to #43 inFIG.10are similar to the states #11 to #13 inFIG.3. In the case where a state #43, which is a state where the person's hand is not touching the object, is reached, an action of alert is performed as indicated by an arrow A41at its head.

After the state #43, which is a state where the person's hand is not touching the object, is reached and the action of alert is performed, it is determined whether the person's face is not facing toward the object.

In a case where it is determined that the person's face is not facing toward the object, the state of the robot1becomes a state #44, which is a state where the person's face is not facing toward the object. In the case where the state #44 is reached, an action of alert is performed as indicated by an arrow A42at its head.

That is, in the failure state transition ofFIG.10, as shown in the region enclosed by a broken line, the action of alert is performed on the basis of a criterion that is stricter than a criterion in the transition ofFIG.3.

In the failure state transition ofFIG.3, the action of alert is performed in a case where it is determined that the person is not looking at the object. In other words, the action of alert is not performed as long the line-of-sight is directed to the object, regardless of the face direction.

Since a criterion that requires more attention of the person is used in that not only the line-of-sight but also the face direction is taken in consideration, it can be regarded that the failure state transition ofFIG.10is a transition that uses a stricter criterion for performing the action of alert.

In the failure state transition ofFIG.10, the transition after the action of alert is performed is similar to the transition described with reference toFIG.3.

Update Example 4

FIG.11is a diagram showing a second example of updating a failure state transition for a task of “passing an object to a person”.

States #51 and #52 inFIG.11are similar to the states #11 and #12 inFIG.3. After the state #12, which is a state where the hand portion13grasping the object is placed in front of the person, it is determined whether the person's hand is not grasping the object.

In a case where it is determined that the person's hand is not grasping the object, the state of the robot1becomes a state #53, which is a state where the person's hand is not grasping the object. In the case where the state #53 is reached, an action of alert is performed as indicated by an arrow A51at its head.

That is, in the failure state transition ofFIG.11, as shown in the region enclosed by a broken line, the action of alert is performed on the basis of a criterion that is stricter than a criterion in the transition ofFIG.3.

In the failure state transition ofFIG.3, the action of alert is performed in a case where it is determined that the person's hand is not touching the object. In other words, the action of alert is not performed as long the hand is touching, not necessarily grasping, the object.

Since the criterion requires more attention of the person in that the object is not only touched but actually grasped, it can be regarded that the failure state transition ofFIG.10is a transition that uses a stricter criterion for performing the action of alert.

In the failure state transition ofFIG.11, the transition after the action of alert is performed is similar to the transition described with reference toFIG.3.

Update Example 5

FIG.12is a diagram showing another example of updating a failure state transition for a task of “grasping an object”.

The failure state transition shown inFIG.12is different from the failure state transition ofFIG.2in that a state where an action of alert is to be performed is added.

States #61 and #62 inFIG.12are similar to the states #1 and #2 inFIG.2. After a state #63, which is a state where the arm portion12is moved, is reached, it is determined whether or not half or more of the object to be grasped is out of the angle of view.

In a case where it is determined that half or more of the object to be grasped is out of the angle of view, the state of the robot1becomes a state #64, which is a state where half or more of the object to be grasped is out of the angle of view.

In the case where the state #64 is reached, an action of alert is performed as indicated by an arrow A61at its head.

That is, in the failure state transition ofFIG.12, as show in the region enclosed by a broken line, the determination of whether or not a state for which the action of alert is to be performed is reached is added, and the action of alert is more likely to be performed than in the transition ofFIG.9.

The transition after the action of alert is performed is similar to the transition after the state #3 inFIG.2.

Thus, as the failure state transition is updated so as to increase the number of states for which an action of alert is to be performed, it is also possible to more reliably prevent the failure of the task.

Example Configuration of Failure State Transition Update Section

FIG.13is a block diagram showing another example functional configuration of the control section31.

The configuration shown inFIG.13is basically the same as the configuration shown inFIG.5except that a failure state transition update section107is added. Overlapping descriptions will be omitted as appropriate.

The failure state transition update section107updates the failure state transition in the above-described manner in a case where it is determined that a timing for updating the failure state transition is reached. Whether or not the timing for updating the failure state transition is reached is determined on the basis of information supplied from the failure state transition determination section103.

The failure state transition update section107outputs failure state transition information indicating an updated failure state transition to the failure state transition storage section104for storage.

FIG.14is a block diagram showing an example configuration of the failure state transition update section107.

As shown inFIG.14, the failure state transition update section107includes a failure information acquisition section131, an update determination section132, an update section133, an update determination criterion information storage section134, and an update transition information storage section135.

The failure information acquisition section131acquires information regarding failure of a task on the basis of information output from the failure state transition determination section103(FIG.13). For example, the failure information acquisition section131identifies the number of times the pre-failure state is reached, and outputs information indicating the identified number of times to the update determination section132.

The update determination section132determines whether or not to update the failure state transition on the basis of the information supplied from the failure information acquisition section131. The update determination criterion information storage section134stores update determination criterion information that is a criterion for determination on an update for each task.

Whether or not to update the failure state transition is performed by, for example, comparing the number of times the pre-failure state is reached and a threshold number of times indicated by the update determination criterion information. In a case where the number of times the pre-failure state is reached for a predetermined task exceeds the threshold number of times, the update determination section132determines to update the failure state transition for the task, and outputs information indicating that to the update section133.

The update section133updates the failure state transition determined to be updated on the basis of information stored in the update transition information storage section135. The update transition information storage section135stores information regarding how to update the failure state transition. The update section133outputs failure state transition information indicating the updated failure state transition to the failure state transition storage section104for storage.

Failure State Transition Update Process

A process of updating a failure state transition by the robot1will be described with reference to the flow chart ofFIG.15.

In step S21, the failure information acquisition section131acquires information regarding failure of a task on the basis of information output from the failure state transition determination section103.

For example, the failure information acquisition section131acquires the number of times the pre-failure state is reached, the content of the failure, and the degree of seriousness of the failure as the information regarding the failure.

The content of the failure is a cause of an action described above. For example, that the object is lost sight of, that the person's hand is not touching the object, that the person is not looking at the object, that half of the object is out of the angle of view, that the person's face is not facing toward the object, that the person's hand is not grasping the object, or the like is acquired as the content of the failure.

Furthermore, the degree of seriousness of the failure is a degree of seriousness of reaching the pre-failure state. For example, the degree of seriousness corresponding to the pre-failure state is acquired, such as whether or not hitting occurs, whether or not the object is dropped, whether or not the object, a wall, or the like is broken, whether or not movement to a limit of the drive region occurs, or whether or not a person is nearby.

In step S22, the update determination section132determines whether or not a condition for updating the failure state transition is satisfied.

Here, for example, it is determined that the update condition is satisfied in a case where the number of times the pre-failure state is reached becomes greater than or equal to a threshold number of times.

Furthermore, it is also determined that the update condition is satisfied in a case where failure of the same content is repeated a threshold number of times or more or a case where a failure with a degree of seriousness greater than a threshold degree occurs.

In a case where it is determined that the update condition is not satisfied in step S22, a return to step S21occurs, and the above process is repeated.

In a case where it is determined that the update condition is satisfied in step S22, in step S23, the update section133updates the failure state transition. The update of the failure state transition is performed as in 1 to 3 below, for example.

1. According to pre-setting, update is performed such that it is determined early that the pre-failure state is reached or such that the criterion for performing an action becomes a stricter criterion as described above.

For example, update is performed such that it is determined early that the pre-failure state is reached in a case where the number of times the pre-failure state is reached becomes greater than or equal to the threshold number of times. Furthermore, update is performed such that the criterion for performing an action becomes a stricter criterion in a case where failure of the same content is repeated a threshold number of times or more or a case where a failure with a degree of seriousness greater than a threshold degree occurs.

2. Update is performed so as to change the failure state transition used according to conditions.

For example, the failure state transition used is switched according to the size of the object to be grasped, such as using the failure state transition inFIG.8in a case where the size of the object to be grasped is smaller than a predetermined size and using the failure state transition inFIG.9in a case where it is larger than the predetermined size.

Furthermore, the failure state transition used is switched according to an attribute of the person to which the object is to be passed, such as using the failure state transition inFIG.10in a case where the person to which the object is to be passed is a child and using the failure state transition inFIG.11in a case where it is an adult.

3. Update is performed according to user setting.

In this case, the user is prompted to update the condition such as by outputting a sound from the speaker43.

In step S24, the update section133causes the failure state transition storage section104to store failure state transition information indicating the updated failure state transition, and ends the process.

Thus, by updating the failure state transition in a case where the condition is satisfied, the robot1can more reliably prevent the failure of the task.

<Examples of Recovery>

A recovery may be performed in a case where a pre-failure state is reached. The recovery is an operation of attempting to perform a task again.

Recovery Example 1

FIG.16is a diagram showing a first example of a failure state transition including a recovery for a task of “grasping an object”.

States #101 to #105 inFIG.16are similar to the states #1 to #5 inFIG.2, respectively.

In the example ofFIG.16, in a case where the state #105, which is a state where the arm portion12is moved, is reached and it is determined that a pre-failure state is reached, the state of the robot1returns to the state #101 again.

That is, in a case of moving the arm portion12with the object being lost sight of, the robot1controls individual parts to reach a state where the arm portion12is raised. In the example ofFIG.16, a recovery is performed by an operation of controlling individual parts to reach a state where the arm portion12is raised.

Recovery Example 2

FIG.17is a diagram showing a second example of a failure state transition including a recovery for a task of “grasping an object”.

States #111 to #115 inFIG.17are similar to the states #1 to #5 inFIG.2, respectively.

In the example ofFIG.17, in a case where the state #115, which is a state where the arm portion12is moved, is reached and it is determined that a pre-failure state is reached, the state of the robot1becomes a state #116, which is a state where the arm portion12is moved to a position where the object has been recognized. The robot1controls individual parts to move the arm portion12to a position immediately before the object is lost sight of.

After the state #116, which is a state where the arm portion12is moved to a position where the object has been recognized, is reached, the state of the robot1becomes a state #117, which is a state where an object tracking parameter is updated.

The object tracking parameter is a parameter for tracking the object on the basis of an image captured by the camera41. In a case where the camera41is provided near the end of the arm portion12, the object tracking parameter includes information such as the moving speed of the arm portion12.

The robot1updates the object tracking parameter such as by decreasing the moving speed of the arm portion12such that the object is not lost sight of. Thereafter, the state of the robot1returns to the state #112, and a similar state transition is continued. In the example ofFIG.17, a recovery is performed by an operation of moving the arm portion12to a position immediately before the object is lost sight of and updating the object tracking parameter.

Recovery Example 3

FIG.18is a diagram showing a first example of a failure state transition including a recovery for a task of “passing an object to a person”.

States #121 to #125 inFIG.18are similar to the states #11 to #15 inFIG.3, respectively.

In the example ofFIG.18, in a case where the state #125, which is a state where the force of the hand portion13grasping the object is reduced, is reached and it is determined that a pre-failure state is reached, the state of the robot1returns to the state #121 again.

That is, in a case where the force of the hand portion13is to be reduced while the person to which the object is to be passed is not looking at the object, the robot1controls individual parts including the hand portion13to reach a state where the object is held. In the example ofFIG.18, a recovery is performed by an operation of controlling individual parts including the hand portion13to reach a state where the object is held.

Recovery Example 4

FIG.19is a diagram showing a second example of a failure state transition including a recovery for a task of “passing an object to a person”.

States #131 to #135 inFIG.19are similar to the states #11 to #15 inFIG.3, respectively.

In the example ofFIG.19, in a case where the state #135, which is a state where the force of the hand portion13grasping the object is reduced, is reached and it is determined that a pre-failure state is reached, the state of the robot1becomes a state #136, which is a state where the object is grasped again. In a case where the force of the hand portion13is to be reduced while the person to which the object is to be passed is not looking at the object, the robot1controls individual parts including the hand portion13to grasp the object again.

After the state #136, which is a state where the object is grasped by the hand portion13again, the state of the robot1becomes a state #137, which is a state where the user is alerted to look at and carefully hold the object. After performing such an alert such as by outputting a sound from the speaker43, the robot1determines whether the person is not looking at the object and repeats a similar state transition.

In the example ofFIG.19, a recovery is performed by an operation of controlling individual parts including the hand portion13to grasp the object again and alerting the user.

As described above, the failure state transition including the recovery is managed in the robot1, and the recovery is performed in a case where the pre-failure state is reached. Therefore, the robot1can prevent the failure of the task and lead the task to success. The recovery may also be performed after the alert is performed.

Example Configuration of Recovery Control Section

FIG.20is a block diagram showing another example functional configuration of the control section31.

The configuration shown inFIG.20is basically the same as the configuration shown inFIG.5except that a recovery control section108is added. Overlapping descriptions will be omitted as appropriate.

In a case where it is determined that a pre-failure state is reached, the recovery control section108controls the drive control section106to perform a recovery. Whether or not the pre-failure state is reached is determined on the basis of information supplied from the failure state transition determination section103or information supplied from the drive control section106.

FIG.21is a block diagram showing an example configuration of the recovery control section108.

As shown inFIG.21, the recovery control section108includes a failure information acquisition section151, a recovery determination section152, a recovery performance section153, a recovery determination criterion information storage section154, and a recovery state transition information storage section155.

The failure information acquisition section151acquires information regarding failure of a task on the basis of information output from the failure state transition determination section103or the drive control section106. For example, the failure information acquisition section151identifies the number of times the pre-failure state is reached, and outputs information indicating the identified number of times to the recovery determination section152.

The recovery determination section152determines whether or not to perform the recovery on the basis of the information supplied from the failure information acquisition section151. The recovery determination criterion information storage section154stores recovery determination criterion information that is a criterion for determination of whether or not to perform the recovery for each task.

Whether or not to perform the recovery is determined by, for example, comparing the number of times the pre-failure state is reached and a threshold number of times indicated by the recovery determination criterion information. In a case where the number of times the pre-failure state is reached for a predetermined task exceeds the threshold number of times, the recovery determination section152determines to perform the recovery, and outputs information indicating that to the recovery performance section153.

The recovery performance section153performs the recovery on the basis of information stored in the recovery state transition information storage section155. The recovery state transition information storage section155stores information regarding the failure state transition including the state of the recovery.

Recovery Performance Process

A process of performing a recovery by the robot1will be described with reference to the flow chart ofFIG.22.

In step S31, the failure information acquisition section151acquires information regarding failure of a task output from the failure state transition determination section103. For example, the failure information acquisition section151acquires the number of times the pre-failure state is reached, the content of the failure, and the degree of seriousness of the failure as the information regarding the failure.

In step S32, the recovery determination section152determines whether or not to perform the recovery.

Here, for example, it is determined to perform the recovery in a case where the number of times the pre-failure state is reached becomes greater than or equal to a threshold number of times. Furthermore, it is determined to perform the recovery in a case where a failure with a lower degree of seriousness than a threshold degree.

In a case where it is determined to perform the recovery in step S32, in step S33, the recovery performance section153performs the recovery.

On the other hand, in a case where it is determined not to perform the recovery in step S32, in step S34, the recovery performance section153controls the drive control section106to perform a predetermined action.

As described above, the robot1performs the recovery before a task fails and can thereby prevent the failure of the task.

Modified Examples

The failure state transition may also be generated in the robot1according to user operation, instead of being generated by the robot1itself. Furthermore, failure state transition information generated according to user operation in an external device such as a PC may be provided to the robot1and used for the processes as described above.

The failure state transition information may be generated on the basis of a failure state transition that occurs in a test stage of the operation of the robot1and provided to the robot1. Information regarding what state transition is a failure state transition is specified by the user, for example.

Furthermore, the user may set a state that should be avoided, on the basis of which the failure state transition information may be generated.

Although processes for cases of performing the tasks of “grasping an object” and “passing an object to a person” have been described above, similar processes are performed in a case of performing another task. In performing another task as well, in a case where the state transition of the robot1follows the same transition as the failure state transition, an action such as emergency stop is performed before failure occurs.

Example of Control System

The operation of the robot1based on the failure state transition may be performed by an external device such as a server on the Internet.

FIG.23is a diagram showing an example configuration of a control system.

The control system inFIG.23is constituted by connecting the robot1and a control server201via a network202such as the Internet. The robot1and the control server201communicate with each other via the network202.

In the control system inFIG.23, the state of the robot1is identified by the control server201on the basis of information sent from the robot1. Information indicating the state of each device of the robot1is sequentially sent from the robot1to the control server201.

In a case where it is detected that the state transition of the robot1follows a failure state transition, the control server201controls the robot1to perform an action corresponding to the state such as an action of alert or an action for the time of task failure.

Thus, the control server201functions as a control device that monitors the state of the robot1and controls the operation of the robot1according to the state. In the control server201, a predetermined program is executed to realize each functional section inFIG.5,FIG.14, orFIG.20.

Example Configuration of Computer

The series of processes described above can be executed by hardware or executed by software. In a case where the series of processes are executed by software, programs constituting the software are installed on a computer integrated into dedicated hardware, a general-purpose personal computer, or the like from a program recording medium.

FIG.24is a block diagram showing an example hardware configuration of a computer that programmatically executes the series of processes described above. The control server201inFIG.23has a configuration similar to the configuration shown in FIG.24.

A central processing unit (CPU)1001, a read only memory (ROM)1002, and a random access memory (RAM)1003are connected to each other by a bus1004.

An input/output interface1005is further connected to the bus1004. An input section1006including a keyboard, a mouse, and the like and an output section1007including a display, a speaker, and the like are connected to the input/output interface1005. Furthermore, a storage section1008including a hard disk, a non-volatile memory, and the like, a communication section1009including a network interface and the like, and a drive1010that drives a removable medium1011are connected to the input/output interface1005.

In the computer configured as described above, the series of processes described above are performed by the CPU1001loading programs stored in the storage section1008on the RAM1003via the input/output interface1005and the bus1004and executing them, for example.

The programs executed by the CPU1001are provided by being recorded in the removable medium1011or via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcast, for example, and installed on the storage section1008.

Note that the programs executed by the computer may be programs in which processes are executed on a time-series basis in the order described herein, or may be programs in which processes are executed in parallel or at a required timing such as when a call occurs.

As used herein, a system refers to a set of a plurality of components (such as devices and modules (parts)), and it does not matter whether or not all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network and one device in which a plurality of modules is housed in one housing are both systems.

The effects described herein are merely examples and are not limited, and there may be other effects.

Embodiments of the present technology are not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the present technology.

For example, the present technology can take a configuration of cloud computing in which one function is distributively and jointly processed by a plurality of devices via a network.

Furthermore, each step described in the flow charts described above can be performed not only by one device but also distributively performed by a plurality of devices.

Moreover, in a case where one step includes a plurality of processes, the plurality of processes included in the one step can be performed not only by one device but also distributively performed by a plurality of devices.

Examples of Combination of Configurations

The present technology can also have the following configurations.

(1)

A control device including:a state transition determination section that controls an operation of a robot to perform a predetermined process before a task results in failure in a case where a state transition of the robot in performing the task follows a failure state transition that is preset as a state transition that results in the failure of the task.
(2)

The control device according to (1) above, in whichthe predetermined process is an alert to a user or an emergency stop of the operation of the robot.
(3)

The control device according to (2) above, in whichthe state transition determination section performs the emergency stop of the operation of the robot in a case where the state transition of the robot follows the failure state transition after the alert to the user is performed as the predetermined process.
(4)

The control device according to any one of (1) to (3), further including:an update section that updates the failure state transition in a case where the state transition of the robot follows the failure state transition.
(5)

The control device according to (4) above, in whichthe update section updates the failure state transition in a case where a number of times the failure state transition is followed exceeds a threshold number of times or according to a content of the state transition of the robot that follows the failure state transition.
(6)

The control device according to (4) or (5) above, in whichthe update section updates the failure state transition such that the state transition of the robot is more likely to follow the failure state transition.
(7)

The control device according to any one of (1) to (6) above, further including:a recovery control section that controls the robot to perform a recovery in a case where the state transition of the robot follows the failure state transition.
(8)

The control device according to (7) above, in whichthe recovery control section causes the recovery to be performed after the predetermined process is performed.
(9)

The control device according to any one of (1) to (8) above, further including:a storage section that stores information indicating the failure state transition.
(10)

A control method including:controlling, by a control device, an operation of a robot to perform a predetermined process before a task results in failure in a case where a state transition of the robot in performing the task follows a failure state transition that is preset as a state transition that results in the failure of the task.
(11)

A program for causing a computer to perform a process of:controlling an operation of a robot to perform a predetermined process before a task results in failure in a case where a state transition of the robot in performing the task follows a failure state transition that is preset as a state transition that results in the failure of the task.

REFERENCE SIGNS LIST

1,2Robot11Base portion12Arm portion13Hand portion31Control section33Drive section41Camera101Operation output section102Operation acquisition section103Failure state transition determination section104Failure state transition storage section105Failure state transition generation section106Drive control section107Failure state transition update section108Recovery control section131Failure information acquisition section132Update determination section133Update section134Update determination criterion information storage section135Update transition information storage section151Failure information acquisition section152Recovery determination section153Recovery performance section154Recovery determination criterion information storage section155Recovery state transition information storage section201Control server202Network