Control device and machine learning device

A control device and a machine learning device enable control for gripping an object having small reaction force. The machine learning device included in the control device includes a state observation unit that observes gripping object shape data related to a shape of the gripping object as a state variable representing a current state of an environment, a label data acquisition unit that acquires gripping width data, which represents a width of the hand of the robot in gripping the gripping object, as label data, and a learning unit that performs learning by using the state variable and the label data in a manner to associate the gripping object shape data with the gripping width data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a new U.S. Patent Application that claims benefit of Japanese Patent Application No. 2017-224275, filed Nov. 22, 2017, the disclosure of this application is being incorporated herein by reference in its entirety for all purposes.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a control device and a machine learning device and especially relates to a control device and a machine learning device that perform control for gripping an object having small reaction force.

2. Description of the Related Art

Gripping an object with a machine through control has been conventionally realized such that a mechanical mechanism part for gripping an object is brought into contact and pressed against the object so as to feed back a current value of a motor and power and moment detected by a force sensor to power for gripping the object (Japanese Patent Application Laid-Open No. 2014-024134, for example).

In general, reaction force from an object is required to detect a current value of a motor and detect power and moment by a force sensor. However, enough reaction force cannot be detected in gripping an object having small reaction force such as a soft object: tofu and a cream puff, for example, so that it is difficult to perform control for gripping the object.

Further, a soft object such as a cream puff and bread has a large error in shape thereof. Accordingly, the object may be too strongly gripped and damaged due to the error in the method in which a current value of a motor for driving a mechanism part and power and moment detected by a force sensor are fed back to power to grip an object.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a control device and a machine learning device that enable control for gripping an object having small reaction force.

The control device according to the present invention solves the above-mentioned problem by controlling a gripping position based on machine learning using a length measuring sensor instead of the method for detecting a current value of a motor and power and moment by a force sensor.

A control device according to an aspect of the present invention that estimates a gripping width of a hand of a robot in gripping a gripping object having small reaction force includes: a machine learning device that learns estimation for the gripping width of the hand of the robot in gripping the gripping object, with respect to a shape of the gripping object; a state observation unit that observes gripping object shape data related to the shape of the gripping object as a state variable representing a current state of an environment; a label data acquisition unit that acquires gripping width data, which represents the gripping width of the hand of the robot in gripping the gripping object, as label data; and a learning unit that performs learning by using the state variable and the label data in a manner to associate the gripping object shape data with the gripping width data.

A machine learning device according to another aspect of the present invention that learns estimation for a gripping width of a hand of a robot in gripping a gripping object with respect to a shape of the gripping object having small reaction force includes: a state observation unit that observes gripping object shape data related to the shape of the gripping object as a state variable representing a current state of an environment; a label data acquisition unit that acquires gripping width data, which represents the gripping width of the hand of the robot in gripping the gripping object, as label data; and a learning unit that performs learning by using the state variable and the label data in a manner to associate the gripping object shape data with the gripping width data.

According to the present invention, a machine can be made securely grip an object having small reaction force without damaging the object.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1is a hardware configuration diagram schematically illustrating substantial parts of a control device according to a first embodiment. A control device1can be mounted as a control device for controlling a robot, for example. Further, the control device1can be mounted as a computer such as a cell computer, a host computer, and a cloud server that is connected with a control device controlling a robot via a network.FIG. 1illustrates an example of a case that the control device1is mounted as a control device controlling a robot.

A CPU11included in the control device1according to the present embodiment is a processor for entirely controlling the control device1. The CPU11reads out a system program stored in a ROM12via a bus20and controls the whole of the control device1in accordance with the system program. A RAM13temporarily stores temporary calculation data and display data and various kinds of data which are inputted by an operator via an input unit, which is not shown, for example.

A non-volatile memory14is backed up by a battery, which is not shown, for example, and thus, the non-volatile memory14is configured as a memory whose storage state is maintained even when the control device1is turned off. The non-volatile memory14stores, for example, control programs which are read in via an interface, control programs which are inputted via a display/MDI unit, and various kinds of data which are acquired from a robot2and a sensor3. The control programs stored in the non-volatile memory14may be developed on the RAM13when the control programs are used. Further, various kinds of system programs required for an operation of the control device1(including a system program for controlling exchange with a machine learning device100) are preliminarily written in the ROM12.

The control device1outputs commands for controlling a joint and a hand, for example, of the robot2with respect to the robot2via an interface18based on the control programs and the like. Further, the control device1acquires data from each unit of the robot2via the interface18.

Further, the control device1acquires data detected by at least one sensor (a length measuring sensor for measuring the length of a gripping object, a camera for taking an image of the gripping object, and sensors for ambient temperature and ambient humidity, for example) attached to the periphery or each unit of the robot via an interface19.

An interface21is an interface for connecting the control device1with the machine learning device100. The machine learning device100includes a processor101that entirely controls the machine learning device100, a ROM102that stores system programs and the like, a RAM103that performs temporary storage in each processing related to machine learning, and a non-volatile memory104that is used for storing learning models and the like. The machine learning device100is capable of observing various information (the length of a gripping object, an appearance of the gripping object, ambient temperature, and ambient humidity, for example) which can be acquired by the control device1, via the interface21. Further, the control device1controls the gripping width of the hand set when the robot2grips a gripping object, for example, based on a value outputted from the machine learning device100.

FIG. 2is a functional block diagram schematically illustrating the control device1and the machine learning device100according to the first embodiment. Functional blocks illustrated inFIG. 2are realized when the CPU11included in the control device1and the processor101of the machine learning device100which are illustrated inFIG. 1execute respective system programs and respectively control an operation of each unit of the control device1and the machine learning device100.

The control device1according to the present embodiment includes a control unit34that controls the robot2based on the control programs stored in the non-volatile memory14and an estimation result, which is outputted from the machine learning device100, of the gripping width of the hand set when the robot2grips a gripping object.

Meanwhile, the machine learning device100included in the control device1includes software (a learning algorithm, for example) and hardware (the processor101, for example) by which the machine learning device100itself learns estimation for the width of the hand of the robot2in gripping a gripping object, with respect to the shape of the gripping object, through so-called machine learning. What the machine learning device100included in the control device1learns corresponds to a model structure representing a correlation between a shape of a gripping object and the gripping width of the hand of the robot2in gripping the gripping object.

As illustrated in the functional block ofFIG. 2, the machine learning device100included in the control device1includes a state observation unit106, a label data acquisition unit108, a learning unit110, and an estimation result output unit122. The state observation unit106observes gripping object shape data S1representing a shape of a gripping object as one of state variables S representing current states of an environment. The label data acquisition unit108acquires label data L including gripping width data L1representing the gripping width of the hand of the robot2in gripping a gripping object. The learning unit110performs learning based on the state variables S and the label data L in a manner to associate the shape of a gripping object with the gripping width of the hand of the robot2in gripping the gripping object. The estimation result output unit122outputs the gripping width of the hand of the robot2in gripping a gripping object, which is estimated based on the shape of a gripping object by using a learned model obtained by the learning unit110.

The state observation unit106acquires the gripping object shape data S1as the state variable S from the sensor3in learning by the learning unit110. Further, the state observation unit106acquires the gripping object shape data S1as the state variable S from the sensor3when the width of the hand of the robot2in gripping a gripping object is estimated by using a learning result of the learning unit110. For either case, the state observation unit106may acquire data via the non-volatile memory14of the control device1, for example, instead of directly acquiring the data from the sensor3.

Among the state variables S observed by the state observation unit106, the length (lateral width) of a gripping object on a gripping position of the robot, for example, may be used as the gripping object shape data S1in the simplest configuration. The length of a gripping object on a gripping position of the robot can be acquired as a value detected by a length measuring sensor attached on the robot2or on a nearby position of the robot2and a value obtained by analyzing an image taken by a camera attached on the robot2or on a nearby position of the robot2, for example. Further, data such as the height of a gripping object and an image of the gripping object taken by a camera attached on the robot2or on a nearby position of the robot2may be used as the gripping object shape data S1.

The label data acquisition unit108acquires the gripping width data L1, which is related to the gripping width of the hand of the robot2in gripping a gripping object, as the label data L from the robot2in learning by the learning unit110. Here, the label data acquisition unit108may acquire data via the non-volatile memory14of the control device1, for example, instead of directly acquiring the data from the robot2. Data for the gripping width of the hand of the robot2, which is set when gripping of the gripping object by the robot2is successful, for example, can be used as the gripping width data L1. Whether gripping of a gripping object by the robot2is successful or failed may be inputted by an operator through an input device which is not illustrated, may be automatically determined by analyzing an image taken by a camera attached on the robot2or on a nearby position of the robot2, or may be determined based on a value detected by a sensor installed on a placing position for the gripping object. Note that the label data acquisition unit108is used in learning by the learning unit110and is not necessarily required as a component of the machine learning device100after the learning by the learning unit110is completed.

The learning unit110learns the label data L (the gripping width data L1representing the width of the hand of the robot2in gripping a gripping object) with respect to the state variable S (the gripping object shape data S1representing the shape of the gripping object) in accordance with arbitrary learning algorithms collectively called machine learning. The learning unit110is capable of learning a correlation between the gripping object shape data S1included in the state variable S and the gripping width data L1included in the label data L, for example. The learning unit110is capable of repeatedly executing learning based on a data set including the state variables S and the label data L.

It is desirable for the learning unit110to execute a plurality of learning cycles based on data respectively obtained for a plurality of robots2, in learning. Through repetition of such a learning cycle, the learning unit110automatically interprets the correlation between the shape of a gripping object (the gripping object shape data S1) and the width of the hand of the robot2in gripping the gripping object (the gripping width data L1). Though the correlation of the gripping width data L1with respect to the gripping object shape data S1is substantially unknown at the start of the learning algorithm, the learning unit110gradually interprets a relation of the gripping width data L1with respect to the gripping object shape data S1as the learning unit110advances the learning. With the learned model consequently obtained, the learning unit110can interpret the correlation of the gripping width data L1with respect to the gripping object shape data S1.

The estimation result output unit122estimates the gripping width of the hand of the robot2in gripping a gripping object from the shape of the gripping object based on a result obtained through learning by the learning unit110(learned model) and outputs the estimated width of the hand of the robot2in gripping the gripping object. More specifically, the gripping width data L1related to the width of the hand of the robot2in gripping a gripping object, which is learned by the learning unit110in a manner to be associated with the gripping object shape data S1representing the shape of a gripping object, represents the width of the hand to be commanded to the robot2by the control unit34when making the robot2grip the gripping object, and this value is outputted in estimation using a learned model obtained by the learning unit110.

As a modification of the machine learning device100included in the control device1, the state observation unit106may observe peripheral state data S2, which represents a peripheral state of the robot2, as the state variable S in addition to the gripping object shape data S1. As the peripheral state data S2, ambient temperature of the robot2, for example, is exemplified. Further, as another example of the peripheral state data S2, ambient humidity of the robot2is cited.

According to the above-described modification, the machine learning device100is capable of performing learning in a manner to associate the gripping object shape data S1and the peripheral state data S2with the gripping width data L1. Therefore, the machine learning device100can highly accurately learn and estimate variation of the proper gripping width of the hand of the robot2when making the robot2grip a gripping object whose softness and so on change depending on ambient temperature (rice cake, for example) and a gripping object whose softness, friction of the surface, and so on change depending on ambient humidity (cream puff, for example).

As another modification of the machine learning device100included in the control device1, the state observation unit106may observe kind data S3, which represents a kind of a gripping object, as the state variable S in addition to the gripping object shape data S1.

According to the above-mentioned modification, the machine learning device100is capable of performing learning in a manner to associate the gripping object shape data S1and the kind data S3with the gripping width data L1, so that the machine learning device100can highly accurately learn and estimate variation of the proper width, which varies depending on change of the kind, of the hand of the robot2.

In the machine learning device100having the above-described configuration, a learning algorithm executed by the learning unit110is not especially limited but a known learning algorithm can be employed as machine learning.FIG. 3illustrates another aspect of the control device1illustrated inFIG. 2, which has the configuration including the learning unit110that executes supervised learning as another example of the learning algorithm. The supervised learning is a method in which known data sets including inputs and outputs respectively corresponding to these inputs (referred to as teacher data) are provided and a feature indicating a correlation between an input and an output is identified based on these teacher data so as to learn a correlation model to be used for estimating a required output with respect to a new input.

In the machine learning device100included in the control device1illustrated inFIG. 3, the learning unit110includes an error calculation unit112and a model update unit114. The error calculation unit112calculates an error E between a correlation model M used for estimating the gripping width of the hand of the robot2in gripping a gripping object based on the shape of the gripping object and a correlation feature identified from teacher data T. The teacher data T is obtained from data of shapes of gripping objects acquired in the past and actual results of the width of the hand of the robot2in gripping gripping objects. The model update unit114updates the correlation model M so as to reduce the error E. The learning unit110learns estimation for the width of the hand of the robot2in gripping a gripping object based on the shape of the gripping object, by repeatedly updating the correlation model M by the model update unit114.

An initial value of the correlation model M is represented by simplifying a correlation between the state variable S and the label data L (for example, by the N-th order function), for example, and is provided to the learning unit110before the start of the supervised learning. In the present invention, data of shapes of gripping objects acquired in the past and data of the actual width of the hand of the robot2in gripping gripping objects can be used as the teacher data T as described above, and the teacher data T is provided to the learning unit110as needed in operation of the control device1. The error calculation unit112identifies a correlation feature indicating a correlation between a shape of a gripping object and the width of the hand of the robot2in gripping the gripping object based on the teacher data T which is provided as need to the learning unit110, and obtains the error E between this correlation feature and the correlation model M corresponding to the state variables S and the label data L in the current state. The model update unit114updates the correlation model M in accordance with predetermined update rules, for example, so that the error E is reduced.

In the following learning cycle, the error calculation unit112estimates the width of the hand of the robot2in gripping a gripping object by using the state variables S in accordance with the updated correlation model M and obtains the error E between the result of the estimation and the label data L which is actually acquired, and the model update unit114updates the correlation model M again. Thus, an unknown correlation between a current state of an environment and estimation corresponding to the state gradually becomes apparent.

When the above-described supervised learning is advanced, a neural network can be used.

FIG. 4Aschematically illustrates a model of a neuron.FIG. 4Bschematically illustrates a model of a three-layered neural network which is configured by combining the neurons illustrated inFIG. 4A. The neural network can be composed of arithmetic devices, storage devices, or the like, for example, in imitation of the model of neurons.

The neuron illustrated inFIG. 4Aoutputs a result y with respect to a plurality of inputs x (input x1to input x3as an example here). Inputs x1to x3are respectively multiplied by weights w (w1to w3) corresponding to these inputs x. Accordingly, the neuron outputs the output y expressed by Formula 1 below. Here, in Formula 1, all of input x, output y, and weight w are vectors. Further, 0 denotes a bias and fkdenotes an activation function.
y=fk(Σi=1nxiwi−θ)  [Formula 1]

In the three-layered neural network illustrated inFIG. 4B, a plurality of inputs x (input x1to input x3as an example here) are inputted from the left side and results y (result y1to result y3as an example here) are outputted from the right side. In the example illustrated inFIG. 4B, inputs x1, x2, x3are each multiplied by corresponding weights (collectively denoted by w1) and each of inputs x1, x2, x3is inputted into three neurons N11, N12, N13.

InFIG. 4B, an output of each of the neurons N11, N12, N13is collectively denoted by z1. z1can be considered as a feature vector obtained by extracting a feature amount of an input vector. In the example illustrated inFIG. 4B, feature vectors z1are each multiplied by corresponding weights (collectively denoted by w2) and each of feature vectors z1is inputted into two neurons N21, N22. Feature vector z1represents a feature between weight w1and weight w2.

InFIG. 4B, an output of each of the neurons N21, N22is collectively denoted by z2. z2can be considered as a feature vector obtained by extracting a feature amount of feature vector z1. In the example illustrated inFIG. 4B, feature vectors z2are each multiplied by corresponding weights (collectively denoted by w3) and each of feature vectors z2is inputted into three neurons N31, N32, N33. Feature vector z2represents a feature between weight w2and weight w3. Finally, neurons N31to N33respectively output results y1to y3.

Here, the method of so-called deep learning in which a neural network having three or more layers is used may be employed as well.

In the machine learning device100included in the control device1, the learning unit110performs calculation of the multilayer structure following the above-described neural network by using the state variable S as an input x, being able to estimate the width (output y) of the hand of the robot2in gripping a gripping object based on the value (input x) of the shape of the gripping object. Here, operation modes of the neural network include a learning mode and a value prediction mode. For example, weight w can be learned by using a learning data set in the learning mode and value determination of an action can be performed by using the learned weight w in the value prediction mode.

Here, detection, classification, inference, and so forth can also be performed in the value prediction mode.

The above-described configuration of the machine learning device100can be described as a machine learning method (or software) each executed by the processor101. This machine learning method is a machine learning method for learning estimation for the width of the hand of the robot2in gripping a gripping object, based on the shape of the gripping object. The machine learning method includes a step in which the processor101observes the shape of the gripping object (the gripping object shape data S1) as the state variable S representing a current state, a step in which the processor101acquires the width of the hand of the robot2in gripping the gripping object (the gripping width data L1) as the label data L, and a step in which the processor101performs learning by using the state variable S and the label data L in a manner to associate the gripping object shape data S1with the width of the hand of the robot2in gripping the gripping object.

A learned model which is obtained through learning by the learning unit110of the machine learning device100is applicable as a program module which is part of software related to machine learning. The learned model according to the present invention can be used in a computer provided with a processor such as a CPU and a GPU and a memory. More specifically, the processor of the computer operates to perform calculation by using a shape of a gripping object as an input in accordance with a command from the learned model stored in the memory and to output an estimation result of the width of the hand of the robot2in gripping the gripping object based on the calculation result. The learned model according to the present invention can be used in a manner to be copied to other computers via an external storage medium and a network, for example.

Further, when the learned model according to the present invention is copied to other computers to be used in a new environment, further learning can be performed with respect to the learned model based on new state variables and new label data obtained in this new environment. In such a case, a learned model derived from the learned model in this environment (referred to below as a derived model) can be obtained. The derived model according to the present invention is the same as the original learned model on the point that the derived model is used for outputting an estimation result for the gripping width of the hand of the robot2in gripping a gripping object based on the shape of the gripping object. However, the derived model is different from the original learned model in that the derived model is used for outputting a result adapted to an environment newer than that of the original learned model. This derived model can be also used in a manner to be copied to other computers via an external storage medium and a network, for example.

Further, it is possible to generate and use a learned model which is obtained by performing learning from the beginning in another machine learning device (referred to below as a distilled model) by using an output, which is obtained with respect to an input to the machine learning device in which the learned model according to the present invention is incorporated (such a learning process is referred to as distillation). In distillation, an original learned model is referred to also as a teacher model and a newly-created distilled model is referred to also as a student model. In general, a distilled model is more suitable to be distributed to other computers via an external storage medium and a network, for example, because the distilled model is smaller in size than an original learned model and exhibits accuracy equivalent to that of the original learned model.

FIG. 5illustrates a system170, according to an embodiment, including the control device1. The system170includes at least one control device1(which is provided with the machine learning device100) which is connected to a network, control devices1′ (which are not provided with the machine learning device100), and a wired/wireless network172which connects the control device1and the control devices1′ to each other.

In the system170having the above-described configuration, the control device1provided with the machine learning device100is capable of automatically and accurately estimating the width of the hand of the robot2in gripping a gripping object with respect to the shape of the gripping object under the control of each of the control device1and the control devices1′, by using a learning result of the learning unit110. Further, the system170may be configured so that the machine learning device100of the control device1learns estimation for the width of the hand of the robot2in gripping a gripping object common to all of the control device1and the control devices1′ based on the state variables S and the label data L, which are obtained from each of the plurality of control devices1and1′, and the learning result is used in control of all of the robots2. According to the system170, speed and reliability in learning for estimation for the gripping width of the hand of the robot2in gripping a gripping object can be improved by using more various data sets (including the state variables S and the label data L) as inputs.

The embodiment of the present invention has been described above, but the present invention can be embodied in various aspects by adding arbitrary alterations, without being limited only to the examples of the above-described embodiment.

For example, the learning algorithm and calculation algorithm executed by the machine learning device100, the algorithm executed by the control device1, and the like are not limited to the above-mentioned algorithms, but various algorithms may be employed.

The above-described embodiment includes the description that the control device1and the machine learning device100are devices including CPUs different from each other, but the machine learning device100may be realized by the CPU11included in the control device1and the system program stored in the ROM12.

The embodiment of the present invention has been described above, but the present invention can be embodied in another aspect by adding arbitrary alterations, without being limited to the examples of the above-described embodiment.