Patent ID: 8886357
Filing Date: 2014-11-11
Classification: G05B,Y10S

Abstract:
1. A reinforcement learning apparatus comprising: a reward function storage unit in which a reward function for outputting a reward can be stored; a first-type environment parameter obtaining unit that obtains a value of one or more first-type environment parameters relating to an environment of a moving control object; a control parameter value calculation unit that calculates a value of one or more control parameters maximizing a reward that is output by the reward function by substituting the value of the one or more first-type environment parameters into the reward function; a control parameter value output unit that outputs the value of the one or more control parameters to the control object; a virtual external force function storage unit in which a virtual external force function for outputting a virtual external force can be stored, a second-type environment parameter obtaining unit that obtains a value of one or more second-type environment parameters relating to the virtual external force; a virtual external force calculation unit that calculates the virtual external force by substituting the value of the one or more second-type environment parameters into the virtual external force function; a virtual external force output unit that outputs the virtual external force to the control object; and a virtual external force approximator, wherein the virtual external force approximator comprises: a function approximator storage unit in which can be stored a function approximator including two or more third-type parameters serving as observable third-type parameters that are at least partially different from the second-type environment parameters and outputting a second virtual external force; a state information obtaining unit that observes the control object, and obtains one or more pieces of state information relating to a state of the control object; a second virtual external force calculation unit that calculates the second virtual external force by substituting the state information into the function approximator; a difference information calculation unit that calculates difference information serving as information relating to a difference between the virtual external force that is output by the virtual external force output unit and the second virtual external force; and an approximation unit that updates a parameter of the function approximator so as to minimize the difference information.