Patent ID: 11886997
Assignee: DEEPMIND TECHNOLOGIES LIMITED
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A computer-implemented method for controlling an agent interacting with an environment to perform a task, comprising:
receiving an observation comprising input state data characterizing a state of the environment;
processing the input state data using an actor neural network to generate an output including output action data defining an output action; and
controlling the agent to perform the output action;
wherein the actor neural network has been trained jointly with a critic neural network that is configured to define a value function that generates an error signal based on an input comprising action data defining an action, state data characterizing the state of the environment, and return data derived from reward data representing a reward from the action performed in a training process; wherein the training process has been performed based on training data including data from a demonstration of the task being performed by an expert agent within the environment, wherein the data from the demonstration includes demonstration transition data for a series of demonstration transitions including demonstration examples of the state data, the action data, the reward data, and new state data representing a new state that have been generated as a result of the expert agent interacting with the environment; wherein during the training process, the actor neural network has been used to operate on the environment to generate operation transition data comprising operational examples of the state data, the action data, the reward data and the new state data, and the actor neural network and the critic neural network have been trained off-policy using the error signal and using stored tuples sampled from a replay buffer comprising tuples from both the operation transition data and the demonstration transition data.