Patent ID: 11907837
Assignee: DEEPMIND TECHNOLOGIES LIMITED
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 23:
24. The computer-readable storage media of claim 22, the operations further comprising:
independently of selecting the current action and storing the current experience tuple in the replay buffer:
obtaining a minibatch of experience tuples from the replay buffer, each experience tuple comprising a training observation representing a training state of the environment, a training action from the discrete set of actions performed by the agent in response to the training observation, a training reward received in response to the agent for performing the training action, and a next training observation representing a next training state of the environment; and
updating the current values of the parameters of the Q network and the actor policy network using the minibatch of experience tuples, comprising:
for each experience tuple in the minibatch:
processing the training observation and a point representing the training action in the experience tuple using the Q network to determine a Q value for the experience tuple in accordance with current values of parameters of the Q network, and
determining a target Q value for the experience tuple from the training reward in the experience tuple and the next training observation in the experience tuple;

updating current values of the parameters of the Q network using errors between the target Q values and the Q values for the experience tuples in the minibatch; and
updating the current values of the parameters of the actor policy network using the updated values of the parameters of the Q network.