Patent ID: 11915130
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 3:
4. The method of claim 3, wherein integrating, into the Q-learning, the soft-consistency penalty used to bias the Q-labels, across the training batches, towards being consistent with the expressible policy comprises:
updating a Q-regressor, wherein updating the Q-regressor is based on:
a given batch of training data including one or more training instances, wherein each training instance includes a current state of the agent, a next state of the agent, an action to transition the agent form the current state to the next state, and a reward;
for each of the one or more training instances, a Q-value based on the current state of the agent and the action to transition the agent from the current state to the next state, a Q-value based on the next state of the agent, and the reward of the training instance;
a value of a Q-regressor generated using one or more previous batches of training data; and
the soft-consistency penalty.