PATENT CLAIM ANALYSIS

Application Number: 15977923
Application Type: Utility
Filing Date: 2018-05
Publication Date: 2018-09
Patent Classification: ["706", "025000"]

Abstract:
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.

Claim (Index 1):
A system for training a deep neural network used to select actions to be performed by an agent that interacts with an environment by performing actions selected from a predetermined set of actions, the system comprising:\n a plurality of workers, wherein each worker is configured to operate independently of each other worker, wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network, and wherein each worker is further configured to repeatedly perform operations comprising:\n determining current values of the parameters of the deep neural network; \n receiving a current observation characterizing a current state of the environment replica interacted with by the actor associated with the worker; \n selecting a current action to be performed by the actor associated with the worker in response to the current observation in accordance with a respective action selection policy for the worker and using one or more outputs generated by the deep neural network in accordance with the current values of the parameters; \n identifying an actual reward resulting from the actor performing the current action when the environment replica is in the current state; \n receiving a next observation characterizing a next state of the environment replica interacted with by the actor, wherein the environment replica transitioned into the next state from the current state in response to the actor performing the current action; \n performing an iteration of a reinforcement learning technique to determine a current gradient using the actual reward and the next observation; \n updating an accumulated gradient with the current gradient to determine an updated accumulated gradient; \n determining whether criteria for updating the current values of the parameters of the deep neural network have been satisfied; and \n when the criteria for updating the current values of the parameters of the deep neural network have been satisfied:\n determining updated values of the parameters using the updated accumulated gradient, and \n storing the updated values in a shared memory accessible to all of the plurality of workers.

Metadata:
- Claim Count in Document: 50.0
- Percentile: 93.0
- Lexical Diversity: 1.58696
- Patent Class: 706.0
- Transitional Phrase Type: open
- Component Type: 1
- Foreign Priority: False
- Related Applications: ['15016173', '15367094', '15349900', '15977913', '15349894']

Analysis Scores:
- 35 USC 101 Eligibility (BERT): 0.3733951519450557
- 35 USC 102 Novelty (BERT): 0.5086253876859591
- Combined Prediction Score: 0.386918175519146
- Mean Citation Score: 225.962432
- Max Citation Score: 296.19855
- Similarity Product: 214.47039454095665

Labels:
- Claim Label 101: 0
- Claim Label 102: 1
- Claim Label 103: 0
- Claim Label 112: 1
- Combined Label: 0
- Label 101 Adjusted: 0

Dataset: test