PATENT CLAIM ANALYSIS

Application Number: 16417522
Application Type: Utility
Filing Date: 2019-05
Publication Date: 2019-11
Patent Classification: ["706", "025000"]

Abstract:
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning using agent curricula. One of the methods includes maintaining data specifying plurality of candidate agent policy neural networks; initializing mixing data that assigns a respective weight to each of the candidate agent policy neural networks; training the candidate agent policy neural networks using a reinforcement learning technique to generate combined action selection policies that result in improved performance on a reinforcement learning task; and during the training, repeatedly adjusting the weights in the mixing data to favor higher-performing candidate agent policy neural networks.

Claim (Index 12):
A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for training a final agent policy neural network that is used to select actions to be performed by an agent interacting with an environment to perform a reinforcement learning task, the operations comprising:\n maintaining data specifying plurality of candidate agent policy neural networks, wherein the plurality of candidate agent policy neural networks includes the final agent policy neural network, and wherein the final agent policy neural network defines an action selection policy for the agent that is more complex than an action selection policy defined by at least one other candidate agent policy neural network; initializing mixing data that assigns a respective weight to each of the candidate agent policy neural networks; training the plurality of candidate agent policy neural networks jointly to perform the reinforcement learning task, comprising, at each of a plurality of training iterations:\n obtaining a training network input comprising an observation of the environment, \n generating, using the candidate agent policy neural networks and in accordance with the weights in the mixing data as of the training iteration, a combined action selection policy using the training network input, and \n training the candidate agent policy neural networks using a reinforcement learning technique to generate combined action selection policies that result in improved performance on the reinforcement learning task; and \n during the training, repeatedly adjusting the weights in the mixing data to favor higher-performing candidate agent policy neural networks.

Metadata:
- Claim Count in Document: 50.0
- Percentile: 100.0
- Lexical Diversity: 1.8
- Patent Class: 706.0
- Transitional Phrase Type: open
- Component Type: 1
- Foreign Priority: False
- Related Applications: ['16380125', '15499832', '14097862', '15704969', '15977923']

Analysis Scores:
- 35 USC 101 Eligibility (BERT): 0.3482038603945876
- 35 USC 102 Novelty (BERT): 0.5022448167568162
- Combined Prediction Score: 0.3636079560308105
- Mean Citation Score: 249.24804
- Max Citation Score: 270.2168
- Similarity Product: 213.8121972335816

Labels:
- Claim Label 101: 1
- Claim Label 102: 1
- Claim Label 103: 0
- Claim Label 112: 1
- Combined Label: 1
- Label 101 Adjusted: 1

Dataset: test