PATENT CLAIM ANALYSIS

Application Number: 16403343
Application Type: Utility
Filing Date: 2019-05
Publication Date: 2019-08
Patent Classification: ["382", "153000"]

Abstract:
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a reinforcement learning system. In one aspect, a method of training an action selection policy neural network for use in selecting actions to be performed by an agent navigating through an environment to accomplish one or more goals comprises: receiving an observation image characterizing a current state of the environment; processing, using the action selection policy neural network, an input comprising the observation image to generate an action selection output; processing, using a geometry-prediction neural network, an intermediate output generated by the action selection policy neural network to predict a value of a feature of a geometry of the environment when in the current state; and backpropagating a gradient of a geometry-based auxiliary loss into the action selection policy neural network to determine a geometry-based auxiliary update for current values of the network parameters.

Claim (Index 12):
A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations for training an action selection policy neural network having a plurality of network parameters for use in selecting actions to be performed by an agent navigating through an environment to accomplish one or more goals, wherein the agent is a mechanical agent interacting with the real-world environment, the operations comprising:\n receiving an observation image characterizing a current state of the environment; processing, using the action selection policy neural network and in accordance with current values of the network parameters, an input comprising the observation image to generate an action selection output controlling the agent to perform the actions; processing, using a geometry-prediction neural network, an intermediate output generated by the action selection policy neural network to predict a value of a feature of a geometry of the environment when in the current state; and determining a gradient of a geometry-based auxiliary loss that is based on an actual value of the feature and the predicted value of the feature; and backpropagating the gradient of the geometry-based auxiliary loss into the action selection policy neural network to determine a geometry-based auxiliary update for the current values of the network parameters.

Metadata:
- Claim Count in Document: 50.0
- Percentile: 100.0
- Lexical Diversity: 2.05
- Patent Class: 382.0
- Transitional Phrase Type: open
- Component Type: 1
- Foreign Priority: False
- Related Applications: ['16380125', '15977923', '15977913', '15349900', '15977891']

Analysis Scores:
- 35 USC 101 Eligibility (BERT): 0.333592469183074
- 35 USC 102 Novelty (BERT): 0.4736260078463816
- Combined Prediction Score: 0.3475958230494048
- Mean Citation Score: 223.102688
- Max Citation Score: 234.53249
- Similarity Product: 212.98089938385544

Labels:
- Claim Label 101: 1
- Claim Label 102: 1
- Claim Label 103: 0
- Claim Label 112: 1
- Combined Label: 1
- Label 101 Adjusted: 1

Dataset: test