PATENT CLAIM ANALYSIS

Application Number: 16127716
Application Type: Utility
Filing Date: 2018-09
Publication Date: 2019-04
Patent Classification: ["706", "025000"]

Abstract:
Exemplary embodiments can maximize long-term value in a machine learning system. The system may employ an offline training process and an online training process. In the offline training process, an initial policy is learned to provide a warm start to the online training process. In the online training process, the system applies concurrent reinforcement learning across multiple environments, with the goal of learning efficient policies in real time from in-flight user data in one environment, and applying the learned policies to other environments. With the combination of offline training and online training, the system is able to improve initial performance through the warm start, while adapting to a changing context through concurrent reinforcement learning.

Claim (Index 21):
An apparatus configured to train an agent, the apparatus comprising:\n a non-transitory device-readable medium storing a model of an environment using historical information about the environment, the historical information about the environment represented by a plurality of dynamics, each dynamic comprising:\n a current state of the environment, \n an action to be performed, \n a value associated with the action, and \n a next state of the environment to which the current state transitions after taking the action; \n a hardware processor circuit; offline training logic configured to train an initial policy using an offline process acting on the model of the environment, the policy comprising a plurality of policy weights that prioritize between available actions to identify a selected action online training logic configured to:\n incorporate the initial policy into an online process acting in a first live environment, the first live environment defined by an online stream of information describing present dynamics of the first live environment, \n identify a plurality of possible actions to be taken in a first live environment, \n use the online process to select an action from among the plurality of possible actions, the action being selected based on the online process' evaluation of a value over time resulting from the actions in view of the present dynamics of the first live environment in view of the policy weights, \n observe a value associated with the selected action, and \n retrain the online process by updating the policy weights based on the observed value; and \n application logic configured to apply the retrained online process in a second live environment defined by a different online stream of information defining different dynamics as compared to the first live environment.

Metadata:
- Claim Count in Document: 59.0
- Percentile: 97.0
- Lexical Diversity: 1.98413
- Patent Class: 706.0
- Transitional Phrase Type: open
- Component Type: 1
- Foreign Priority: False
- Related Applications: ['15997371', '15723539', '15986037', '15897263', '16023949']

Analysis Scores:
- 35 USC 101 Eligibility (BERT): 0.3567313829368226
- 35 USC 102 Novelty (BERT): 0.5050176125626566
- Combined Prediction Score: 0.3715600058994061
- Mean Citation Score: 180.807792
- Max Citation Score: 190.55603
- Similarity Product: 107.86254710923552

Labels:
- Claim Label 101: 1
- Claim Label 102: 1
- Claim Label 103: 1
- Claim Label 112: 1
- Combined Label: 1
- Label 101 Adjusted: 1

Dataset: test