PATENT CLAIM ANALYSIS

Application Number: 16021971
Application Type: Utility
Filing Date: 2018-06
Publication Date: 2018-11
Patent Classification: ["706", "015000"]

Abstract:
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output sequence from an input sequence. In one aspect, one of the systems includes an encoder neural network configured to receive the input sequence and generate encoded representations of the network inputs, the encoder neural network comprising a sequence of one or more encoder subnetworks, each encoder subnetwork configured to receive a respective encoder subnetwork input for each of the input positions and to generate a respective subnetwork output for each of the input positions, and each encoder subnetwork comprising: an encoder self-attention sub-layer that is configured to receive the subnetwork input for each of the input positions and, for each particular input position in the input order: apply an attention mechanism over the encoder subnetwork inputs using one or more queries derived from the encoder subnetwork input at the particular input position.

Claim (Index 25):
The system of  claim 24 , wherein each decoder self-attention sub-layer comprises a plurality of decoder self-attention layers, and wherein each decoder self-attention layer is configured to, at each generation time step:\n apply a learned query linear transformation to the input at each output position preceding the corresponding output position to generate a respective query for each output position, apply a learned key linear transformation to each input at each output position preceding the corresponding output position to generate a respective key for each output position, apply a learned value linear transformation to each input at each output position preceding the corresponding output position to generate a respective key for each output position, and for each output position preceding the corresponding output position,\n determine a respective output-position specific weight for each of the output positions by applying a comparison function between the query for the output position and the keys, and \n determine an initial decoder attention output for the output position by determining a weighted sum of the values weighted by the corresponding output-position specific weights for the output position.

Metadata:
- Claim Count in Document: 80.0
- Percentile: 94.0
- Lexical Diversity: 2.65574
- Patent Class: 706.0
- Transitional Phrase Type: none
- Component Type: 0
- Foreign Priority: False
- Related Applications: ['15396091', '15237575', '15815686', '15970662', '15055476']

Analysis Scores:
- 35 USC 101 Eligibility (BERT): 0.3510160258130333
- 35 USC 102 Novelty (BERT): 0.4898177911779829
- Combined Prediction Score: 0.3648962023495282
- Mean Citation Score: 196.154436
- Max Citation Score: 209.22903
- Similarity Product: 156.7033799625635

Labels:
- Claim Label 101: 1
- Claim Label 102: 1
- Claim Label 103: 0
- Claim Label 112: 0
- Combined Label: 1
- Label 101 Adjusted: 0

Dataset: test