Patent ID: 11941346
Assignee: SALESFORCE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 7:
8. A system for document summarization, the system comprising:
a memory that stores a summarization model;
a communication interface that receives a text document comprising a plurality of tokens; and
one or more hardware processors that:
computes, at an attention layer, a first set of token representations by attending the plurality of tokens to respective nearby tokens within a pre-defined encoding window;
generates, by a pooling layer, a set of segment representations from the computed first set of token representations from the attention layer;
updates the set of segment representations with a full self-attention layer;
generates a second set of token representations by applying cross attention upon the first set of token representations and the updated set of segment representations after the full self-attention layer; and
sends the generated second set of token representations of the text document to a decoder for generating a summary output based on the second set of token representations.