Patent ID: 11893012
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A system, comprising:
one or more computing devices that implement an analytics service of a provider network;
wherein the one or more computing devices include instructions that upon execution on or across the one or more computing devices cause the one or more computing devices to:
store, in a repository of the analytics service, respective sets of descriptors corresponding to content identified within images of individual ones of one or more reference documents, wherein a particular descriptor comprises information about at least a text key within an image of a reference document;
in response to a first request to extract content from an image of a particular document, perform a first content extraction on the image of the particular document, to:
identify, using a first query directed to the repository, at least a first reference document, wherein the particular document is distinct from the one or more reference documents including the first reference document, and wherein a first set of descriptors of the first reference document meets a similarity criterion with respect to a second set of descriptors obtained from the image of the particular document, wherein the second set of descriptors is obtained using a first set of one or more machine learning models to which the image of the particular document has been provided as input, and wherein the first set of one or more machine learning models comprises an optical character recognition model; and
based at least in part on a result of the first query, perform a layout-aware matching analysis of the particular document and the first reference document to project respective portions of the image of the particular document onto the first reference document, wherein the layout-aware matching analysis comprises computing an estimate of a difference in location of (a) a first key within a projected representation of the image of the particular document and (b) the first key within the first reference document, and the projected representation is used to extract a first predicted value corresponding to the first key from the particular document; and

in response to a second request to extract content from an image of another document after the respective sets of descriptors have been stored, perform a second content extraction on the image of the other document, to:
execute a second query on the repository, wherein a result of the second query indicates that the repository does not comprise a reference document whose descriptors meet the similarity criterion with respect to descriptors obtained from the other document; and
based at least in part on the result of the second query, utilize a second set of one or more machine learning models to extract, from the other document, a second predicted value corresponding to a second key, wherein the second set of one or more machine learning models comprises at least one model which is not used for the layout-aware matching analysis;

wherein the first content extraction achieves a performance improvement based at least in part on use of the layout-aware matching analysis of the particular document and the first reference document, to (a) improve an accuracy of extracted content or (b) reduce an amount of compute, memory, or storage resources used to extract content.