Patent ID: 11915500
Assignee: SALESFORCE, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. A computer system comprising:
one or more computer processors; and
a non-transitory computer readable storage medium storing instructions that when executed by the one or more processors causes the one or more processors to perform operations comprising:
receiving a request to recognize text in an input image, the input image comprising a scene with embedded text;
providing the input image to a convolutional neural network component to generate visual features and multi-scale visual features based on the input image, the visual features represented as a feature vector of lower dimension than the input image;
providing the multi-scale visual features output by the convolutional neural network component to an attention extractor neural network component, wherein the attention extractor neural network component:
summarizes the multi-scale visual features into summarized visual features:
generates attention features based on the summarized visual features of the input image, each attention feature corresponding to an attention map for a visual feature, and
adds attention maps to corresponding attention features to generate added attention features;

combining the visual features and the added attention features to obtain mixed features;
providing the mixed features as input to a character recognizer neural network component, wherein the character recognizer neural network component generates an attention score based on hidden features of the character recognizer neural network, the character recognizer neural network component outputting a sequence of characters representing predicted text recognized from the input image; and
providing the recognized text from the input image for display.