Patent ID: 11880657
Assignee: WELLS FARGO BANK, N.A.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 9:
10. An apparatus for automatically determining accuracy of entity recognition of text, the apparatus comprising:
segmentation circuitry configured to segment a service entity recognition analysis result of the text and a gold entity recognition analysis result of the text into common superstrings that define entity spans;
classification circuitry configured to classify each of the entity spans based on an accuracy of entity recognition in the service entity recognition analysis result of the text corresponding to the entity spans using a classification system that differentiates accurately identified but improperly bounded entities into at least three subcategories to obtain an entity accuracy classification, wherein classifying an entity span of the entity spans further comprises:
identifying, by the classification circuitry, a difference between the service entity recognition analysis result and the gold entity recognition analysis result as an error of the service entity recognition analysis result, wherein the difference includes one of the accurately identified but improperly bounded entities,
classifying, by the classification circuitry, the error into one category of a group of categories, and
using, by scoring circuitry, the error that is classified into the one category in obtaining a score report for a service that generated the service entity recognition analysis result of the text; and

the scoring circuitry configured to:
obtain, for the service that generated the service entity recognition analysis result of the text, the score report based on the entity accuracy classification, and
perform an action set based on the entity accuracy classification.