Patent ID: 11900703
Assignee: HISTOWIZ, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 25:
26. A non-transitory computer-readable storage media encoded with a computer program including instructions executable by a processor to create a whole slide histopathology image multi-tagging application comprising:
a) an intake module configured to receive a digital whole slide image and sample a plurality of non-background patches from the whole slide image at a plurality of magnifications;
b) a machine learning model to predict multiple slide-level tags from the plurality of non-background patches, wherein the machine learning model comprises:
i. a visual feature extraction module configured to extract a visual feature representation of each patch;
ii. a patch transformation module configured to modulate patch features by providing the extracted visual feature representations as inputs to a first attention aggregation mechanism performing operations comprising: mapping each visual feature representation into different attention-weighted features through multi-head attentions and outputting a weighted patch feature comprising an aggregation of the visual feature representations and the attention-weighted features;
wherein the machine learning model operates in a multi-scale mode utilizing the patches at the plurality of magnifications to identify both patch features and magnification levels most discriminant for the multi-tagging;

iii. a multi-tag attention module configured to construct slide-level tags identifying tag classifications by providing the weighted patch features to a plurality of second attention aggregation mechanisms, one per output head, each performing operations comprising: mapping each weighted patch feature into different attention-weighted features through multi-head attentions, and outputting a slide-level tag comprising an aggregation of the attention-weighted features; and

c) a tagging module configured to associate the multiple slide-level tags with the digital whole slide image.