Patent Document ID: 10102454
Application ID: 15813401

Base Claim:
1. A computer-implemented method comprising: utilizing two or more classifiers to calculate, for an input image, probability scores for a plurality of classes based on visual information extracted from the input image and semantic relationships in a classification hierarchy, wherein each of the two or more classifiers is associated with a given one of two or more levels in the classification hierarchy with each level in the classification hierarchy comprising a subset of the plurality of classes; classifying the input image based on the calculated probability scores; training the two or more classifiers to calculate probability scores for respective subset of the plurality of classes; and performing label inference to refine classification probabilities in the two or more classifiers based on semantic relationships in the classification hierarchy; wherein training the two or more classifiers and performing label inference comprises at least one of: taking as input a graph structure having initial values for nodes corresponding to classification probabilities in the two or more classifier and outputting the graph structure with modified values for the nodes; utilizing a multi-task learning based loss function that jointly optimizes classifiers associated with each of the two or more levels in the classification hierarchy; and utilizing a random walk process that smooths classification probabilities over two or more classes in a same semantic path in the classification hierarchy.

---

Claim 3:
3. The method of claim 1 , wherein the classification hierarchy comprises a tree structure and performing label inference to refine the classification probabilities in the two or more classifiers comprises, for two or more leaf nodes having classification probabilities within a designated threshold from one another, adjusting the classification probabilities for the two or more leaf nodes based on classification probabilities for parent nodes in a higher level of the classification hierarchy relative to the two or more leaf nodes.