# Zero-Shot Recognition Enhancement by Distance-Weighted Contextual Inference
## 3 Method
### Contextual Inference
#### 3.4.1 Similarity-Based Association Measurements  
To grasp the context of an unseen object, we determine associations through similarities between the class label of the target object and those of its surrounding objects. Note that we assume that the surrounding objects are recognized before beginning ZSR for the unseen object, as explained in Section 3.2. When measuring similarities for contextual inference, we consider not only the top-1 classified label of the surrounding objects, but also all potentially-ranked labels of their candidates. This helps to alleviate the problem of recognizing misclassified surrounding objects as the representative context and allows our system to consider multiple potential contexts. External knowledge sources--a knowledge graph, a semantic embedding space, and both together--are used for three different similarity measurements: semantic similarity (SM), cosine similarity (CS), and the harmony of both (HM), respectively. * Semantic Similarity Measurement  
The SM metric is defined over documents and is based on the likeness of conceptual meanings [47]. In this paper, we use a hierarchical knowledge graph, ontology, representing the hierarchical concepts of objects for the SM measurement. Various measures [48] can be used with a knowledge graph, such as path and depth measures, information content-based measures, feature measures, and hybrid measures. We adopt three typical path and depth-based measures for our experiments, as presented in Section 4.2.2. A SM-based association, \(S^{SM}\), between one unseen class label and a single surrounding object, is given by  
\[S^{SM}(e,si)=\max_{1\leq k\leq N^{cam}}p_{k}\cdot sem(c_{e},c_{k}), \tag{5}\]  
where the inputs are a semantic embedding \(e\) of an unseen class label \(c_{e}\) and the surrounding information \(si\) for a single surrounding object. \(c_{k}\) denotes a class label of the \(k\)-th predicted candidate for the surrounding object. Each SM value between \(c_{e}\) and \(c_{k}\) is weighted by \(p_{k}\), a prediction probability for the candidate that implies the reliability of the measured similarity. An association's output is the maximization of the weighted similarities for candidates because finding the best combination of an unseen class label and all candidates for all surrounding objects is equivalent to finding and combining the best candidate for each surrounding object. * Cosine Similarity Measurement  
To measure associations, we next use CS, which is available in semantic embedding spaces. Various types of semantic embedding spaces are used in ZSR, such as manually-annotated attributes [11; 12], text descriptions of images [49], word embeddings [26; 27], and rdf graph embeddings [50]. Among those, the attribute space represents fine-grained concepts of each class label with a binary value depicting the presence/absence of an attribute, based on human annotated description (e.g., for attributes horse-like, stripe, and green, \(e_{zebra}=[1,1,0]\), \(e_{tiger}=[0,1,0]\), and \(e_{watermelon}=[0,1,1]\), respectively). The word embedding space contains vectors learned by neural net with a certain feature dimension according to the mutual frequency of words in the context in thetext corpus (e.g., due to simultaneous occurrence of words, \"monkey\" and \"banana\", the magnitude and direction of \(e_{monkey}\) and \(e_{banana}\) can be similar). In terms of ZSR performance, embedding spaces based on manual construction such as attributes and text descriptions of images are generally more effective than a word embedding space [6; 12; 51; 52]. However, a word embedding space constructed in an unsupervised manner still has higher versatility and utility than the other options because it costs much less and enables large-scale recognition with many class labels [53]. We thus use word embedding spaces as the semantic embedding space \(E\) for the evaluation, although our proposed model works independently on the type of semantic space.