I SOMETRIC P ROPAGATION N ETWORK FOR G ENERALIZED Z ERO - SHOT L EARNING
Anonymous authors
Paper under double-blind review
A BSTRACT
Zero-shot learning (ZSL) aims to classify images of an unseen class only based on a few attributes describing that class but no access to any training sample.
A popular strategy is to learn a mapping between the semantic space of class attributes and the visual space of images based on the seen classes and their data.
Thus, an unseen class image can be ideally mapped to its corresponding class attributes.
The key challenge is how to align the representations in the two spaces.
For most ZSL settings, the attributes for each seen/unseen class are only represented by a vector while the seen-class data provide much more information.
Thus, the imbalanced supervision from the semantic and the visual space can make the learned mapping easily overﬁtting to the seen classes.
To resolve this problem, we propose Isometric Propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces.
Speciﬁcally, IPN learns to propagate the class representations on an auto-generated graph within each space.
In contrast to only aligning the resulted static representation, we regularize the two dynamic propagation procedures to be isometric in terms of the two graphs’ edge weights per step by minimizing a consistency loss between them.
IPN achieves state-of-the-art performance on three popular ZSL benchmarks.
To evaluate the generalization capability of IPN, we further build two larger benchmarks with more diverse unseen classes, and demonstrate the advantages of IPN on them.
I NTRODUCTION
One primary challenge on the track from artiﬁcial intelligence to human-level intelligence is to improve the generalization capability of machine learning models to unseen problems.
While most supervised learning methods focus on generalization to unseen data from training task/classes, zeroshot learning (ZSL) (Larochelle et al., 2008; Lampert et al., 2014; Xian et al., 2019a) has a more ambitious goal targeting the generalization to new tasks and unseen classes.
In the context of image classiﬁcation, given images from some training classes, ZSL aims to classify new images from unseen classes with zero training image available.
Without training data, it is impossible to directly learn a mapping from the input space to the unseen classes.
Hence, recent works introduced learning to map between the semantic space and visual space (Zhu et al., 2019a; Li et al., 2017; Jiang et al., 2018)
so that the query image representation and the class representation can be mapped to a shared space for comparison.
However, learning to align the representation usually leads to overﬁtting on seen classes (Liu et al., 2018; Jiang et al., 2020).
One of the reasons is that in most zero-shot learning settings, the number of images per class is hundreds while the semantic information provided for one class is only limited to one vector.
Thus, the mapping function is easily overﬁtting to the seen classes when trained using the small set of attributes.
The learned model typically has imbalanced performance on samples from the seen classes and unseen classes, i.e., strong when predicting samples from the seen classes but struggles in the prediction for samples in the unseen classes.
In this paper, we propose “ Isometric Propagation Network (IPN)” which learns to dynamically interact between the visual space and the semantic space.
Within each space, IPN uses an attention module to generate a category graph, and then perform multiple steps of propagation on the graph.
In every step, the distribution of the attention scores generated in each space are regularized to be
[Figure]
isometric so that implicitly aligns the relationships between classes in two spaces.
Due to different motifs of the generated graphs, the regularizer for the isometry between the two distributions are provided with sufﬁcient training supervisions and can potentially generalize better on unseen classes.
To get more diverse graph motifs, we also apply episodic training which samples different subset of classes as the nodes for every training episode rather than the whole set of seen classes all the time.
To evaluate IPN, we compared it with several state-of-the-art ZSL methods on three popular ZSL benchmarks, i.e., AWA2 (Xian et al., 2019a), CUB (Welinder et al., 2010) and aPY (Farhadi et al., 2009).
To test the generalization ability on more diverse unseen classes rather than similar classes to the seen classes, e.g., all animals for AWA2 and all birds for CUB, we also evaluate IPN on two new large-scale datasets extracted from tiered ImageNet (Ren et al., 2018), i.e., tiered ImageNet-Mixed and tiered ImageNet-Segregated.
IPN consistently achieves state-of-the-art performance in terms of the harmonic mean of the accuracy on unseen and seen classes.
We show the ablation study of each component in our model and visualize the ﬁnal learned representation in each space in Figure 1.
It shows the class representations in each space learned by IPN are evenly distributed and connected smoothly between the seen and unseen classes.
The two spaces can also reﬂect each other.
R ELATED W ORKS
Most ZSL methods can be categorized as either discriminative or generative.
Discriminative methods generate the classiﬁers for unseen classes from its attributes by learning a mapping from a input image’s visual features to the associated class’s attributes on training data of seen classes (Frome et al., 2013; Akata et al., 2015a;b; Romera-Paredes & Torr, 2015; Kodirov et al., 2017; Xian et al., 2016; Socher et al., 2013; Cacheux et al., 2019; Li et al., 2019c), or by relating similar classes according to their shared properties, e.g., semantic features (Norouzi et al., 2013) or phantom classes (Changpinyo et al., 2016), etc.
In contrast, generative methods employ the recently popular image generation models (Goodfellow et al., 2014) to synthesize images of unseen classes, which reduces ZSL to supervised learning (Xian et al., 2018; Huang et al., 2019b) but usually increases the training cost and model complexity.
Alignment between two spaces in ZSL Frome et al.
; Socher et al.
; Zhang et al.
(2017) proposed to transform representations either to the visual space or the semantic space for distance for comparison.
Li et al.
(2017) introduces how to optimize the semantic space so that the two spaces can be better aligned.
Jiang et al.
(2018) tries to map the class representations from two spaces to another common space.
Comparatively, our IPN does not align two spaces by feature transformation but by regularizing the isometry of the intermediate output from two propagation model.
Graph in ZSL Graph structures have already been explored for their beneﬁts to some discriminative
ZSL models, e.g., Wang et al.
Kampffmeyer et al.
(2019) learn to generate the weights of a fully-connected classiﬁer for a given class based on its semantic features computed on a pre-deﬁned semantic graph using a graph convolutional networks (GCN).
They require (1) a pre-trained CNN whose last fully-connected layer provides the ground truth of the classiﬁers ZSL aims to generate; and (2) an existing knowledge graph as the input of GCN.
By contrast, our method learns to automatically generate the graph and the class prototypes composing the support set of a KNN classiﬁer.
Hence, our method does not require a pre-trained CNN to provide a ground truth for the KNN classiﬁer.
In addition, we train a modiﬁed graph attention network (GAN) (rather than a GCN) to generate the prototypes, so it learns both the class adjacency and edges by itself and can provide more accurate modeling of class dependency in scenarios when no knowledge graph is provided.
Tong et al.
and Xie et al.
(2020) also leverage the idea of graph to transform the feature extracted from the CNN while our graph serves as the paths for propagation.
Zhang et al.
(2019) uses GAN to generate data and label propagation for prediction
It requires an off-the-shelf graph generated by ConceptNet (Speer et al while IPN applies prototype propagation and learns to construct the graph in end-
-end manner.
Attention in ZSL The attention module has been applied for zero-shot learning for localization.
Liu et al. (2019a) introduces localization on the semantic information and Xie et al.
(2019) proposes localization on the visual feature.
Zhu et al.
(2019c) further extends it to multi-localization guided by semantic.
Different from them, our attention serves as generating the weights for propagation.
Our episodic training strategy is inspired by the meta-learning method from (Santoro et al., 2016).
Similar methods have been broadly studied in meta-learning and few-shot learning approaches (Finn et al., 2017; Snell et al., 2017; Sung et al., 2018) but rarely studied for ZSL in previous works.
In this paper, we show that training ZSL model as a meta-learner can improve the generalization performance and mitigate overﬁtting.
P ROBLEM S ETTING
We consider the following setting of ZSL.
The training set is composed of data-label pairs ( x , y ) , where x ∈ X is a sample from class y , and y must be one of the seen classes in Y seen .
During the test stage, given a test sample x of class y , the ZSL model achieved in training is expected to produce the correct prediction of y .
In order to relate the unseen classes in test to the seen classes in training, we are also given the semantic embedding s y of each class y ∈ Y seen ∪ Y unseen .
We further assume that seen and unseen classes are disjoint, i.e., Y seen ∩Y unseen = ∅ .
The test samples come from both Y seen and Y unseen in generalized zero-shot learning.
I SOMETRIC PROPAGATION N ETWORK
In this paper, we bridge the two spaces via class prototypes in both the semantic and visual spaces, i.e., a semantic prototype P sy and a visual prototype P vy for each class y .
In order to generate high-quality prototypes, we develop a learnable scheme called “ Isometric Propagation Network (IPN)” to iteratively reﬁne the prototypes by propagating them on a graph of classes so each prototype is dynamically determined by the samples and attributes of its related classes.
In this section, we will ﬁrst introduce the architecture of IPN.
An illustration of IPN is given in
Figure 2.
Then, we will elaborate the episodic training strategy, in which a consistency loss is minimized for isometric alignment between the two spaces.
4. P ROTOTYPE I NITIALIZATION
Given a task T deﬁned on a subset of classes Y T with training data D T , IPN ﬁrst initializes the semantic and visual prototype for each class y ∈ Y T .
The semantic prototype of class y is transformed from its attributes s y .
Speciﬁcally, we have drawn on the expert modules from (Zhang & Shi, 2019) to initialize the semantic prototypes P sy [0] .
In previous works (Snell et al., 2017; Santoro et al., 2016), the visual prototype of a class is usually computed by averaging the features of images from the class.
In IPN, for each seen class y ∈ Y T ∩ Y seen , we initialize its visual prototype P vy [0] by
[Equation]
where f cnn ( · ) is a backbone convolutional neural nets (CNN) extracting features from raw images, and W is a linear transformation aiming to change the dimension of the visual prototype to be the same as semantic prototypes.
We argue that W is necessary because (1) the visual feature produced by the CNN is usually in high dimension, and reducing its dimension can save signiﬁcant computations during propagation; (2) keeping the dimensions of the visual and semantic prototypes the same can avoid the dominance of either of them in the model.
During test, we initialize the visual prototype of each unseen class y ∈ Y T ∩Y unseen as the weighted sum of its K -nearest classes from Y seen in the semantic space 1 , i.e.,
[Equation]
where W is the same transformation from Eq.
(1), and N y is the set of the top- K ( K is a hyperparameter) seen classes 2 with the largest similarities
c s ( P sy [0] , P sz [0]) to class y in semantic space.
4. C ATEGORY G RAPH G ENERATION
Although the propagation needs to be conducted on a category graph, IPN does not rely on any given graph structure as many previous works (Kampffmeyer et al., 2019;
Wang et al., 2018)
so it has less limitations when applied to more general scenarios.
In IPN, we inference the category graph of Y T by applying thresholding to the attention scores between the initialized prototypes.
In the visual space, the set of edges is deﬁned by
[Equation]
where (cid:15) is the threshold.
Thereby, the category graph is generated as G T,v = ( Y T , E T,v ) .
Similarly, in the semantic space, we generate another graph G T,s =
( Y T , E T,s ) .
4. I SOMETRIC P ROPAGATION
Given the initialized prototypes and generated category graphs in the two spaces, we apply the following prototype propagation in each space for τ steps in order to reﬁne the prototype of each class.
In the visual space, each propagation step updates
[Equation]
In the semantic space, we apply similar propagation steps over classes on graph G T,s by using the attention module associated with a s ( · , · ) .
Intuitively, the attention module computes the similarity between the two prototypes.
In the visual space, given the prototypes P vy and P vz for class y and z , the attention module ﬁrst computes the following cosine similarity:
[Equation]
where h v ( · ) is a learnable transformation and (cid:107) · (cid:107) 2 represents the (cid:96) 2 norm.
For a class y on a category graph, we can use the attention module to aggregate messages from its neighbor classes
N y .
In this case, we need to further normalize the similarities of y to each z ∈ N y as follows.
[Equation]
where γ is a temperature parameter controlling the smoothness of the normalization.
In the semantic space, we have similar deﬁnitions of h s ( · ) , c s ( · , · ) and a s ( · , · ) .
In the following, we will use P vy [ t ] and P sy [ t ] to represent the visual and semantic prototype of class y at the t th step of propagation.
We will use | · | to represent the size of a set and ReLU( · ) (cid:44) max {· , 0 } .
After τ steps of propagation, we concatenate the prototypes achieved in the two spaces for each class to form the ﬁnal prototypes:
[Equation]
The formulation of a s ( · , · ) and c s ( · , · ) follows the same way as a v ( · , · ) and c v ( · , · ) .
To be simple, we don’t rewrite the formulations here.
4. C LASSIFIER GENERATED FROM P ROTOTYPES
Given the ﬁnal prototypes and a query/test image x , we apply a fully-connected layer to predict the similarity between x and each class’s prototype.
For each class y ∈ Y T , it computes
[Equation]
Intuitively, the above classiﬁer can be interpreted as a soft KNN classiﬁer with similarity metric f ( · , · ) .
When applied to a query image x , the predicted class is the most probable class, i.e., ˆ y = arg max y ∈Y T Pr( y | x ; P ) , where Pr denotes the probability.
4. T RAINING S TRATEGIES
Episodic training on subgraphs.
In each episode, we sample a subset of training classes Y T ⊆
Y seen , and then build a training (or support) set D Ttrain and a validation (or query) set D Tvalid from the training samples of Y T .
They are two disjoint sets of samples from the same subset of training classes.
We only use D Ttrain to generate the prototypes P by the aforementioned procedures, and then apply backpropagation to minimize the loss of D Tvalid .
This equals to an empirical risk minimization, i.e., min (cid:88)
[Equation]
This follows the same idea of meta-learning Ravi & Larochelle (2016): it aims to minimize the validation loss instead of the training loss and thus can further improve generalization and restrain overﬁtting.
Moreover, since we train the IPN model on multiple random tasks T and optimize the propagation scheme over different subgraphs (each generated for a random subset of classes), the generalization capability of IPN to unseen classes and new tasks can be substantially enhanced by this meta-training strategy.
Moreover, it effectively reduces the propagation costs since computing the attention needs O ( |Y T | 2 ) operations.
In addition, as shown in the experiments, episodic training is effective in mitigating the problem of data imbalance and overﬁtting to seen classes, which harasses many ZSL methods.
Consistency loss for alignment of visual and semantic space.
In ZSL, the interaction and alignment between different spaces can provide substantial information about class dependency and is critical to the generation of accurate class prototypes.
In IPN, we apply prototype propagation in two separate spaces on their own category graphs by using two untied attention modules.
In order to keep the propagation in the two spaces consistent, we add an extra consistency loss to the training objective in Eq.
It computes the KL-divergence between two distributions deﬁned in the two spaces for each class y ∈ Y T at every propagation step t :
[Equation]
where
[Equation]
The consistency loss enforces the two attention modules in the two spaces to generate the same attention scores over classes, even when the prototypes and attention module’s parameter are different in the two spaces.
In this way, they share similar class dependency over the propagation steps.
So the generated dual prototypes are well aligned and can provide complementary information when generating the classiﬁer.
The ﬁnal optimization we used to train IPN becomes:
[Equation]
where λ is weight for the consistency loss, and P , p vy [ t ] , and p sy [ t ] are computed only based on
D Ttrain , the training (or support) set of task- T for classes in Y T .
E XPERIMENTS
We evaluate IPN and compare it with several state-of-the-art ZSL models in both ZSL setting (where the test set is only composed of unseen classes) and generalized ZSL setting (where the test set is composed of both seen and unseen classes).
We report three standard ZSL evaluation metrics, and provide ablation study of several variants of IPN in the generalized ZSL setting.
The visualization of the learned class representation is shown in Figure 1.
5. D ATASETS AND E VALUATION CRITERIA
Small and Medium benchmarks.
Our evaluation includes experiments on three standard benchmark datasets widely used in previous works, i.e., AWA2 (Xian et al., 2019a), CUB (Welinder et al., 2010) and aPY (Farhadi et al., 2009).
The former two are extracted from ImageNet-1K, where
AWA2 contains 50 animals classes and CUB contains 200 bird species.
aPY includes 32 classes.
Detailed statistics of them can be found in Table 1.
Two new large-scale benchmarks.
The three benchmark datasets above might be too small for modern DNN models since the training may suffers from large variance and end-to-end training usually fails.
Previous ZSL methods therefore ﬁne-tune a pre-trained CNN to achieve satisfactory performance on these datasets, which is costly and less practical.
Given these issues, we proposed two new large-scale datasets extracted from tiered ImageNet (Ren et al., 2018), i.e., tiered ImageNetSegregated and tiered ImageNet-Mixed, whose statistics are given in Table 1.
Comparing to the three benchmarks, they are over 10 × larger, and training a ResNet101 (He et al., 2016) from scratch on their training sets can achieve a validation accuracy of 65 % , indicating that end-to-end training without ﬁne-tuning is possible.
These two benchmarks differ in that the “Mixed” version allows a seen class and an unseen class to belong to the same super category, while the “Segregated” version does not.
The class attributes are the word embeddings of the class names provided by GloVe (Pennington et al., 2014).
We removed the classes that do not have embeddings in GloVe.
Following the most popular setting of ZSL (Xian et al., 2019a), we report the per-class accuracy and harmonic mean of the accuracy on seen and unseen classes. [Table]
[Table]
5. I MPLEMENTATION AND H YPERPARAMETER D ETAILS
For a fair comparison, we follow the setting in (Xian et al., 2019a) and use a pre-trained ResNet101 (He et al., 2016) to extract 2048-dimensional image features without ﬁne-tuning.
For the three standard benchmarks, the model is pre-trained on ImageNet-1K. For the two new datasets, it is pre-trained on the training set used to train ZSL models.
All hyperparameters were chosen on the validation sets provided by Xian et al. (2019a).
We use the same hyperparameters tuned on AWA2 for other datasets since they are of similar data type, except for aPY on which we choose learning rate 1 .
× 10 − 3 and weight decay 5 .
× 10 − 4 .
For the rest of the datasets, we train IPN by Adam (Kingma & Ba, 2015) for 360 epochs with a weight decay factor 1 .
× 10 − 4 .
The learning rate starts from 2 .
× 10 − 5 and decays by a multiplicative factor 0.1 every 240 epochs.
In each epoch, we update IPN on multiple N -way- K -shot tasks ( N -classes with K samples per class), each corresponding to an episode mentioned in Section 4.5.
In our experiments, the number of episodes in each epoch is n / NK , where n is the total size of the training set, N =and K = 1 (a small K can mitigate the imbalance between seen and unseen classes); h v and h s are linear transformations; we set temperature γ = 10 , threshold (cid:15) =
cos 40 o for the cosine similarities between class prototypes, and weight for consistency loss λ = 1 .
We use propagation steps τ = 2.
5. M AIN R ESULTS
The results are reported in Table 2.
Although most previous works (Xian et al., 2019b;a; Sariyildiz & Cinbis, 2019) show that the more complicated generative models usually outperforms the discriminative ones.
Our experiments imply that IPN, as a simple discriminative model, can signiﬁcantly outperform the generative models requiring much more training efforts.
Most baselines achieve a high ACC seen but much lower ACC unseen since they suffer from and more sensitive to the imbalance between seen and unseen classes.
In contrast, IPN always achieves the highest H-metric among
[Table]
In scaling the experiments to the two large datasets, discriminative models perform better than generative models, which are easily prone to overﬁtting or divergence.
Moreover, during the test stage, generative models require to synthesize data for unseen classes, which may lead to expensive computations when applied to tasks with many classes.
In contrast, IPN scales well to these large datasets, outperforming state-of-the-art baselines by a large margin.
The performance of most models on tiered ImageNet-Segregated is generally worse than tiered ImageNet-Mixed because that the unseen classes in the former do not share the same ancestor classes with the seen classes, while in the latter such sharing is allowed.
5. A BLATION S TUDY
Are two prototypes necessary?
We compare IPN with its two variants, each using only one out of the two types of prototypes from the two spaces when generating the classiﬁer.
The results are shown in Table 3a.
It shows that semantic prototype (only) performs much better than visual prototype (only), especially on the unseen classes.
This is because that the unseen classes’ semantic embedding is given while its visual information is missing (zero images) during the test stage.
Although IPN is capable of generating a visual prototype based on other classes’ images and class dependency, it is much less accurate than the semantic prototype.
However, it also shows that the visual prototypes (even the ones of unseen classes) can provide extra information and further improve the performance of the semantic prototype, e.g., they improve the unseen classes’ accuracy from 61.1% to 67.5%.
One possible reason is that the visual prototype bridges the query image’s visual feature with the semantic features of classes, and thus provides an extra “visual context” for the localization of the correct class.
In addition, we observe that IPN with only the visual prototype converges faster than
IPN with only a semantic prototype (30 epochs vs. 200 epochs).
This might be an artifact caused by the pre-trained CNN, which already produces stable visual prototypes since the early stages.
However, it raises a caveat that heavily relying on a pre-trained model might discourage the learning and results in a bad local optimum.
How’s the beneﬁt of propagation?
We then evaluate the effects of the propagation scheme in the two spaces.
In particular, we compare IPN with its variants that apply the propagation steps in only one of the two spaces, and the variant that does not apply any propagation after initialization.
Their performance is reported in Table 3b.
Without any propagation, the classiﬁcation performance heavily relies on pre-trained CNN.
Hence, the accuracy of seen classes can be much higher than that of the unseen classes.
Compared to IPN without any propagation, applying propagation in either the visual space or the semantic space can improve the accuracy of unseen classes by 2 ∼ 3%, while the accuracy on the seen classes stays almost the same.
When equipped with dual propagation, we observe a signiﬁcant improvement of the accuracy of the unseen classes but a degradation on the seen classes.
However, they lead to the highest H-metric, which indicates that the dual propagation achieves a desirable trade-off between overﬁtting on seen classes and generalization to unseen classes.
Effect of Isometric Alignment In IPN, we minimize a consistency loss during training for alignment of the visual and semantic space.
Here, we consider two other options for alignment that exchange the attention modules used in the two spaces, i [Figure] e., using the visual attention module in the semantic space
[Figure]
or using the semantic attention module in the visual space.
We also include the
IPN trained without consistency loss or any alignment as a baseline.
Their results are reported in
Table 3c.
Compared to IPN with/without consistency loss, sharing a single attention module between the two spaces suffers from the overﬁtting problem on the seen classes.
This is because that the visual features and semantic embeddings are two different types of features so they should not share the same similarity metric.
In contrast, the consistency loss only enforces the output similarity scores of the two attention modules to be close, while the models/functions computing the similarity can be different.
Therefore, to produce the same similarity scores, it does not require each class having the same features in the two spaces.
Hence, it leads to a much softer regularization
[Figure]
S ENSITIVITY OF HYPERPARAMETERS
[Figure] [Figure] [Figure] The proposed IPN has some hyperparameters as mentioned in Section 5.2.
To analyze the sensitivity of those hyperparameters, such as threshold, temperature, N and K for N-way-K-shot task, propagation steps, etc.
We try different values and show the performance on the AWA2 dataset in
Figure 3.
The temperature γ is not sensitive for the accuracy.
The threshold (cid:15) needs to be bigger than cosine(30 o ) to give IPN some space for the ﬁltering.
The propagation steps τ is best at 2.
Weight decay is a sensitive hyperparameter and this is also found in previous ZSL literature (Zhang & Shi, 2019; Liu et al., 2019b).
C ONCLUSIONS
In this work, we propose an Isometric Propagation Network, which learns to relate classes within each space and across two spaces.
IPN achieves state-of-the-art performance and we show the importance of each component and visualize the interpretable relationship of the learned class representations.
In future works, we will study other possibilities to initialize the visual prototypes of unseen classes in IPN.
A E VALUATION C RITERIA
Considering the imbalance between test classes by following the most recent works (Xian et al., 2019a) given a test set D , we evaluate ZSL methods’ performance based on the per-class accuracy averaged over a targeted set of classes Y , i.e.,
[Equation]
Following Xian et al.
(2019a)
, we report ACC Y when Y
= Y seen and Y
=
Y unseen , and the harmonic mean of them, i.e.,
[Equation]
Methods
CUB
[Table]
M
[Table]
GenerativeModels f-CLSWGAN (Xian et al., 2018) [Table] [Table] LisGAN (Li et al., 2019b) 58.
[Table] [Table] f-VAEGAN-D2 (Xian et al., 2019b) 61. [Table]
DiscriminativeModels
SAE (Kodirov et al., 2017) 33.
[Table]
DEM (Zhang et al., 2017) 51.
[Table]
RN (Sung et al., 2018) 55. GAFE (Liu et al., 2019b) 52. [Table] [Table]
B Z ERO - SHOT L EARNING
The comparison between IPN to other baselines on the setting of Zero-shot Learning is shown in
Table 4.
On CUB, AWA2, aPU, tiered -M , and tiered -S , our IPN outperforms other algorithms by a large margin.
C M ORE E XPERIMENTS ON L ARGE - SCALE D ATASETS
In our work, we follow the train/test split suggested by Frome et al.
(2013), who proposed to use the 21K ImageNet dataset for zero-shot evaluation.
Here, 2-hops and 3-hops represent that the evaluation is over the classes which are 2/3 hops away from ImageNet-1k.
Different from experiments on
AWA2, we use N =20 and K =5 for the full ImageNet.
Besides, since the edges between each class are provided on the full ImageNet, we directly use them instead of calculating by Eq. Methods [Table]
ConSE (Norouzi et al., 2013)
[Table]
SYNC (Changpinyo et al., 2016)
[Table]
EXEM (Changpinyo et al., 2017)
[Table] [Table] [Table] [Table]
GCNZ (Wang et al., 2018) 19.
[Table]
[Table]
[Table]
[Table] DGP (Kampffmeyer et al., 2019) [Table] [Table]
[Table]
[Table]
IPN (ours)
[Table]