Original: A DAPTIVE AND G ENERATIVE Z ERO -S HOT L EARNING
Revised: A DAPTIVE AND G ENERATIVE Z ERO -S HOT L EARNING

Original: Paper ID

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We address the problem of generalized zero-shot learning (GZSL) where the task is to predict the class label of a target image whether its label belongs to the seen or unseen category.
Revised: We address the problem of generalized zero-shot learning (GZSL) where the task is to predict the class label of a target image whether its label belongs to the seen or unseen category.

Original: Similar to ZSL, the learning setting assumes that all class-level semantic features are given, while only the images of seen classes are available for training.
Revised: Similar to ZSL, the learning setting assumes that all class-level semantic features are given, while only the images of seen classes are available for training.

Original: By exploring the correlation between image features and the corresponding semantic features, the main idea of the proposed approach is to enrich the semantic-to-visual (S2V) embeddings via a seamless fusion of adaptive and generative learning.
Revised: By exploring the correlation between image features and the corresponding semantic features, the main idea of the proposed approach is to enrich the semantic-to-visual (S2V) embeddings via a seamless fusion of adaptive and generative learning.

Original: To this end, we extend the semantic features of each class by supplementing image-adaptive attention so that the learned S2V embedding can account for not only inter-class but also intra-class variations.
Revised: To this end, we extend the semantic features of each class by supplementing image-adaptive attention so that the learned S2V embedding can account for not only inter-class but also intra-class variations.

Original: In addition, to break the limit of training with images only from seen classes, we design a generative scheme to simultaneously generate virtual class labels and their visual features by sampling and interpolating over seen counterparts.
Revised: In addition, to break the limit of training with images only from seen classes, we design a generative scheme to simultaneously generate virtual class labels and their visual features by sampling and interpolating over seen counterparts.

Original: In inference, a testing image will give rise to two different S2V embeddings, seen and virtual .
Revised: In inference, a testing image will give rise to two different S2V embeddings, seen and virtual.

Original: The former is used to decide whether the underlying label is of the unseen category or otherwise a speciﬁc seen class; the latter is to predict an unseen class label.
Revised: The former is used to decide whether the underlying label is of the unseen category or otherwise a speciﬁc seen class; the latter is to predict an unseen class label.

Original: To demonstrate the effectiveness of our method, we report state-of-the-art results on four standard
Revised: To demonstrate the effectiveness of our method, we report state-of-the-art results on four standard

Original: GZSL datasets, including an ablation study of the proposed modules.
Revised: GZSL datasets, including an ablation study of the proposed modules.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Different from conventional learning tasks, zero-shot learning (ZSL) by Lampert et al.
Revised: Different from conventional learning tasks, zero-shot learning (ZSL) by Lampert et al.

Original: Palatucci et al.
Revised: Palatucci et al.

Original: ; Akata et al.
Revised: ; Akata et al.

Original: (2013) explores the extreme case of performing inference only over samples of unseen classes.
Revised: (2013) explores the extreme case of performing inference only over samples of unseen classes.

Original: To make the scenario more realistic, generalized zero-shot learning (GZSL) (Chao et al., 2016; Xian et al., 2017) is subsequently proposed so that inference can concern samples of both seen and unseen classes.
Revised: To make the scenario more realistic, generalized zero-shot learning (GZSL) (Chao et al., 2016; Xian et al., 2017) is subsequently proposed so that inference can concern samples of both seen and unseen classes.

Original: Nevertheless.
Revised: Nevertheless.

Original: the learning setting in ZSL/GZSL is essentially the same where sample classes are divided into two categories, seen and unseen , but only those samples of seen classes are accessible to training.
Revised: the learning setting in ZSL/GZSL is essentially the same where sample classes are divided into two categories, seen and unseen , but only those samples of seen classes are accessible to training.

Original: In addition, each of all the classes under consideration is characterized by semantic features such as attributes (Xian et al., 2018b) or text descriptions (Zhu et al., 2018) to specify and relate seen and unseen classes.
Revised: In addition, each of all the classes under consideration is characterized by semantic features such as attributes (Xian et al., 2018b) or text descriptions (Zhu et al., 2018) to specify and relate seen and unseen classes.

Original: The lack of training samples from unseen classes has prompted generative approaches (Chen et al., 2018;
Revised: The lack of training samples from unseen classes has prompted generative approaches (Chen et al., 2018;

Original: Felix et al., 2018; Kumar Verma et al., 2018; Mishra et al., 2018) to creating synthetic data from semantic features of unseen classes.
Revised: Felix et al., 2018; Kumar Verma et al., 2018; Mishra et al., 2018) to creating synthetic data from semantic features of unseen classes.

Original: The strategy could enable learning semantic-visual alignment on unseen classes implicitly, and thus improves the ability to classify unseen classes.
Revised: The strategy could enable learning semantic-visual alignment on unseen classes implicitly, and thus improves the ability to classify unseen classes.

Original: However, such generative models are indeed trained on seen samples, and the quality of synthesized unseen samples is predominantly inﬂuenced by seen classes.
Revised: However, such generative models are indeed trained on seen samples, and the quality of synthesized unseen samples is predominantly inﬂuenced by seen classes.

Original: If the number of training samples of each seen class is small, it is hard for generative models to adequately synthesize samples of unseen classes, leading to unsatisfactory zero-shot learning.
Revised: If the number of training samples of each seen class is small, it is hard for generative models to adequately synthesize samples of unseen classes, leading to unsatisfactory zero-shot learning.

Original: To better address the issue, we propose to synthesize visual and semantic features of virtual classes rather than those of the unseen classes.
Revised: To better address the issue, we propose to synthesize visual and semantic features of virtual classes rather than those of the unseen classes.

Original: An interesting analogy is that childhood experience and relevant study (Greene, 1995) suggest the behavior of using human imagination to produce new object concepts could assist our cognitive capability.
Revised: An interesting analogy is that childhood experience and relevant study (Greene, 1995) suggest the behavior of using human imagination to produce new object concepts could assist our cognitive capability.

Original: To mimic people utilizing imagination for exploring new knowledge, we create virtual classes by the integration of past ”experience” (seen classes).
Revised: To mimic people utilizing imagination for exploring new knowledge, we create virtual classes by the integration of past ”experience” (seen classes).

Original: In detail, we extend the mixup technique by Zhang et al.
Revised: In detail, we extend the mixup technique by Zhang et al.

Original: (2018) to generate virtual classes, with a subtle difference that mixing is conducted on the semantic features (in addition to the visual ones), rather than the class label vectors.
Revised: (2018) to generate virtual classes, with a subtle difference that mixing is conducted on the semantic features (in addition to the visual ones), instead of the class label vectors.

Original: In ZSL/GZSL, each seen or unseen class is typically described by a single semantic feature vector.
Revised: In ZSL/GZSL, each seen or unseen class is typically described by a single semantic feature vector.

Original: The practice is useful in differentiating different classes in a principled way, but may not be sufﬁcient to reﬂect the inter-class and intra-class visual discrepancies, not to mention the ambiguities caused by different backgrounds, view orientations, or occlusion in images.
Revised: The practice is useful in differentiating different classes in a principled way, but may not be sufﬁcient to reﬂect the inter-class and intra-class visual discrepancies, not to mention the ambiguities caused by different backgrounds, view orientations, or occlusion in images.

Original: The concern of inefﬁcient class-level representation can also be observed from how the semantic feature vectors are constructed.
Revised: The concern of inefﬁcient class-level representation can also be observed from how the semantic feature vectors are constructed.

Original: Take, for example, the Attribute Pascal and Yahoo ( aPY ) dataset (Farhadi et al., 2009), where each instance is annotated by 64 attributes.
Revised: Take, for example, the Attribute Pascal and Yahoo ( aPY ) dataset (Farhadi et al., 2009), where each instance is annotated by 64 attributes.

Original: The semantic features of each class in aPY are obtained by averaging the attribute vectors of all its instances.
Revised: The semantic features of each class in aPY are obtained by averaging the attribute vectors of all its instances.

Original: We are thus motivated to introduce an imageadaptive class representation, integrating the original semantic features for inter-class discrimination with an image-speciﬁc attention vector for intra-class variations.
Revised: We are thus motivated to introduce an imageadaptive class representation, integrating the original semantic features for inter-class discrimination with an image-speciﬁc attention vector for intra-class variations.

Original: With the addition of virtual training data and the image-adaptive class representation, our method is designed to learn two classiﬁcation experts: one for seen classes and the other for unseen classes.
Revised: With the addition of virtual training data and the image-adaptive class representation, our method is designed to learn two classiﬁcation experts: one for seen classes and the other for unseen classes.

Original: Both experts project the image-adaptive semantic feature vectors to the visual space and use cosine similarity to ﬁnd the class label most similar to the given visual feature vector.
Revised: Both experts project the image-adaptive semantic feature vectors to the visual space and use cosine similarity to ﬁnd the class label most similar to the given visual feature vector.

Original: The seen expert is trained with the provided training (seen) data, while the class prediction is over all possible classes, including seen and unseen.
Revised: The seen expert is trained with the provided training (seen) data, while the class prediction is over all possible classes, including seen and unseen.

Original: In inference, if its predicted class is not within the seen category.
Revised: In inference, if its predicted class is not within the seen category.

Original: The testing sample is deemed to be from the unseen category, whose label is then decided by the unseen expert.
Revised: The testing sample is deemed to be from the unseen category, whose label is then decided by the unseen expert.

Original: The unseen expert is trained with the virtual data only, and the process indeed resembles meta-learning.
Revised: The unseen expert is trained with the virtual data only, and the process indeed resembles meta-learning.

Original: However, the effectiveness of meta-learning is boosted by the design of the imageadaptive mechanism in that ﬁne-tuning is not needed in performing zero-shot classiﬁcation over unseen classes.
Revised: However, the effectiveness of meta-learning is boosted by the design of the imageadaptive mechanism in that ﬁne-tuning is not needed in performing zero-shot classiﬁcation over unseen classes.

Original: We characterize the main contributions of this work as follows.
Revised: We characterize the main contributions of this work as follows

Original: • Instead of generating synthetic data of unseen classes, we propose to yield virtual classes and data by mixup interpolations.
Revised: • Instead of generating synthetic data of unseen classes, we propose to yield virtual classes and data by mixup interpolations.

Original: The virtual classes of synthetic data can then be seamlessly coupled with meta-learning to improve the inference on unseen testing samples.
Revised: The virtual classes of synthetic data can then be seamlessly coupled with meta-learning to improve the inference on unseen testing samples.

Original: • We introduce the concept of representing each class with image-adaptive semantic features that could vary among intra-class samples.
Revised: • We introduce the concept of representing each class with image-adaptive semantic features that could vary among intra-class samples.

Original: While the adaptive mechanism improves classifying the seen classes, it manifests the advantage in boosting the effect of meta-learning over virtual data to zero-shot inference over unseen classes.
Revised: While the adaptive mechanism improves classifying the seen classes, it manifests the advantage in boosting the effect of meta-learning over virtual data to zero-shot inference over unseen classes.

Original: • We demonstrate state-of-the-art results of zero-shot learning over four popular benchmark datasets and justify the design of our method with a thorough ablation study.
Revised: • We demonstrate state-of-the-art results of zero-shot learning over four popular benchmark datasets and justify the design of our method with a thorough ablation study.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: We review related works in this section.
Revised: We review relevant literature in this section.

Original: First, we study generative approaches for ZSL/GZSL that synthesize unseen images for training.
Revised: First, we describe generative approaches for ZSL/GZSL that synthesize unseen images for training.

Original: To improve GZSL performance, we  coupleproposed virtual class generation with meta-learning for mimicking the inference scenario.
Revised: To improve GZSL performance, we propose to couple virtual class generation with meta-learning for mimicking the inference scenario.

Original: Finally, we discuss attention approaches that extract discriminating features from images to help classiﬁcation.
Revised: Next, we discuss attention approaches that extract discriminating features from images to help classiﬁcation.

Original: 2. G ENERATIVE A PPROACHES FOR ZSL/GZSL
Revised: 2. G ENERATIVE A PPROACHES FOR ZSL/GZSL

Original: Arguably one of the most important problems in ZSL/GZSL is to prevent models from being biased to seen classes.
Revised: Arguably one of the most important problems in ZSL/GZSL is to prevent models from being biased to seen classes.

Original: Generative approaches (Chen et al., 2018; Felix et al., 2018; Kumar Verma et al., 2018;
Revised: Generative approaches (Chen et al., 2018; Felix et al., 2018; Kumar Verma et al., 2018;

Original: Mishra et al., 2018; Schonfeld et al., 2019; Paul et al., 2019; Xian et al., 2019) tackle the problem by synthesizing visual features of unseen classes from their semantic features with generative models like Generative Adversarial Networks (GAN; Goodfellow et al., 2014) or Variational Autoencoders (VAE) (VAE; Kingma & Welling, 2014).
Revised: Mishra et al., 2018; Schonfeld et al., 2019; Paul et al., 2019; Xian et al., 2019) tackle the problem by synthesizing visual features of unseen classes from their semantic features with generative models like Generative Adversarial Networks (GAN; Goodfellow et al., 2014) or Variational Autoencoders (VAE) (VAE; Kingma & Welling, 2014).

Original: The synthetic visual features act as pseudo-examples of unseen classes, and effectively reduce the ZSL/GZSL problem to a supervised learning one.
Revised: The synthetic visual features act as pseudo-examples of unseen classes, and effectively reduce the ZSL/GZSL problem to a supervised learning one.

Original: One key issue behind generative approaches comes from the insufﬁcient amount of data to learn a good generative model.
Revised: One key issue behind generative approaches comes from the insufﬁcient amount of data to learn a good generative model.

Original: As a consequence, some semantic features that seem important during training maybe cause overﬁtting, and others that seem less important can be  dropped.
Revised: As a consequence, some semantic features that seem important during training may cause overﬁtting, and others that seem less important may be completely dropped.

Original: Therefore, many works propose new constraints or losses to preserve semantic features and regularize the generative model.
Revised: Therefore, several prior techniques propose new constraints or losses to preserve semantic features and regularize the generative model.

Original: For instance, (Chen et al., 2018) avoids the dropping of semantic information by disentangling the semantic space into two subspaces, one for classiﬁcation and the other for reconstruction; Felix et al.
Revised: For instance, (Chen et al., 2018) avoids the dropping of semantic information by disentangling the semantic space into two subspaces, one for classiﬁcation and the other for reconstruction [Table]

Original: (2018) enforces visual-semantic feature consistency by requiring synthesized visual features to reconstruct semantic features accurately.
Revised: ) enforces visual-semantic feature consistency by requiring synthesized visual features to reconstruct semantic features accurately.

Original: Even with the new constraints or losses, the success of generative approaches still highly depends on whether there is enough data/variation in the seen classes to synthesize diverse visual features.
Revised: However, even with the new constraints or losses, the success of generative approaches still highly depends on whether there is enough data/variation in the seen classes to synthesize diverse visual features.

Original: [Figure]

Original: Otherwise, those features will be too close to the few seen classes and cannot help ZSL/GZSL much.
Revised: Otherwise, the yielded features will be too close to those few seen classes and cannot help ZSL/GZSL much.

Original: For example, in AWA2 dataset (Xian et al., 2018a), the unseen samples of “rat” are easily misclassiﬁed as seen classes of “mouse” or “hamster” because synthetic visual features for “rat” is doomed to be confused with seen visual features of those similar classes.
Revised: For example, in AWA2 dataset (Xian et al., 2018a), the unseen samples of “rat” are easily misclassiﬁed as seen classes of “mouse” or “hamster” because synthetic visual features for “rat” are doomed to be confused with seen visual features of those similar classes.

Original: The phenomenon inspires us to generate synthetic visual features for not only the given unseen classes but also the virtual unseen classes.
Revised: The phenomenon inspires us to generate synthetic visual features for not only the given unseen classes but also the virtual unseen classes.

Original: These virtual classes provide a wider spectrum of support for unseen scenarios to improve ZSL/GZSL performance.
Revised: These virtual classes provide a wider spectrum of support for unseen scenarios to improve ZSL/GZSL performance.

Original: Besides, we combine the idea of meta-learning (Vinyals et al., 2016;
Revised: Our other focal effort is to connect the concept of meta-learning (Vinyals et al., 2016;

Original: Finn et al., 2017) with virtual classes in our training phase for the unseen expert.
Revised: Finn et al., 2017) with virtual classes in the training phase of the unseen expert.

Original: The most closely related works in ZSL are Li et al.
Revised: The most closely related approaches in ZSL are Li et al.

Original: (2019b); Yu et al.
Revised: (2019b); Yu et al.

Original: (2020); Verma et al.
Revised: (2020); Verma et al.

Original: Sung et al.
Revised: Sung et al.

Original: ; Hu et al.
Revised: ; Hu et al.

Original: Verma et al.
Revised: Verma et al.

Original: (2020) combines meta-learning and generative models to conquer limits in generative models.
Revised: (2020) combines meta-learning and generative models to conquer limits in generative models.

Original: Li et al.
Revised: Li et al.

Original: (2019b) mimics the inference scenario by randomly selecting seen classes as ”fake” new classes in each episode.
Revised: (2019b) mimics the inference scenario by randomly selecting seen classes as ”fake” new classes in each episode.

Original: Yu et al.
Revised: Yu et al.

Original: (2020) randomly splits seen classes into two sets to train and reﬁne the model.
Revised: (2020) randomly splits seen classes into two sets to train and reﬁne the model.

Original: In contrast, we simulate the ZSL inference scenario in each episode by creating virtual classes from seen classes with mixup.
Revised: In contrast, we simulate the ZSL inference scenario in each episode by creating virtual classes from seen classes using mixup.

Original: With the training scenario closer to the inference setting in ZSL, our model achieve better S2V embedding on unseen classes and obtain state-of-the-art ZSL performance on most datasets.
Revised: Owing to the training scenario more resembling the inference setting in ZSL, our model achieves better S2V embedding on unseen classes and obtains state-of-the-art ZSL performance on most datasets.

Original: Besides, Sung et al.
Revised: Besides, Sung et al.

Original: (2018) learns a deep distance metric to classify samples.
Revised: (2018) learns a deep distance metric to classify samples.

Original: Hu et al.
Revised: Hu et al.

Original: (2018) utilizes the correction module to assist classiﬁcation.
Revised: (2018) utilizes the correction module to assist classiﬁcation.

Original: On the contrary, our model do not have to learn complex relationship between classes and additional assistant module to achieve good performance.
Revised: On the contrary, our model does not need to learn complex relationship between classes and additional assistant module to achieve good performance.

Original: 2. ATTENTION
Revised: 2. ATTENTION

Original: Attention mechanism is widely used on ZSL (Ji et al., 2018; Xie et al., 2019; Huynh & Elhamifar, 2020; Min et al., 2020; Liu et al., 2019).
Revised: Attention mechanism is widely used in ZSL (Ji et al., 2018; Xie et al., 2019; Huynh & Elhamifar, 2020; Min et al., 2020; Liu et al., 2019).

Original: Highlighting important local features and reducing noisy features inﬂuence generates a better mapping between visual and semantic domains.
Revised: Highlighting important local features and reducing noisy feature inﬂuence generate a more effective mapping between visual and semantic domains.

Original: Ji et al.
Revised: Ji et al.

Original: (2018) proposes S 2 GA which utilizes semantic features to focus on most informative visual local features.
Revised: (2018) proposes S 2 GA which utilizes semantic features to emphasize most informative visual local features.

Original: (Xie et al., 2019) employs AREN and ACSE to pay attention to the most important region in images.
Revised: (Xie et al., 2019) employs AREN and ACSE to focus on the most important region in images.

Original: Huynh & Elhamifar (2020) applies dense attribute-attention to ﬁnd the most discriminating image part and embedding them to semantic features individually.
Revised: Huynh & Elhamifar (2020) applies dense attribute-attention to ﬁnd the most discriminating image parts and embed them to semantic features individually.

Original: DVBE (Min et al., 2020) utilizes spatial and channel attention to maximize inter-class margin.
Revised: DVBE (Min et al., 2020) utilizes spatial and channel attention to maximize inter-class margin.

Original: Although the attention to visual features  proved usable by the above works, seldom work mention the importance of attention on semantic features.
Revised: While the attention to visual features is proved useful by the above-mentioned research efforts, relatively few attempts assess the importance of attention on semantic features.

Original: In the ZSL setting, each classonly contains  one semantic feature, so the intra-class variance loses in the  semantic feature.
Revised: In the ZSL setting, each class contains only one semantic feature vector such that intra-class variations are neglected from the single semantic representation.

Original: For example,sometimes the object is occluded partly in the image or in the front of different backgrounds, and a unique  semantic feature  is hard to reﬂect this variance.
Revised: For example, the object of interest can be occluded partly in the image or in the front of different backgrounds, and a unique class-wise semantic feature vector is hard to reﬂect such variances.

Original: Thus, attention to semantic features is also very important in ZSL.
Revised: Thus, attention to semantic features is worth further exploring in ZSL.

Original: The most similar work  is LFGAA (Liu et al., 2019) who pays attention to the semantic prediction of samples.
Revised: Along this line, the most relevant work to ours is LFGAA (Liu et al., 2019), which considers semantic prediction of samples.

Original: Different from LFGAA, we put attention on discriminating dimensions of semantic features and map them to the visual domain.
Revised: Different from LFGAA, our method puts attention on discriminating dimensions of semantic features and maps them to the visual domain.

Original: The effect of our proposed attention  is to attract a visual representation of ground true class and repel others.
Revised: The effect of the proposed attention mechanism is to attract the visual representation of ground-truth class and repel others.

Original: Besides, the other  difference between us and LFGAA is the shortcut  in the attention model.
Revised: Besides, our other main difference from LFGAA is the shortcut design in the attention model.

Original: We add a shortcut to prevent the model from over-ﬁtting on training data and retain discriminating dimensions for unseen class classiﬁcation.[Figure]
Revised: We add a shortcut to prevent the model from overﬁtting on training data and retain discriminating dimensions for unseen class classiﬁcation.

Original:  M ETHOD
Revised:  M ETHOD

Original: We coin the proposed method as AGZSL in that its main idea is the fusion of Adaptive and Generative mechanisms in solving ZSL/GZSL.
Revised: We coin the proposed method as AGZSL in that its main idea is the fusion of Adaptive and Generative mechanisms in solving ZSL/GZSL.

Original: (See Figure 1.)
Revised: (See Figure 1.)

Original: We describe below the details of AGZSL.
Revised: We describe below the details of AGZSL.

Original: 3. N OTATIONS AND F ORMULATION
Revised: 3. N OTATIONS AND F ORMULATION

Original: We denote training data as D = D S ∪ D U where D S
Revised: We denote training data as D = D S ∪ D U where D S

Original: = { ( x s ∈ X s ,
Revised: = { ( x s ∈ X s ,

Original: a s ∈
Revised: a s ∈

Original: A s , y s ∈ Y s ) } comprises seen data and D U
Revised: A s , y s ∈ Y s ) } comprises seen data and D U

Original: = { ( a u ∈ A u , y u ∈ Y u ) } includes unseen class labels y u and the corresponding  semantic/attribute feature vectors a u .
Revised: = { ( a u ∈ A u , y u ∈ Y u ) } includes unseen class labels y u and the corresponding [Figure] semantic/attribute feature vectors a u .

Original: The dimensions of visual features and semantic features are respectively expressed as x ∈ R d and a ∈ R k , while the number of total classes is n .
Revised: The dimensions of visual features and semantic features are respectively expressed as x ∈ R d and a ∈ R k , while the number of total classes is n .

Original: We also assume that the seen category Y s and the unseen category Y u are disjoint, i.e., Y s ∩Y u
Revised: We also assume that the seen category Y s and the unseen category Y u are disjoint, i.e., Y s ∩Y u

Original: = ∅
Revised: = ∅

Original: and Y s ∪Y u = Y .
Revised: and Y s ∪Y u = Y .

Original: Besides, we deﬁne A s ∪ A u =
Revised: Besides, we deﬁne A s ∪ A u =

Original: A ∈ R k × n as semantic features of all classes.
Revised: A ∈ R k × n as semantic features of all classes.

Original: Finally, the virtual data via mixup interpolations is denoted as D V = { ( x v ∈ X v , a v ∈
Revised: Finally, the virtual data via mixup interpolations is denoted as D V = { ( x v ∈ X v , a v ∈

Original: A v , y v ∈ Y v ) } .
Revised: A v , y v ∈ Y v ) } .

Original: As we explain later, y v is simply a new label of virtual class, rather than from label interpolation as in Zhang et al.
Revised: As we explain later, y v is simply a new label of virtual class, rather than from label interpolation as in Zhang et al.

Original: To perform the classiﬁcation task, we explore the similarity correlations in the visual domain and consider semantic-to-visual (S2V) embeddings.
Revised: To perform the classiﬁcation task, we explore the similarity correlations in the visual domain and consider semantic-to-visual (S2V) embeddings.

Original: Analogous to COSMO (Atzmon & Chechik, 2019), our method includes seen and unseen experts, f s and f u , to classify testing samples as shown in
Revised: Analogous to COSMO (Atzmon & Chechik, 2019), our method includes seen and unseen experts, f s and f u , to classify testing samples as shown in

Original: Figure 2; however, the algorithmic designs are fundamentally different.
Revised: Figure 2; however, the algorithmic designs are fundamentally different.

Original: The novel component of our method is the introduction of Image-Adaptive Semantics (IAS) in both experts to expand the spectrum of S2V embeddings.
Revised: The novel component of our method is the introduction of Image-Adaptive Semantics (IAS) in both experts to expand the spectrum of S2V embeddings.

Original: Speciﬁcally, in training, IAS diversiﬁes the total mappings of S2V embedding from the number of different semantic vectors/classes to the number of samples.
Revised: Speciﬁcally, in training, IAS diversiﬁes the total mappings of S2V embedding from the number of different semantic vectors/classes to the number of samples.

Original: Its effect is twofold.
Revised: Its effect is twofold.

Original: First, it boosts the classiﬁcation performance of the seen expert.
Revised: First, it boosts the classiﬁcation performance of the seen expert.

Original: Second, it generalizes the meta-learning of the unseen expert with virtual data and yields good performance in the inference of samples from unseen classes.
Revised: Second, it generalizes the meta-learning of the unseen expert with virtual data and yields good performance in the inference of samples from unseen classes.

Original: Given a testing sample x in inference, we apply the two learned experts by
Revised: Given a testing sample x in inference, we apply the two learned experts by

Original: [Equation]
Revised: [Equation]

Original: To decide the class label ˆ y of x , we use the seen expert to discriminate whether the sample is from the seen category.
Revised: To decide the class label ˆ y of x , we use the seen expert to discriminate whether the sample is from the seen category.

Original: If it is the case then ˆ y is decided by f s , and otherwise by f u .
Revised: If it is the case then ˆ y is decided by f s , and otherwise by f u .

Original: That is, we have
Revised: That is, we have

Original: [Equation]
Revised: [Equation]

Original: otherwise

Original: [Equation]

Original: [Figure]

Original: 3. I MAGE A DAPTIVE S EMANTICS (IAS)
Revised: 3. I MAGE A DAPTIVE S EMANTICS (IAS)

Original: As ZSL/GZSL assumes one semantic feature vector for each class, the conventional semantic-tovisual architecture leads to one single class-level visual feature vector, which is common to all samples of the same class.
Revised: As ZSL/GZSL assumes one semantic feature vector for each class, the conventional semantic-tovisual architecture leads to one single class-level visual feature vector, which is common to all samples of the same class.

Original: To learn such a single embedding vector with the constraint of being simultaneously similar to all visual features of samples from the same class tends to yield unsatisfactory classiﬁcation outcomes in that it simply ignores the intra-class variations in visual features.
Revised: To learn such a single embedding vector with the constraint of being simultaneously similar to all visual features of samples from the same class tends to yield unsatisfactory classiﬁcation outcomes in that it simply ignores the intra-class variations in visual features.

Original: Diversifying semantic-to-visual embeddings with IAS establishes the core of our model as shown in
Revised: Diversifying semantic-to-visual embeddings with IAS establishes the core of our model as shown in

Original: Figure 3.
Revised: Figure 3.

Original: To realize the concept, the network module learns to map image-adaptive semantic features to the corresponding visual features, according to the cosine similarity.
Revised: To realize the concept, the network module learns to map image-adaptive semantic features to the corresponding visual features, according to the cosine similarity.

Original: To this effect, IAS leverages the given visual features x to focus on discriminative dimensions of the respective semantic features.
Revised: To this effect, IAS leverages the given visual features x to focus on discriminative dimensions of the respective semantic features.

Original: Speciﬁcally, we apply IAS to obtain modiﬁed semantic features ˆ A ( x ) ∈ R k × n by
Revised: Speciﬁcally, we apply IAS to obtain modiﬁed semantic features ˆ A ( x ) ∈ R k × n by

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Revised: [Figure]

Revised: SeenExpert

Revised: UnseenExpert

Original:  where softmax( g ( x ))∈ R k represents using an linear layer g to predict semantic attention and (cid:12) denotes Hadamard product to combine attention and semantic features.
Revised: Training phase Inference phase SeenExpert UnseenExpert where softmax( g ( x ))

Revised: ∈

Revised: R k represents using a linear layer g to predict semantic attention and (cid:12) denotes Hadamard product to combine attention and semantic features.

Original: Note that the shortcut implementation in (3) is pivotal as the softmax function often puts high attention weights on sparse dimensions and draw others near to zero.
Revised: Note that the shortcut implementation in (3) is pivotal as the softmax function often puts high attention weights on sparse dimensions and draws others near to zero.

Original: Without the shortcut, the modiﬁed ˆ A would miss some part of semantic features and degenerate the classiﬁcation performance.
Revised: Without the shortcut, the modiﬁed ˆ A would miss some part of semantic features and degenerate the classiﬁcation performance.

Original: Also, it is worthy to point out that compared to LFGAA (Liu et al., 2019), our method pays attention to the semantic features of classes, not the semantic prediction of the model.
Revised: Also, it is worthy to point out that compared to LFGAA (Liu et al., 2019), our method pays attention to the semantic features of classes, not the semantic prediction of the model.

Original: With (3), the process to drive the image-adaptive class-level visual features V ( x ) can be expressed by
Revised: With (3), the process to drive the image-adaptive class-level visual features V ( x ) can be expressed by

Original: [Equation]
Revised: [Equation]

Original: Once we have obtained the class-level visual features, we then use the cosine similarity and softmax to calculate the probability of each class.
Revised: Once we have obtained the class-level visual features, we then use the cosine similarity and softmax to calculate the probability of each class.

Original: We have the class probability of prediction on x :
Revised: We have the class probability of prediction on x :

Original: [Equation]
Revised: [Equation]

Original: where σ is  learnable scale and f is indeed the general form of an classiﬁcation expert.
Revised: where σ is the learnable scale and f is indeed the general form of a classiﬁcation expert.

Original: We conclude by stating that the Cross-Entropy(CE) loss is adopted to update the model:
Revised: We conclude by stating that the Cross-Entropy(CE) loss is adopted to update the model:

Original: [Equation]
Revised: [Equation]

Original: where p y ( x ) is the predicted probability of ground-truth label.
Revised: where p y ( x ) is the predicted probability of ground-truth label.

Original: 3. S EEN E XPERT
Revised: 3. S EEN E XPERT

Original: The objective of seen expert f s is to recognize seen classes and otherwise decide whether a testing sample belongs to unseen classes in inference phase.
Revised: The objective of seen expert f s is to recognize seen classes and otherwise decide whether a testing sample belongs to unseen classes in inference phase.

Original: In zero-shot learning, especially GZSL, the capability to single out testing samples of unseen classes is crucial to the classiﬁcation performance.
Revised: In zero-shot learning, especially GZSL, the capability to single out testing samples of unseen classes is crucial to the classiﬁcation performance.

Original: If one can establish reasonable split mechanisms such as in Min et al.
Revised: If one can establish reasonable split mechanisms such as in Min et al.

Original: and
Revised: and

Original: Chen et al.
Revised: Chen et al.

Original: (2020), the more challenging GZSL would be simpliﬁed into ZSL.
Revised: (2020), the more challenging GZSL would be simpliﬁed into ZSL.

Original: For example, COSMO (Atzmon & Chechik, 2019) utilizes Conﬁdence Base Gating to discriminate between seen and unseen classes, while DVBE (Min et al., 2020) considers the entropy threshold.
Revised: For example, COSMO (Atzmon & Chechik, 2019) utilizes Conﬁdence Base Gating to discriminate between seen and unseen classes, while DVBE (Min et al., 2020) considers the entropy threshold.

Original: In our formulation, the seen expert learns to predict all possible classes, including seen and unseen, although the training data assume only seen class labels.
Revised: In our formulation, the seen expert learns to predict all possible classes, including seen and unseen, although the training data assume only seen class labels.

Original: That is, f s is trained to map (image adaptive) seen semantic features to their corresponding visual features.
Revised: That is, f s is trained to map (image adaptive) seen semantic features to their corresponding visual features.

Original: Such optimization can be thought of as one-category anomaly detection over only seen classes.
Revised: Such optimization can be thought of as one-category anomaly detection over only seen classes.

Original: Thus, in inference, if the highest cosine similarity value by the seen expert f s is not yielded by one of the seen classes, the testing sample is considered abnormal from the unseen category and its classiﬁcation should be decided by the unseen expert discussed next. 
Revised: Thus, in inference, if the highest cosine similarity value by the seen expert f s is not yielded by one of the seen classes, the testing sample is considered abnormal from the unseen category and its classiﬁcation should be decided by the unseen expert discussed next. [Figure]

Original: 3. UNSEEN EXPERT
Revised: 3. U NSEEN E XPERT

Original: So far we have stated the objective of the unseen expert f u is to classify the labels of those that cannot be decided by the seen expert, as in (2).
Revised: So far we have stated the objective of the unseen expert f u is to classify the labels of those that cannot be decided by the seen expert, as in (2).

Original: To resolve the dilemma of zero-shot setting, we apply meta-learning to train f u over virtual classes by mixup interpolations.
Revised: To resolve the dilemma of zero-shot setting, we apply meta-learning to train f u over virtual classes by mixup interpolations.

Original: The exact number of virtual classes in each episode is decided by the pilot study (Li et al., 2019b), and an ablation study in appendix is provided to analyze its effect on the performance of zero-shot classiﬁcation.
Revised: The exact number of virtual classes in each episode is decided by the pilot study (Li et al., 2019b), and an ablation study in appendix is provided to analyze its effect on the performance of zero-shot classiﬁcation.

Original: To begin with, in each episode we randomly select m pairs of seen classes.
Revised: To begin with, in each episode we randomly select m pairs of seen classes.

Original: Then, we create m virtual classes out of these pairs by mixup.
Revised: Then, we create m virtual classes out of these pairs by mixup.

Original: The semantic and visual features for these virtual classes are deﬁned as
Revised: The semantic and visual features for these virtual classes are deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where a i and a j are semantic features of paired classes, and x i and x j are randomly chosen from visual samples of the paired classes.
Revised: where a i and a j are semantic features of paired classes, and x i and x j are randomly chosen from visual samples of the paired classes.

Original: Following Zhang et al.
Revised: Following Zhang et al.

Original: (2018), λ ∈ [0 , 1] is sampled from the beta distribution.
Revised: (2018), λ ∈ [0 , 1] is sampled from the beta distribution.

Original: We then deﬁne a new virtual class y v , with a v as semantic features, to be associated with virtual visual features x v .
Revised: We then deﬁne a new virtual class y v , with a v as semantic features, to be associated with virtual visual features x v .

Original: Previous generative models (Schonfeld et al., 2019; Xian et al., 2019) synthesize unseen samples by GAN/VAE or their variants.
Revised: Previous generative models (Schonfeld et al., 2019; Xian et al., 2019) synthesize unseen samples by GAN/VAE or their variants.

Original: In comparison, the proposed meta-learning has at least three advantages for ZSL.
Revised: In comparison, the proposed meta-learning has at least three advantages for ZSL.

Original: First, the small number of classes in some datasets, such asin AWA2, may fall short to provide sufﬁcient support set such that the generative model is hard to be optimized, while episodic meta-learning over virtual classes can overcome this problem.
Revised: First, the small number of classes in some datasets, such asin AWA2, may fall short to provide sufﬁcient support set such that the generative model is hard to be optimized, while episodic meta-learning over virtual classes can overcome this problem.

Original: Second, some part of semantic features which are important to classify unseen classes do not have enough training classes in the datasets.
Revised: Second, some part of semantic features which are important to classify unseen classes do not have enough training classes in the datasets.

Original: With mixup semantic features as virtual classes, our model can learn underlying discriminating semantic features well.
Revised: With mixup semantic features as virtual classes, our model can learn underlying discriminating semantic features well.

Original: Third, the fusion of image-adaptive mechanism and meta-learning is ﬂexible.
Revised: Third, the fusion of image-adaptive mechanism and meta-learning is ﬂexible.

Original: As a result, we do not need online ﬁne-tuning in applying the unseen expert f u yielded by meta-learning to predict the label of unseen class for testing samples.
Revised: As a result, we do not need online ﬁne-tuning in applying the unseen expert f u yielded by meta-learning to predict the label of unseen class for testing samples.

Original:  E XPERIMENT 4. E XPERIMENTAL S ETTING
Revised:  E XPERIMENT 4. E XPERIMENTAL S ETTING

Original: Datasets.
Revised: Datasets.

Original: In GZSL, there are four widely used benchmark datasets for evaluation, which are AWA (Xian et al., 2018a), USCD Birds-200-2011 ( CUB )
Revised: In GZSL, there are four widely used benchmark datasets for evaluation, which are AWA (Xian et al., 2018a), USCD Birds-200-2011 ( CUB )

Original: (Welinder et al., 2010), SUN (Patterson & Hays, 2012), and aPY (Farhadi et al., 2009).
Revised: (Welinder et al., 2010), SUN (Patterson & Hays, 2012), and aPY (Farhadi et al., 2009).

Original: AWA2 contains 37K images and 50 kinds of animals with 85dimension attributes.
Revised: AWA2 contains 37K images and 50 kinds of animals with 85dimension attributes.

Original: CUB consists of 11K images and 200 bird species described with 312-dimension attributes.
Revised: CUB consists of 11K images and 200 bird species described with 312-dimension attributes.

Original: SUN is composed of 14K images 717 classes labeled with 102-dimension attributes.
Revised: SUN is composed of 14K images 717 classes labeled with 102-dimension attributes.

Original: aPY includes 15K images and 32 different classes with 64-dimension attributes.
Revised: aPY includes 15K images and 32 different classes with 64-dimension attributes.

Original: The attributes of class in datasets will be taken as the semantic features in our work.
Revised: The attributes of class in datasets will be taken as the semantic features in our work.

Original: We follow existing GZSL works (Min et al., 2020; Li et al., 2019b) and extract 2048 -dimensional visual features for each image by the ResNet101 backbone (He et al., 2016) pretrained on ImageNet-1K. We then split the data into seen and unseen classes according to the benchmark procedure from Xian et al. 
Revised: We follow existing GZSL approaches (Min et al., 2020; Li et al., 2019b) and extract 2048 -dimensional visual features for each image by the ResNet101 backbone (He et al., 2016) pretrained on ImageNet-1K. We then split the data into seen and unseen classes according to the benchmark procedure from Xian et al. [Table]

Original: Evaluation.
Revised: Evaluation.

Original: We evaluate our approach for both ZSL and GZSL settings.
Revised: We evaluate our approach for both ZSL and GZSL settings.

Original: For the ZSL setting, the average per-class top-1 accuracy ( T ) on unseen classes is taken as the evaluation metric.
Revised: For the ZSL setting, the average per-class top-1 accuracy ( T ) on unseen classes is taken as the evaluation metric.

Original: For
Revised: For

Original: GZSL
Revised: GZSL

Original: (Xian et al., 2017), the evaluation metrics include seen classes accuracy S , unseen class accuracy U , and their harmonic mean H
Revised: (Xian et al., 2017), the evaluation metrics include seen classes accuracy S , unseen class accuracy U , and their harmonic mean H

Original: =
Revised: =

Original: (2 U · S ) /
Revised: (2 U · S ) /

Original: ( U + S ) .
Revised: ( U + S ) .

Original: [Table]

Original: Implementation.
Revised: Implementation.

Original: Followed suggestion in Li et al. (2019b) , we normalized the visual and semantic features into [0 , 1] .
Revised: As suggested in Li et al. (2019b), we normalize the visual and semantic features into [0 , 1] .

Original: The architecture of semantic-to-visual embedding contains two layers linear model with 1600 hidden units and utilize ReLU on the hidden and output layer.
Revised: The architecture of semantic-to-visual embedding contains a two-layer linear model with 1,600 hidden units and utilizes ReLU on the hidden and output layer.

Original: The seen and unseen experts are trained by Adam optimizer with a learning rate 5 ∗ 10 − 5 and 5 ∗ 10 − 4 respectively for all datasets.
Revised: The seen and unseen experts are trained by Adam optimizer with a learning rate 5 × 10 − 5 and 5 × 10 − 4 respectively for all datasets.

Original: We apply 200,000 episodes to train  unseen expert.
Revised: We apply 200,000 episodes to train the unseen expert.

Original: In each episode, we randomly generated 16 or 20 (based on the dataset) virtual classes and 4 samples for each class.
Revised: In each episode, we randomly generate 16 or 20 (based on the dataset) virtual classes and 4 samples for each class.

Original: We follow  setting in (Min et al., 2020) to ﬁne-tune the backbone.
Revised: We follow the setting in (Min et al., 2020) to ﬁne-tune the backbone.

Original: 4. C OMPARING S TATE - OF - THE -A RT A PPROACHES
Revised: 4. C OMPARISON WITH S TATE - OF - THE -A RT A PPROACHES

Original: We compare our method with 13 recent GZSL approaches.
Revised: We compare our method with 13 recent GZSL approaches.

Original: These contain SP-AEN (Chen et al., 2018),
Revised: These include SP-AEN (Chen et al., 2018),

Original: DLFZRL (Tong et al., 2019), PSR (Annadani & Biswas, 2018), CDL (Jiang et al., 2018), PQZSL (Li et al., 2019a), f-VAEGAM-D2 (Xian et al., 2019), LsrGAN (Vyas et al., 2020), TF-VAEGAN (Narayan et al., 2020), OCD-CVAE (Keshari et al., 2020) , ZSML Softmax (Verma et al., 2020),
Revised: DLFZRL (Tong et al., 2019), PSR (Annadani & Biswas, 2018), CDL (Jiang et al., 2018), PQZSL (Li et al., 2019a), f-VAEGAM-D2 (Xian et al., 2019), LsrGAN (Vyas et al., 2020), TF-VAEGAN (Narayan et al., 2020), OCD-CVAE (Keshari et al., 2020), ZSML Softmax (Verma et al., 2020),

Original: E-PGN (Yu et al., 2020), Relation Net (Sung et al., 2018), Correlation Net (Hu et al., 2018), GXE Li et al.(2019b), LFGAA+Hybrid (Liu et al., 2019), AREN (Xie et al., 2019), COSMO (Atzmon &
Revised: E-PGN (Yu et al., 2020), Relation Net (Sung et al., 2018), Correlation Net (Hu et al., 2018), GXE Li et al.

Original:  Chechik, 2019), DVBE (Min et al., 2020).
Revised: (2019b), LFGAA+Hybrid (Liu et al., 2019), AREN (Xie et al., 2019), COSMO (Atzmon & Chechik, 2019), DVBE (Min et al., 2020).

Original: As shown in our Table 1, our method AGZSL achieves the bestperformance of GZSL  on  four datasets.
Revised: Table 1 shows that the proposed AGZSL achieves the best GZSL performance on all four datasets.

Original: The proposed AGZSL improves harmonic mean by 7.4 % on AWA2 , 1.9 % on CUB , 0.2 % on SUN , and 0.5 % on aPY  respectively.
Revised: It improves harmonic mean by 7.4 % on AWA2 , 1.9 % on CUB , 0.2 % on SUN , and 0.5 % on aPY , respectively.

Original: Among the compared approaches, DVBE searches entropy threshold τ and  embedding model for the best harmonic means according to each dataset.
Revised: Particularly, among those methods in comparison, DVBE needs to search the entropy threshold τ and the embedding model for the best harmonic means according to each dataset.

Original: In contrast to DVBE, without entropy threshold and speciﬁc model searching, our model achieves over 4 % improvement on most datasets.
Revised: In contrast to DVBE, without entropy threshold and speciﬁc model searching, our model achieves over 4 % improvement on most datasets.

Original: The main reason  is that AGZSL provides better visual discriminating  representation for  seen and unseen classes.
Revised: The main reason for better performance is that AGZSL provides more discriminating visual representation for both seen and unseen classes.

Original: Therefore, our model can  separate seen and unseen correctly and achieve better performance in the harmonic mean.
Revised: Therefore, our model can effectively separate seen and unseen samples and achieve better performance in the harmonic mean.

Original: Notably,the AGZSL drop performance on aPY after ﬁne-tuning backbone.
Revised: Notably, AGZSL drops performance on aPY after backbone ﬁne-tuning.

Original: The main reason is that the number of seen class and training images are fewest among the four datasets.
Revised: It could be due to that its number of seen classes and training images are fewest among the four datasets.

Original: After ﬁne-tuning the backbone, the visual features over-ﬁt on training samples and become less discriminating for correct classiﬁcation. 
Revised: Fine-tuning the backbone causes the visual features to overﬁt training samples and become less discriminating for correct classiﬁcation. [Table]

Original: Furthermore, generative models achieve higher performance on CUB and SUN , because  there are many seen classes to synthesize better quality unseen sample in these two datasets.
Revised: Furthermore [Table] generative models achieve higher performance on CUB and SUN , because in these two datasets there are many seen classes for synthesizing unseen samples of better quality.

Original: Compared to them, our model overcomes the inﬂuence of few seen classes and also achieves the best performance on AWA2 and aPY .
Revised: In comparison [Table] our model overcomes the inﬂuence of few seen classes and also achieves the best performance on AWA2 and aPY .

Original: Thus, the  virtual classes  are indeed helpful to classify unseen classes.
Revised: Our results empirically support that the introduction of virtual classes and samples are indeed helpful to classify unseen classes.

Original: Notably, ImageNet-1K used to pretrain backbone are very different from  ZSL ﬁne-grained databases, e.g.,
Revised: Finally, we remark that since the backbone is pretrained on ImageNet-1K, which is rather different from the ZSL ﬁne-grained databases, e [Table]

Original:  CUB and SUN .
Revised: g., CUB and SUN .

Original: Thus, the harmonic mean improves a lot after ﬁne-tuning the backbone on these two datasets.[Table] [Figure] m p l e s
Revised: Thus, the harmonic mean improves a lot after ﬁne-tuning the backbone on these two datasets.

Original: [Figure]

Original: 4. A BLATION STUDY
Revised: 4. A BLATION S TUDY

Original: To qualify the beneﬁts of virtul classes and IAS [Figure] we carry  an ablation study on  four datasets.
Revised: To evaluate the beneﬁts of tackling ZSL/GZSL with virtual classes and IAS, we carry out an ablation study on the four datasets.

Original: We add each  componentstep by step to show their effect.
Revised: We incrementally include each key component to assess their effect.

Original: As shown in Table 2, the virtual classes improve ZSL accuracy extremely.
Revised: As shown in Table 2, the virtual classes improve ZSL accuracy signiﬁcantly.

Original:  ZSL improve 4.6 % on AWA2 , 2.4 % on CUB , 1.7 % on SUN , and 2.8 % on aPY  respectively.
Revised: The ZSL performance improves 4.6 % on AWA2 , 2.4 % on CUB , 1.7 % on SUN , and 2.8 % on aPY , respectively.

Original: The result means that the virtual classes are advantageous for ZSL.
Revised: The result implies that the virtual classes are advantageous to ZSL.

Original: After adding IAS to our model,  seen and unseen accuracy simultaneously and improve averaged 1.5 % H.
Revised: After adding IAS to our model, both seen and unseen accuracy increas, and the harmonic mean H improves about 1.5 % on average.

Original: The result exempliﬁes  IAS can provide better semantic-to-visual embedding and achieve higher performance on GZSL.
Revised: The consistent gain exempliﬁes that IAS can provide better semantic-to-visual embedding and achieve higher performance on GZSL.

Original: 4. C OSINE SIMILARITY MARGIN
Revised: 4. C OSINE S IMILARITY M ARGIN

Original: Figure 4 illustrates the effect of our model.
Revised: Figure 4 illustrates the effect of the proposed AGZSL model.

Original: The cosine similarity margin means the distance between the correct class and the other nearest class.
Revised: The cosine similarity margin means the inner product value between the correct class and the other nearest class.

Original: If the margin is negative, the model will predict the false class.
Revised: A negative margin indicates that the model predicts a false class.

Original: Besides, a larger margin mean model has better embedding, so it can distinguish the correct class from others clearly.
Revised: On the other hand, a large positive margin implies that the model results in a proper embedding distinguishing the correct class from others effectively.

Original: Thus, the positive and larger margin is a very important objective in GZSL.
Revised: Thus, seeking a positive and larger margin is a crucial and important objective in GZSL.

Original: As  the ﬁrst row in Figure 4, the margin of unseen samples sift right on  four datasets.
Revised: As shown in the ﬁrst row of Figure 4, the margins of unseen samples shift right on all four datasets by our method.

Original: It shows our methods provide better embedding on  visual space and make the model easier to predict the correct class.
Revised: It shows that AGZSL indeed generates better embedding on the visual space and makes the model easier to predict the correct class.

Original: Moreover,  the second row in Figure 4 shows our methods improve the margin of seen samples dramatically on most datasets.
Revised: Moreover, from the second row in Figure 4, the plots show that our method improves the margins of seen samples dramatically on most datasets.

Original: The above experiment explains our models obtain a higher margin in most samples, so we achieve better  performance on  four  datasets.virtual classes unseen classes seen classes (a) AWA 0. virtual classes unseen classes seen classes (b) CUB 0.6 0.4 0.2 0.0 0.2 0.4 0.6 0. virtual classes unseen classes seen classes (c) SUN 0.4 0.2 0.0 0.2 0.4 0.6 0.8 0.
Revised: The above experiment explains that AGZSL can achieve a higher margin for most samples and achieves better classiﬁcation performance on the four testing datasets.

Original: [Figure]

Original: 4. T HE DISTRIBUTION OF VIRTUAL CLASSES
Revised: 4. T HE D ISTRIBUTION OF V IRTUAL C LASSES

Original: To explicitly exhibit the distributions of seen, unseen and virtual classes, we use PCA to map the class-wise semantic features to a two-dimensional space, as shown in Figure 5.
Revised: To explicitly exhibit the distributions of seen, unseen and virtual classes, we apply PCA to map the class-wise semantic features to a two-dimensional space, as shown in Figure 5.

Original: The features of virtual classes scatter around seen classes, because we have used B (5,1) to generate virtual classes.
Revised: The features of virtual classes scatter around seen classes, because we have used B (5,1) to generate virtual classes.

Original: Advantageously, some virtual classes are very close to unseen classes in the semantic domain and turn out to be helpful for enabling the unseen expert to classify samples of unseen classes.
Revised: Advantageously, some virtual classes are very close to unseen classes in the semantic domain and turn out to be helpful for enabling the unseen expert to classify samples of unseen classes.

Original: With meta-learning over these numerous virtual classes and samples, the proposed S2V can better learn the embedding between semantic and visual domains, and results in an effective unseen expert. 
Revised: With meta-learning over these numerous virtual classes and samples, the proposed S2V can better learn the embedding between semantic and visual domains, and results in an effective unseen expert. [Figure] [Figure]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose IAS and  virtual classes  to solve the ZSL/GZSL.
Revised: In this paper [Figure] we propose Image Adaptive Semantics (IAS) and meta-learning with virtual classes and samples to solve the ZSL/GZSL problem.

Original: To reﬂect the intra-class visual discrepancies, IAS  emphasizeson the most discriminating dimension in semantic features according to each visual feature.
Revised: To deal with the intra-class visual discrepancies, IAS adaptively emphasizes the most discriminating dimensions in semantic features with respect to the underlying visual features.

Original: On the other hand, we apply generative learning in our model to classify unseen samples.
Revised: To better classify samples of the unseen classes in inference, we propose a novel formulation of generative meta-learning.

Original: Different from previous generative models that synthesize unseen samples for training the model, we create virtual classes as unseen classes in the training phase.
Revised: Different from previous generative models that focus on synthesizing unseen samples for training the model, we create virtual classes and their respective virtual samples in the training phase.

Original: To imitate the ZSL inference scenario, we apply meta-learning with  virtual classes.
Revised: Further, to imitate the ZSL inference scenario, we carry out meta-learning with these virtual data to extend our model.

Original: In our experiment,  AGZSL is beneﬁcial to tackle the ZSL/GZSL. Consequently, we achieve remarkable advantages over other methods on theZSL/GZSL four  benchmark datasets
Revised: In our experiment, we demonstrate that AGZSL is beneﬁcial to tackle the challenging ZSL/GZSL problem and achieves signiﬁcant advantages over those in comparison on the four ZSL/GZSL benchmark datasets. A CKNOWLEDGEMENTS

Revised: This work was supported in part by the MOST, Taiwan under Grants 110-2634-F-001-009 and 107-2628-E-002-008-MY3.

Revised: We are grateful to the National Center for High-performance Computing for providing computational resources and facilities.

Original: A A PPENDIX
Revised: A A PPENDIX

Original: [Figure]
Revised: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Shortcut in IAS Table 3 shows that shortcut in IAS improves performance a lot.
Revised: Shortcut in IAS Table 3 shows that shortcut in IAS improves performance a lot.

Original: The unseen class accuracy increase around 50 % on AWA2 and SUN .
Revised: The unseen class accuracy increase around 50 % on AWA2 and SUN .

Original: Besides, the harmonic mean also enhances 20 % to 60 % on four datasets with the shortcut.
Revised: Besides, the harmonic mean also enhances 20 % to 60 % on four datasets with the shortcut.

Original: The main reason is that the softmax usually puts high attention weight on few dimensions and decreases others near to zero.
Revised: The main reason is that the softmax usually puts high attention weight on few dimensions and decreases others near to zero.

Original: Thus, without shortcut, the model is prone to over-ﬁtting on training data.
Revised: Thus, without shortcut, the model is prone to overﬁtting on training data.

Original: Cosine similarity enhancement Figure 6 illustrates better embedding for seen and unseen classes in our model.
Revised: Cosine similarity enhancement Figure 6 illustrates better embedding for seen and unseen classes in our model.

Original: Since we apply cosine similarity to classify samples, the higher score means visual representation of each class is closer to their visual samples.
Revised: Since we apply cosine similarity to classify samples, the higher score means visual representation of each class is closer to their visual samples.

Original: The ﬁrst row in 6 shows effect of our model on unseen classes.
Revised: The ﬁrst row in 6 shows effect of our model on unseen classes.

Original: With generative learning and IAS, cosine similarity increase averaged 0.05 to 0.1 on different datasets.
Revised: With virtual classes and IAS, cosine similarity increase averaged 0.05 to 0.1 on different datasets.

Original: The increasing amount shows our model provides better visualto-semantic embedding for unseen classes and samples are easier to classify correctly.
Revised: The increasing amount shows our model provides better visual-to-semantic embedding for unseen classes and samples are easier to classify correctly.

Original: Therefore, our model can achieve good ZSL performance.
Revised: Therefore, our model can achieve good ZSL performance.

Original: Besides, the second row in 6 illustrates the effect of IAS on seen classes.
Revised: Besides, the second row in 6 illustrates the effect of IAS on seen classes.

Original: Most of the testing seen samples increase their cosine similarity around 0. on four datasets.
Revised: Most of the testing seen samples increase their cosine similarity around 0.02 on four datasets.

Original: With the IAS and virtual classes, the model can achieve better visual-to-semantic embedding on all classes and obtain better the generalized seen accuracy on four datasets.
Revised: With the IAS and virtual classes, the model can achieve better visual-to-semantic embedding on all classes and obtain better the generalized seen accuracy on four datasets.

Original: virtual classes number per episode To search the class number in each episode for best ZSL, we search 4 different numbers, i.e, 4,8,12, and 20 per episode.
Revised: virtual classes number per episode To search the class number in each episode for best ZSL, we search 4 different numbers, i.e, 4,8,12, and 20 per episode.

Original: Table 4 shows that the best class number setting is related to the class number of classes.
Revised: Table 4 shows that the best class number setting is related to the class number of classes.

Original: For example, the AWA2 and aPY which include fewer unseen classes obtain the best ZSL accuracy with 16 virtual classes.
Revised: For example, the AWA2 and aPY which include fewer unseen classes obtain the best ZSL accuracy with 16 virtual classes.

Original: On the other hand, the CUB and SUN which include more unseen classes achieve the best performance with 20 virtual classes.
Revised: On the other hand, the CUB and SUN which include more unseen classes achieve the best performance with 20 virtual classes.

Original: The reason is that the advantage of meta-learning is to simulate inference scenarios while training (Li et al., 2019b).
Revised: The reason is that the advantage of meta-learning is to simulate inference scenarios while training (Li et al., 2019b).

Original: The virtual class number set closer to the unseen class number is favorable in the meta-learning setting.
Revised: The virtual class number set closer to the unseen class number is favorable in the meta-learning setting.

Original: Therefore, the best virtual class number is different according to the unseen number of the dataset.
Revised: Therefore, the best virtual class number is different according to the unseen number of the dataset.

Original: Random Variable
Revised: Random Variable

Original: B ( , )
Revised: B ( , )

Original: B(5,5)
Revised: B(5,5)

Original: B(1,1)
Revised: B(1,1)

Original: B(5,1)
Revised: B(5,1)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Effect of beta distribution on ZSL accuracy To explore the best beta distribution setting for ZSL, our experiments apply three different α and β sets.
Revised: Effect of beta distribution on ZSL accuracy To explore the best beta distribution setting for ZSL [Table] our experiments apply three different α and β sets.

Original: As shown in Figure 7, B(1,1) means virtual classes distribute uniformly between selected class pairs.
Revised: As shown in Figure 7, B(1,1) means virtual classes distribute uniformly between selected class pairs.

Original: Then, B(5,5) means most of the virtual classes distribute among the middle of selected class pairs.
Revised: Then, B(5,5) means most of the virtual classes distribute among the middle of selected class pairs.

Original: On the other hand, B(5,1) means most virtual classes distribute closer to one of the classes in selected class pairs.
Revised: On the other hand, B(5,1) means most virtual classes distribute closer to one of the classes in selected class pairs.

Original: Table 5 demonstrates that B(5,1) achieves best performance on each dataset.
Revised: Table 5 demonstrates that B(5,1) achieves best performance on each dataset.

Original: In our speculation, the virtual classes created based on one class are closer to real-world classes.
Revised: In our speculation, the virtual classes created based on one class are closer to real-world classes.

Original: For example, the zebra is very similar to the horse in shape.
Revised: For example, the zebra is very similar to the horse in shape.

Original: Thus, the virtual classes distributing among seen classes enhance ZSL accuracy are very beneﬁcial for training ZSL.
Revised: Thus, the virtual classes distributing among seen classes enhance ZSL accuracy are very beneﬁcial for training ZSL.