Original: A DAPTIVE AND G ENERATIVE Z ERO -S HOT L EARNING
Revised: A DAPTIVE AND G ENERATIVE Z ERO -S HOT L EARNING

Original: A BSTRACT
Revised: A BSTRACT

Original: We address the problem of generalized zero-shot learning (GZSL) where the task is to predict the class label of a target image whether its label belongs to the seen or unseen category.
Revised: We address the problem of generalized zero-shot learning (GZSL) where the task is to predict the class label of a target image whether its label belongs to the seen or unseen category.

Original: Similar to ZSL, the learning setting assumes that all class-level semantic features are given, while only the images of seen classes are available for training.
Revised: Similar to ZSL, the learning setting assumes that all class-level semantic features are given, while only the images of seen classes are available for training.

Original: By exploring the correlation between image features and the corresponding semantic features, the main idea of the proposed approach is to enrich the semantic-to-visual (S2V) embeddings via a seamless fusion of adaptive and generative learning.
Revised: By exploring the correlation between image features and the corresponding semantic features, the main idea of the proposed approach is to enrich the semantic-to-visual (S2V) embeddings via a seamless fusion of adaptive and generative learning.

Original: To this end, we extend the semantic features of each class by supplementing image-adaptive attention so that the learned S2V embedding can account for not only inter-class but also intra-class variations.
Revised: To this end, we extend the semantic features of each class by supplementing image-adaptive attention so that the learned S2V embedding can account for not only inter-class but also intra-class variations.

Original: In addition, to break the limit of training with images only from seen classes, we design a generative scheme to simultaneously generate virtual class labels and their visual features by sampling and interpolating over seen counterparts.
Revised: In addition, to break the limit of training with images only from seen classes, we design a generative scheme to simultaneously generate virtual class labels and their visual features by sampling and interpolating over seen counterparts.

Original: In inference, a testing image will give rise to two different S2V embeddings, seen and virtual.
Revised: In inference, a testing image will give rise to two different S2V embeddings, seen and virtual.

Original: The former is used to decide whether the underlying label is of the unseen category or otherwise a speciﬁc seen class; the latter is to predict an unseen class label.
Revised: The former is used to decide whether the underlying label is of the unseen category or otherwise a speciﬁc seen class; the latter is to predict an unseen class label.

Original: To demonstrate the effectiveness of our method, we report state-of-the-art results on four standard
Revised: To demonstrate the effectiveness of our method, we report state-of-the-art results on four standard

Original: GZSL datasets, including an ablation study of the proposed modules.
Revised: GZSL datasets, including an ablation study of the proposed modules.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Different from conventional learning tasks, zero-shot learning (ZSL) by Lampert et al.
Revised: Different from conventional learning tasks, zero-shot learning (ZSL) by Lampert et al.

Original: Palatucci et al.
Revised: Palatucci et al.

Original: ; Akata et al.
Revised: ; Akata et al.

Original: (2013) explores the extreme case of performing inference only over samples of unseen classes.
Revised: (2013) explores the extreme case of performing inference only over samples of unseen classes.

Original: To make the scenario more realistic, generalized zero-shot learning (GZSL) (Chao et al., 2016; Xian et al., 2017) is subsequently proposed so that inference can concern samples of both seen and unseen classes.
Revised: To make the scenario more realistic, generalized zero-shot learning (GZSL) (Chao et al., 2016; Xian et al., 2017) is subsequently proposed so that inference can concern samples of both seen and unseen classes.

Original: Nevertheless.
Revised: Nevertheless.

Original: the learning setting in ZSL/GZSL is essentially the same where sample classes are divided into two categories, seen and unseen , but only those samples of seen classes are accessible to training.
Revised: the learning setting in ZSL/GZSL is essentially the same where sample classes are divided into two categories, seen and unseen , but only those samples of seen classes are accessible to training.

Original: In addition, each of all the classes under consideration is characterized by semantic features such as attributes (Xian et al., 2018b) or text descriptions (Zhu et al., 2018) to specify and relate seen and unseen classes.
Revised: In addition, each of all the classes under consideration is characterized by semantic features such as attributes (Xian et al., 2018b) or text descriptions (Zhu et al., 2018) to specify and relate seen and unseen classes.

Original: The lack of training samples from unseen classes has prompted generative approaches (Chen et al., 2018;
Revised: The lack of training samples from unseen classes has prompted generative approaches (Chen et al., 2018;

Original: Felix et al., 2018; Kumar Verma et al., 2018; Mishra et al., 2018) to creating synthetic data from semantic features of unseen classes.
Revised: Felix et al., 2018; Kumar Verma et al., 2018; Mishra et al., 2018) to creating synthetic data from semantic features of unseen classes.

Original: The strategy could enable learning semantic-visual alignment on unseen classes implicitly, and thus improves the ability to classify unseen classes.
Revised: The strategy could enable learning semantic-visual alignment on unseen classes implicitly, and thus improves the ability to classify unseen classes.

Original: However, such generative models are indeed trained on seen samples, and the quality of synthesized unseen samples is predominantly inﬂuenced by seen classes.
Revised: However, such generative models are indeed trained on seen samples, and the quality of synthesized unseen samples is predominantly inﬂuenced by seen classes.

Original: If the number of training samples of each seen class is small, it is hard for generative models to adequately synthesize samples of unseen classes, leading to unsatisfactory zero-shot learning.
Revised: If the number of training samples of each seen class is small, it is hard for generative models to adequately synthesize samples of unseen classes, leading to unsatisfactory zero-shot learning.

Original: To better address the issue, we propose to synthesize visual and semantic features of virtual classes rather than those of the unseen classes.
Revised: To better address the issue, we propose to synthesize visual and semantic features of virtual classes rather than those of the unseen classes.

Original: An interesting analogy is that childhood experience and relevant study (Greene, 1995) suggest the behavior of using human imagination to produce new object concepts could assist our cognitive capability.
Revised: An interesting analogy is that childhood experience and relevant study (Greene, 1995) suggest the behavior of using human imagination to produce new object concepts could assist our cognitive capability.

Original: To mimic people utilizing imagination for exploring new knowledge, we create virtual classes by the integration of past ”experience” (seen classes).
Revised: To mimic people utilizing imagination for exploring new knowledge, we create virtual classes by the integration of past ”experience” (seen classes).

Original: In detail, we extend the mixup technique by Zhang et al.
Revised: In detail, we extend the mixup technique by Zhang et al.

Original: (2018) to generate virtual classes, with a subtle difference that mixing is conducted on the semantic features (in addition to the visual ones), instead of the class label vectors.
Revised: (2018) to generate virtual classes, with a subtle difference that mixing is conducted on the semantic features (in addition to the visual ones), instead of the class label vectors.

Original: In ZSL/GZSL, each seen or unseen class is typically described by a single semantic feature vector.
Revised: In ZSL/GZSL, each seen or unseen class is typically described by a single semantic feature vector.

Original: The practice is useful in differentiating different classes in a principled way, but may not be sufﬁcient to reﬂect the inter-class and intra-class visual discrepancies, not to mention the ambiguities caused by different backgrounds, view orientations, or occlusion in images.
Revised: The practice is useful in differentiating different classes in a principled way, but may not be sufﬁcient to reﬂect the inter-class and intra-class visual discrepancies, not to mention the ambiguities caused by different backgrounds, view orientations, or occlusion in images.

Original: The concern of inefﬁcient class-level representation can also be observed from how the semantic feature vectors are constructed.
Revised: The concern of inefﬁcient class-level representation can also be observed from how the semantic feature vectors are constructed.

Original: Take, for example, the Attribute Pascal and Yahoo ( aPY ) dataset (Farhadi et al., 2009), where each instance is annotated by 64 attributes.
Revised: Take, for example, the Attribute Pascal and Yahoo ( aPY ) dataset (Farhadi et al., 2009), where each instance is annotated by 64 attributes.

Original: The semantic features of each class in aPY are obtained by averaging the attribute vectors of all its instances.
Revised: The semantic features of each class in aPY are obtained by averaging the attribute vectors of all its instances.

Original: We are thus motivated to introduce an imageadaptive class representation, integrating the original semantic features for inter-class discrimination with an image-speciﬁc attention vector for intra-class variations.
Revised: We are thus motivated to introduce an imageadaptive class representation, integrating the original semantic features for inter-class discrimination with an image-speciﬁc attention vector for intra-class variations.

Original: With the addition of virtual training data and the image-adaptive class representation, our method is designed to learn two classiﬁcation experts: one for seen classes and the other for unseen classes.
Revised: With the addition of virtual training data and the image-adaptive class representation, our method is designed to learn two classiﬁcation experts: one for seen classes and the other for unseen classes.

Original: Both experts project the image-adaptive semantic feature vectors to the visual space and use cosine similarity to ﬁnd the class label most similar to the given visual feature vector.
Revised: Both experts project the image-adaptive semantic feature vectors to the visual space and use cosine similarity to ﬁnd the class label most similar to the given visual feature vector.

Original: The seen expert is trained with the provided training (seen) data, while the class prediction is over all possible classes, including seen and unseen.
Revised: The seen expert is trained with the provided training (seen) data, while the class prediction is over all possible classes, including seen and unseen.

Original: In inference, if its predicted class is not within the seen category.
Revised: In inference, if its predicted class is not within the seen category.

Original: The testing sample is deemed to be from the unseen category, whose label is then decided by the unseen expert.
Revised: The testing sample is deemed to be from the unseen category, whose label is then decided by the unseen expert.

Original: The unseen expert is trained with the virtual data only, and the process indeed resembles meta-learning.
Revised: The unseen expert is trained with the virtual data only, and the process indeed resembles meta-learning.

Original: However, the effectiveness of meta-learning is boosted by the design of the imageadaptive mechanism in that ﬁne-tuning is not needed in performing zero-shot classiﬁcation over unseen classes.
Revised: However, the effectiveness of meta-learning is boosted by the design of the imageadaptive mechanism in that ﬁne-tuning is not needed in performing zero-shot classiﬁcation over unseen classes.

Original: We characterize the main contributions of this work as follows
Revised: We characterize the main contributions of this work as follows

Original: • Instead of generating synthetic data of unseen classes, we propose to yield virtual classes and data by mixup interpolations.
Revised: • Instead of generating synthetic data of unseen classes, we propose to yield virtual classes and data by mixup interpolations.

Original: The virtual classes of synthetic data can then be seamlessly coupled with meta-learning to improve the inference on unseen testing samples.
Revised: The virtual classes of synthetic data can then be seamlessly coupled with meta-learning to improve the inference on unseen testing samples.

Original: • We introduce the concept of representing each class with image-adaptive semantic features that could vary among intra-class samples.
Revised: • We introduce the concept of representing each class with image-adaptive semantic features that could vary among intra-class samples.

Original: While the adaptive mechanism improves classifying the seen classes, it manifests the advantage in boosting the effect of meta-learning over virtual data to zero-shot inference over unseen classes.
Revised: While the adaptive mechanism improves classifying the seen classes, it manifests the advantage in boosting the effect of meta-learning over virtual data to zero-shot inference over unseen classes.

Original: • We demonstrate state-of-the-art results of zero-shot learning over four popular benchmark datasets and justify the design of our method with a thorough ablation study.
Revised: • We demonstrate state-of-the-art results of zero-shot learning over four popular benchmark datasets and justify the design of our method with a thorough ablation study.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: We review relevant literature in this section.
Revised: We review relevant literature in this section.

Original: First, we describe generative approaches for ZSL/GZSL that synthesize unseen images for training.
Revised: First, we describe generative approaches for ZSL/GZSL that synthesize unseen images for training.

Original: To improve GZSL performance, we propose to couple virtual class generation with meta-learning for mimicking the inference scenario.
Revised: To improve GZSL performance, we propose to couple virtual class generation with meta-learning for mimicking the inference scenario.

Original: Next, we discuss attention approaches that extract discriminating features from images to help classiﬁcation.
Revised: Next, we discuss attention approaches that extract discriminating features from images to help classiﬁcation.

Original: 2. G ENERATIVE A PPROACHES FOR ZSL/GZSL
Revised: 2. G ENERATIVE A PPROACHES FOR ZSL/GZSL

Original: Arguably one of the most important problems in ZSL/GZSL is to prevent models from being biased to seen classes.
Revised: Arguably one of the most important problems in ZSL/GZSL is to prevent models from being biased to seen classes.

Original: Generative approaches (Chen et al., 2018; Felix et al., 2018; Kumar Verma et al., 2018;
Revised: Generative approaches (Chen et al., 2018; Felix et al., 2018; Kumar Verma et al., 2018;

Original: Mishra et al., 2018; Schonfeld et al., 2019; Paul et al., 2019; Xian et al., 2019) tackle the problem by synthesizing visual features of unseen classes from their semantic features with generative models like Generative Adversarial Networks (GAN; Goodfellow et al., 2014) or Variational Autoencoders (VAE) (VAE; Kingma & Welling, 2014).
Revised: Mishra et al., 2018; Schonfeld et al., 2019; Paul et al., 2019; Xian et al., 2019) tackle the problem by synthesizing visual features of unseen classes from their semantic features with generative models like Generative Adversarial Networks (GAN; Goodfellow et al., 2014) or Variational Autoencoders (VAE) (VAE; Kingma & Welling, 2014).

Original: The synthetic visual features act as pseudo-examples of unseen classes, and effectively reduce the ZSL/GZSL problem to a supervised learning one.
Revised: The synthetic visual features act as pseudo-examples of unseen classes, and effectively reduce the ZSL/GZSL problem to a supervised learning one.

Original: One key issue behind generative approaches comes from the insufﬁcient amount of data to learn a good generative model.
Revised: One key issue behind generative approaches comes from the insufﬁcient amount of data to learn a good generative model.

Original: As a consequence, some semantic features that seem important during training may cause overﬁtting, and others that seem less important may be completely dropped.
Revised: As a consequence, some semantic features that seem important during training may cause overﬁtting, and others that seem less important may be completely dropped.

Original: Therefore, several prior techniques propose new constraints or losses to preserve semantic features and regularize the generative model.
Revised: Therefore, several prior techniques propose new constraints or losses to preserve semantic features and regularize the generative model.

Original: For instance, (Chen et al., 2018) avoids the dropping of semantic information by disentangling the semantic space into two subspaces, one for classiﬁcation and the other for reconstruction
Revised: For instance, (Chen et al., 2018) avoids the dropping of semantic information by disentangling the semantic space into two subspaces, one for classiﬁcation and the other for reconstruction

Original: [Table]
Revised: [Table]

Original: ) enforces visual-semantic feature consistency by requiring synthesized visual features to reconstruct semantic features accurately.
Revised: ) enforces visual-semantic feature consistency by requiring synthesized visual features to reconstruct semantic features accurately.

Original: However, even with the new constraints or losses, the success of generative approaches still highly depends on whether there is enough data/variation in the seen classes to synthesize diverse visual features.
Revised: However, even with the new constraints or losses, the success of generative approaches still highly depends on whether there is enough data/variation in the seen classes to synthesize diverse visual features.

Original: Otherwise, the yielded features will be too close to those few seen classes and cannot help
Revised: Otherwise, the yielded features will be too close to those few seen classes and cannot help

Original: ZSL/GZSL much.
Revised: ZSL/GZSL much.

Original: For example, in AWA2 dataset (Xian et al., 2018a), the unseen samples of “rat” are easily misclassiﬁed as seen classes of “mouse” or “hamster” because synthetic visual features for “rat” are doomed to be confused with seen visual features of those similar classes.
Revised: For example, in AWA2 dataset (Xian et al., 2018a), the unseen samples of “rat” are easily misclassiﬁed as seen classes of “mouse” or “hamster” because synthetic visual features for “rat” are doomed to be confused with seen visual features of those similar classes.

Original: The phenomenon inspires us to generate synthetic visual features for not only the given unseen classes but also the virtual unseen classes.
Revised: The phenomenon inspires us to generate synthetic visual features for not only the given unseen classes but also the virtual unseen classes.

Original: These virtual classes provide a wider spectrum of support for unseen scenarios to improve ZSL/GZSL performance.
Revised: These virtual classes provide a wider spectrum of support for unseen scenarios to improve ZSL/GZSL performance.

Original: Our other focal effort is to connect the concept of meta-learning (Vinyals et al., 2016;
Revised: Our other focal effort is to connect the concept of meta-learning (Vinyals et al., 2016;

Original: Finn et al., 2017) with virtual classes in the training phase of the unseen expert.
Revised: Finn et al., 2017) with virtual classes in the training phase of the unseen expert.

Original: The most closely related approaches in
Revised: The most closely related approaches in

Original: ZSL are Li et al.
Revised: ZSL are Li et al.

Original: (2019b); Yu et al.
Revised: (2019b); Yu et al.

Original: (2020); Verma et al.
Revised: (2020); Verma et al.

Original: Sung et al.
Revised: Sung et al.

Original: ; Hu et al.
Revised: ; Hu et al.

Original: Verma et al.
Revised: Verma et al.

Original: (2020) combines meta-learning and generative models to conquer limits in generative models.
Revised: (2020) combines meta-learning and generative models to conquer limits in generative models.

Original: Li et al.
Revised: Li et al.

Original: (2019b) mimics the inference scenario by randomly selecting seen classes as ”fake” new classes in each episode.
Revised: (2019b) mimics the inference scenario by randomly selecting seen classes as ”fake” new classes in each episode.

Original: Yu et al.
Revised: Yu et al.

Original: (2020) randomly splits seen classes into two sets to train and reﬁne the model.
Revised: (2020) randomly splits seen classes into two sets to train and reﬁne the model.

Original: In contrast, we simulate the ZSL inference scenario in each episode by creating virtual classes from seen classes using mixup.
Revised: In contrast, we simulate the ZSL inference scenario in each episode by creating virtual classes from seen classes using mixup.

Original: Owing to the training scenario more resembling the inference setting in ZSL, our model achieves better S2V embedding on unseen classes and obtains state-of-the-art ZSL performance on most datasets.
Revised: Owing to the training scenario more resembling the inference setting in ZSL, our model achieves better S2V embedding on unseen classes and obtains state-of-the-art ZSL performance on most datasets.

Original: Besides, Sung et al.
Revised: Besides, Sung et al.

Original: (2018) learns a deep distance metric to classify samples.
Revised: (2018) learns a deep distance metric to classify samples.

Original: Hu et al.
Revised: Hu et al.

Original: (2018) utilizes the correction module to assist classiﬁcation.
Revised: (2018) utilizes the correction module to assist classiﬁcation.

Original: On the contrary, our model does not need to learn complex relationship between classes and additional assistant module to achieve good performance.
Revised: On the contrary, our model does not need to learn complex relationship between classes and additional assistant module to achieve good performance.

Original: 2. ATTENTION
Revised: 2. ATTENTION

Original: Attention mechanism is widely used in ZSL (Ji et al., 2018; Xie et al., 2019; Huynh & Elhamifar, 2020; Min et al., 2020; Liu et al., 2019).
Revised: Attention mechanism is widely used in ZSL (Ji et al., 2018; Xie et al., 2019; Huynh & Elhamifar, 2020; Min et al., 2020; Liu et al., 2019).

Original: Highlighting important local features and reducing noisy feature inﬂuence generate a more effective mapping between visual and semantic domains.
Revised: Highlighting important local features and reducing noisy feature inﬂuence generate a more effective mapping between visual and semantic domains.

Original: Ji et al.
Revised: Ji et al.

Original: (2018) proposes S 2 GA which utilizes semantic features to emphasize most informative visual local features.
Revised: (2018) proposes S 2 GA which utilizes semantic features to emphasize most informative visual local features.

Original: (Xie et al., 2019) employs AREN and ACSE to focus on the most important region in images.
Revised: (Xie et al., 2019) employs AREN and ACSE to focus on the most important region in images.

Original: Huynh & Elhamifar (2020) applies dense attribute-attention to ﬁnd the most discriminating image parts and embed them to semantic features individually.
Revised: Huynh & Elhamifar (2020) applies dense attribute-attention to ﬁnd the most discriminating image parts and embed them to semantic features individually.

Original: DVBE (Min et al., 2020) utilizes spatial and channel attention to maximize inter-class margin.
Revised: DVBE (Min et al., 2020) utilizes spatial and channel attention to maximize inter-class margin.

Original: While the attention to visual features is proved useful by the above-mentioned research efforts, relatively few attempts assess the importance of attention on semantic features.
Revised: While the attention to visual features is proved useful by the above-mentioned research efforts, relatively few attempts assess the importance of attention on semantic features.

Original: In the ZSL setting, each class contains only one semantic feature vector such that intra-class variations are neglected from the single semantic representation.
Revised: In the ZSL setting, each class contains only one semantic feature vector such that intra-class variations are neglected from the single semantic representation.

Original: For example, the object of interest can be occluded partly in the image or in the front of different backgrounds, and a unique class-wise semantic feature vector is hard to reﬂect such variances.
Revised: For example, the object of interest can be occluded partly in the image or in the front of different backgrounds, and a unique class-wise semantic feature vector is hard to reﬂect such variances.

Original: Thus, attention to semantic features is worth further exploring in ZSL.
Revised: Thus, attention to semantic features is worth further exploring in ZSL.

Original: Along this line, the most relevant work to ours is LFGAA (Liu et al., 2019), which considers semantic prediction of samples.
Revised: Along this line, the most relevant work to ours is LFGAA (Liu et al., 2019), which considers semantic prediction of samples.

Original: Different from LFGAA, our method puts attention on discriminating dimensions of semantic features and maps them to the visual domain.
Revised: Different from LFGAA, our method puts attention on discriminating dimensions of semantic features and maps them to the visual domain.

Original: The effect of the proposed attention mechanism is to attract the visual representation of ground-truth class and repel others.
Revised: The effect of the proposed attention mechanism is to attract the visual representation of ground-truth class and repel others.

Original: Besides, our other main difference from
Revised: Besides, our other main difference from

Original: LFGAA is the shortcut design in the attention model.
Revised: LFGAA is the shortcut design in the attention model.

Original: We add a shortcut to prevent the model from overﬁtting on training data and retain discriminating dimensions for unseen class classiﬁcation.
Revised: We add a shortcut to prevent the model from overﬁtting on training data and retain discriminating dimensions for unseen class classiﬁcation.

Original:  M ETHOD
Revised:  M ETHOD

Original: We coin the proposed method as AGZSL in that its main idea is the fusion of Adaptive and Generative mechanisms in solving ZSL/GZSL.
Revised: We coin the proposed method as AGZSL in that its main idea is the fusion of Adaptive and Generative mechanisms in solving ZSL/GZSL.

Original: (See Figure 1.)
Revised: (See Figure 1.)

Original: We describe below the details of AGZSL.
Revised: We describe below the details of AGZSL.

Original: 3. N OTATIONS AND F ORMULATION
Revised: 3. N OTATIONS AND F ORMULATION

Original: We denote training data as D = D S ∪ D U where D S
Revised: We denote training data as D = D S ∪ D U where D S

Original: = { ( x s ∈ X s ,
Revised: = { ( x s ∈ X s ,

Original: a s ∈
Revised: a s ∈

Original: A s , y s ∈ Y s ) } comprises seen data and D U
Revised: A s , y s ∈ Y s ) } comprises seen data and D U

Original: = { ( a u ∈ A u , y u ∈ Y u ) } includes unseen class labels y u and the corresponding
Revised: = { ( a u ∈ A u , y u ∈ Y u ) } includes unseen class labels y u and the corresponding

Original: [Figure]
Revised: [Figure]

Original: semantic/attribute feature vectors a u .
Revised: semantic/attribute feature vectors a u .

Original: The dimensions of visual features and semantic features are respectively expressed as x ∈ R d and a ∈ R k , while the number of total classes is n .
Revised: The dimensions of visual features and semantic features are respectively expressed as x ∈ R d and a ∈ R k , while the number of total classes is n .

Original: We also assume that the seen category Y s and the unseen category Y u are disjoint, i.e., Y s ∩Y u
Revised: We also assume that the seen category Y s and the unseen category Y u are disjoint, i.e., Y s ∩Y u

Original: = ∅
Revised: = ∅

Original: and Y s ∪Y u = Y .
Revised: and Y s ∪Y u = Y .

Original: Besides, we deﬁne A s ∪ A u =
Revised: Besides, we deﬁne A s ∪ A u =

Original: A ∈ R k × n as semantic features of all classes.
Revised: A ∈ R k × n as semantic features of all classes.

Original: Finally, the virtual data via mixup interpolations is denoted as D V = { ( x v ∈ X v , a v ∈
Revised: Finally, the virtual data via mixup interpolations is denoted as D V = { ( x v ∈ X v , a v ∈

Original: A v , y v ∈ Y v ) } .
Revised: A v , y v ∈ Y v ) } .

Original: As we explain later, y v is simply a new label of virtual class, rather than from label interpolation as in Zhang et al.
Revised: As we explain later, y v is simply a new label of virtual class, rather than from label interpolation as in Zhang et al.

Original: To perform the classiﬁcation task, we explore the similarity correlations in the visual domain and consider semantic-to-visual (S2V) embeddings.
Revised: To perform the classiﬁcation task, we explore the similarity correlations in the visual domain and consider semantic-to-visual (S2V) embeddings.

Original: Analogous to COSMO (Atzmon & Chechik, 2019), our method includes seen and unseen experts, f s and f u , to classify testing samples as shown in
Revised: Analogous to COSMO (Atzmon & Chechik, 2019), our method includes seen and unseen experts, f s and f u , to classify testing samples as shown in

Original: Figure 2; however, the algorithmic designs are fundamentally different.
Revised: Figure 2; however, the algorithmic designs are fundamentally different.

Original: The novel component of our method is the introduction of Image-Adaptive Semantics (IAS) in both experts to expand the spectrum of S2V embeddings.
Revised: The novel component of our method is the introduction of Image-Adaptive Semantics (IAS) in both experts to expand the spectrum of S2V embeddings.

Original: Speciﬁcally, in training, IAS diversiﬁes the total mappings of S2V embedding from the number of different semantic vectors/classes to the number of samples.
Revised: Speciﬁcally, in training, IAS diversiﬁes the total mappings of S2V embedding from the number of different semantic vectors/classes to the number of samples.

Original: Its effect is twofold.
Revised: Its effect is twofold.

Original: First, it boosts the classiﬁcation performance of the seen expert.
Revised: First, it boosts the classiﬁcation performance of the seen expert.

Original: Second, it generalizes the meta-learning of the unseen expert with virtual data and yields good performance in the inference of samples from unseen classes.
Revised: Second, it generalizes the meta-learning of the unseen expert with virtual data and yields good performance in the inference of samples from unseen classes.

Original: Given a testing sample x in inference, we apply the two learned experts by
Revised: Given a testing sample x in inference, we apply the two learned experts by

Original: [Equation]
Revised: [Equation]

Original: To decide the class label ˆ y of x , we use the seen expert to discriminate whether the sample is from the seen category.
Revised: To decide the class label ˆ y of x , we use the seen expert to discriminate whether the sample is from the seen category.

Original: If it is the case then ˆ y is decided by f s , and otherwise by f u .
Revised: If it is the case then ˆ y is decided by f s , and otherwise by f u .

Original: That is, we have
Revised: That is, we have

Original: [Equation]
Revised: [Equation]

Original: 3. I MAGE A DAPTIVE S EMANTICS (IAS)
Revised: 3. I MAGE A DAPTIVE S EMANTICS (IAS)

Original: As ZSL/GZSL assumes one semantic feature vector for each class, the conventional semantic-tovisual architecture leads to one single class-level visual feature vector, which is common to all samples of the same class.
Revised: As ZSL/GZSL assumes one semantic feature vector for each class, the conventional semantic-tovisual architecture leads to one single class-level visual feature vector, which is common to all samples of the same class.

Original: To learn such a single embedding vector with the constraint of being simultaneously similar to all visual features of samples from the same class tends to yield unsatisfactory classiﬁcation outcomes in that it simply ignores the intra-class variations in visual features.
Revised: To learn such a single embedding vector with the constraint of being simultaneously similar to all visual features of samples from the same class tends to yield unsatisfactory classiﬁcation outcomes in that it simply ignores the intra-class variations in visual features.

Original: Diversifying semantic-to-visual embeddings with IAS establishes the core of our model as shown in
Revised: Diversifying semantic-to-visual embeddings with IAS establishes the core of our model as shown in

Original: Figure 3.
Revised: Figure 3.

Original: To realize the concept, the network module learns to map image-adaptive semantic features to the corresponding visual features, according to the cosine similarity.
Revised: To realize the concept, the network module learns to map image-adaptive semantic features to the corresponding visual features, according to the cosine similarity.

Original: To this effect, IAS leverages the given visual features x to focus on discriminative dimensions of the respective semantic features.
Revised: To this effect, IAS leverages the given visual features x to focus on discriminative dimensions of the respective semantic features.

Original: Speciﬁcally, we apply IAS to obtain modiﬁed semantic features ˆ A ( x ) ∈ R k × n by
Revised: Speciﬁcally, we apply IAS to obtain modiﬁed semantic features ˆ A ( x ) ∈ R k × n by

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Figure]
Revised: [Figure]

Original: SeenExpert
Revised: SeenExpert

Original: UnseenExpert
Revised: UnseenExpert

Original: Training phase
Revised: Training phase

Original: Inference phase
Revised: Inference phase

Original: SeenExpert
Revised: SeenExpert

Original: UnseenExpert where softmax( g ( x ))
Revised: UnseenExpert where softmax( g ( x ))

Original: ∈
Revised: ∈

Original: R k represents using a linear layer g to predict semantic attention and (cid:12) denotes Hadamard product to combine attention and semantic features.
Revised: R k represents using a linear layer g to predict semantic attention and (cid:12) denotes Hadamard product to combine attention and semantic features.

Original: Note that the shortcut implementation in (3) is pivotal as the softmax function often puts high attention weights on sparse dimensions and draws others near to zero.
Revised: Note that the shortcut implementation in (3) is pivotal as the softmax function often puts high attention weights on sparse dimensions and draws others near to zero.

Original: Without the shortcut, the modiﬁed ˆ A would miss some part of semantic features and degenerate the classiﬁcation performance.
Revised: Without the shortcut, the modiﬁed ˆ A would miss some part of semantic features and degenerate the classiﬁcation performance.

Original: Also, it is worthy to point out that compared to LFGAA (Liu et al., 2019), our method pays attention to the semantic features of classes, not the semantic prediction of the model.
Revised: Also, it is worthy to point out that compared to LFGAA (Liu et al., 2019), our method pays attention to the semantic features of classes, not the semantic prediction of the model.

Original: With (3), the process to drive the image-adaptive class-level visual features V ( x ) can be expressed by
Revised: With (3), the process to drive the image-adaptive class-level visual features V ( x ) can be expressed by

Original: [Equation]
Revised: [Equation]

Original: Once we have obtained the class-level visual features, we then use the cosine similarity and softmax to calculate the probability of each class.
Revised: Once we have obtained the class-level visual features, we then use the cosine similarity and softmax to calculate the probability of each class.

Original: We have the class probability of prediction on x :
Revised: We have the class probability of prediction on x :

Original: [Equation]
Revised: [Equation]

Original: where σ is the learnable scale and f is indeed the general form of a classiﬁcation expert.
Revised: where σ is the learnable scale and f is indeed the general form of a classiﬁcation expert.

Original: We conclude by stating that the Cross-Entropy(CE) loss is adopted to update the model:
Revised: We conclude by stating that the Cross-Entropy(CE) loss is adopted to update the model:

Original: [Equation]
Revised: [Equation]

Original: where p y ( x ) is the predicted probability of ground-truth label.
Revised: where p y ( x ) is the predicted probability of ground-truth label.

Original: 3. S EEN E XPERT
Revised: 3. S EEN E XPERT

Original: The objective of seen expert f s is to recognize seen classes and otherwise decide whether a testing sample belongs to unseen classes in inference phase.
Revised: The objective of seen expert f s is to recognize seen classes and otherwise decide whether a testing sample belongs to unseen classes in inference phase.

Original: In zero-shot learning, especially GZSL, the capability to single out testing samples of unseen classes is crucial to the classiﬁcation performance.
Revised: In zero-shot learning, especially GZSL, the capability to single out testing samples of unseen classes is crucial to the classiﬁcation performance.

Original: If one can establish reasonable split mechanisms such as in Min et al.
Revised: If one can establish reasonable split mechanisms such as in Min et al.

Original: and
Revised: and

Original: Chen et al.
Revised: Chen et al.

Original: (2020), the more challenging GZSL would be simpliﬁed into ZSL.
Revised: (2020), the more challenging GZSL would be simpliﬁed into ZSL.

Original: For example, COSMO (Atzmon & Chechik, 2019) utilizes Conﬁdence Base Gating to discriminate between seen and unseen classes, while DVBE (Min et al., 2020) considers the entropy threshold.
Revised: For example, COSMO (Atzmon & Chechik, 2019) utilizes Conﬁdence Base Gating to discriminate between seen and unseen classes, while DVBE (Min et al., 2020) considers the entropy threshold.

Original: In our formulation, the seen expert learns to predict all possible classes, including seen and unseen, although the training data assume only seen class labels.
Revised: In our formulation, the seen expert learns to predict all possible classes, including seen and unseen, although the training data assume only seen class labels.

Original: That is, f s is trained to map (image adaptive) seen semantic features to their corresponding visual features.
Revised: That is, f s is trained to map (image adaptive) seen semantic features to their corresponding visual features.

Original: Such optimization can be thought of as one-category anomaly detection over only seen classes.
Revised: Such optimization can be thought of as one-category anomaly detection over only seen classes.

Original: Thus, in inference, if the highest cosine similarity value by the seen expert f s is not yielded by one of the seen classes, the testing sample is considered abnormal from the unseen category and its classiﬁcation should be decided by the unseen expert discussed next.
Revised: Thus, in inference, if the highest cosine similarity value by the seen expert f s is not yielded by one of the seen classes, the testing sample is considered abnormal from the unseen category and its classiﬁcation should be decided by the unseen expert discussed next.

Original: [Figure]
Revised: [Figure]

Original: 3. U NSEEN E XPERT
Revised: 3. U NSEEN E XPERT

Original: So far we have stated the objective of the unseen expert f u is to classify the labels of those that cannot be decided by the seen expert, as in (2).
Revised: So far we have stated the objective of the unseen expert f u is to classify the labels of those that cannot be decided by the seen expert, as in (2).

Original: To resolve the dilemma of zero-shot setting, we apply meta-learning to train f u over virtual classes by mixup interpolations.
Revised: To resolve the dilemma of zero-shot setting, we apply meta-learning to train f u over virtual classes by mixup interpolations.

Original: The exact number of virtual classes in each episode is decided by the pilot study (Li et al., 2019b), and an ablation study in appendix is provided to analyze its effect on the performance of zero-shot classiﬁcation.
Revised: The exact number of virtual classes in each episode is decided by the pilot study (Li et al., 2019b), and an ablation study in appendix is provided to analyze its effect on the performance of zero-shot classiﬁcation.

Original: To begin with, in each episode we randomly select m pairs of seen classes.
Revised: To begin with, in each episode we randomly select m pairs of seen classes.

Original: Then, we create m virtual classes out of these pairs by mixup.
Revised: Then, we create m virtual classes out of these pairs by mixup.

Original: The semantic and visual features for these virtual classes are deﬁned as
Revised: The semantic and visual features for these virtual classes are deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where a i and a j are semantic features of paired classes, and x i and x j are randomly chosen from visual samples of the paired classes.
Revised: where a i and a j are semantic features of paired classes, and x i and x j are randomly chosen from visual samples of the paired classes.

Original: Following Zhang et al.
Revised: Following Zhang et al.

Original: (2018), λ ∈ [0 , 1] is sampled from the beta distribution.
Revised: (2018), λ ∈ [0 , 1] is sampled from the beta distribution.

Original: We then deﬁne a new virtual class y v , with a v as semantic features, to be associated with virtual visual features x v .
Revised: We then deﬁne a new virtual class y v , with a v as semantic features, to be associated with virtual visual features x v .

Original: Previous generative models (Schonfeld et al., 2019; Xian et al., 2019) synthesize unseen samples by GAN/VAE or their variants.
Revised: Previous generative models (Schonfeld et al., 2019; Xian et al., 2019) synthesize unseen samples by GAN/VAE or their variants.

Original: In comparison, the proposed meta-learning has at least three advantages for ZSL.
Revised: In comparison, the proposed meta-learning has at least three advantages for ZSL.

Original: First, the small number of classes in some datasets, such asin AWA2, may fall short to provide sufﬁcient support set such that the generative model is hard to be optimized, while episodic meta-learning over virtual classes can overcome this problem.
Revised: First, the small number of classes in some datasets, such asin AWA2, may fall short to provide sufﬁcient support set such that the generative model is hard to be optimized, while episodic meta-learning over virtual classes can overcome this problem.

Original: Second, some part of semantic features which are important to classify unseen classes do not have enough training classes in the datasets.
Revised: Second, some part of semantic features which are important to classify unseen classes do not have enough training classes in the datasets.

Original: With mixup semantic features as virtual classes, our model can learn underlying discriminating semantic features well.
Revised: With mixup semantic features as virtual classes, our model can learn underlying discriminating semantic features well.

Original: Third, the fusion of image-adaptive mechanism and meta-learning is ﬂexible.
Revised: Third, the fusion of image-adaptive mechanism and meta-learning is ﬂexible.

Original: As a result, we do not need online ﬁne-tuning in applying the unseen expert f u yielded by meta-learning to predict the label of unseen class for testing samples.
Revised: As a result, we do not need online ﬁne-tuning in applying the unseen expert f u yielded by meta-learning to predict the label of unseen class for testing samples.

Original:  E XPERIMENT 4. E XPERIMENTAL S ETTING
Revised:  E XPERIMENT 4. E XPERIMENTAL S ETTING

Original: Datasets.
Revised: Datasets.

Original: In GZSL, there are four widely used benchmark datasets for evaluation, which are AWA (Xian et al., 2018a), USCD Birds-200-2011 ( CUB )
Revised: In GZSL, there are four widely used benchmark datasets for evaluation, which are AWA (Xian et al., 2018a), USCD Birds-200-2011 ( CUB )

Original: (Welinder et al., 2010), SUN (Patterson & Hays, 2012), and aPY (Farhadi et al., 2009).
Revised: (Welinder et al., 2010), SUN (Patterson & Hays, 2012), and aPY (Farhadi et al., 2009).

Original: AWA2 contains 37K images and 50 kinds of animals with 85dimension attributes.
Revised: AWA2 contains 37K images and 50 kinds of animals with 85dimension attributes.

Original: CUB consists of 11K images and 200 bird species described with 312-dimension attributes.
Revised: CUB consists of 11K images and 200 bird species described with 312-dimension attributes.

Original: SUN is composed of 14K images 717 classes labeled with 102-dimension attributes.
Revised: SUN is composed of 14K images 717 classes labeled with 102-dimension attributes.

Original: aPY includes 15K images and 32 different classes with 64-dimension attributes.
Revised: aPY includes 15K images and 32 different classes with 64-dimension attributes.

Original: The attributes of class in datasets will be taken as the semantic features in our work.
Revised: The attributes of class in datasets will be taken as the semantic features in our work.

Original: We follow existing GZSL approaches (Min et al., 2020; Li et al., 2019b) and extract 2048 -dimensional visual features for each image by the ResNet101 backbone (He et al., 2016) pretrained on ImageNet-1K. We then split the data into seen and unseen classes according to the benchmark procedure from Xian et al.
Revised: We follow existing GZSL approaches (Min et al., 2020; Li et al., 2019b) and extract 2048 -dimensional visual features for each image by the ResNet101 backbone (He et al., 2016) pretrained on ImageNet-1K. We then split the data into seen and unseen classes according to the benchmark procedure from Xian et al.

Original: [Table]
Revised: [Table]

Original: Evaluation.
Revised: Evaluation.

Original: We evaluate our approach for both ZSL and GZSL settings.
Revised: We evaluate our approach for both ZSL and GZSL settings.

Original: For the ZSL setting, the average per-class top-1 accuracy ( T ) on unseen classes is taken as the evaluation metric.
Revised: For the ZSL setting, the average per-class top-1 accuracy ( T ) on unseen classes is taken as the evaluation metric.

Original: For
Revised: For

Original: GZSL
Revised: GZSL

Original: (Xian et al., 2017), the evaluation metrics include seen classes accuracy S , unseen class accuracy U , and their harmonic mean H
Revised: (Xian et al., 2017), the evaluation metrics include seen classes accuracy S , unseen class accuracy U , and their harmonic mean H

Original: =
Revised: =

Original: (2 U · S ) /
Revised: (2 U · S ) /

Original: ( U + S ) .
Revised: ( U + S ) .

Original: Implementation.
Revised: Implementation.

Original: As suggested in Li et al. (2019b), we normalize the visual and semantic features into [0 , 1] .
Revised: As suggested in Li et al. (2019b), we normalize the visual and semantic features into [0 , 1] .

Original: The architecture of semantic-to-visual embedding contains a two-layer linear model with 1,600 hidden units and utilizes ReLU on the hidden and output layer.
Revised: The architecture of semantic-to-visual embedding contains a two-layer linear model with 1,600 hidden units and utilizes ReLU on the hidden and output layer.

Original: The seen and unseen experts are trained by Adam optimizer with a learning rate 5 × 10 − 5 and 5 × 10 − 4 respectively for all datasets.
Revised: The seen and unseen experts are trained by Adam optimizer with a learning rate 5 × 10 − 5 and 5 × 10 − 4 respectively for all datasets.

Original: We apply 200,000 episodes to train the unseen expert.
Revised: We apply 200,000 episodes to train the unseen expert.

Original: In each episode, we randomly generate 16 or 20 (based on the dataset) virtual classes and 4 samples for each class.
Revised: In each episode, we randomly generate 16 or 20 (based on the dataset) virtual classes and 4 samples for each class.

Original: We follow the setting in (Min et al., 2020) to ﬁne-tune the backbone.
Revised: We follow the setting in (Min et al., 2020) to ﬁne-tune the backbone.

Original: 4. C OMPARISON WITH S TATE - OF - THE -A RT A PPROACHES
Revised: 4. C OMPARISON WITH S TATE - OF - THE -A RT A PPROACHES

Original: We compare our method with 13 recent GZSL approaches.
Revised: We compare our method with 13 recent GZSL approaches.

Original: These include SP-AEN (Chen et al., 2018),
Revised: These include SP-AEN (Chen et al., 2018),

Original: DLFZRL (Tong et al., 2019), PSR (Annadani & Biswas, 2018), CDL (Jiang et al., 2018), PQZSL (Li et al., 2019a), f-VAEGAM-D2 (Xian et al., 2019), LsrGAN (Vyas et al., 2020), TF-VAEGAN (Narayan et al., 2020), OCD-CVAE (Keshari et al., 2020), ZSML Softmax (Verma et al., 2020),
Revised: DLFZRL (Tong et al., 2019), PSR (Annadani & Biswas, 2018), CDL (Jiang et al., 2018), PQZSL (Li et al., 2019a), f-VAEGAM-D2 (Xian et al., 2019), LsrGAN (Vyas et al., 2020), TF-VAEGAN (Narayan et al., 2020), OCD-CVAE (Keshari et al., 2020), ZSML Softmax (Verma et al., 2020),

Original: E-PGN (Yu et al., 2020), Relation Net (Sung et al., 2018), Correlation Net (Hu et al., 2018), GXE
Revised: E-PGN (Yu et al., 2020), Relation Net (Sung et al., 2018), Correlation Net (Hu et al., 2018), GXE

Original: Li et al.
Revised: Li et al.

Original: (2019b),
Revised: (2019b),

Original: LFGAA+Hybrid (Liu et al., 2019), AREN (Xie et al., 2019), COSMO (Atzmon & Chechik, 2019), DVBE (Min et al., 2020).
Revised: LFGAA+Hybrid (Liu et al., 2019), AREN (Xie et al., 2019), COSMO (Atzmon & Chechik, 2019), DVBE (Min et al., 2020).

Original: Table 1 shows that the proposed AGZSL achieves the best GZSL performance on all four datasets.
Revised: Table 1 shows that the proposed AGZSL achieves the best GZSL performance on all four datasets.

Original: It improves harmonic mean by 7.4 % on AWA2 , 1.9 % on CUB , 0.2 % on SUN , and 0.5 % on aPY , respectively.
Revised: It improves harmonic mean by 7.4 % on AWA2 , 1.9 % on CUB , 0.2 % on SUN , and 0.5 % on aPY , respectively.

Original: Particularly, among those methods in comparison, DVBE needs to search the entropy threshold τ and the embedding model for the best harmonic means according to each dataset.
Revised: Particularly, among those methods in comparison, DVBE needs to search the entropy threshold τ and the embedding model for the best harmonic means according to each dataset.

Original: In contrast to DVBE, without entropy threshold and speciﬁc model searching, our model achieves over 4 % improvement on most datasets.
Revised: In contrast to DVBE, without entropy threshold and speciﬁc model searching, our model achieves over 4 % improvement on most datasets.

Original: The main reason for better performance is that AGZSL provides more discriminating visual representation for both seen and unseen classes.
Revised: The main reason for better performance is that AGZSL provides more discriminating visual representation for both seen and unseen classes.

Original: Therefore, our model can effectively separate seen and unseen samples and achieve better performance in the harmonic mean.
Revised: Therefore, our model can effectively separate seen and unseen samples and achieve better performance in the harmonic mean.

Original: Notably, AGZSL drops performance on aPY after backbone ﬁne-tuning.
Revised: Notably, AGZSL drops performance on aPY after backbone ﬁne-tuning.

Original: It could be due to that its number of seen classes and training images are fewest among the four datasets.
Revised: It could be due to that its number of seen classes and training images are fewest among the four datasets.

Original: Fine-tuning the backbone causes the visual features to overﬁt training samples and become less discriminating for correct classiﬁcation.
Revised: Fine-tuning the backbone causes the visual features to overﬁt training samples and become less discriminating for correct classiﬁcation.

Original: [Table]
Revised: [Table]

Original: Furthermore
Revised: Furthermore

Original: [Table]
Revised: [Table]

Original: generative models achieve higher performance on CUB and SUN , because in these two datasets there are many seen classes for synthesizing unseen samples of better quality.
Revised: generative models achieve higher performance on CUB and SUN , because in these two datasets there are many seen classes for synthesizing unseen samples of better quality.

Original: In comparison
Revised: In comparison

Original: [Table]
Revised: [Table]

Original: our model overcomes the inﬂuence of few seen classes and also achieves the best performance on
Revised: our model overcomes the inﬂuence of few seen classes and also achieves the best performance on

Original: AWA2 and aPY .
Revised: AWA2 and aPY .

Original: Our results empirically support that the introduction of virtual classes and samples are indeed helpful to classify unseen classes.
Revised: Our results empirically support that the introduction of virtual classes and samples are indeed helpful to classify unseen classes.

Original: Finally, we remark that since the backbone is pretrained on ImageNet-1K, which is rather different from the ZSL ﬁne-grained databases, e
Revised: Finally, we remark that since the backbone is pretrained on ImageNet-1K, which is rather different from the ZSL ﬁne-grained databases, e

Original: [Table]
Revised: [Table]

Original: g., CUB and SUN .
Revised: g., CUB and SUN .

Original: Thus, the harmonic mean improves a lot after ﬁne-tuning the backbone on these two datasets.
Revised: Thus, the harmonic mean improves a lot after ﬁne-tuning the backbone on these two datasets.

Original: 4. A BLATION S TUDY
Revised: 4. A BLATION S TUDY

Original: To evaluate the beneﬁts of tackling ZSL/GZSL with virtual classes and IAS, we carry out an ablation study on the four datasets.
Revised: To evaluate the beneﬁts of tackling ZSL/GZSL with virtual classes and IAS, we carry out an ablation study on the four datasets.

Original: We incrementally include each key component to assess their effect.
Revised: We incrementally include each key component to assess their effect.

Original: As shown in Table 2, the virtual classes improve ZSL accuracy signiﬁcantly.
Revised: As shown in Table 2, the virtual classes improve ZSL accuracy signiﬁcantly.

Original: The ZSL performance improves 4.6 % on AWA2 , 2.4 % on CUB , 1.7 % on SUN , and 2.8 % on aPY , respectively.
Revised: The ZSL performance improves 4.6 % on AWA2 , 2.4 % on CUB , 1.7 % on SUN , and 2.8 % on aPY , respectively.

Original: The result implies that the virtual classes are advantageous to ZSL.
Revised: The result implies that the virtual classes are advantageous to ZSL.

Original: After adding IAS to our model, both seen and unseen accuracy increas, and the harmonic mean H improves about 1.5 % on average.
Revised: After adding IAS to our model, both seen and unseen accuracy increases, and the harmonic mean H improves about 1.5 % on average.

Original: The consistent gain exempliﬁes that IAS can provide better semantic-to-visual embedding and achieve higher performance on GZSL.
Revised: The consistent gain exempliﬁes that IAS can provide better semantic-to-visual embedding and achieve higher performance on GZSL.

Original: 4. C OSINE S IMILARITY M ARGIN
Revised: 4. C OSINE S IMILARITY M ARGIN

Original: Figure 4 illustrates the effect of the proposed AGZSL model.
Revised: Figure 4 illustrates the effect of the proposed AGZSL model.

Original: The cosine similarity margin means the inner product value between the correct class and the other nearest class.
Revised: The cosine similarity margin means the inner product value between the correct class and the other nearest class.

Original: A negative margin indicates that the model predicts a false class.
Revised: A negative margin indicates that the model predicts a wrong class.

Original: On the other hand, a large positive margin implies that the model results in a proper embedding distinguishing the correct class from others effectively.
Revised: On the other hand, a large positive margin implies that the model results in a proper embedding distinguishing the correct class from others effectively.

Original: Thus, seeking a positive and larger margin is a crucial and important objective in GZSL.
Revised: Thus, seeking a positive and larger margin is a crucial and important objective in GZSL.

Original: As shown in the ﬁrst row of Figure 4, the margins of unseen samples shift right on all four datasets by our method.
Revised: As shown in the ﬁrst row of Figure 4, the margins of unseen samples shift right on all four datasets by our method.

Original: It shows that AGZSL indeed generates better embedding on the visual space and makes the model easier to predict the correct class.
Revised: It shows that AGZSL indeed generates better embedding on the visual space and makes the model easier to predict the correct class.

Original: Moreover, from the second row in Figure 4, the plots show that our method improves the margins of seen samples dramatically on most datasets.
Revised: Moreover, from the second row in Figure 4, the plots show that our method improves the margins of seen samples dramatically on most datasets.

Original: The above experiment explains that AGZSL can achieve a higher margin for most samples and achieves better classiﬁcation performance on the four testing datasets.
Revised: The above experiment explains that AGZSL can achieve a higher margin for most samples and achieves better classiﬁcation performance on the four testing datasets.

Original: 4. T HE D ISTRIBUTION OF V IRTUAL C LASSES
Revised: 4. T HE D ISTRIBUTION OF V IRTUAL C LASSES

Original: To explicitly exhibit the distributions of seen, unseen and virtual classes, we apply PCA to map the class-wise semantic features to a two-dimensional space, as shown in Figure 5.
Revised: To explicitly exhibit the distributions of seen, unseen and virtual classes, we apply PCA to map the class-wise semantic features to a two-dimensional space, as shown in Figure 5.

Original: The features of virtual classes scatter around seen classes, because we have used B (5,1) to generate virtual classes.
Revised: The features of virtual classes scatter around seen classes, because we have used B (5,1) to generate virtual classes.

Original: Advantageously, some virtual classes are very close to unseen classes in the semantic domain and turn out to be helpful for enabling the unseen expert to classify samples of unseen classes.
Revised: Advantageously, some virtual classes are very close to unseen classes in the semantic domain and turn out to be helpful for enabling the unseen expert to classify samples of unseen classes.

Original: With meta-learning over these numerous virtual classes and samples, the proposed S2V can better learn the embedding between semantic and visual domains, and results in an effective unseen expert.
Revised: With meta-learning over these numerous virtual classes and samples, the proposed S2V can better learn the embedding between semantic and visual domains, and results in an effective unseen expert.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper
Revised: In this paper

Original: [Figure]
Revised: [Figure]

Original: we propose Image Adaptive Semantics (IAS) and meta-learning with virtual classes and samples to solve the ZSL/GZSL problem.
Revised: we propose Image Adaptive Semantics (IAS) and meta-learning with virtual classes and samples to solve the ZSL/GZSL problem.

Original: To deal with the intra-class visual discrepancies, IAS adaptively emphasizes the most discriminating dimensions in semantic features with respect to the underlying visual features.
Revised: To deal with the intra-class visual discrepancies, IAS adaptively emphasizes the most discriminating dimensions in semantic features with respect to the underlying visual features.

Original: To better classify samples of the unseen classes in inference, we propose a novel formulation of generative meta-learning.
Revised: To better classify samples of the unseen classes in inference, we propose a novel formulation of generative meta-learning.

Original: Different from previous generative models that focus on synthesizing unseen samples for training the model, we create virtual classes and their respective virtual samples in the training phase.
Revised: Different from previous generative models that focus on synthesizing unseen samples for training the model, we create virtual classes and their respective virtual samples in the training phase.

Original: Further, to imitate the ZSL inference scenario, we carry out meta-learning with these virtual data to extend our model.
Revised: Further, to imitate the ZSL inference scenario, we carry out meta-learning with these virtual data to extend our model.

Original: In our experiment, we demonstrate that AGZSL is beneﬁcial to tackle the challenging ZSL/GZSL problem and achieves signiﬁcant advantages over those in comparison on the four ZSL/GZSL benchmark datasets.
Revised: In our experiment, we demonstrate that AGZSL is beneﬁcial to tackle the challenging ZSL/GZSL problem and achieves signiﬁcant advantages over those in comparison on the four ZSL/GZSL benchmark datasets.

Original: A CKNOWLEDGEMENTS
Revised: A CKNOWLEDGEMENTS

Original: This work was supported in part by the MOST, Taiwan under Grants 110-2634-F-001-009 and 107-2628-E-002-008-MY3.
Revised: This work was supported in part by the MOST, Taiwan under Grants 110-2634-F-001-009 and 107-2628-E-002-008-MY3.

Original: We are grateful to the National Center for High-performance Computing for providing computational resources and facilities.
Revised: We are grateful to the National Center for High-performance Computing for providing computational resources and facilities.

Original: A A PPENDIX
Revised: A A PPENDIX

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Shortcut in IAS Table 3 shows that shortcut in IAS improves performance a lot.
Revised: Shortcut in IAS Table 3 shows that shortcut in IAS improves performance a lot.

Original: The unseen class accuracy increase around 50 % on AWA2 and SUN .
Revised: The unseen class accuracy increase around 50 % on AWA2 and SUN .

Original: Besides, the harmonic mean also enhances 20 % to 60 % on four datasets with the shortcut.
Revised: Besides, the harmonic mean also enhances 20 % to 60 % on four datasets with the shortcut.

Original: The main reason is that the softmax usually puts high attention weight on few dimensions and decreases others near to zero.
Revised: The main reason is that the softmax usually puts high attention weight on few dimensions and decreases others near to zero.

Original: Thus, without shortcut, the model is prone to overﬁtting on training data.
Revised: Thus, without shortcut, the model is prone to overﬁtting on training data.

Original: Cosine similarity enhancement Figure 6 illustrates better embedding for seen and unseen classes in our model.
Revised: Cosine similarity enhancement Figure 6 illustrates better embedding for seen and unseen classes in our model.

Original: Since we apply cosine similarity to classify samples, the higher score means visual representation of each class is closer to their visual samples.
Revised: Since we apply cosine similarity to classify samples, the higher score means visual representation of each class is closer to their visual samples.

Original: The ﬁrst row in 6 shows effect of our model on unseen classes.
Revised: The ﬁrst row in 6 shows effect of our model on unseen classes.

Original: With virtual classes and IAS, cosine similarity increase averaged 0.05 to 0.1 on different datasets.
Revised: With virtual classes and IAS, cosine similarity increase averaged 0.05 to 0.1 on different datasets.

Original: The increasing amount shows our model provides better visual-to-semantic embedding for unseen classes and samples are easier to classify correctly.
Revised: The increasing amount shows our model provides better visual-to-semantic embedding for unseen classes and samples are easier to classify correctly.

Original: Therefore, our model can achieve good ZSL performance.
Revised: Therefore, our model can achieve good ZSL performance.

Original: Besides, the second row in 6 illustrates the effect of IAS on seen classes.
Revised: Besides, the second row in 6 illustrates the effect of IAS on seen classes.

Original: Most of the testing seen samples increase their cosine similarity around 0.02 on four datasets.
Revised: Most of the testing seen samples increase their cosine similarity around 0.02 on four datasets.

Original: With the IAS and virtual classes, the model can achieve better visual-to-semantic embedding on all classes and obtain better the generalized seen accuracy on four datasets.
Revised: With the IAS and virtual classes, the model can achieve better visual-to-semantic embedding on all classes and obtain better the generalized seen accuracy on four datasets.

Original: virtual classes number per episode To search the class number in each episode for best ZSL, we search 4 different numbers, i.e, 4,8,12, and 20 per episode.
Revised: virtual classes number per episode To search the class number in each episode for best ZSL, we search 4 different numbers, i.e, 4,8,12, and 20 per episode.

Original: Table 4 shows that the best class number setting is related to the class number of classes.
Revised: Table 4 shows that the best class number setting is related to the class number of classes.

Original: For example, the AWA2 and aPY which include fewer unseen classes obtain the best ZSL accuracy with 16 virtual classes.
Revised: For example, the AWA2 and aPY which include fewer unseen classes obtain the best ZSL accuracy with 16 virtual classes.

Original: On the other hand, the CUB and SUN which include more unseen classes achieve the best performance with 20 virtual classes.
Revised: On the other hand, the CUB and SUN which include more unseen classes achieve the best performance with 20 virtual classes.

Original: The reason is that the advantage of meta-learning is to simulate inference scenarios while training (Li et al., 2019b).
Revised: The reason is that the advantage of meta-learning is to simulate inference scenarios while training (Li et al., 2019b).

Original: The virtual class number set closer to the unseen class number is favorable in the meta-learning setting.
Revised: The virtual class number set closer to the unseen class number is favorable in the meta-learning setting.

Original: Therefore, the best virtual class number is different according to the unseen number of the dataset.
Revised: Therefore, the best virtual class number is different according to the unseen number of the dataset.

Original: Random Variable
Revised: Random Variable

Original: B ( , )
Revised: B ( , )

Original: B(5,5)
Revised: B(5,5)

Original: B(1,1)
Revised: B(1,1)

Original: B(5,1)
Revised: B(5,1)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Effect of beta distribution on ZSL accuracy To explore the best beta distribution setting for ZSL
Revised: Effect of beta distribution on ZSL accuracy To explore the best beta distribution setting for ZSL

Original: [Table]
Revised: [Table]

Original: our experiments apply three different α and β sets.
Revised: our experiments apply three different α and β sets.

Original: As shown in Figure 7, B(1,1) means virtual classes distribute uniformly between selected class pairs.
Revised: As shown in Figure 7, B(1,1) means virtual classes distribute uniformly between selected class pairs.

Original: Then, B(5,5) means most of the virtual classes distribute among the middle of selected class pairs.
Revised: Then, B(5,5) means most of the virtual classes distribute among the middle of selected class pairs.

Original: On the other hand, B(5,1) means most virtual classes distribute closer to one of the classes in selected class pairs.
Revised: On the other hand, B(5,1) means most virtual classes distribute closer to one of the classes in selected class pairs.

Original: Table 5 demonstrates that B(5,1) achieves best performance on each dataset.
Revised: Table 5 demonstrates that B(5,1) achieves best performance on each dataset.

Original: In our speculation, the virtual classes created based on one class are closer to real-world classes.
Revised: In our speculation, the virtual classes created based on one class are closer to real-world classes.

Original: For example, the zebra is very similar to the horse in shape.
Revised: For example, the zebra is very similar to the horse in shape.

Original: Thus, the virtual classes distributing among seen classes enhance ZSL accuracy are very beneﬁcial for training ZSL.
Revised: Thus, the virtual classes distributing among seen classes enhance ZSL accuracy are very beneﬁcial for training ZSL.