Original: C LOSED - FORM S AMPLE P ROBING FOR L EARNING G ENERATIVE M ODELS IN Z ERO - SHOT L EARNING
Revised: C LOSED - FORM S AMPLE P ROBING FOR L EARNING G ENERATIVE M ODELS IN Z ERO - SHOT L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Generative model based approaches have led to signiﬁcant advances in zero-shot learning (ZSL) over the past few-years.
Revised: Generative model based approaches have led to signiﬁcant advances in zero-shot learning (ZSL) over the past few-years.

Original: These approaches typically aim to learn a conditional generator that synthesizes training samples of classes conditioned on class deﬁnitions.
Revised: These approaches typically aim to learn a conditional generator that synthesizes training samples of classes conditioned on class deﬁnitions.

Original: The ﬁnal zero-shot learning model is then obtained by training a supervised classiﬁcation model over the real and/or synthesized training samples of seen and unseen classes, combined.
Revised: The ﬁnal zero-shot learning model is then obtained by training a supervised classiﬁcation model over the real and/or synthesized training samples of seen and unseen classes, combined.

Original: Therefore, naturally, the generative model needs to produce not only relevant samples, but also those that are sufﬁciently rich for classiﬁer training purposes, which is handled by various heuristics in existing works.
Revised: Therefore, naturally, the generative model needs to produce not only relevant samples, but also those that are sufﬁciently rich for classiﬁer training purposes, which is handled by various heuristics in existing works.

Original: In this paper, we introduce a principled approach for training generative models directly for training data generation purposes.
Revised: In this paper, we introduce a principled approach for training generative models directly for training data generation purposes.

Original: Our main observation is that the use of closed-form models opens doors to end-to-end training thanks to differentiability of the solvers.
Revised: Our main observation is that the use of closed-form models opens doors to end-to-end training thanks to differentiability of the solvers.

Original: In our approach, at each generative model update step, we ﬁt a task-speciﬁc closed-form ZSL model from generated samples, and measure its loss on novel samples all within the compute graph, a procedure that we refer to as sample probing .
Revised: In our approach, at each generative model update step, we ﬁt a task-speciﬁc closed-form ZSL model from generated samples, and measure its loss on novel samples all within the compute graph, a procedure that we refer to as sample probing .

Original: In this manner, the generator receives feedback directly based on the value of its samples for model training purposes.
Revised: In this manner, the generator receives feedback directly based on the value of its samples for model training purposes.

Original: Our experimental results show that the proposed sample probing approach improves the
Revised: Our experimental results show that the proposed sample probing approach improves the

Original: ZSL results even when integrated into state-of-the-art generative models.
Revised: ZSL results even when integrated into state-of-the-art generative models.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Zero-shot Learning (ZSL) has recently received a great interest for being one of the promising paradigms towards building very large vocabulary (visual) understanding models with limited training data.
Revised: Zero-shot Learning (ZSL) has recently received a great interest for being one of the promising paradigms towards building very large vocabulary (visual) understanding models with limited training data.

Original: The problem of ZSL can be summarized as the task of transferring information across classes such that the instances of unseen classes, with no training examples, can be recognized at test time, based on the training samples of seen classes.
Revised: The problem of ZSL can be summarized as the task of transferring information across classes such that the instances of unseen classes, with no training examples, can be recognized at test time, based on the training samples of seen classes.

Original: Similarly, the term Generalized Zero-Shot
Revised: Similarly, the term Generalized Zero-Shot

Original: Learning (GZSL) (Chao et al., 2016; Xian et al., 2018a) is used to refer to a practically more valuable variant of ZSL where both seen and unseen classes may appear at test time.
Revised: Learning (GZSL) (Chao et al., 2016; Xian et al., 2018a) is used to refer to a practically more valuable variant of ZSL where both seen and unseen classes may appear at test time.

Original: GZSL brings in additional challenges since GZSL models need to produce conﬁdence scores that are comparable across all classes.
Revised: GZSL brings in additional challenges since GZSL models need to produce conﬁdence scores that are comparable across all classes.

Original: Recent work shows that hallucinating unseen class samples through statistical generative models can be an effective strategy, e.g .
Revised: Recent work shows that hallucinating unseen class samples through statistical generative models can be an effective strategy, e.g .

Original: (Mishra et al., 2017; Xian et al., 2018b; 2019; Sariyildiz & Cinbis, 2019; Zhu et al., 2018; Schonfeld et al., 2019; Arora et al., 2018;
Revised: (Mishra et al., 2017; Xian et al., 2018b; 2019; Sariyildiz & Cinbis, 2019; Zhu et al., 2018; Schonfeld et al., 2019; Arora et al., 2018;

Original: Narayan et al., 2020).
Revised: Narayan et al., 2020).

Original: These approaches rely on generative models conditioned on class embeddings , obtained from auxiliary semantic knowledge, such as visual attributes (Xian et al., 2019), class name word embeddings (Schonfeld et al., 2019), or textual descriptions (Zhu et al., 2018).
Revised: These approaches rely on generative models conditioned on class embeddings , obtained from auxiliary semantic knowledge, such as visual attributes (Xian et al., 2019), class name word embeddings (Schonfeld et al., 2019), or textual descriptions (Zhu et al., 2018).

Original: The resulting synthetic examples, typically in combination with existing real examples, are used for training a supervised classiﬁer.
Revised: The resulting synthetic examples, typically in combination with existing real examples, are used for training a supervised classiﬁer.

Original: In generative GZSL approaches, the quality of class-conditional samples is crucial for building accurate recognition models.
Revised: In generative GZSL approaches, the quality of class-conditional samples is crucial for building accurate recognition models.

Original: It is not straight-forward to formally deﬁne the criteria of good training samples .
Revised: It is not straight-forward to formally deﬁne the criteria of good training samples .

Original: Arguably, however, samples need to be (i) realistic ( e.g . free from unwanted artifacts), (ii) relevant ( i.e . belong to the desired class distribution) and (iii) informative ( i.e . contain examples deﬁning class boundaries) to train an accurate classiﬁer.
Revised: Arguably, however, samples need to be (i) realistic ( e.g . free from unwanted artifacts), (ii) relevant ( i.e . belong to the desired class distribution) and (iii) informative ( i.e . contain examples deﬁning class boundaries) to train an accurate classiﬁer.

Original: Clearly, a primary factor affecting the quality of generated samples is the loss driving the conditional generative model training process.
Revised: Clearly, a primary factor affecting the quality of generated samples is the loss driving the conditional generative model training process.

Original: In this work, we aim to address the problem of training data generating models via an end-to-end mechanism that we call sample probing .
Revised: In this work, we aim to address the problem of training data generating models via an end-to-end mechanism that we call sample probing .

Original: 1 Our main goal is to directly evaluate the ability of a generative model in synthesizing training examples.
Revised: 1 Our main goal is to directly evaluate the ability of a generative model in synthesizing training examples.

Original: To this end, we observe that we can leverage classiﬁcation models with closed-form solvers to efﬁciently measure the quality of training samples, in an end-to-end manner.
Revised: To this end, we observe that we can leverage classiﬁcation models with closed-form solvers to efﬁciently measure the quality of training samples, in an end-to-end manner.

Original: More speciﬁcally, we formulate a simple yet powerful meta-learning approach: at each training iteration, (i) take a set of samples from the generative model for a randomly selected subset of classes, (ii) train a zero-shot probing model using only the synthesized samples, and (iii) evaluate the probing model on real samples from the training set.
Revised: More speciﬁcally, we formulate a simple yet powerful meta-learning approach: at each training iteration, (i) take a set of samples from the generative model for a randomly selected subset of classes, (ii) train a zero-shot probing model using only the synthesized samples, and (iii) evaluate the probing model on real samples from the training set.

Original: We then use the loss value as an end-to-end training signal for updating the generative model parameters.
Revised: We then use the loss value as an end-to-end training signal for updating the generative model parameters.

Original: Since we speciﬁcally focus on probing models with exact closed-form solutions, the probing model optimization simpliﬁes into a differentiable linear algebraic expression and takes part as a differentiable unit within the compute graph.
Revised: Since we speciﬁcally focus on probing models with exact closed-form solutions, the probing model optimization simpliﬁes into a differentiable linear algebraic expression and takes part as a differentiable unit within the compute graph.

Original: A graphical summary of the proposed training scheme is given in Figure 1.
Revised: A graphical summary of the proposed training scheme is given in Figure 1.

Original: In the rest of the paper, we ﬁrst provide an overview of related work, and explain our approach in detail.
Revised: In the rest of the paper, we ﬁrst provide an overview of related work, and explain our approach in detail.

Original: We then present a through experimental evaluation on GZSL benchmarks in Section 4, where the results show that sample probing yields improvements when introduced into state-of-theart baselines.
Revised: We then present a through experimental evaluation on GZSL benchmarks in Section 4, where the results show that sample probing yields improvements when introduced into state-of-theart baselines.

Original: We conclude with ﬁnal remarks in Section 5.
Revised: We conclude with ﬁnal remarks in Section 5.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: The generalized zero-shot learning problem has been introduced by Xian et al.
Revised: The generalized zero-shot learning problem has been introduced by Xian et al.

Original: (2017) and Chao et al.
Revised: (2017) and Chao et al.

Original: The extensive study in Xian et al.
Revised: The extensive study in Xian et al.

Original: (2018a) has shown that the success of methods can greatly vary across zero-shot and generalized zero-shot learning problems.
Revised: (2018a) has shown that the success of methods can greatly vary across zero-shot and generalized zero-shot learning problems.

Original: The additional challenge in the generalized case is the need for deciding whether an input test sample belongs to a seen or unseen class.
Revised: The additional challenge in the generalized case is the need for deciding whether an input test sample belongs to a seen or unseen class.

Original: Discriminative training of ZSL models, such as those based on bilinear compatibility functions (Weston et al., 2011; Frome et al., 2013), are likely to yield higher conﬁdence scores for seen classes.
Revised: Discriminative training of ZSL models, such as those based on bilinear compatibility functions (Weston et al., 2011; Frome et al., 2013), are likely to yield higher conﬁdence scores for seen classes.

Original: To alleviate this problem, a few recent works have proposed ways to regularize discriminative models towards producing comparable conﬁdence scores across all classes and avoid over-ﬁtting, e.g . Jiang et al.
Revised: To alleviate this problem, a few recent works have proposed ways to regularize discriminative models towards producing comparable conﬁdence scores across all classes and avoid over-ﬁtting, e.g . Jiang et al.

Original: Liu et al.
Revised: Liu et al.

Original: ; Chou et al.
Revised: ; Chou et al.

Original: (2021b).
Revised: (2021b).

Original: Generative approaches to zero-shot learning naturally address the conﬁdence score calibration problem.
Revised: Generative approaches to zero-shot learning naturally address the conﬁdence score calibration problem.

Original: However, in general, generative modeling corresponds to a more sophisticated task than learning discriminant functions only (Vapnik & Vapnik, 1998).
Revised: However, in general, generative modeling corresponds to a more sophisticated task than learning discriminant functions only (Vapnik & Vapnik, 1998).

Original: In this context, the problem is further complicated by the need of predicting zero-shot class distributions.
Revised: In this context, the problem is further complicated by the need of predicting zero-shot class distributions.

Original: To tackle this challenging task, a variety of techniques have been proposed (Mishra et al., 2017; Xian et al., 2018b; 2019; Sariyildiz & Cinbis, 2019; Zhu et al., 2018; Schonfeld et al., 2019; Arora et al., 2018; Elhoseiny & Elfeki, 2019;
Revised: To tackle this challenging task, a variety of techniques have been proposed (Mishra et al., 2017; Xian et al., 2018b; 2019; Sariyildiz & Cinbis, 2019; Zhu et al., 2018; Schonfeld et al., 2019; Arora et al., 2018; Elhoseiny & Elfeki, 2019;

Original: Narayan et al., 2020) by adapting generative models, such as VAEs (Kingma & Welling,
Revised: Narayan et al., 2020) by adapting generative models, such as VAEs (Kingma & Welling,

Original: 2014; Rezende et al., 2014) and GANs (Goodfellow et al., 2014).
Revised: 2014; Rezende et al., 2014) and GANs (Goodfellow et al., 2014).

Original: To enforce class conditioning, the state-of-the-art approaches use a mixture of heuristics: Xian et al.
Revised: To enforce class conditioning, the state-of-the-art approaches use a mixture of heuristics: Xian et al.

Original: (2018b) uses the loss of a pretrained classiﬁer over the generated samples during training.
Revised: (2018b) uses the loss of a pretrained classiﬁer over the generated samples during training.

Original: Narayan et al.
Revised: Narayan et al.

Original: (2020) additionally uses the loss of a sample-to-attribute regressor, in combination with a feedback mechanism motivated from feedback networks (Zamir et al., 2017).
Revised: (2020) additionally uses the loss of a sample-to-attribute regressor, in combination with a feedback mechanism motivated from feedback networks (Zamir et al., 2017).

Original: Sariyildiz & Cinbis (2019) uses projection discriminator (Miyato & Koyama, 2018) and gradient matching loss as a gradient based similarity measure for comparing real versus synthetic data distributions.
Revised: Sariyildiz & Cinbis (2019) uses projection discriminator (Miyato & Koyama, 2018) and gradient matching loss as a gradient based similarity measure for comparing real versus synthetic data distributions.

Original: None of these generative ZSL approaches, however, directly measure the value of the generated samples for training classiﬁcation models.
Revised: None of these generative ZSL approaches, however, directly measure the value of the generated samples for training classiﬁcation models.

Original: The main difﬁculty lies in the need for back-propagating over long compute chains, which is both inefﬁcient and prone to gradient vanishing problems.
Revised: The main difﬁculty lies in the need for back-propagating over long compute chains, which is both inefﬁcient and prone to gradient vanishing problems.

Original: To the best of our knowledge, we introduce the ﬁrst end-to-end solution to this problem by the idea of using probing models with closed-form solvers to monitor the sample quality for model training purposes.
Revised: To the best of our knowledge, we introduce the ﬁrst end-to-end solution to this problem by the idea of using probing models with closed-form solvers to monitor the sample quality for model training purposes.

Original: Our approach is effectively a meta-learning (Snell et al., 2017) scheme.
Revised: Our approach is effectively a meta-learning (Snell et al., 2017) scheme.

Original: Meta-learning is a prominent idea in few-shot learning, where the goal is learning to build predictive models from a limited number of samples.
Revised: Meta-learning is a prominent idea in few-shot learning, where the goal is learning to build predictive models from a limited number of samples.

Original: The main motivation is the idea that general-purpose classiﬁcation models may behave suboptimally when only few training samples are provided.
Revised: The main motivation is the idea that general-purpose classiﬁcation models may behave suboptimally when only few training samples are provided.

Original: A variety of meta-learning driven few-shot learning models have been proposed, such as meta-models that transform few samples to classiﬁers (Gidaris & Komodakis, 2018;
Revised: A variety of meta-learning driven few-shot learning models have been proposed, such as meta-models that transform few samples to classiﬁers (Gidaris & Komodakis, 2018;

Original: Snell et al., 2017), set-to-set transformations for improving sample representations (Ye et al., 2020; Bronskill et al., 2020), fast adaptation networks for few examples (Finn et al., 2017; Rusu et al., 2019; Nichol et al., 2018).
Revised: Snell et al., 2017), set-to-set transformations for improving sample representations (Ye et al., 2020; Bronskill et al., 2020), fast adaptation networks for few examples (Finn et al., 2017; Rusu et al., 2019; Nichol et al., 2018).

Original: In contrast to such mainstream learning to classify and learning to adapt approaches, we aim to address the problem of learning to generate training examples for GZSL.
Revised: In contrast to such mainstream learning to classify and learning to adapt approaches, we aim to address the problem of learning to generate training examples for GZSL.

Original: There are only few and recent studies that aim to tackle generative ZSL via meta-learning principles.
Revised: There are only few and recent studies that aim to tackle generative ZSL via meta-learning principles.

Original: Verma et al.
Revised: Verma et al.

Original: (2020; 2021) embrace the learning-to-adapt framework of MAML (Finn et al., 2017), which originally aims to learn the optimal initialization for few-shot adaptation.
Revised: (2020; 2021) embrace the learning-to-adapt framework of MAML (Finn et al., 2017), which originally aims to learn the optimal initialization for few-shot adaptation.

Original: The MAML steps are incorporated by iteratively applying a single-step update to the generative model using the generative model (VAE/GAN) loss terms, and then back-propagating over the re-computed generative loss terms on new samples from a disjoint subset of classes using the single-step updated model.
Revised: The MAML steps are incorporated by iteratively applying a single-step update to the generative model using the generative model (VAE/GAN) loss terms, and then back-propagating over the re-computed generative loss terms on new samples from a disjoint subset of classes using the single-step updated model.

Original: Our approach differs fundamentally, as we propose to use the discriminative guidance of
Revised: Our approach differs fundamentally, as we propose to use the discriminative guidance of

Original: ZSL models fully-trained directly from generated sample batches.
Revised: ZSL models fully-trained directly from generated sample batches.

Original: In another work, Yu et al.
Revised: In another work, Yu et al.

Original: (2020) proposes an episodic training -like approach with periodically altered training sets and losses during training, to learn non-stochastic mappings between the class-embeddings and class-centers.
Revised: (2020) proposes an episodic training -like approach with periodically altered training sets and losses during training, to learn non-stochastic mappings between the class-embeddings and class-centers.

Original: Chou et al.
Revised: Chou et al.

Original: (2021a), similarly inspired from meta-learning and mixup regularization (Zhang et al., 2018), proposes to train a novel discriminative ZSL model over episodically deﬁned virtual training classes obtained by linearly mixing classes.
Revised: (2021a), similarly inspired from meta-learning and mixup regularization (Zhang et al., 2018), proposes to train a novel discriminative ZSL model over episodically deﬁned virtual training classes obtained by linearly mixing classes.

Original: Neither of these approaches learn sample generating models, therefore, they have no direct relation to our work focusing on the problem of measuring the sample quality for ZSL model training purposes, with end-to-end discriminative guidance.
Revised: Neither of these approaches learn sample generating models, therefore, they have no direct relation to our work focusing on the problem of measuring the sample quality for ZSL model training purposes, with end-to-end discriminative guidance.

Original: The use of recognition models with closed-form solvers has attracted prior interest in various contexts.
Revised: The use of recognition models with closed-form solvers has attracted prior interest in various contexts.

Original: Notably, Romera-Paredes & Torr (2015) proposes the ESZSL model as a simple and effective
Revised: Notably, Romera-Paredes & Torr (2015) proposes the ESZSL model as a simple and effective

Original: ZSL formulation.
Revised: ZSL formulation.

Original: We leverage the closed-form solvability of the ESZSL model as part of our approach.
Revised: We leverage the closed-form solvability of the ESZSL model as part of our approach.

Original: Bertinetto et al.
Revised: Bertinetto et al.

Original: (2019) utilizes ridge-regression based task-speciﬁc few-shot learners within a discriminative meta-learning framework.
Revised: (2019) utilizes ridge-regression based task-speciﬁc few-shot learners within a discriminative meta-learning framework.

Original: In a similar fashion, Bhat et al.
Revised: In a similar fashion, Bhat et al.

Original: ; Liu et al.
Revised: ; Liu et al.

Original: (2020) tackle the problem of video object segmentation (VOS) and use ridge regression based task-speciﬁc segmentation models within a meta-learning framework.
Revised: (2020) tackle the problem of video object segmentation (VOS) and use ridge regression based task-speciﬁc segmentation models within a meta-learning framework.

Original: None of these approaches aim to use recognition models with closed form solvers to form guidance for generative model training.
Revised: None of these approaches aim to use recognition models with closed form solvers to form guidance for generative model training.

Original: Another related research topic is generative few-shot learning (FSL), where the goal is learning to synthesize additional in-class samples from few examples, e.g . Hariharan & Girshick (2016);
Revised: Another related research topic is generative few-shot learning (FSL), where the goal is learning to synthesize additional in-class samples from few examples, e.g . Hariharan & Girshick (2016);

Original: Wang et al.
Revised: Wang et al.

Original: ; Gao et al.
Revised: ; Gao et al.

Original: ; Schwartz et al.
Revised: ; Schwartz et al.

Original: ; Lazarou et al.
Revised: ; Lazarou et al.

Original: Among these, Wang et al.
Revised: Among these, Wang et al.

Original: (2018) is particularly related for following a similar motivation of learning to generate good training examples.
Revised: (2018) is particularly related for following a similar motivation of learning to generate good training examples.

Original: This is realized by feeding generated samples to a meta-learning model to obtain a classiﬁer, apply it to real query samples, and use its query loss to update the generative model.
Revised: This is realized by feeding generated samples to a meta-learning model to obtain a classiﬁer, apply it to real query samples, and use its query loss to update the generative model.

Original: Apart from the main difference in the problem deﬁnition (GZSL vs FSL), our work differs mainly by fully-training a closed-form solvable ZSL model from scratch at each training step, instead of a few-shot meta-learners that is jointly trained progressively with the generative model.
Revised: Apart from the main difference in the problem deﬁnition (GZSL vs FSL), our work differs mainly by fully-training a closed-form solvable ZSL model from scratch at each training step, instead of a few-shot meta-learners that is jointly trained progressively with the generative model.

Original: Finally, we note that meta-learning based approaches have also been proposed for a variety of different problems, such as learning-to-optimize (Li & Malik, 2016;
Revised: Finally, we note that meta-learning based approaches have also been proposed for a variety of different problems, such as learning-to-optimize (Li & Malik, 2016;

Original: Chen et al., 2017) and long-tail classiﬁcation (Liu et al., 2019; Ren et al., 2020).
Revised: Chen et al., 2017) and long-tail classiﬁcation (Liu et al., 2019; Ren et al., 2020).

Original:  M ETHOD
Revised:  M ETHOD

Original: In this section, we ﬁrst formally deﬁne the generalized zero-shot learning problem and then deﬁne a mathematical framework to summarize the core training dynamics of mainstream generative GZSL approaches.
Revised: In this section, we ﬁrst formally deﬁne the generalized zero-shot learning problem and then deﬁne a mathematical framework to summarize the core training dynamics of mainstream generative GZSL approaches.

Original: We then express our approach in the context of this mathematical framework.
Revised: We then express our approach in the context of this mathematical framework.

Original: 3. P ROBLEM DEFINITION
Revised: 3. P ROBLEM DEFINITION

Original: In zero-shot learning, the goal is to learn a classiﬁcation model that can recognize the test instances of unseen classes Y u , which has no training examples, based on the model learned over the training examples provided for the disjoint set of seen classes Y s .
Revised: In zero-shot learning, the goal is to learn a classiﬁcation model that can recognize the test instances of unseen classes Y u , which has no training examples, based on the model learned over the training examples provided for the disjoint set of seen classes Y s .

Original: We refer to the class-limited training set by D tr , which consists of sample and class label pairs ( x ∈ X , y ∈ Y s ) .
Revised: We refer to the class-limited training set by D tr , which consists of sample and class label pairs ( x ∈ X , y ∈ Y s ) .

Original: In our work, we focus on
Revised: In our work, we focus on

Original: ZSL models where X is the space of image representations extracted using a pre-trained ConvNet.
Revised: ZSL models where X is the space of image representations extracted using a pre-trained ConvNet.

Original: In generalized zero-shot learning, the goal is to build the classiﬁcation model using the training data set D tr , such that the model can recognize both seen and unseen class samples at test time.
Revised: In generalized zero-shot learning, the goal is to build the classiﬁcation model using the training data set D tr , such that the model can recognize both seen and unseen class samples at test time.

Original: For simplicity, we restrict our discussion to the GZSL problem setting below.
Revised: For simplicity, we restrict our discussion to the GZSL problem setting below.

Original: In order to enable the recognition of unseen class instances, it is necessary to have visually-relevant prior knowledge about classes so that classes can visually be related to each other.
Revised: In order to enable the recognition of unseen class instances, it is necessary to have visually-relevant prior knowledge about classes so that classes can visually be related to each other.

Original: Such prior knowledge is delivered by the mapping ψ : Y → A , where A expresses the prior knowledge space.
Revised: Such prior knowledge is delivered by the mapping ψ : Y → A , where A expresses the prior knowledge space.

Original: In most cases, the prior knowledge is provided as d ψ -dimensional vector-space embeddings of classes, obtained using visual attributes, taxonomies, class names combined with word embedding models, the textual descriptions of classes combined with language models, see e.g .
Revised: In most cases, the prior knowledge is provided as d ψ -dimensional vector-space embeddings of classes, obtained using visual attributes, taxonomies, class names combined with word embedding models, the textual descriptions of classes combined with language models, see e.g .

Original: Akata et al.
Revised: Akata et al.

Original: Following the common terminology, we refer to ψ as the class embedding function.
Revised: Following the common terminology, we refer to ψ as the class embedding function.

Original: Generative GZSL.
Revised: Generative GZSL.

Original: In our work, we focus on generative approaches to GZSL.
Revised: In our work, we focus on generative approaches to GZSL.

Original: The main goal is to learn a conditional generative model G : A × Z → X , which takes some class embedding a ∈ A and stochasticity-inducing noise input z , and yields a synthetic sample x ∈ X .
Revised: The main goal is to learn a conditional generative model G : A × Z → X , which takes some class embedding a ∈ A and stochasticity-inducing noise input z , and yields a synthetic sample x ∈ X .

Original: Once such a generative model is learned, synthetic training examples for all classes can simply be sampled from the G -induced distribution P G , and the ﬁnal classiﬁer over Y can be obtained using any standard supervised classiﬁcation model.
Revised: Once such a generative model is learned, synthetic training examples for all classes can simply be sampled from the G -induced distribution P G , and the ﬁnal classiﬁer over Y can be obtained using any standard supervised classiﬁcation model.

Original: We refer to the trainable parameters of the model G by θ G .
Revised: We refer to the trainable parameters of the model G by θ G .

Original: As summarized in Section 2, existing approaches vary greatly in terms of their generative model details.
Revised: As summarized in Section 2, existing approaches vary greatly in terms of their generative model details.

Original: For the purposes of our presentation, most of the GZSL works (if not all) can be summarized as the iterative minimization of some loss function that acts on the outputs of the generative model:
Revised: For the purposes of our presentation, most of the GZSL works (if not all) can be summarized as the iterative minimization of some loss function that acts on the outputs of the generative model:

Original: [Equation]
Revised: [Equation]

Original: where z x refers to the noise input associated with the training sample ( x, y ) and (cid:96) G is the generative model learning loss.
Revised: where z x refers to the noise input associated with the training sample ( x, y ) and (cid:96) G is the generative model learning loss.

Original: ( x, a ) ∼ D tr is shorthand notation for ( x, ψ ( y ))
Revised: ( x, a ) ∼ D tr is shorthand notation for ( x, ψ ( y ))

Original: ∼ D tr At each iteration the goal is to reduce L G approximated over a mini-batch of real samples and their class embeddings.
Revised: ∼ D tr At each iteration the goal is to reduce L G approximated over a mini-batch of real samples and their class embeddings.

Original: In our notation, we deliberately keep certain details simple.
Revised: In our notation, we deliberately keep certain details simple.

Original: Noticeably, z x greatly varies across models.
Revised: Noticeably, z x greatly varies across models.

Original: For example, in the case of a conditional GAN model, z x ∼ p
Revised: For example, in the case of a conditional GAN model, z x ∼ p

Original: ( z ) can simply be a sample from a simple prior distribution p ( z ) , e.g . as in (Xian et al., 2018b; Sariyildiz & Cinbis, 2019;
Revised: ( z ) can simply be a sample from a simple prior distribution p ( z ) , e.g . as in (Xian et al., 2018b; Sariyildiz & Cinbis, 2019;

Original: Elhoseiny & Elfeki, 2019).
Revised: Elhoseiny & Elfeki, 2019).

Original: In contrast, in variational training, z x is the latent code sampled from a variational posterior, i.e .
Revised: In contrast, in variational training, z x is the latent code sampled from a variational posterior, i.e .

Original: z x ∼ q ( z | x ) , where the variational posterior q ( z | x ) is given by a variational encoder trained jointly with G , e.g . as in (Narayan et al., 2020; Xian et al., 2019).
Revised: z x ∼ q ( z | x ) , where the variational posterior q ( z | x ) is given by a variational encoder trained jointly with G , e.g . as in (Narayan et al., 2020; Xian et al., 2019).

Original: Another important simpliﬁcation that we intentionally make in Eq.
Revised: Another important simpliﬁcation that we intentionally make in Eq.

Original: (1) is the fact that we deﬁne the generative model learning loss (cid:96) G as a function of generator output and class embedding, to emphasize its sample-realisticity and class-relevance estimation goals.
Revised: (1) is the fact that we deﬁne the generative model learning loss (cid:96) G as a function of generator output and class embedding, to emphasize its sample-realisticity and class-relevance estimation goals.

Original: However, the exact domain of (cid:96) G heavily depends on its details, which typically consists of multiple terms and/or (adversarially) trained models.
Revised: However, the exact domain of (cid:96) G heavily depends on its details, which typically consists of multiple terms and/or (adversarially) trained models.

Original: In most of the state-of-the-art approaches, this term is a combination of VAE reconstruction loss (Xian et al., 2019), conditional or unconditional adversarial discriminator network (Xian et al., 2018b; Narayan et al., 2020; Sariyildiz & Cinbis, 2019; Xian et al.,
Revised: In most of the state-of-the-art approaches, this term is a combination of VAE reconstruction loss (Xian et al., 2019), conditional or unconditional adversarial discriminator network (Xian et al., 2018b; Narayan et al., 2020; Sariyildiz & Cinbis, 2019; Xian et al.,

Original: 2019), a sample-to-class classiﬁer for measuring class relevance (Xian et al., 2018b) and sample-toembedding mappings (Narayan et al., 2020).
Revised: 2019), a sample-to-class classiﬁer for measuring class relevance (Xian et al., 2018b) and sample-toembedding mappings (Narayan et al., 2020).

Original: The loss L G may also incorporate additional regularization terms, such as (cid:96) 2 regularization or a gradient penalty term (Sariyildiz & Cinbis, 2019).
Revised: The loss L G may also incorporate additional regularization terms, such as (cid:96) 2 regularization or a gradient penalty term (Sariyildiz & Cinbis, 2019).

Original: In the following, we explain our sample probing approach as a loss term that can, in principle, be used in conjunction with virtually any of the mainstream generative zero-shot learning formulations.
Revised: In the following, we explain our sample probing approach as a loss term that can, in principle, be used in conjunction with virtually any of the mainstream generative zero-shot learning formulations.

Original: 3. S AMPLE PROBING AS GENERATIVE MODEL GUIDANCE
Revised: 3. S AMPLE PROBING AS GENERATIVE MODEL GUIDANCE

Original: The problem that we aim to address is enforcement of G to learn to produce samples maximally beneﬁcial for zero-shot model training purposes.
Revised: The problem that we aim to address is enforcement of G to learn to produce samples maximally beneﬁcial for zero-shot model training purposes.

Original: We approach this problem through a learning to generate training samples perspective, where we aim to monitor the quality of the generative model through the synthetic class samples it provides.
Revised: We approach this problem through a learning to generate training samples perspective, where we aim to monitor the quality of the generative model through the synthetic class samples it provides.

Original: In construction of our approach, at each training iteration t , we ﬁrst randomly select a subset of
Revised: In construction of our approach, at each training iteration t , we ﬁrst randomly select a subset of

Original: Y t pb-tr ⊂ Y of seen classes.
Revised: Y t pb-tr ⊂ Y of seen classes.

Original: We refer to these classes as probe-train classes.
Revised: We refer to these classes as probe-train classes.

Original: This subset deﬁnes the set of classes that are used for training the iteration-speciﬁc probing model over the synthetic samples.
Revised: This subset deﬁnes the set of classes that are used for training the iteration-speciﬁc probing model over the synthetic samples.

Original: More speciﬁcally, we ﬁrst take samples from the model G with the parameters θ tG for these classes, and fully train a temporary ZSL model over them using regularized loss minimization:
Revised: More speciﬁcally, we ﬁrst take samples from the model G with the parameters θ tG for these classes, and fully train a temporary ZSL model over them using regularized loss minimization:

Original: [Equation]
Revised: [Equation]

Original: where f pb is the scoring function of the temporary probing model parameterized by Γ and (cid:96) pb is its training loss.
Revised: where f pb is the scoring function of the temporary probing model parameterized by Γ and (cid:96) pb is its training loss.

Original: A t pb-tr is the set of class embeddings of classes in Y t pb-tr .
Revised: A t pb-tr is the set of class embeddings of classes in Y t pb-tr .

Original: Regularization term over Γ is not shown explicitly for brevity.
Revised: Regularization term over Γ is not shown explicitly for brevity.

Original: The result of Eq.
Revised: The result of Eq.

Original: (2), gives us a purely synthetic sample driven model Γ t , which we leverage as a way to estimate the success of the generator in synthesizing training examples.
Revised: (2), gives us a purely synthetic sample driven model Γ t , which we leverage as a way to estimate the success of the generator in synthesizing training examples.

Original: For this purpose, we sample real examples from the training set D tr as validation examples for the probe model.
Revised: For this purpose, we sample real examples from the training set D tr as validation examples for the probe model.

Original: Since we use a (G)ZSL model as the probing model, we can evaluate the model on examples of the classes not used for training the model.
Revised: Since we use a (G)ZSL model as the probing model, we can evaluate the model on examples of the classes not used for training the model.

Original: Therefore, we sample these probe-validation examples from the remaining classes Y pb-val =
Revised: Therefore, we sample these probe-validation examples from the remaining classes Y pb-val =

Original: Y s \ Y t pb-tr , i.e . the classes with real training examples but unused for probe model training, and use softmax cross-entropy loss over these samples as the probing loss:
Revised: Y s \ Y t pb-tr , i.e . the classes with real training examples but unused for probe model training, and use softmax cross-entropy loss over these samples as the probing loss:

Original: [Equation]
Revised: [Equation]

Original: where D pb-val ⊂ D tr is the data subset of classes Y pb-val .
Revised: where D pb-val ⊂ D tr is the data subset of classes Y pb-val .

Original: p ( y | x ; Γ t ) is the target class likelihood obtained by applying softmax to f pb ( x, ψ ( y ); Γ t ) scores over the set of target class set.
Revised: p ( y | x ; Γ t ) is the target class likelihood obtained by applying softmax to f pb ( x, ψ ( y ); Γ t ) scores over the set of target class set.

Original: Here, as target class set, one can use only the classes in Y pb-val (ZSL probing) or those in both Y pb-tr and
Revised: Here, as target class set, one can use only the classes in Y pb-val (ZSL probing) or those in both Y pb-tr and

Original: Y pb-val (GZSL probing).
Revised: Y pb-val (GZSL probing).

Original: We treat this decision as a hyper-parameter and tune on the validation set.
Revised: We treat this decision as a hyper-parameter and tune on the validation set.

Original: We use a weighted combination of L pb and L G , as our ﬁnal loss function.
Revised: We use a weighted combination of L pb and L G , as our ﬁnal loss function.

Original: Therefore, the gradients ∇ θ G L pb act effectively as the training signal for guiding G towards yielding training examples that results in (G)ZSL probing models with minimal empirical loss.
Revised: Therefore, the gradients ∇ θ G L pb act effectively as the training signal for guiding G towards yielding training examples that results in (G)ZSL probing models with minimal empirical loss.

Original: 3. C LOSED - FORM PROBE MODEL
Revised: 3. C LOSED - FORM PROBE MODEL

Original: A critical part of construction is the need for a probing model where minimization of Eq.
Revised: A critical part of construction is the need for a probing model where minimization of Eq.

Original: (2) is both efﬁcient and differentiable, so that the solver itself can be a part of the compute graph.
Revised: (2) is both efﬁcient and differentiable, so that the solver itself can be a part of the compute graph.

Original: Probing models that require iterative gradient descent based optimization are unlikely to be suitable as one would need to make a large number of probing model updates for each single G update step, which is both inefﬁcient and prone to gradient vanishing problems.
Revised: Probing models that require iterative gradient descent based optimization are unlikely to be suitable as one would need to make a large number of probing model updates for each single G update step, which is both inefﬁcient and prone to gradient vanishing problems.

Original: We address this problem through the use of a ZSL model that can be efﬁciently ﬁt using a closed-form solution.
Revised: We address this problem through the use of a ZSL model that can be efﬁciently ﬁt using a closed-form solution.

Original: For this purpose, we opt to use the ESZSL (Romera-Paredes & Torr, 2015) as the main closed-form probe model in our experiments.
Revised: For this purpose, we opt to use the ESZSL (Romera-Paredes & Torr, 2015) as the main closed-form probe model in our experiments.

Original: The model is formalized by the following minimization problem:
Revised: The model is formalized by the following minimization problem:

Original: [Equation]
Revised: [Equation]

Original: where X ∈ R d x × m and A ∈ R d ψ × k represent the feature and class embeddings corresponding to m input training examples and k classes, Y is the { 0 , 1 } m × k matrix of groundtruth labels, Γ ∈ R d x × d ψ
Revised: where X ∈ R d x × m and A ∈ R d ψ × k represent the feature and class embeddings corresponding to m input training examples and k classes, Y is the { 0 , 1 } m × k matrix of groundtruth labels, Γ ∈ R d x × d ψ

Original: [Figure]
Revised: [Figure]

Original: is the compatibility model matrix, and Ω (Γ) is the regularization function, deﬁned as:
Revised: is the compatibility model matrix, and Ω (Γ) is the regularization function, deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where λ x , λ a , λ n correspond to term weights.
Revised: where λ x , λ a , λ n correspond to term weights.

Original: When the regularization term weights are set such that λ n = λ x λ a , the optimal solution to Eq.
Revised: When the regularization term weights are set such that λ n = λ x λ a , the optimal solution to Eq.

Original: (4) can be computed in a closed-form:
Revised: (4) can be computed in a closed-form:

Original: [Equation]
Revised: [Equation]

Original: This approach was originally proposed as a standalone label-embedding based ZSL model in
Revised: This approach was originally proposed as a standalone label-embedding based ZSL model in

Original: Romera-Paredes & Torr (2015), with the practical advantage of having an efﬁcient solver.
Revised: Romera-Paredes & Torr (2015), with the practical advantage of having an efﬁcient solver.

Original: Here, we re-purpose this approach as a probing model in our framework, where the fact that the model is solvable in closed-form is critically important, enabling the idea of end-to-end sample probing.
Revised: Here, we re-purpose this approach as a probing model in our framework, where the fact that the model is solvable in closed-form is critically important, enabling the idea of end-to-end sample probing.

Original: For this purpose, we utilize the solver given by Eq.
Revised: For this purpose, we utilize the solver given by Eq.

Original: (6) as the implementation of Eq.
Revised: (6) as the implementation of Eq.

Original: (2), which takes a set of synthetic training samples and estimates the corresponding probing model parameters.
Revised: (2), which takes a set of synthetic training samples and estimates the corresponding probing model parameters.

Original: Alternative probe models.
Revised: Alternative probe models.

Original: While we utilize ESZSL in our main experiments, we demonstrate the possibility of using the proposed approach with different probe models using two additional alternatives.
Revised: While we utilize ESZSL in our main experiments, we demonstrate the possibility of using the proposed approach with different probe models using two additional alternatives.

Original: The ﬁrst one, which we call Vis2Sem , is the regression model from visual features to their corresponding class embeddings, deﬁned as follows (using the same notation as in ESZSL):
Revised: The ﬁrst one, which we call Vis2Sem , is the regression model from visual features to their corresponding class embeddings, deﬁned as follows (using the same notation as in ESZSL):

Original: [Equation]
Revised: [Equation]

Original: A discussion of the Vis2Sem model can be found in Kodirov et al.
Revised: A discussion of the Vis2Sem model can be found in Kodirov et al.

Original: The second one, which we call Sem2Vis , is the class embeddings to visual features regression model of Shigeto et al.
Revised: The second one, which we call Sem2Vis , is the class embeddings to visual features regression model of Shigeto et al.

Original: [Equation]
Revised: [Equation]

Original: Both models, just like ESZSL, are originally deﬁned as non-generative ZSL models, and we repurpose them to deﬁne our data-dependent generative model training losses.
Revised: Both models, just like ESZSL, are originally deﬁned as non-generative ZSL models, and we repurpose them to deﬁne our data-dependent generative model training losses.

Original: Unlike the bi-linear compatibility model of ESZSL, however, these models rely on distance based classiﬁcation, and do not directly yield class probability estimates.
Revised: Unlike the bi-linear compatibility model of ESZSL, however, these models rely on distance based classiﬁcation, and do not directly yield class probability estimates.

Original: While one can still obtain a probability distribution over classes, e.g . by applying softmax to negative (cid:96) 2 distances, for simplicity, we directly use the
Revised: While one can still obtain a probability distribution over classes, e.g . by applying softmax to negative (cid:96) 2 distances, for simplicity, we directly use the

Original: Sem2Vis and Vis2Sem based distance predictions between the visual features of probe-validation samples and their corresponding class embeddings to compute L pb as a replacement of Eq.
Revised: Sem2Vis and Vis2Sem based distance predictions between the visual features of probe-validation samples and their corresponding class embeddings to compute L pb as a replacement of Eq.

Original: Summary.
Revised: Summary.

Original: A summary of the ﬁnal approach from a compute graph point of view, is given in
Revised: A summary of the ﬁnal approach from a compute graph point of view, is given in

Original: Figure 2.
Revised: Figure 2.

Original: The proposed approach aims to realize the goal of learning to generate good training samples by evaluating the synthesis quality through the lens of a closed-form trainable probe model, the prediction loss of which is used as a loss for the G updates.
Revised: The proposed approach aims to realize the goal of learning to generate good training samples by evaluating the synthesis quality through the lens of a closed-form trainable probe model, the prediction loss of which is used as a loss for the G updates.

Original: Therefore, G is expected to be progressively guided towards producing realistic, relevant and informative samples, through the reinforcement of which may vary depending on the inherent nature of the chosen probe model.
Revised: Therefore, G is expected to be progressively guided towards producing realistic, relevant and informative samples, through the reinforcement of which may vary depending on the inherent nature of the chosen probe model.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Datasets.
Revised: Datasets.

Original: We use the four mainstream GZSL benchmark datasets: Caltech-UCSD-Birds (CUB)
Revised: We use the four mainstream GZSL benchmark datasets: Caltech-UCSD-Birds (CUB)

Original: (Wah et al., 2011), SUN Attribute (SUN) (Patterson & Hays, 2012), Animals with Attributes(AWA2, more simply AWA) (Xian et al., 2018a) and Oxford Flowers (FLO) (Nilsback & Zisserman, 2008).
Revised: (Wah et al., 2011), SUN Attribute (SUN) (Patterson & Hays, 2012), Animals with Attributes(AWA2, more simply AWA) (Xian et al., 2018a) and Oxford Flowers (FLO) (Nilsback & Zisserman, 2008).

Original: CUB is a medium scale ﬁne-grained dataset consisting of 200 bird species classes, 11. images and 312 dimensional attribute annotations.
Revised: CUB is a medium scale ﬁne-grained dataset consisting of 200 bird species classes, 11. images and 312 dimensional attribute annotations.

Original: SUN is a ﬁne-grained dataset with 14.340 images and 102 dimensional attributes.
Revised: SUN is a ﬁne-grained dataset with 14.340 images and 102 dimensional attributes.

Original: AWA is a coarse-grained dataset with 30.475 images of 50 animal classes with 85-dimensional attributes.
Revised: AWA is a coarse-grained dataset with 30.475 images of 50 animal classes with 85-dimensional attributes.

Original: FLO is a medium-scale ﬁne-grained dataset with 102 classes and 1024-dimensional attributes.
Revised: FLO is a medium-scale ﬁne-grained dataset with 102 classes and 1024-dimensional attributes.

Original: Following the state-of-the-art, we use the class embeddings and the version-2 splits deﬁned by Xian et al.
Revised: Following the state-of-the-art, we use the class embeddings and the version-2 splits deﬁned by Xian et al.

Original: (2018a).
Revised: (2018a).

Original: We evaluate the results in terms of GZSL-u, GZSL-s and h-score (H) values (Xian et al., 2018a).
Revised: We evaluate the results in terms of GZSL-u, GZSL-s and h-score (H) values (Xian et al., 2018a).

Original: The h-score, i.e .
Revised: The h-score, i.e .

Original: the harmonic mean of GZSL-u and
Revised: the harmonic mean of GZSL-u and

Original: GZSL-s scores, aims to measure how well a model recognizes seen and unseen classes collectively.
Revised: GZSL-s scores, aims to measure how well a model recognizes seen and unseen classes collectively.

Original: In our experiments, we use the image features extracted from ResNet-101 backbone pretrained on
Revised: In our experiments, we use the image features extracted from ResNet-101 backbone pretrained on

Original: ImageNet 1K.
Revised: ImageNet 1K.

Original: In the experiments based on ﬁne-tuned representations, we use the backbone ﬁnetuned with the training images of seen classes, as in Xian et al.
Revised: In the experiments based on ﬁne-tuned representations, we use the backbone ﬁnetuned with the training images of seen classes, as in Xian et al.

Original: ; Narayan et al.
Revised: ; Narayan et al.

Original: Hyper-parameter tuning policy.
Revised: Hyper-parameter tuning policy.

Original: In our preliminary studies, we observe that ﬁnal GZSL performance, especially in terms of h-score, of most models strongly depends on the selection of the hyperparameters.
Revised: In our preliminary studies, we observe that ﬁnal GZSL performance, especially in terms of h-score, of most models strongly depends on the selection of the hyperparameters.

Original: We also observe that there is no widely-accepted policy on how the hyper-parameters of
Revised: We also observe that there is no widely-accepted policy on how the hyper-parameters of

Original: GZSL models shall be tuned.
Revised: GZSL models shall be tuned.

Original: It is a rather common practice in the GZSL literature to either directly report the hyper-parameters used in experiments without an explanation on the tuning strategy or simply refer to tuning on validation set , which we ﬁnd a vaguely-deﬁned policy as (i) Xian et al.
Revised: It is a rather common practice in the GZSL literature to either directly report the hyper-parameters used in experiments without an explanation on the tuning strategy or simply refer to tuning on validation set , which we ﬁnd a vaguely-deﬁned policy as (i) Xian et al.

Original: (2018a) deﬁnes an unseen-class only validation split, which does not allow monitoring the h-score, and, (ii) it is unclear which metric one should use for GZSL model selection purposes.
Revised: (2018a) deﬁnes an unseen-class only validation split, which does not allow monitoring the h-score, and, (ii) it is unclear which metric one should use for GZSL model selection purposes.

Original: Therefore, to obtain comparable results within our experiments, we use the following policy to tune the hyper-parameters of our approach and our baselines: we ﬁrst leave-out 20% of train class samples as val-seen samples.
Revised: Therefore, to obtain comparable results within our experiments, we use the following policy to tune the hyper-parameters of our approach and our baselines: we ﬁrst leave-out 20% of train class samples as val-seen samples.

Original: We periodically train a supervised classiﬁer by taking synthetic samples from the generative model, and evaluate it on the validation set, consisting of the aforementioned val-seen samples plus the val-unseen samples with respect to the benchmark splits.
Revised: We periodically train a supervised classiﬁer by taking synthetic samples from the generative model, and evaluate it on the validation set, consisting of the aforementioned val-seen samples plus the val-unseen samples with respect to the benchmark splits.

Original: We choose the hyper-parameter combination with the highest h-score on the validation set.
Revised: We choose the hyper-parameter combination with the highest h-score on the validation set.

Original: We obtain ﬁnal models by re-training the generative model from scratch on the training and validation examples combined.
Revised: We obtain ﬁnal models by re-training the generative model from scratch on the training and validation examples combined.

Original: 4. M AIN RESULTS
Revised: 4. M AIN RESULTS

Original: In this subsection, we discuss our main experimental results.
Revised: In this subsection, we discuss our main experimental results.

Original: As we observe that the results are heavily inﬂuenced from hyper-parameter tuning strategy, our main goal throughout our experiments is the validation of the proposed sample probing idea by integrating it into strong generative GZSL baselines, and then comparing results using the same tuning methodology.
Revised: As we observe that the results are heavily inﬂuenced from hyper-parameter tuning strategy, our main goal throughout our experiments is the validation of the proposed sample probing idea by integrating it into strong generative GZSL baselines, and then comparing results using the same tuning methodology.

Original: Using this principle, we present two main types of analysis: (i) the evaluation of the proposed approach using ESZSL as the probe model in combination with a number of generative GZSL models, and (ii) the evaluation of alternative closed-form probe models within our framework.
Revised: Using this principle, we present two main types of analysis: (i) the evaluation of the proposed approach using ESZSL as the probe model in combination with a number of generative GZSL models, and (ii) the evaluation of alternative closed-form probe models within our framework.

Original: Generative GZSL models with sample probing.
Revised: Generative GZSL models with sample probing.

Original: To evaluate the sample probing approach as a general technique to improve generative model training, we integrate it to four recent generative GZSL approaches: conditional Wasserstein GAN (cWGAN) (Arjovsky et al., 2017; Miyato &
Revised: To evaluate the sample probing approach as a general technique to improve generative model training, we integrate it to four recent generative GZSL approaches: conditional Wasserstein GAN (cWGAN) (Arjovsky et al., 2017; Miyato &

Original: Koyama, 2018), LisGAN (Li et al., 2019), TF-VAEGAN (Narayan et al., 2020) and FREE (Chen et al., 2021).
Revised: Koyama, 2018), LisGAN (Li et al., 2019), TF-VAEGAN (Narayan et al., 2020) and FREE (Chen et al., 2021).

Original: We additionally report results for the variant of TF-VAEGAN with the ﬁne-tuned representations ( TF-VAEGAN-FT ), as it is the only one among them with reported ﬁne-tuning results.
Revised: We additionally report results for the variant of TF-VAEGAN with the ﬁne-tuned representations ( TF-VAEGAN-FT ), as it is the only one among them with reported ﬁne-tuning results.

Original: For the cWGAN, we follow the implementation details described in Sariyildiz & Cinbis (2019), and tune hyper-parameters using our policy.
Revised: For the cWGAN, we follow the implementation details described in Sariyildiz & Cinbis (2019), and tune hyper-parameters using our policy.

Original: For LisGAN, TF-VAEGAN and FREE models, we use the ofﬁcial repositories shared by their respective authors.
Revised: For LisGAN, TF-VAEGAN and FREE models, we use the ofﬁcial repositories shared by their respective authors.

Original: We use the version of TF-VAEGAN without feedback loop (Narayan et al., 2020), for simplicity, as the model yields excellent performance with and without feedback loops.
Revised: We use the version of TF-VAEGAN without feedback loop (Narayan et al., 2020), for simplicity, as the model yields excellent performance with and without feedback loop.

Original: In all models (except cWGAN), we only re-tune the number of training iterations of the original models using our hyper-parameter tuning policy, to make the re-
Revised: In all models (except cWGAN), we only re-tune the number of training iterations of the original models using our hyper-parameter tuning policy, to make the re-

Original: [Table] [Table]
Revised: [Table] [Table]

Original: sults comparable, as it is unclear how the original values were obtained.
Revised: sults comparable, as it is unclear how the original values were obtained.

Original: 2 We keep all remaining hyper-parameters unchanged to remain as close as possible to the original implementations
Revised: 2 We keep all remaining hyper-parameters unchanged to remain as close as possible to the original implementations

Original: [Table]
Revised: [Table]

Original: The results over the four benchmark datasets are presented in Table 1.
Revised: The results over the four benchmark datasets are presented in Table 1.

Original: In terms of the h-scores, we observe improvements in 17 out of 19 cases, at varying degrees (up to 4 .
Revised: In terms of the h-scores, we observe improvements in 17 out of 19 cases, at varying degrees (up to 4 .

Original: 6 points ).
Revised: 6 points ).

Original: Only in two cases we observe a slight degradation (maximum of 0 .
Revised: Only in two cases we observe a slight degradation (maximum of 0 .

Original: 2 points) in performance.
Revised: 2 points) in performance.

Original: Overall, these improvements over already strong and state-of-the-art (or competitive) baselines validate the effectiveness of the proposed sample probing approach, suggesting that it is a valid method towards end-to-end learning of generative GZSL models directly optimized for synthetic train data generation purposes
Revised: Overall, these improvements over already strong and state-of-the-art (or competitive) baselines validate the effectiveness of the proposed sample probing approach, suggesting that it is a valid method towards end-to-end learning of generative GZSL models directly optimized for synthetic train data generation purposes

Original: [Table]
Revised: [Table]

Original: Sample probing with alternative closed-form models.
Revised: Sample probing with alternative closed-form models.

Original: We now evaluate our approach with different closed-form probe models, speciﬁcally ESZSL, Sem2Vis (Shigeto et al., 2015) and
Revised: We now evaluate our approach with different closed-form probe models, speciﬁcally ESZSL, Sem2Vis (Shigeto et al., 2015) and

Original: Vis2Sem (Kodirov et al., 2017), as described in Section 3.3.
Revised: Vis2Sem (Kodirov et al., 2017), as described in Section 3.3.

Original: For these experiments, we use the
Revised: For these experiments, we use the

Original: TF-VAEGAN as the base generative model.
Revised: TF-VAEGAN as the base generative model.

Original: The results with four conﬁgurations over four benchmark datasets are presented in Table 2.
Revised: The results with four conﬁgurations over four benchmark datasets are presented in Table 2.

Original: First of all, in terms of h-scores, we observe considerable performance variations across the probe models and datasets: Sem2Vis performs the best on CUB ( +0 . 9 over the baseline), ESZSL provides a clear gain on FLO ( +4 . 6 ) and a relative improvement on AWA ( +0 . 3 ), and Vis2Sem improves the most on SUN ( +1 . 8 ).
Revised: First of all, in terms of h-scores, we observe considerable performance variations across the probe models and datasets: Sem2Vis performs the best on CUB ( +0 . 9 over the baseline), ESZSL provides a clear gain on FLO ( +4 . 6 ) and a relative improvement on AWA ( +0 . 3 ), and Vis2Sem improves the most on SUN ( +1 . 8 ).

Original: These results suggest that sample probe alternatives have their own advantages and disadvantages, and their performances can be data dependent.
Revised: These results suggest that sample probe alternatives have their own advantages and disadvantages, and their performances can be data dependent.

Original: Therefore, in a practical application, probe model options can be incorporated into the model selection process.
Revised: Therefore, in a practical application, probe model options can be incorporated into the model selection process.

Original: More in-depth understanding of closed-form model characteristics for sample probing purposes, and the formulation and evaluation of other probe models can be important future work directions.
Revised: More in-depth understanding of closed-form model characteristics for sample probing purposes, and the formulation and evaluation of other probe models can be important future work directions.

Original: Overall, the fact that we observe equivalent ( 2 ) or better ( 9 ) h-scores in 11 out of 12 sample probing experiments indicates the versatility of the approach in terms of compatibility with various closed-form probe models.
Revised: Overall, the fact that we observe equivalent ( 2 ) or better ( 9 ) h-scores in 11 out of 12 sample probing experiments indicates the versatility of the approach in terms of compatibility with various closed-form probe models.

Original: Comparison to other generative GZSL approaches.
Revised: Comparison to other generative GZSL approaches.

Original: Performance comparisons across independent experiment results can be misleading due to differences in formulation-agnostic implementation and model selection details.
Revised: Performance comparisons across independent experiment results can be misleading due to differences in formulation-agnostic implementation and model selection details.

Original: Nevertheless, an overall comparison can be found in Appendix A.
Revised: Nevertheless, an overall comparison can be found in Appendix A.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: A NALYSIS
Revised: A NALYSIS

Original: ZSL vs GZSL loss in sample probing.
Revised: ZSL vs GZSL loss in sample probing.

Original: In Table
Revised: In Table

Original: [Table]
Revised: [Table]

Original: we present a comparison of our approach, using TF-VAEGAN as the generative model and ESZSL as the probe model, when two different types of losses ( zsl-loss and gzsl-loss ) are used as L pb in Eq.
Revised: we present a comparison of our approach, using TF-VAEGAN as the generative model and ESZSL as the probe model, when two different types of losses ( zsl-loss and gzsl-loss ) are used as L pb in Eq.

Original: They differ from each other in terms of classes among which the real examples of probe-validation classes are classiﬁed, during the evaluation of sample probing ZSL model.
Revised: They differ from each other in terms of classes among which the real examples of probe-validation classes are classiﬁed, during the evaluation of sample probing ZSL model.

Original: zsl-loss and gzsl-loss indicate that the examples of probe-validation classes are classiﬁed among only probe-validation classes
Revised: zsl-loss and gzsl-loss indicate that the examples of probe-validation classes are classiﬁed among only probe-validation classes

Original: [Table]
Revised: [Table]

Original: and both probe-train and probe-validation classes, respectively.
Revised: and both probe-train and probe-validation classes, respectively.

Original: We observe that using either one during the evaluation of sample probing ZSL model, brings its own characteristic results (Table 3).
Revised: We observe that using either one during the evaluation of sample probing ZSL model, brings its own characteristic results (Table 3).

Original: On all datasets, using zsl-loss increases the seen accuracy while using gzsl-loss increases the unseen accuracy.
Revised: On all datasets, using zsl-loss increases the seen accuracy while using gzsl-loss increases the unseen accuracy.

Original: We choose among these two options using our same hyper-parameter tuning policy, on the validation set.
Revised: We choose among these two options using our same hyper-parameter tuning policy, on the validation set.

Original: [Figure]
Revised: [Figure]

Original: Effect of sample probing loss weight.
Revised: Effect of sample probing loss weight.

Original: Figure 3 shows the validation and test set h-score values as a function of  probing loss weight.
Revised: Figure 3 shows the validation and test set h-score values as a function of sample probing loss weight.

Original: In the test set results, we observe an overall increasing performance trend with larger loss weights, up to the weight 6 , highlighting the contribution of sample probing.
Revised: In the test set results, we observe an overall increasing performance trend with larger loss weights, up to the weight 6 , highlighting the contribution of sample probing.

Original: The optimal weight with respect to the validation and the test sets, however, differs.
Revised: The optimal weight with respect to the validation and the test sets, however, differs.

Original: This observation is an example for the difﬁculty of tuning the ZSL model based on validation set.
Revised: This observation is an example for the difﬁculty of tuning the ZSL model based on validation set.

Original: We set the loss weight to 5 following our hyper-parameter tuning policy, which yields almost 0 .
Revised: We set the loss weight to 5 following our hyper-parameter tuning policy, which yields almost 0 .

Original: 5 lower than the maximum testset score observed for this single hyper-parameter.
Revised: 5 lower than the maximum test-set score observed for this single hyper-parameter.

Original: Quantitative analysis of sample quality.
Revised: Quantitative analysis of sample quality.

Original: We quantitatively evaluate the sample quality using Fr´echet (Wasserstein-2) distance, which is also used in the FID metric for evaluating GAN models.
Revised: We quantitatively evaluate the sample quality using Fr´echet (Wasserstein-2) distance, which is also used in the FID metric for evaluating GAN models.

Original: In Table 4 we provide a comparison for TF-VAEGAN and our approach for respective mean per-class Fr´echet distances between real and synthetic samples (200 synthetic samples per class) of unseen classes on three datasets.
Revised: In Table 4, we provide a comparison for TF-VAEGAN and our approach for respective mean per-class Fr´echet distances between real and synthetic samples (200 synthetic samples per class) of unseen classes on three datasets.

Original: The results show that sample probing helps generator to generate more realistic samples compared to TF-VAEGAN.
Revised: The results show that sample probing helps generator to generate more realistic samples compared to TF-VAEGAN.

Original: We further investigate the qualitative analysis of the sample quality via t-SNE visualizations in Appendix B.
Revised: We further investigate the qualitative analysis of the sample quality via t-SNE visualizations in Appendix B.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We propose a principled GZSL approach, which makes use of closed-form ZSL models in generative model training to provide a sample-driven and end-to-end feedback to the generator.
Revised: We propose a principled GZSL approach, which makes use of closed-form ZSL models in generative model training to provide a sample-driven and end-to-end feedback to the generator.

Original: Extensive experiments over four benchmark datasets show that the proposed sample probing scheme consistently improves the GZSL results and the sample quality, can easily be integrated to existing generative
Revised: Extensive experiments over four benchmark datasets show that the proposed sample probing scheme consistently improves the GZSL results and the sample quality, can easily be integrated to existing generative

Original: GZSL approach and can be utilized with various closed-form probe models.
Revised: GZSL approach and can be utilized with various closed-form probe models.

Original: Reproducibility statement.
Revised: Reproducibility statement.

Original: We discuss the problem of model selection in GZSL, and the difﬁculty of making fair comparison across GZSL models in Section 4.
Revised: We discuss the problem of model selection in GZSL, and the difﬁculty of making fair comparison across GZSL models in Section 4.

Original: We believe that these are open problems in GZSL research.
Revised: We believe that these are open problems in GZSL research.

Original: We also clearly explain our hyper-parameter tuning policy, which we use in our experiments.
Revised: We also clearly explain our hyper-parameter tuning policy, which we use in our experiments.

Original: We provide our source code implementing the main model, as a supplementary material to provide reproducibility.
Revised: We provide our source code implementing the main model, as a supplementary material to provide reproducibility.

Original: You can ﬁnd
Revised: You can ﬁnd

Original: how we process the data in meta dset.py and util.py, the ESZSL implementation in eszsl.py, ﬁnal classiﬁer in classiﬁer.py, main training code in train tfvaegan inductive sample probing.py, and the models in model.py.
Revised: how we process the data in meta dset.py and util.py, the ESZSL implementation in eszsl.py, ﬁnal classiﬁer in classiﬁer.py, main training code in train tfvaegan inductive sample probing.py, and the models in model.py.

Original: README.txt includes a brief explanation on how to run an experiment.
Revised: README.txt includes a brief explanation on how to run an experiment.

Original: We are committed to releasing a complete repository upon acceptance
Revised: We are committed to releasing a complete repository upon acceptance

Original: A PPENDICES A C OMPARISON TO THE STATE - OF - THE - ART
Revised: A PPENDICES A C OMPARISON TO THE STATE - OF - THE - ART

Original: In Table we compare our results with the state-of-the-art generative approaches for GZSL.
Revised: In Table we compare our results with the state-of-the-art generative approaches for GZSL.

Original: During training, we select our best model based on the validation results and report test results on models that give best validation scores.
Revised: During training, we select our best model based on the validation results and report test results on models that give best validation scores.

Original: For consistency and to keep the baseline comparable to our results, we again report our own results for LisGAN, TF-VAEGAN (without feedback loop) and FREE using our hyper-parameter tuning policy, but do acknowledge that the original papers typically report higher results.
Revised: For consistency and to keep the baseline comparable to our results, we again report our own results for LisGAN, TF-VAEGAN (without feedback loop) and FREE using our hyper-parameter tuning policy, but do acknowledge that the original papers typically report higher results.

Original: The upper part of the table contains results with the original image representations, and the lower part contains those based on ﬁne-tuned representations.
Revised: The upper part of the table contains results with the original image representations, and the lower part contains those based on ﬁne-tuned representations.

Original: From the results without ﬁne-tuning, we observe that the proposed sample probing based generative model yields state-of-the-art h-scores in all CUB, FLO, SUN and AWA datasets.
Revised: From the results without ﬁne-tuning, we observe that the proposed sample probing based generative model yields state-of-the-art h-scores in all CUB, FLO, SUN and AWA datasets.

Original: We also observe competitive results in terms of individual unseen and seen class accuracy values.
Revised: We also observe competitive results in terms of individual unseen and seen class accuracy values.

Original: When compared against results using ﬁne-tuned representations, we again observe state-of-the-art h-scores on CUB and FLO datasets, with a close second on SUN.
Revised: When compared against results using ﬁne-tuned representations, we again observe state-of-the-art h-scores on CUB and FLO datasets, with a close second on SUN.

Original: On AWA, we observe that f-VAEGAN achieves the highest results with a signiﬁcant margin over our TF-VAEGAN based baseline, where the sample probing improves the baseline yet still achieves a score below that of f-VAEGAN.
Revised: On AWA, we observe that f-VAEGAN achieves the highest results with a signiﬁcant margin over our TF-VAEGAN based baseline, where the sample probing improves the baseline yet still achieves a score below that of f-VAEGAN.

Original: Overall, while it is hardly fair to compare models with signiﬁcant implementation details, these results suggest the overall competitiveness of the obtained data generating models with sample probing.
Revised: Overall, while it is hardly fair to compare models with signiﬁcant implementation details, these results suggest the overall competitiveness of the obtained data generating models with sample probing.

Original: B V ISUALIZATION OF UNSEEN DATA
Revised: B V ISUALIZATION OF UNSEEN DATA

Original: To provide additional qualitative insight into the improvements that can be gained using sample probing, we present t-SNE visualizations of synthetic class samples in Figure 4.
Revised: To provide additional qualitative insight into the improvements that can be gained using sample probing, we present t-SNE visualizations of synthetic class samples in Figure 4.

Original: In the ﬁgure, each plot corresponds to an unseen class on the FLO dataset, and the points correspond to the t-SNE
Revised: In the ﬁgure, each plot corresponds to an unseen class on the FLO dataset, and the points correspond to the t-SNE

Original: [Figure]
Revised: [Figure]

Original: embeddings of real samples ( × points), generated samples using TF-VAEGAN with sample probing ( (cid:32) points) and those using the baseline TF-VAEGAN model without sample probing ( (cid:78) points).
Revised: embeddings of real samples ( × points), generated samples using TF-VAEGAN with sample probing ( (cid:32) points) and those using the baseline TF-VAEGAN model without sample probing ( (cid:78) points).

Original: From the plots, we can observe that the generative model trained with sample probing tends to yield samples much more aligned with the corresponding true class distributions, compared to those of the baseline model.
Revised: From the plots, we can observe that the generative model trained with sample probing tends to yield samples much more aligned with the corresponding true class distributions, compared to those of the baseline model.

Original: Overall, these plots demonstrate how sample probing can improve the overall sample quality of a generative model, and possibly lead to superior recognition models when the generated samples are used for classiﬁer training.
Revised: Overall, these plots demonstrate how sample probing can improve the overall sample quality of a generative model, and possibly lead to superior recognition models when the generated samples are used for classiﬁer training.