Closed-form Sample Probing for Learning Generative Models in Zero-shot Learning
Authors: Samet Cetin
Orhun Buğra Baran
Ramazan Gokberk Cinbis
Venue: ICLR 2022 Poster (2022)
Keywords: zero-shot learning
generative zero-shot learning
generative models

Abstract: Generative model based approaches have led to significant advances in zero-shot learning (ZSL) over the past few years. These approaches typically aim to learn a conditional generator that synthesizes training samples of classes conditioned on class definitions. The final zero-shot learning model is then obtained by training a supervised classification model over the real and/or synthesized training samples of seen and unseen classes, combined. Therefore, naturally, the generative model needs to produce not only relevant samples, but also those that are sufficiently rich for classifier training purposes, which is handled by various heuristics in existing works. In this paper, we introduce a principled approach for training generative models {\em directly} for training data generation purposes. Our main observation is that the use of closed-form models opens doors to end-to-end training thanks to the differentiability of the solvers. In our approach, at each generative model update step, we fit a task-specific closed-form ZSL model from generated samples, and measure its loss on novel samples all within the compute graph, a procedure that we refer to as {\em sample probing}. In this manner, the generator receives feedback directly based on the value of its samples for model training purposes. Our experimental results show that the proposed sample probing approach improves the ZSL results even when integrated into state-of-the-art generative models.


Published as a conference paper at ICLR 2022
CLOSED-FORM SAMPLE PROBING FOR LEARNING GENERATIVE MODELS IN ZERO-SHOT LEARNING
Samet Cetin, Orhun Bugra Baran & Ramazan Gokberk Cinbis
Middle East Technical University
Department of Computer Engineering
Ankara, Turkey (cetin. samet, bugra baran, geinbis} edu. tr
ABSTRACT
Generative model based approaches have led significant advances in zero-shot learning (ZSL) over the past few years. These approaches typically aim to learn a conditional generator that synthesizes training samples of classes conditioned on class definitions. The final zero-shot learning model is then obtained by training a supervised classification model over the real and/or synthesized training samples of seen and unseen classes, combined. Therefore, naturally, the generative model needs to produce not only relevant samples, but also those that are sufficiently rich for classifier training purposes, which is handled by various heuristics in existing works. In this paper, we introduce a principled approach for training generative models directly for training data generation purposes. Our main observation is that the use of closed-form models opens doors to end-to-end training thanks to the differentiability of the solvers. In our approach, each generative model up- date step, we fit a task-specific closed-form ZSL model from generated samples, and measure its loss on novel samples all within the compute graph, a procedure that we refer to as sample probing. In this manner, the generator receives feed- back directly based on the value of its samples for model training purposes. Our experimental results show that the proposed sample probing approach improves the ZSL results even when integrated into state-of-the-art generative models.
INTRODUCTION
Zero-shot Learning (ZSL) has recently received great interest for being one of the promising paradigms towards building very large vocabulary (visual) understanding models with limited train- ing data. The problem of ZSL can be summarized the task of transferring information across classes such that the instances of unseen classes, with no training examples, can be recognized at test time, based on the training samples of seen classes. Similarly, the term Generalized Zero-Shot Learning (GZSL) (Chao et al. 2016 Xian et al. 2018a) is used to refer to practically more valu- able variant of ZSL where both seen and unseen classes may appear at time. GZSL brings in additional challenges since GZSL models need to produce confidence scores that are comparable across all classes.
Recent work shows that hallucinating unseen class samples through statistical generative models can be effective strategy, e.g. (Mishra et al. 2017 Xian et al., 2018b 2019; Sariyildiz Cin- bis 2019: Zhu et al.) 2018 [Schonfeld et al. 2019 Arora et al. [2018] ( Narayan et al. 2020). These approaches rely on generative models conditioned on class embeddings, obtained from auxiliary se- mantic knowledge, such as visual attributes (Xian et al. (2019), class name word embeddings (Schon-) feld et al. 2019), or textual descriptions (Zhu et al. 2018). The resulting synthetic examples, typi- used for training supervised classifier.
cally in combination with existing real examples,
In generative GZSL approaches, the quality of class-conditional samples is crucial for building accurate recognition models. is not traight-forwart to formally define the criteria of good training samples. Arguably, however, samples need to (i) realistic (e.g. free from unwanted artifacts), (ii) relevant (i.e. belong to the desired class distribution) and (iii) informative (i.e. contain examples

---

Published as a conference paper at ICLR 2022
Class embadiinge
Synthtic training samples
Conditional Generative Model
Closed-form solver
Real examples of probe-validationn classes
Bample probing loss
Clase (irtbbure))
probing ZSL model
Figure 1: Illustration of the proposed framework for the end-to-end sample probing of conditional generative models. At each training iteration. we take synthetic training examples for some subset of seen classes (probe-train classes) from the conditional generative models, train a closed-form solvable zero-shot learning model (sample probing 7SL. model) over them and evaluate it on the real examples of different subset of seen classes (probe-validatios classes). The resulting cross-entropy loss of the probing model is used as a loss term for the generative model update.
defining class boundaries) train an accurate classifier. Clearly, a primary factor affecting the quality of generated samples is the loss driving the conditional generative model training process.
In this work, we aim to address the problem of training data generating models via an end-to-end mechanism that we call sample probing D Our main goal is to directly evaluate the ability of generative model in synthesizing training examples. To this end, we observe that we can leverage classification models with closed-fore foreddffoommmoo solvers to efficiently measure the quality of training samples, in an end-to-end manner. More specifically, we formulate a simple yet powerful meta-learning ap- proach: at each training iteration, (i) take set of samples from the generative model for randomly selected subset of classes, (ii) train zero-shot probing model using only the synthesized samples, and (iii) evaluate the probing model on real samples from the training set. We then use the loss value as an end-to-end training signal for updating the generative model parameters. Since we specifically focus on probing models with exact closed-form solutions, the probing model optimization simpli- fies into a differentiable linear algebraic expression and takes part as a differentiable unit within the compute graph. A graphical summary of the proposed training scheme is given in Figure
In the rest of the paper, we first provide an overview of related work, and explain our approach in detail. We then present a thorough experimental evaluation on GZSL benchmarks in Section [ where the results show that sample probing yields improvements when introduced into state-of-the- art baselines. We conclude with final remarks in Section
RELATED WORK
The generalized zero-shot learning problem has been introduced by Xian al. (2017) and Chao ct al. (2016). The extensive study in Xian et al. (2018a) has shown that the success of methods can greatly vary across zero-shot and generalized zero-shot learning problems. The additional challenge in the generalized case is the need for deciding whether an input test sample belongs to a seen or unseen class. Discriminative training of ZSL models, such as those based on bilinear compatibility functions (Weston et al. 2011 Frome et al., 2013). are likely to yield higher confidence scores for seen classes. To alleviate this problem, a few recent works have proposed ways to regularize discriminative models towards producing comparable confidence scores across all classes and avoid over-fitting e.g. Diang al. (2019); Liu et al. (2018): Chou et al. (2021b).
Generative approaches to zero-shot learning naturally address the confidence score calibration prob- lem. However, in general, generative modeling corresponds to a more sophisticated task than learn- ing discriminant functions only (Vapnik & Vapnik, 1998). In this context, the problem is further complicated by the need of predicting zero-shot class distributions. To tackle this challenging task, variety of techniques have been proposed (Mishra al. 2017 Xian et al. 2018b 2019 Sariyildiz
Our use of the sample probing term is not closely related to the natural language model analysis technique Peters Peters 2018 Hewitt Liang 2019).
known as probing [Helinkov al. [2077]
2

---

Published as a conference paper at ICLR 2022
& Cinbis 2019: Zhu et al. 2018: Schonfeld et al. 2019 Arora al. 2018 Elhoseiny & Elfeki Narayan et al. 20200) by adapting generative models. such as VAES Kingma & Welling
2019
2014 Rezende al. 2014) and GANs (Goodfellow et al. 2014). To enforce class conditioning, the
state-of-the-art approaches use a mixture of heuristics: Xian et al. (2018b) uses the loss of pre- trained classifier over the generated samples during training. Narayan al. (2020) additionally uses the loss of a sample-to-attribut regressor, in combination with a feedback mechanism motivated from feedback networks (Zamir al., 2017). (Sariyildiz & Cinbis (2019) uses projection discrimi- nator (Miyato& & Koyama| (0188) and gradient matching loss as gradient based similarity measure for comparing real versus synthetic data distributions. None of these generative ZSL approaches, however, directly measure the value of the generated samples for training classification models. The main difficulty lies the need for back-propage pack-propagating over long compute chains, which is both ineffi cient and prone to gradient vanishing problems. To the best of our knowledge, we introduce the first end-to-end solution to this problem by the idea of using probing models with closed-for form solvers to monitor the sample quality for model training purposes.
Our approach is effectively a meta-learning (Snell et al., 2017) scheme. Meta-learning is a promi- nent idea in few-shot learning, where the goal is learning build predictive models from limited number of samples. The main motivation is the idea that purppose classification models may behave suboptimally when only a few training samples are provided. A variety of meta-learning driven few-shot learning models have been proposed, such as meta-models that transform few sam- ples to classifiers (Gidaris & Komodakis 2018 [nell et al. (017), set-to-set transformations for improving sample representations (Ye et al. 2020 Bronskill et al. 2020), fast adaptation networks for few examples (Finn et al., [017] Rusu et al. 2019 Nichol et al. 2018). In contrast to such mainstream learning to classify and learning to adapt approaches, we aim to address the problem of learning to generate training examples for GZSL.
There are only few and recent studies that aim to tackle generative ZSL via meta-learning prin- ciples. Verma al. (2020) 202T) embrace the learning-to-adapt framework of MAMI. (Finn et al.) 2017]. which originally aims to learn the optimal initialization for few-shot adaptation. The MAMI. steps are incorporated by iteratively applying a single-step update to the generative model using the generative model (VAE/GAN) loss terms, and then broppproggg over the re-computed gen- erative loss terms on new samples from a disjoint subset of classes using the single-step updated model. Our approach differs fundamentally, as propose to the discriminative guidance of ZSL models fully-traine directly from generated sample batches. In another work, Yu et al. (2020) proposes an episodic training-like approach with periodically altered training sets and losses during training, to learn non-stochastic mappings between the class-embeddings and class-centers. Chou al. (2021a). similarly inspired by meta-learning and mixup regularization (Zhang et al. 2018), proposes to train a novel discriminative ZSL model over episodically defined virtual training classes obtained by linearly mixing classes. Neither of these approaches learn sample generating models, therefore, they have no direct relation to our work focusing on the problem of measuring the sample quality for ZSL model training purposes, with end-to-end discriminative guidance.
The use of recognition models with closed-form solvers has attracted prior interest in various con- texts. Notably, [Romemaa Paredes & Torr (015) proposes the ESZSL model as a simple and effective 7SL. formulation. We leverage the closed-form solvability of the ESZSI model as part of our ap- proach. Bertinetto et al. (2019) utilizes ridge-regression based task-specific few-shot learners within discriminative meta- meta-learning framework. In similar fashion, Bhatt et al. (2020); Liu et al. (2020) tackle the problem of video object segmentation (VOS) and use ridge regression based task-specific segmentation models within meta-learnig framework. None of these approaches aim nition models with closed form solvers to form guidance for generative model training.
recog-
Another related research topic is generative few-shot learning (FSL), where the goal is learning to synthesize additional in-class samples from a few examples, e.g. Hariharan & Girshick Girshick((016)) Wang
al. 2018); Gao al. (2018): Schwwrtz et al. (2018); Lazarou et al. (2021). Among these, Wang et al. 2018)) is particularly related for following a similar motivation of learning to generate good training examples. This is realized by feeding generated samples a meta-learning model obtain a classifier, apply it to real query samples, and use its query loss to update the generative model. Apart from the main difference in the problem definition (GZSL vs FSL), our work differs mainly by fully-training closed-form solvable ZSL model from scratch at each training step, instead of few-shot meta-learners that are jointly trained progressively with the generative model.
3

---

Published as a conference paper at ICLR 2022
Finally, we note that meta-leaming based approaches have also been proposed for variety of dif- ferent problems, such as learning-tor coaming-to-optimeee to-oopimizee (Li & Malik 2016; Chen et al. 2017) and long-tail
classification (Liu al. 2019 Ren al. 2020).
METHOD
In this section, we first formally define the generalized zero-shot learning problem and then define a mathematical framework to summarize the core training dynamics of mainstream generative GZSL approaches. We then express our approach the context of this mathematical framework.
3.1 PROBLEM DEFINITION
In zero-shot learning, the goal is to learn a classification model that can recognize the test instances of unseen classes V'., which has no training examples, based on the model learned over the training examples provided for the disjoint set of seen classes V'.. We refer to the class-limited training set by Dus which consists of sample and class label pairs (I E X, y V.)) In our work, focus ZSL models where X' is the space of image representations extracted using a pre-trained ConvNet. In generalized zero-shot learning, the goal is to build the classification model using the training data set 'D., such that the model can recognize both seen and unseen class samples at test time. For simplicity, restrict discussion the GZSL, problem setting below.
In order to enable the recognition of unseen class instances, it is necessary to have visually-relevant prior knowledge about classes that classes visually be related to each other. Such prior knowledge is delivered by the mapping v/ y + A, where A expresses the prior knowledge space. In most cases, the prior knowledge provided as de de-dimensional vector-space embeddings of classes, obtained using visual attributes, taxonomies, class names combined with word embedding models, the textual descriptions of classes combined with language models, see e.g. AAkaa et al. 2015). Following the common terminology, we refer to va as the class embedding function.
Generative GZSL. In our work, we focus on generative approaches to GZSL. The main goal is to learn conditional generative model G A Z X, which takes some class embedding E A and stochasticity-indeele generative model learned, synthetic training examples for all classes can simply be sampled from the G-induced distribution Pc. and the final classifier over ) can be obtained using any standard supervised classification model. We refer to the trainable parameters of the model G by 0g.
noise input 2, and yields synthetic sample E X. Once such a
As summarized in Section details. For the purposes of our presentation, most the GZSL works (if not all) can summarized as the iterative minimization of some loss function that acts the outputs of the generative model:
2 existing approaches vary greatly in terms of their generative model
L
E (a,a)~D,
[&e (G(a, a))
where 2, refers the noise input associated with the training sample (x,, y) and &a is the generative model learning loss. (z, a) Dr is shorthand notation for (z, vvv)) ~ Dur At each iteration, the goal is to reduce LG approximated over mini-batch of real samples and their class embeddings.
In our notation, we deliberately keep certain details simple. Noticeably, Zy greatly varies across models. For example, in the case of conditional GAN model, Zg ~ p(z) can simply be sample from simple prior distribution p(2), e.g. as in [ian et al. [0118b Sariyildiz & Cinbis 2019 Elhoseiny Elfeki 2019). In contrast, in variational training, 2g the latent code sampled from variational posterior, i.e. En ((](rr), where the variational posterior y(= ar) is given by variational encoder trained jointly with G, e.g. as in (Narayan et al., 2020 Xian et al. 2019).
Another important simplification that we intentionally make in Eq. 11 is the fact that we define the generative model learning loss la as a function of generator output and class embedding, to emphasize its sample-rellistictyy and class-relevance estimation goals. However, the exact do- main of & heavily depends on its details, which typically consists of multiple terms and/or (ad- versarially) trained models. In most of the state-of-the-a approaches, this term is combination of VAE reconstruction loss (Xian et al.) (201)). conditional or unconditional adversarial discrim-
inator network (Xian al. 2018h Narayan al. 2020 Sariyildiz & Cinbis. 2019 Xian et al.
4

---

In the construction of our approach, at each training iteration L, we first randomly select subset Published as a conference paper at ICLR 2022
2019). sample-to-clase classifier for measuring class relevance (Xian et al. 2018b) and sample-to- embedding mappings (Narayan et al. 2020). The loss LG may also incorporate additional regular- ization terms, such as regularization
gradient penalty term (Sariyildiz & Cinbis 2019).
In the following, we explain our sample probing approach as a loss term that can, in principle, be used in conjunction with virtually any of the mainstream generative zero-shot learning formulations.
3.2 SAMPLE PROBING AS GENERATIVE MODEL GUIDANCE
The problem that we aim to address is the enforcement of G to learn to produce samples maximally beneficial for zero-shot model training purposes. We approach this problem through a learning to generate training samples perspective, where we aim to monitor the quality of the generative model through the synthetic class samples provides.
of Y pb-ir C y' of seen classes. We refer to these classes as probe-train classes. This subset defines the set of classes that are used for training the iteration-specific probing model over the synthetic samples. More specifically, we first take samples from the model G with the parameters 0% for these classes, and fully train a temporary ZSL model over them using regularized loss minimization:
arg arg min I
E a= G=
Up (fro (z), a)]
where fob is the scoring function of the temporary probing model parameterized by I' and Epb is its training loss. A/beer is the set of class embeddings of classes in yt2 not shown explicitly for brevity.
pb--tr Regularization term over I is
The result of Eq. (2). gives us a purely synthetic sample driven model 1 which we leverage as a way to estimate the success of the generator in synthesizing training examples. For this purpose, we sample real examples from the training set Du as validation examples for the probe model. Since we use (G)ZSL model as the probing model, we can evaluate the model on examples of the classes not used for training the model. Therefore, we sample these vale-vvaii examples from the remaining classes Yph-val = V. Ypbur Lee the classes with real training examples but unused for probe model training, and use softmax cross-entropy loss over these samples as the probing loss:
EE Lpb (r.y))~DDaow
p(ogp(y)>;; [t]]
where Dpt-val C. Dir is the data subset of classes Ypo-val- ply|æ; I't) is the target class likelihood obtained by applying softmax to fT v'(y); 1nt) scores over the set of target class set. Here, as the target class set, one can use only the classes in Ypb-val (ZSL probing) or those in both Ypb-rr and Yat-val (GZSL probing). We treat this decision as a hyper-parameter and tune the validation set.
We use weighted combination of Lpb and LGs as our final loss function. Therefore, the gradients Voa act effectively as the training signal for guiding G towards yielding training examples that result in (G)ZSL. probing models with minimal empirical loss.
3.3 CLOSED-FORM PROBE MODEL
critical part of the construction the need for probing model where minimization of Eq. (2) is both efficient and differentiable, so that the solver itself can be part of the compute graph. Probing models that require iterative gradient descent based optimization are unlikely to be suitable as one would need to make a large number of probing model updates for cach single G' update step, which both inefficient and prone to gradient vanishing problems. We address this problem through the of ZSL model that can be efficiently fit using closed-fome solution.
For this purpose, we opt to use the ESZSL (Romera- Paanera-Paredes & Torr 2015) as the main closed-for form probe model in our experiments. The model is formalized by the following minimization problem:
min X A II + 2 (T)
(4)
where E Rd Xm and A E Rd represent the feature and class embeddings corresponding to m input training examples and k classes, Y is the (0, 1) j matrix of groundtruth labels, I E Rds xdg
5

---

Published as a conference paper at ICLR 2022
Other losses (e.g.. GAN discriminator)
LG
G(a,,
an Arrit
Voc
Proba model
Training over synthetic samples
Generator Inputs
Parameters Iteration
Probing model parameters
(1-1)) Paoooal
log p(y ar; l
Proba-validation
Sample probiig
examples
Figure 2: The compute graph view of the proposed approach, some training iteration t. The upper half shows sampling from the generative model and the lower half shows sample probing model loss. Circles denote generator and probing model parameters. Blue arrows show the back-propagation path for updating the generative model. Best viewed in color.
is the compatibility model matrix, and Ω (T) is the regularization function, defined as:
Ω (T) Ar A + Xa A AT.
(5)
where Az das An correspond term weights, When the regularization term weights are set such that An = Anda, the optimal solution to Eq. (4) can be computed in a closed-form:
I (XXT - AYA" Aa1)
(6)
This approach was originally proposed as standalone label-embedding based 7SL. model in Romera-Paredee & Torr (2015), with the practical advantage of having efficient solver. Here, we re-purpose this approach as a probing model in our framework, where the fact that the model is solvable in closed-form is critically important, enabling the idea of end-to-end sample probing. For
this purpose, we utilize the solver given by Eq. (6) as the implementation of Eq. (2) which takes a set of synthetic training samples and estimates the corresponding probing model parameters.
Alternative probe models. While we utilize ESZSL in our main experiments, we demonstrate the possibility using the proposed approach with different probe models using two additional alternatives. The first one, which call Vis2Sem, is the regression model from visual features to their corresponding class embeddings. defined as follows (using the same notation as in ESZSL):
X 1 + X
A discussion of the Vis2Sem model can be found in Kodirov et al. (017)) The second one, which call Sem2Vis, the class embeddings visual features regression model Shigeto al. (2015):
min X TAYT|E + A,, [
Both models, just like ESZSL, are originally defined as non-generative ZSL models, and we re- purpose them to define our data-dependent generative model training losses. Unlike the bi-linear compatibility model of ESZSL, however, these models rely on distance based classification, and do not directly yield class probability estimates. While one still obtain probability distribution over classes, e.g. by applying softmax to negative lz distances, for simplicity. we directly use the Sem2Vis and Vis2Sem based distance predictions between the visual features of samples and their corresponding class embeddings to compute Lyb as a replacement of Eq.
Summary. A summary of the final approach from a compute graph point view, is given in Figure 2 The proposed approach aims to realize the goal of learning to generate good training samples by evaluating the synthesis quality through the lens of a closed-form trainable probe model, the prediction loss of which is used loss for the G updates. Therefore, G is expected to be progressively guided towards producing realistic, relevant and informative samples, through the reinforcement of which may vary depending on the inherent nature of the chosen probe model.

---

Published as a conference paper at ICLR 2022
4 EXPERIMENTS
Datasets. We use the four mainstream GZSL benchmark datasets: Caltech-UCSD- (CUB) Wah et al.) 2011). SUN Attribute (SUN) Paatterson & Hays, 2012). Animals with Attributes (AWA2, more simply AWA) (Xian et al. 201 and Oxford Flowers (FLO) (Nilsback & Zisserman 2008]. CUB is a medium scale fine-ge graing dataset consisting of 200 bird species classes, 11.758 images and 312 dimensional attribute annotations. SUN is a fine-grained dataset with 14.340 images and 102 dimensional attributes. AWA is a coarse-grained dataset with 30.475 images of 50 animal classes with 85-dimensional attributes. FLO is medium-scale fine- grainernnndd dataset with 102 classes and 1024-dimensional attributes. Following the state-of-the 2ersion-2 splits defined by Xiaa al. (2018a). We evaluate the results in terms of GZSL-u, GZSL-s and h-score (H) values (Xian et al.) 2018a). The h-score, i.e. the harmonic mean of GZSL-u and G7SL-s scores, aims measure how well model recognizes seen and unseen classes collectively.
we use the class embeddings and the
In our experiments, we use the image features extracted from ResNet-101 backbone pretrained on ImageNet 1K. In the experiments based on fine-tuned representations, we use the backbone fine- tuned with the training images of seen classes, as in Xian et al. (2019) Narayan et al. (2020).
Hyper-parameter tuning policy. In our preliminary studies, we observe that the final GZSL perfor- mance, especially in terms of score,, of most models strongly depends the selection of the hyper- parameters. We also observe that there is no widely-accepted policy on how the GZSL models shall be tuned. It is rather common practice in the GZSL literature either directly report the hyper-parameters parameters used in experiments without an explanation on the tuning strategy or simply refer to tuning on the validation ser, which we find raguely-define policy as (i) Xian al.) (2018a) defines an unseen-class only validation split, which does not allow monitoring the h-score, and, (ii) is unclear which metric one should use for GZSL model selection purposes.
of
Therefore, to obtain comparable results within our experiments, we use the following policy to tune the hyper-parameters of our approach and our baselines: we first leave-out 20% of train class sam- ples as val-seen samples. We periodically train supervised classifier by taking synthetic samples from the generative model, and evaluate it on the validation set, consisting of the aforementioned val-seen samples plus the val-unseer samples with respect to the benchmark splits. We choose the combination with the highest h-score on the validation set. We obtain final models by re-training the generative model from scratch on the training and validation examples combined.
4.1
MAIN RESULTS
In this subsection, we discuss our main experimental results. As we observe that the results heavily influenced by hyper-parameter tuning strategy, our main goal throughout our experiments is the validation of the proposed sample probing idea by integrating it into strong generative GZSL. baselines, and then comparing results using the same tuning methodology. Using this principle, we present two main types of analysis: (i) the evaluation of the proposed approach using ESZSL as the probe model in combination with number of generative GZSL models, and (ii) the evaluation of alternative closed-fone form probe models within our framework.
Generative GZSL models with sample probing. To evaluate the sample probing approach as a general technique to improve generative model training, we integrate it to four recent genera- tive GZSL approaches: conditional Wasserstein GAN (cWGAN) (Arjovsky et al. 2017: Miyato & Koyama 2018), LisGAN (Li et al. 2019). TA-VAEGAN (Narayan et al.) 2020) and FREE (Chen et al. (021). We additionally report results for the variant of VA-EEGNN with the fine-tuned rep- resentations (TF-VAEGAN-FT) as it is the only one among them with reported fine-tuning results. For the eWGAN, we follow the implementation details described in Sariyildiz & Cinbis (2019) and tune saper-pprameters using our policy. For LisGAN, TA-VAGAAN and FREE models, we use the official repositories shared by their respective authors. We use the version of TAEGAN with- out feedback loop (Narayan et al., 2020), for simplicity, as the model yields excellent performance with and without feedback loop. In all models (except cWGAN), we only re-tune the number of training iterations of the original models using our hyper-parameter tuning policy, to make the
7

---

Published as a conference paper at ICLR 2022
Table 1: Evaluation of sample probing with multiple generative GZSL models on four bench- mark datasets. Each row pair shows the effect of adding sample probing to particular generative GZSL model, using ESZSL as the closed-form probe model. We use the same hyper-paramett optimization policy in all cases make results comparable. We observe h-score improvements at varying degrees in 17 out of 19 model, feature & dataset variations.
Sample probing
CUB
u
II
u
cWGAN Miyato Koyama, 2018
(ESZSL)
N
45.1
53.1
48.7
50.7
74.3
60.3
61.0
44.4
41.6
57.3
39.3
LiGGAN
Li et 2019
N (ESZSL)
40.9
60.5
48.8
83.1
81.7
64.4
41.5
36.6
38.9
44.2
77.0
56.1
44.2
59.2
59.6
36.7
77.8
65.6
44.0
35.4
39.2
46.2
71.5
56.2
TF- VABGAN Narayan et al.
2020
(HSZSL))
51.1
63.3
58.4
56.6
63.5
83.2
67.5
72.1
44.0
39.7
41.7
55.2
74.7
63.5
TI-VEAGAN-FT Narayan
N
64.2
72.7
68.2
70.0
91.3
79.2
46.5
41.7
44.0
41.7
90.2
57.0
(BSZSL.)
76.1
91.7
79.5
45.6
87.6
FRFE Chen al.
2121
(ESZSL)
N
51.2
51.6
61.5
60.4
55.9
55.7
62.8
82.2
80.7
70.6
72,9
46.2
48.2
36.5
57.2
41.5
41.2
51.3
18.2
78.7
78.0
59.8
61.8
Table 2: Sample probing with alternative closed-form models, based on TF-VAEGAN.
Closed-form pmhhe model
CUB
FLO
SUN
AWA
u
8
H
u
8
u
H
53.9
58.4
56.0
59.4
78.3
67.5
42,9
39.3
41.0
54.4
75.2
63.2
ESZSL
51.1
63.3
56.6
63.5
83.2
72.1
44.0
39.7
41.7
55.2
74.7
63.5
Sem2Vis
51.9
63,0
56.9
58.6
80.9
68.0
44.7
38.4
41.3
54.9
74.6
63,2
Vis2Sem
37.1
70.4
48.6
58.3
80.1
67.5
46.0
40.1
42.8
55.3
74.3
63.4
sults comparable. as it is unclear how the original values were obtained We keep all remaining hyper-parameters unchanged to remain as close as possible to the original implementations.
The results over the four benchmark datasets are presented in Table T In terms of the h-scores, we observe improvements in 17 out of 19 cases, at varying degrees (up to 4.6 points). ). Only in two cases observe slight degradation (maximum of 0.2 points) in performance. Overall, these improve- ments over already strong and state-of-the-art (or competitive) baselines validate the effectiveness of the proposed sample probing approach, suggesting that it is valid method towards end-to-end learning of generative GZSL models directly optimized for synthetic train data generation purposes.
Sample probing with alternative closed-form models. We now evaluate our approach with different closed-form probe models, specifically ESZSL, Sem2Vis (Shigeto al. 2015) and Vis2Sem (Kodirov et al.] (017), as described in Section 3.3 For these experiments, we use the TF-VAEGAN the base generative model.
The results with four configurations over four benchmark datasets are presented in Table First of all, in terms of h-scores, we observe considerable performance variations across the probe models and datasets: Sem2Vis performs the best on CUB (+0.9 over the baseline), ESZSL provides clear gain on FLO (+4.6) and a relative improvement on AWA (+0.3), and Vis2Sem improves the most on SUN (+1.8). These results suggest that sample probe alternatives have their own advantages and disadvantages, and their performances can be data dependent. Therefore, in a practical application, probe model options can be incorporated into the model selection process. More in-depth under- standing of closed-form model characteristies for sample probing purposes, and the formulation and evaluation of other probe models can be important future work directions. Overall, the fact that we observe equivalent (2) or better (9) h-scores in 11 out of 12 sample probing experiments indicates the versatility of the approach in terms of compatibility with various closed-form probe models.
Comparison to other generative GZSL approaches. Performance comparisons across indepen- dent experiment results can be misleading due to differences in formulation-agnostte implementation and model selection details. Nevertheless, an overall comparison can be found in Appendix A
We also tune LisGAN for AWA2 the original paper reports AWAI results instead.
8

---

Published as a conference paper at ICLR 2022
Table ZSL GZSL based sample probing losses (using TF-VAEGAN and ESZSL).
Sample probing
Baseline
zsl-loss
gzsl-loss
8
CUB
53,9
58.4
56.0
50.5
63.6
56.3
51.1
63.3
56.6
FLO
59.4
78.3
67.5
62.4
83.8
71.5
63.5
83.2
72,1
SUN
42.9
39.3
41.0
44.0
39.7
41.7
46.0
36.9
41.0
AWA
54.4
75.2
63.2
55.2
74.7
63.5
55.6
72.8
63.0
Table 4: Comparison of mean per-class Fréchet Distance between real and generated unseen class
samples on CUB, AWA and FLOO datasets for TF-VAEGAN and Our approach. Lower better.
CUB
FLO
AWA
Baseline
21.5
31.0
18.5
Ours
19.8
30.2
17.9
4.2 ANALYSIS
ZSL vs GZSL, loss in sample probing. In Table 3 we present a comparison of our approach, using TF-VAEAAN as the generative model and ESZSL the probe model, when two different types of losses (zsl-loss and gzsl-loss) are used as Lpb in Eq. 33 They differ from each other in terms of classes among which the real examples probe-valication classes are classified, during the evaluation of sample probing ZSL model. zsl-loss and gesl-loss indicate that the examples of probe- valldaalio classes are classified among only mobe-validation classes, and both probe-trait train and grobe-validation classes, respectively. We observe that using either one during the evaluation of sample probing 7SL. model, brings its own characteristic results (Table On all datasets, using 2sl-loss increases the seen accuracy while using gzsl-loss increases the unseen accuracy. We choose among these two options using our same dyper-paramete tuning policy, on the validation set.
Effect of sample probing loss weight. Figure shows the validation and test set h-score values function of sample probing loss weight. all increasing performance trend with larger loss weights, up to the weight 6, highlighting the contribution sample prob- ing. The optimal weight with respect to the validation and the test sets, however, differs. This observation is an example for the difficulty of tuning the 7SL model based validation set. We set the loss weight to 5 following our hyper-parameter tun- ing policy, which yields almost 0.5 lower than the maximum test-set score observed for this single syper-parameter.
test set results, we observe an over-
Samplh procing weight
Figure 3: The effect of sample probing weight on the CUB dataset.
Quantitative analysis of sample quality. We quantitatively
evaluate the sample quality using Fréchet (Wasserstein-22) distance, which is also used in the FID metric for evaluating GAN models. In Table 4 we provide a comparison for TF-VAEGAN and our approach for respective mean per-class Fréchet distances between real and synthetic samples (200 synthetic samples per class) of unseen classes on three datasets. The results show that sample probing helps generator generate more realistic samples compared TF-VAEGAN, We further investigate the qualitative analysis of the sample quality via t-SNE visualizations in Appendix B
5 CONCLUSIONS
We propose principled GZSL approach, which makes of closed-form ZSL models in genera- tive model training to provide sample-driven and end-to-end feedback to the generator. Extensive experiments over four benchmark datasets show that the proposed sample probing scheme consis- tently improves the GZSL results and the sample quality, can easily be integrated into the existing generative GZSL. approach and can be utilized with various dlosed-form probe models.

---

Published as a conference paper at ICLR 2022
Acknowledgement. This work was supported in part by the TUBITAK Grant 119E597. The nu- merical calculations were partially performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).
Reproducibility statement. We discuss the problem of model selection in G7SL, and the difficulty of making fair comparison across GZSL models in Section 4 We believe that these are open prob- lems in GZSL research. We also clearly explain syper-parameter tuning policy, which we use in our experiments. We will provide our source code on a public repository.
REFERENCES
Zeynep Akata, Scott Reed, Daniel Walter, Honglak Lee. and Bernt Schiele. Evaluation of Output Embeddings for Fine-Grained Image Classification. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., pp. 2927-2936, 2015.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Proc. Conf. Mach. Learn.. pp. 314-223, PMLR, 2017.
Gundeep Arora, Vinay Kumar Verma, Ashish Mishra, and Piyush Rai. Generalized zero-shot leamnngg via synthesized examples. Proc. IEEE Conf. Comput. Vis. Pattern Recog., pp. 4281 1-899499, 2018.
Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. What do neural machine translation models learn about morphology? arXiv preprint urXiv:1704.034477 2017.
Luca Bertinetto, Philip H S Tomr, João Henriques, and Andrea Vedaldi. Meta-Learning with Differentiable Closed-Form Solvers. ICLR, pp. 15, 2019.
Goutam Bhat, Felix Järemo Lawin, Martin Danelljan, Andreas Robinson, Michael Felsberg, Luc Van Gool, and Radu Timofte. Learning What to Learn for Video Object Segmentation. Proc. European Conf. on Computer Vision, May 2020.
John Bronskill, Jonathan Gordon, James Requeima, Sebastian Nowozin, and Richard Turner. Tasknorm: Rethinking batch normalization for meta-learning. In Proc. Int. Conf. Mach. Learn., volume 119 of Pro- ceedings of Machine Learning Research, pp. 1153-1164. PMLR, 2020. URL http: http:::/procceeinige
m1 m. p.rrss/s//// v1 feronski112000 html
Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, and Fei Sha. An Empirical Study and Analysis of General- ized Zero-Shot Learning for Object Recognition in the Wild. In Proc, European Conf. Computer Vision, 2016.
Shiming Chen, Wenjie Wang, Beihao Xia, Qinmu Peng, Xinge You, Feng Zheng, and Ling Shao. FREE: Feature refinement for generalized zero-shot learning. Proc. IEEE Int. Conf. on Computer Vision, pp. 122-131, 2021.
Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lillicrap, Matt Botvinick, and Nando Freitas. Learning to learn without gradient descent by gradient descent. In Proc. Int. Conf. Mach. Learn., pp. 748-756, 2017.
Yu-Ying Chou, Hsuan-Tien Lin, and Tyng-Luh Liu. Adaptive and generative zero-shot learning. Proc. Int. Conf. Learn. Represent., 2021a.
Yu- Yu-Ying Chou, Hsuan- Tien Lin, and Lung-Luh Liu, Adaptive and Generative Zero-Shot Learning, In Proc. Int. Conf. Learn. Represent., 14, 2021b.
Mohamed Elhoseiny and Mohamed Elfeki. Creativity Inspired Zero-Shot Learning. In Proc. IEFE Int, Conf. on Computer Vision, April 2019.
Rafacl Felix, B. G. Vijay Kumar, Ian Reid, and Gustavo Carnciro. Multi-modal Cycle-consistent Generalized Zero-Shot Learning. In Proc. European Conf. on Computer Vision, July 2018.
Chelsea Finn, Picter Abbeel, and Sergey Levine. Model-agnostic meta-lcarning for fast adaptation of deep networks. In Doina Precup and Yee whye Tch (cds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 1126- 1126-1135. PMLR, 06-11 Aug 2017. uR http:///oceednnn...e mwr. www.mppa.m.mmmmm www...pprsss..m httpa////////iinn//
Andrea Frome, Greg S. Corrado, Jon Shlens, Samy Bengio, Jeff Dean, and Tomas Mikolov. DeViS A Deep Visual- Semantic Embedding Model. In Proc. Adv. Neural Inf. Process. Syst., pp. 2121-2129, 2013.
10

---

Published as a conference paper at ICLR 2022
Hang Gao, Zheng Shou, Alireza Zareian, Hanwang Zhang, and Shih-Fu Chang. Low-shot Learning via Covariance-Preserving Preserving Adversarial Augmentation Networks. In Advances in Neural Information Processing Systems 31, pp. 983-993 2018.
Covariance-
Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proc. IEEE Conf. Comput. Vis. Pattem Recog.. pp. 4367-4375, 2018.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinherger (eds.), Proc. Adv. Neural Inf. Process. Syst., volume 27. Cur- ran Associates, Inc., 2014. URL http:: /ttpttpp:///ccceediiie ngs. wttprrip..cppppprrr.eeee
Bharath Hariharan and Ross Girshick. Low-shot Visual Recognition by Shrinking and Hallucinating Features. arXiv e-prints, art. 160.:1666600819 June 2016.
John Hewitt and Percy Liang, Designing and Interpreting Probes with Control Tasks, In Proc. of the Empiricial Methods in Natural Language Processing, September 2019.
Huajie Jiang, Ruiping Wang, Shiguang Shan, and Xilin Chen. Transferable Contrastive Network for General- ized Zero-Shot Learning. In Proc. IEFE Int. Conf. on Computer Vision, August 2019.
Diederik P Kingma and Max Welling. Stochastic gradient vb and the variational auto- outodencoder. In Proc. Int. Conf. Learn. Represent. volume 19, 2014.
Elyor Kodirov, Tao Xiang, and Shaogang Gong. Semantic autoencoder for zero-shot learning. 2017 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp- 4447-4456, 2017.
Michalis Lazarou, Yannis Avrithis, and Tania Stathaki. Few-shot learning via tensor hallucination. In ICLR2021 workshop: "Synthetic Data Generation: Quality, Privacy, Bias", April 2021.
Jingjing Li, Mengmeng Jing, Ke Lu, Zhengming Ding, Lei Zhu, and 7i Huang. Leveraging the invariant side of generative zero-shot learning. In Proc. IEEE Conf. Comput. Vis. Pattern Recog.. pp. 7402-7411, 2019.
Ke Li and Jitendra Malik. Learning to optimize. arXiv preprint arXiv:1606.0183 2016.
Shichen Liu, Mingsheng Long, Jianmin Wang, and Michael I Jordan. Generalized Zero-Shot Learning with Deep Calibration Network. Proc. Adv. Neural Inf. Process. Syst., pp. 2005-2015, 2018.
Yu Liu, Lingqiao Liu, Haokui Zhang, Hamid Rezatofighi, Qingsen Yan, and Ian Reid. Meta learning with differentiable closed-form solver for fast video object segmentation. In TEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 8439-8446 IEEE, 2020.
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang. Boqing Gong, and Stella X. Yu. Large-Seate Scale Long- Tailed Recognition in an Open World. Proc. IEEE Conf. Comput. Vis. Puttern Recog.. pp. 2532 2541, Long Beach, CA, USA, June 2019.
Ashish Mishra, M. Shiva Krishna Reddy, Anurag Mittal, and Hema A. Murthy. A Generative Model For Zero Shot Learning Using Conditional Variational Autoencoders. Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops, September 2017.
Takeru Miyato and Masanori Koyama. eGANs with Projection Discriminator. In Proc. Int. Conf. Learn Represent. February 2018.
Sanath Narayan, Akshita Gupta. Fahad Shahbaz Khan, Cees G. M. Snoek, and Ling Shao. Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification. In Proc. European Conf. on Computer Vision, pp. 479-495, 2020.
Alex Nichol, Joshua Achiam, and John Schulman. On First-Order Meta-Learming Algorithms. arXiv e-prints, art. arXiv: March 2018
Maria- Elena Nilsback and Andrew Zisserman. Automated flower classification over large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing, Dec 2008
G. Patterson and J. Hays. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In Proc. IEEE Conf. Comput. Vis. Patiern Recog.. pp. 2751-2758, 2012. doi: 10.1 11O.////...00..22277777..
Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, Octoher-Novemher 2018.

---

Published as a conference paper at ICLR 2022
Jiawei Ren, Cunjun Yu, Shunan Sheng, Xiao Ma, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Balanced Meta- Softmax for Long-Tailen Visual Recognition. Proc. Adv. Neural Inf. Process. Syst., November 2020.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. Proc. Int. Conf. Mach. Learn. pp. 1278-1286. PMIR, 2014.
Bernardino Romera-Paredee and Philip Torr. An embarrassingly simple approach to zero-shot learning. In Proc. Int. Conf. Mach. Learn.. pp. 2152-2161, 2015.
Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learnin with latent embedding optimization. In Proc. Int. Conf. Learn. Represent. 2019.
URL htt.:http:://eo /tttps://openrreview net www.ooodddnnnuuaaa....
Mert Bulent Sariyildiz and Ramazan Gokberk Cinbis. Gradient matching generative networks for zero-shot learning. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., pp. 2168 2178, 2019.
Edgar Schonfeld, Sayna Ebrahimi, Samarth Sinha, Trevor Damelll and Zeynep Akata. Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders. In Proc. TEFE Conf. Comput. Vis. Pattern Recog., pp. 8239-8247, Long Beach, CA. USA, June 2019.
Eli Schwartz, Leonid Karlinsky, Joseph Shtok, Sivan Harary, Mattias Marder, Abhishek Kumar, Rogerio Feris, an effective sample synthesis method for few-shot object
Raja Giryes, and Alex Bronstein. recognition. In NewrIPS, pp. 2845 2855, 2018.
Yutaro Shigeto, Ikumi Suzuki, Kazuo Hara, Masashi Shimbo, and Yuji Matsumoto. Ridge regression, hubness, and zero-shot learning. ECMI/PKDD, 2015.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In I. Guyon, U. V. Luxburg. S. Bengio, Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Proc. Adv. Neural Inf. Process. Syst. volume 30. Curran Associates, Inc., 2017. URL /ttpp:://rrcceedinggg
neurips.
ce/ / paperapppa// 201 wwt.///ile/eeeeeeeeeeeeeeeeeeeee www.a////////eeeeeeeeeeeeaaaeeea /11////46661222 0114441122222eee 22 429004 wwweee//cbboo -acbbc2--apper paf
V Vapnik and V Vapnik. Statistical Learning Theory 160. Wilcy, New York, 1998.
Vinay Kumar Verma, Dhanajit Brahma, and Piyush Rai. Meta-Learning for Generalized Zero-Shot Learning. AAAI Conference on Artificial Intelligence, 34(04))602-66069, April 2020. ISSN 2374-3468, 2159-53999.
Vinay Kumar Verma, Ashish Mishra, Anubha Pandey, Hema A Murthy, and Piyush Rai. Towards Zero-Shot Learning With Fewer Seen Class Examples. In WACV, pp. 11, 2021.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech- Birds-200-2011 Dataset. Technical Report CNS-TR TTR-201--0000 California Institute of Technology. 2011.
Yu-Xiong Wang, Ross Girshick, Martial Hebert, and Bharath Hariharan. Low-Shot Learning from Imaginary Data. arXiv: 1801000400011000 [cs], January 2018
Jason Weston, Samy Bengio, and Nicolas Usunier. Wsabie: Scaling up to large vocabulary image annotation. 2011.
Yongqin Xian, Bernt Schiele, and Zeynep Akata. Zero-Shot Learning The Good, the Bad and the Ugly. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., March 2017.
Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata, Zero-shot learning comprehensive evaluation of the good, the bad and the ugly. IEEE Trans. Pattern Anal. Mach. Intell., 41(9):2251 2265, 2018a.
Yongqin Xian, Tobias Lorenz, Bernt Schicle, and Zeynep Akata. Feature generating networks for zero-shot learning. In Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2018b.
Yongqin Xian, Saurabh Sharma, Bernt Schiele, and Zeynep Akata, f-vaegan-d2: A feature generating frame- work for any-shot learning. In Proc. IEEE Conf. Comput. Vis. Pattern Recog.. pp. 10275-10284, 2019.
Han-Jia Ye. Hexiang Hu, De-Chuan Zhan, and Fei Sha. Few-shot learning via embedding adaptation with set-to-set functions. In Proc. IEEE Conf. Comput, Vis. Pattern Recog., pp. 8808 0008-8817.00 2020.
Yunlong Yu, Zhong Ji, Jungong Han, and Zhongfei Zhang. Episode-Bard Based Prototype Generating Network for Shoo Learning. In Proc. IEFE Conf. Comput. Vis. Pattern Recog., pp. 14032-1400 14441 Seattle, WA, USA, June 2020.
12

---

Published as a conference paper at ICLR 2022
Table 5: Comparison against state-of-the-art generative model based GZSL on CUB, FLO, SUN and AWA datasets. Results obtained with the proposed features are reported, together with the results obtained with fine-tuned features under fine-tuned (FT). The results are reported in terms of top-1 accuracy of unseen (u) and seen (s) classes, together with their harmonic mean (H),
CUB
FLO
SUN
AWA
H
H
u
H
u
H
ECLSWGAN Xian et al.
2018b
3.7
57.7
49.7
59.0
73.8
65.6
42.6
36.6
39.4
57.9
61.4
99.6
Cycle-WGAN TTllx
2018
47.9
59.3
53.0
61.6
69.2
65.2
47.2
33.8
39.4
59.6
63.4
59.8
LisGAN (Li et ar 2019
40.9
60.5
48.8
53.1
81.7
64.4
41.5
36.6
38.9
44.2
77.0
56.1
EVARGAN Xiin
48.4
53.6
56.8
74,9
64.6
45.1
38.0
41.3
57.6
70.6
63.5
TA-VABAN Naravan
2020
53.9
58.4
56.0
59.4
78.3
67.5
42.9
39.3
41.0
54.4
75.2
63.2
Metn- VGAN Verma ol 2021
55.2
48.0
53.2
57.4
70.5
63.5
FREE (Chen
51.2
61.5
55.9
62.8
80.7
70.6
46,2
37.2
41.2
48.2
78.7
59.8
Ours (hased on TP VAFGIAN)
51.1
63.3
56.6
63,5
83,2
72.1
44.0
39.7
41.7
55.2
74.7
63.5
E-VAEGAN (Xian et al. 2019
63.2
75.6
68.9
50.1
37.8
43.1
57.1
76.1
65.2
FT
VARGANT Naraaan 2020
64,2
72.7
70.0
91.3
79.2
46.5
41.7
41.7
90,2
57.0
Ours (based on TF- VA-AAAN
63.1
76.1
69.0
70.2
91.7
79.5
47.8
40.6
43,9
45.6
87.6
60.0
Amir Zamir, Te-Lin Wu, Lin Sun, William Shen, Bertram Shi, Jitendra Malik, Silvio Savarese. Feedback networks. In Proc. IEFE Conf. Comput. Vis. Puttern Recog, pp. 1308-1317, 2017.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In Proc. Int. Conf. Learn. Represent, 2018
Yizhe Zhu, Mohamed Elhoseiny. Bingehen Liu, Xi Peng, and Ahmed Elgammal. A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts. Proc. IEEE Conf. Comput. Vis. Puttern Recog., pp. 1004-1013, Salt Lake City, UT, USA, June 2018.
APPENDICES
A COMPARISON TO THE STATE-OF-THE-ARY
In Table compare our results with the state-of-the-at generative approaches for GZSL. During training, we select our best model based on the validation results and report test results on models that give best validation scores. For consistency and to keep the baseline comparable to our results, we again report our own results for LisGAN, TF-VAEGAN (without feedback loop) and FREE using our hyper-parameter tuning policy, but do acknowledge that the original papers typically report higher results. The upper part of the table contains results with the original image representations, and the lower part contains those based on fine-tuned representations.
From the results without fine-tuning, we observe that the proposed sample probing based generative model yields state-of-the-an h-scores in all CUB, FLO. SUN and AWA datasets. We also observe competitive results in terms of individual unseen and seen class accuracy values. When compared against results using fine-tuned representations, we again observe state-of-the-an h-scores on CUB and FLO datasets, with close second on SUN. On AWA, we observe that f-VAEGAN achieves the highest results with a significant margin over our VAEGAN based baseline, where the sample probing improves the baseline yet still achieves is hardly fair to compare models with significant implementation details, these results suggest the overall competitiveness of the obtained data generating models with sample probing.
below that of f-VAEGAN. Overall, while it
B VISUALIZATION OF UNSEEN DATA
To provide additional qualitative insight into the improvements that can be gained using sample probing. we present t-SNE visualizations of synthetic class samples in Figure In the figure, each plot corresponds to unseen class the FLO dataset, and the points correspond to the t-SNE embeddings of real samples (x points), generated samples using TF- VAEGAN with sample probing points) and those using the baseline TF-VAEGAN model without sample probing points).
13

---

Published as a conference paper at ICLR 2022
Probing
(a)
(b)
Peal
Sample Probing
Samp Probing R
T-JAECAN
TFAAEGAN
(d)
Real
Real
Sample Prebing
Sample Probina T-JAEGAN
V-JAEGAN
(c)
(f)
Figure 4: t-SNE visualization of different unseen classes from FLO dataset. Each plot shows t-SNE embeddings of real samples (x points), generated samples using TF- VAEGAN with sample probing points) and those using the baseline TF-VAEGAN (Narayan al. 2020) model without sample probing AA points).
From the plots, we can observe that the generative model trained with sample probing tends to yield samples much more aligned with the corresponding true class distributions, compared to those of the baseline model. Overall, these plots demonstrate how sample probing can improve the overall sample quality of a generative model, and possibly lead to superior recognition models when the generated samples are used for classifier training.
14


Reviews:
[{"review_id": "vLs5a7WVBOc", "reviewer": "Reviewer_UDYK", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Paper Strengths：\n\nThe authors tackle an important and challenging problem of generative modeling based generalized zero-shot learning. The proposed approach is simple. Experimental evaluations demonstrate the effect by introducing the sample probing method and end-to-end training.\n\nPaper Weaknesses：\n\n1) In the related work section, the authors discussed meta-learning and few-shot classification. However, there was a lack of discussion on using generative models to perform data augmentation for meta-learning and few-shot classification, such as [Low-shot visual recognition by shrinking and hallucinating features, ICCV, 2017] [MetaGAN: An adversarial approach to few-shot learning, NeurIPS, 2018] [Low-shot learning from imaginary data, CVPR, 2018] [Delta-encoder: an effective sample synthesis method for few-shot object recognition, NeurIPS, 2018] [Image deformation meta-networks for one-shot learning, CVPR, 2019]. While zero-shot and few-shot learning are different, they are related. In particular, I feel that the generative model framework in this paper is similar to [Low-shot learning from imaginary data, CVPR, 2018], where the classification loss is used to train the generator in an end-to-end fashion and the generator is integrated into the meta-training framework. An in-depth discussion on this is needed.\n\n2) The proposed approach relies on a closed-form zero-shot model, which is also the main technical contribution of this paper. For this purpose, the ESZSL model [Romera-Paredes & Torr, 2015] is used. However, it seems that such requirement makes the method restrictive to this zero-shot model, and thus is not general and cannot be combined with more state-of-the-art zero-shot models. Even for ESZSL [Romera-Paredes & Torr, 2015], its closed-form solution only applies for some particular parameter setting.\n\n3) From the results, like in Table 1 and Table 3, the improvements of the proposed approach seem quite marginal, especially when the features are fine-tuned. Also, the proposed approach is not consistently better than existing methods. For example, in Table 3, for the fine-tuning setting, on 2 out of 4 benchmarks, the proposed approach is worse than the state of the art.\n\n4) It would be interesting to see the ablation study regarding the different loss components.\n\n5) Somehow it is a little vague – is the paper synthesizing raw images or feature vectors in the pre-trained feature space? This is only explicitly mentioned until it talks about the implementation details. It would be better to make this explicit starting in the introduction.\n\n6) Following the previous comment, it would be interesting to visualize the synthesized features, like using t-SNE visualizations, and analyze why they are helpful.\n\n7) There are a bunch of hyper-parameters involved in the proposed approach. The authors also mentioned the difficulty in consistently setting up these hyper-parameters. Moreover, how is the hyper-parameter sensitivity?\n\n8) There are several grammar issues and typos in the paper. Please proofread.", "summary_of_the_paper": "This paper aims to address the problem of generalized zero-shot classification based on generative models. The main contribution is that it considers training and evaluating a generative model in synthesizing training examples that are helpful to improve classification performance. To this end, it leverages the zero-shot learning model of ESZSL that can be efficiently fit using a closed-form solution. This then serves as a sample probing mechanism, so that the generator receives training signal directly based on the value of its samples for classifier training and thus enables end-to-end training. The corresponding sample probing loss function is added into the standard generative model training loss. The final training procedure is performed in a way that is similar to meta-training. The approach is tested on standard generalized zero-shot classification setups, including CUB, SUN, AWA2, and FLO, and compared with state-of-the-art results.", "summary_of_the_review": "The paper introduces sample probing to train a generative model that synthesizes data in feature space for generalized zero-shot learning. The proposed approach is restrictive to certain zero-shot learning models. Some ablation studies and analysis are missing.\n\n=================\n\npost rebuttal:\n\nI thank the authors for the extensive experiments and clarification made in the rebuttal. The rebuttal has addressed most of my concerns, especially showing the generalizability of the proposed approach (different types of generative models and different types of closed-form ZSL models), so I raised my score. However, I am still a little concerned about the novelty of the proposed approach and its marginal improvements.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "lLE1yDjpze", "reviewer": "Reviewer_NR3H", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Paper strengths:\n\n1) The paper is well written and easy to follow and understand. \n2) The motivations for the work are clear, as is the method itself.\n3) Providing greater detail on hyperparameter choices is great, and as the authors demonstrate - it is also crucial.\n4) The proposed method is also general in the sense that it can be readily applied to future work. In my opinion, this is a major selling point of the work.\n5) Results appear to indicate an improvement over the state of the art. \n\nNote, however, see point (1) in the weaknesses section in regards to points 4 and 5.\n\nPaper weaknesses:\n\nWhile the method is promising and the suggested approach makes intuitive sense, I feel like the experimental results do not currently support it well enough.\n\nIn particular (listed in order of importance for my evaluation):\n\n1) \n\nAs the author's note in their closing remarks: \"Our method works in an end-to-end manner and it can\nbe easily integrated into any mainstream generative zero-shot learning framework.\". This is, in my opinion, one of the most significant selling points of the paper. Alas, it is not investigated. The method is applied only to a single baseline model*, where it shows improved performance only when the baseline is trained with a different set of hyper-parameters than in it's original implementation.\n\n*There is an additional experiment on a single dataset (out of 4) with a different, very basic baseline.\n\n1.1) What were your results when you used the same hyper-parameters as the baseline model? Why do you think your model outperforms the baseline with a specific number of iterations, but underperforms with another? Perhaps your model is simply more efficient and converges faster, but introduces other problems which mitigate the advantage in the long-run?\n\n1.2) If things are highly hyper-parameter sensitive, why did you optimize only their number of iterations? What about the other parameters?\n\n1.3) Can your method be applied to other generative ZSL models? I would have more confidence in the general applicability of the method if it demonstrated improvements when integrated into multiple existing works.\n\nIn this context, Table 3 (where most of the results reside) is largely irrelevant. Most of these numbers are simply an indication that TF-VAEGAN is better than its competition. (I am not advocating the table's removal - but applying your method to the models listed there would be a considerable improvement).\n\n2) \n\nDo you have any intuition as to why the choice of ZSL / GZSL in your loss (i.e. table 4) is so crucial that a wrong choice may make your model perform equally to, or worse than the baseline on some datasets? Is this merely a function of how well the original model performs on seen vs unseen classes? (this is something that could be seen if we had more adapted baselines to compare with!)\n\n3) \n\nHow much of an effect does your method have on training times? Does the closed-form solution of EZSL have a noticeable impact on the time required per training iteration?\n\n4) \n\nA natural alternative to your approach may be the use of meta-learned ZSL model in place of the closed-form solution. This would allow updating the model with a single training iteration which may produce sufficiently strong gradients for your generator. My knowledge of recent works in the field is limited, are you aware of any works doing something similar? If so, comparisons to them could strengthen your work.\n\n5) \n\nGiven the aim of increasing reproducibility by reporting hyper-parameter tuning methodologies, I would add answers to the following questions (at the very least to the released code):\n\n5.1) Are you using the same 20% validation split for all experiments on a given dataset? If so, can this split be released?\n\n5.2) Did you use a single split, or cross-validate?\n\n6) \n\nSample quality metrics - Fréchet distance is typically sensitive to the number of generated samples used in the comparison. These should be reported to facilitate future comparisons.\n", "summary_of_the_paper": "The paper outlines a method for improving generative zero-shot learning (ZSL) approaches. Rather than simply training the generative model with the goal of reproducing some real data distribution, the work proposes to train it with an additional goal of synthesizing samples that directly improve the performance of the downstream classification model.\n\nThe authors propose to do so through a novel sample-probing loss in which generated samples are used to train a closed-form ZSL model with a differentiable solution. The ZSL model is then evaluated directly on the classification task - and gradients are back-propagated to the generative model's parameters. By applying this approach to an existing generative ZSL model, the authors demonstrate improved sample quality for their synthetic data and increased classification performance across multiple datasets.\n\nIn summary, the paper's contributions are:\n1) A major contribution: An approach to improving existing generative ZSL methods with a loss that maximizes their performance on the downstream task.\n2) A minor contribution: A more detailed and rigorous reporting of the methodologies used to fine-tune model hyperparameters, aimed at increasing reproducibility.", "summary_of_the_review": "Overall, I enjoyed the paper and would have liked to recommend acceptance. The approach makes intuitive sense, and can no doubt be extended with multiple future works, offering the community a parallel line of investigation into improving GZSL results.\n\nHowever, I think the flaws outlined in point (1) of the weakness section are fundamental enough that I am worried that any future works which try to build on this one might simply be wasting their time. I do not believe this to be the case, but I would like to see additional experimental results that would convince me otherwise.\n\n\n*I have marked my confidence as 3 due to limited familiarity with related work. As an extension to that, I may have missed prior art which already suggested similar ideas.\n\n******************************\nPost rebuttal update:\n\nThe authors conducted an extensive set of experiments and addressed my primary concerns (the method's generalizability to additional baselines). I am still concerned about some aspects of the evaluation (considerably worse results for all baselines compared to their originally reported values). However, since the paper suggests a method for improving other models, the relative improvements are what matters most. As such, I am willing to accept the current demonstration of a (fairly) consistent improvement when using the same hyper-parameter selection approach across the board. \n\nEven if accepted, I urge the authors to better highlight and explain the difference between their experimental results and the original baseline values.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "jzMLX_iCxh", "reviewer": "Reviewer_vUDu", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strong Points:  1- The presentation is clear, and the research problem is well motivated. \n2- The proposed method is evaluated on four relatively comprehensive benchmark datasets.\n\n\nWeaknesses: 1- The paper could not highlight its novelty well. The idea to improve model generalization ability with cross-validation is not new. The proposed method seems to be an integration of generative models and the existing closed-form solution.  The closed-form probe model is borrowed from [a],[b],[c].\n\n2- To prove the better efficacy of the proposed model, it should be trained using a few examples (5/10 samples) per seen class.\n\n3- The experiments for a large dataset (ImageNet) should be included for better evaluation.\n\n\n[a]- Meta-Learning for Generalized Zero-Shot Learning, AAAI 2020.\n[b]- Episode-Based Prototype Generating Network for Zero-Shot Learning, CVPR 2020.\n[c]- Towards Zero-Shot Learning With Fewer Seen Class Examples, WACV 2021.\n", "summary_of_the_paper": "\nThe paper addresses the task of GZSL – more specifically, they provide a way to improve the quality of the generative samples in generative GZSL. A closed-form probe model is introduced to provide an efficient and differentiable solution in compute graph. In this manner, the generator receives feedback directly based on the value of its samples for model training purposes. It shows the results on two different settings, with fine-tuning features and without fine-tuning the features.\n\n", "summary_of_the_review": "\nDue to lack of novelty and the performance of the proposed model is marginally above the existing approaches.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "eToE-dS7Dwd", "reviewer": "Reviewer_yvpj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work attempts to address a major concern in current generative ZSL models, i.e., the quality of the synthesized training examples used to train the final GZSL classifier, as the final GZSL performance highly depends on those generated samples. By measuring the quality of those samples during the training process, more informative samples can be obtained, leading to improved performance. Overall, the paper is well-written and easy to follow, with adequate technical details for re-implementation. Below please find the detailed suggestions and questions:\n\n1) In the last paragraph on Page 1, the authors claimed that samples need to be realistic, relevant and informative. However, in the method section, there are no detailed discussions regarding how the proposed solution endows the synthetic samples with these three properties. More clarifications on this point need to be added.\n\n2) Another concern is the performance gain over TF-VAEGAN. From the tables, we can only observe less than 1% increase w.r.t. the harmonic mean in most cases on the benchmarks. One could have expected to see more gains as measuring the sample quality provides important additional information for the generative model.\n\n3) Does the overall framework highly depend on the solvers it adopts? Is it possible to adopt other solvers like the one in SAE [R1]? If employing this solver, can we obtain even more improved GZSL performance as SAE shows the advantage over ESZSL?\n\n4) From Fig. 3, it is interesting to see that the validation set and the test one exhibits the almost opposite trend w.r.t. the harmonic mean. Does this indicate that the hyper-parameter tuning policy is not suitable for the GZSL task?\n\n5) Table 2 only depicts the results on the CUB dataset - how about the performance on the other three datasets?\n\n[R1] Kodirov et al., Semantic Autoencoder for Zero-Shot Learning, CVPR 2017.", "summary_of_the_paper": "This paper proposes sample probing, a meta-learning scheme for zero-shot learning (ZSL), to measure the quality of the synthetic samples provided by certain generative models. Specifically, an existing closed-form ZSL solver is plugged into an existing generative ZSL framework. Owing to the differentiability of the solver, the whole pipeline is end-to-end trainable. Experiments were conducted on four standard benchmarks, where we can observe the state-of-the-art performance achieved by the proposed sample probing based approach.", "summary_of_the_review": "This paper addresses an important issue existing in current generative ZSL models. The paper is well-motivated and the solution seems to be effective. The major concern lies in the performance gain and some details regarding the method need further clarifications.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ZTOCCIxrJTy", "reviewer": "Reviewer_dVi5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n\n1- The paper is well written.\n\n2- This approach tries to generate realistic, relevant, and informative features train an accurate classifier using a sample probing mechanism.\n\n3- It also uses feature fine-tune to improve the performance further.\n\nWeaknesses:\n\n1.  The novelty of the proposed approach is limited. The main contribution of this paper is the sample probing technique to generate features. However, prior works ([1],[2],[3]) have proposed a very similar approach. Therefore, given [1],[2],[3] methods, the contribution of this paper is not novel.\n\n2. Considering prior works ([3], and other results from Table 1 of [3]), the accuracy improvements are either not significant or are lower.\n\n\n[1] Episode-Based Prototype Generating Network for Zero-Shot Learning\n\n[2] Meta-Learning for Generalized Zero-Shot Learning\n\n[3] Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning", "summary_of_the_paper": "This paper proposes a generalized zero-shot learning approach by providing feedback for generator with the evaluation of real samples of unseen classes.", "summary_of_the_review": "Considering the weaknesses of the paper, I would give this paper marginally below the acceptance threshold.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes to improve (generalized) zero-shot learning, by training a generator jointly with the classification task, such that it generates samples that reduce the classification loss.  To achieve this, they use a zero shot model that has a (differentiable) closed form solution (ESZSL), so the full model can be optimized end-to-end. The approach is evaluated on the standard benchmarks of GZSL. 

Reviewers had some concerns regarding novelty compared with previous work and quality of experiments and evaluations. The authors answered most of these concerns in their rebuttal including discussion with previous work and additional evaluations.  As a result, the paper would be interesting for the ICLR audience.
Decision: Accept (Poster)