Generator Born from Classifier
Authors: Runpeng Yu
Xinchao Wang
Venue: NeurIPS 2023 poster (2023)
Keywords: Generative Model

Abstract: In this paper, we make a bold attempt toward an ambitious task: given a pre-trained classifier, we aim to reconstruct an image generator, without relying on any data samples. From a black-box perspective, this challenge seems intractable, since it inevitably involves identifying the inverse function for a classifier, which is, by nature, an information extraction process. As such, we resort to leveraging the knowledge encapsulated within the parameters of the neural network. Grounded on the theory of Maximum-Margin Bias of gradient descent, we propose a novel learning paradigm, in which the generator is trained to ensure that the convergence conditions of the network parameters are satisfied over the generated distribution of the samples. Empirical validation from various image generation tasks substantiates the efficacy of our strategy.

Generator Born from Classifier
Runpeng Yu
Xinchao Wang
National University of Singapore yu@u. mus.edu xinchaodnus edu. ag
Abstract
In this paper, we make a bold attempt toward an ambitious task: given pre-trained classifier, we aim to reconstruct an image generator, without relying on any data samples. From a black-box perspective, this challenge seems intractable, since it inevitably involves identifying the inverse function for a classifier, which is, by nature, an information extraction process. As such, we resort to leveraging the knowledge encapsulated within the parameters of the neural network. Grounded the theory of Maximum-Man Margnn Bias of gradient descent, learning paradigm, in which the generator is trained to ensure that the convergence conditions of the network parameters satisfied over the generated distribution of the samples. Empirical validation from various image generation tasks substantiates the efficacy of our strategy.
propose a novel
1 Introduction
The majority of machine learning research bifurcates into two distinct branches of study: the predictive task and the generative task. Given the input and the label y, the former one focuses the training of a nigh-performing classifier or regressor, which approximates p(y) x)) [Vaswani et al., 2023, Dosovitskiy al., 2021, Jing et al., 2023], whereas the latter one aims train generative model capable of sampling from p( (y) or p(x, y) [Goodfellow et al., 2014]. The gap between the predictive and the generative models, predictive models about the marginal distribution p(æ). In the realm of deep neural networks, however, the preeraaraettiiiitionn leads the overfitting on the training distribution and the memorization of the training samples [Feldman and Zhang, 2020, Daniely, 2020, Arpit et al., 2017], which, in turn, make the network implicitly retain information about p(æ).. With this component in hand, it prompts the question of whether it is feasible to derive generative model from a predictive one.
result, predominantly arises from the lack of information in the
In this paper, explore this novel task, which attempts learn a generator directly from a pre- trained classifier, without the assistance of any training data. Unarguably, this a highly ambitious task with substantial difficulty, either explicitly extracting information about p(æ) from a pre- trained classifier or directly solving this inverse problem from classifier to generator poses significant challenges. Despite these challenges, the value of this task lies in its potential to offer to training generators that mitigates the direct dependence on large volumes of training data. This provides possible solution for learning tasks in scenarios where data is scarce unavailable. Moreover, this task presents a novel way to utilize and analyze the pre-trained predictive models, facilitating our understanding of the encoded information within the parameters.
approach
To this end, we propose a novel learning scheme. Our approach is grounded in the theory of Maaimum--Mrrgnn Bias of gradient descent, which demonstrates that the parameters of neural network trained via gradient descent will converge to the solution of a specific optimization problem. This optimization problem minimizes the norm of the neural network parameters while maximizing the classification margin on the training dataset. The necessary condition for the solution of this
Corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

0
1
2
3
7
9
01
1
2
3
7
9
(a) MNIST
(b) CelebA
Figure I: Images produced by the generator trained only using pre-trained classifier. The generated images, positioned in the first row, accompanied by their nearest neighbors from the original dataset, displayed in the second row.
optimization problem constructs system of equations, describing the relationship between the pre-trained neural network parameters and the training data distribution.
Since aim is to learn generator from the pre-trained classifier parameters, the generator therefore expected to approximate a distribution that satisfies the necessary condition of this optimization problem. To accomplish this, we design the loss function for training the generator and the corresponding training algorithm based on the necessary condition. The entire training process does not rely on any training data; all available information related the pre-trained data is encapsulated within the parameters of the pre-trained classifier.
The intuition behind our design is twofold: pre-trained classifier performs well under the data distribution it approximated, and on the other hand, the generator should ensure that the current classifier parameters are the convergence point of the gradient descent algorithm under the data distribution it approximated. It's noteworthy that the original data distribution naturally satisfies these conditions. Therefore, anticipate that employing the proposed method will guide the generator to discover the original data distribution.
hand, the generator should guarantee that the
We conduct experiments on commonly used image datasets. Fig. 1 shows some generated images of MNIST and CelebA datasets. Remarkably, even trained without access to the original data, the generator is able to perform conditional sampling and generate the digits and faces.
Our contribution is therefore novel approach that, for the first time, attempts to train generator from a pre-trained classifier without utilizing training data. The proposed approach produces encouraging
results on synthetic and real-world images.
list of symbols utilized this paper and corresponding descriptions can be found in the Appendix.
2 Related Work
Generative Adversarial Networks. Generative Adversarial Network (GAN) consists of a generator and a discriminator collectively optimizing a minimax problem to learn and replicate the original data distribution [Goodfellow et al., 2014]. Numerous extensions of the original GAN have been investigated, including functionality enhancement [Chen et al., 2016, Mirza and Osindero, 2014, Odena et al., 2017, Donahue et al., 2017]; architecture optimization and scaling [Karras et al., 2018, Brock et al., 2019, Denton et al., 2015]; and training loss design [Nowozin et al., 2016, Arjovsky et al., 2017, Wei et al., 2018). Owing their superior generation quality, GANs have found wide-ranging applications image synthesis [Yang et al., 2017b], blending [Wu et al., 2017], inpainting [Yeh et al., 2017, Yang et al., 2017a, Yu et al., 2018], super-resolution [Ledig al., 2017], denoising [Linh et al., 2020]; image-to-image translation [Zhu et al., 2017, Isola et al., 2017]; 3D object generation [Wu al., 2016]; video generation [Vondrick al., 2016, Tulyakov al., 2018]; etc.
Both our work and GAN require an additional classifier to guide the training of the generation and provide measure of authenticity for generated data. However, in the GAN framework, the classifier is trained concurrently with the generator, with the explicit goal of discerning the quality of generated results. In contrast, method utilizes pre-trained classifier, which can be arbitrary, and its training objective is to maximize classification accuracy, not to judge the quality of generated results. This imbues our approach with considerable flexibility. Furthermore, during the training of GAN, the generator has access to training data. However, in our task, the training data is not available to the generator, which makes our task harder.
Feature Visualization and Model Inversion. Besides our task, neural network feature visualization and model inversion share the objective of extracting information relevant to the training data from

---

pre- tre-trained classifiers. Neural network feature visualization is technique aimed at identifying imput data that maximally activates specific neurons or layers within the network, thereby providing insights into the patterns or features that the network is primed to recognize. [Engstrom et al., 2019, Olah et al., 2017, Nguyen et al., 2016] Model inversion in neural networks refers to the process of inferring or reconstructing input data given the trained model. [Yin et al., 2020] In the realm of adversarial attacks, model inversion is deployed to discover sensitive information about the training data from the model's 's outputs. [He et al., 2019, Zhao et al., 2021, Fredrikson et al., 2015]
Unlike these tasks, where an independent gradient optimization is required for each generation, our goal is to develop generator capable of sampling from the training data distribution. While there is research in neural network feature visualization and model inversion that utilizes a generative model to assist in the restoration of specific training data, these works typically employ the generator more as a prior for the recovery process. [Nguyen et al., 2017, Jeon et al., 2021, Yang et al., 2019, Zhang et al., 2020] The training of such generator necessitates additional training data, which should be similar to or encompass the original training data used for the classifier. In contrast, our approach directly derives generator from the classifier without the utilization of extraneous training data.
Energy-Based Model. Within the framework of energy-based models, pre-trained classifiers have also been demonstrated to be capable of acting as a conditional probability distribution, generating data samples. [LeCun et al., 2006, Grathwohl et al., 2020, Guo et al., 2023] Although the energy- based models theoretically bridge predictive and generative models, practical implementation for sample generation based on it still relies on an optimization target and executes a multi-step gradient optimization process, akin to model inversion. In contrast, our objective is to develop generator with the ability for random or condition- -aased sampling, procuring data samples directly through the forward pass of the neural network.
Maximum-Margin Bias of Gradient Descent. Our work is based on the study of the Maximum- Margin Bias of gradient descent. These investigations primarily seek to elucidate why gradient descent algorithms are capable of learning models with robust generalization capabilities, even in the absence of additional regularization constraints. It has been discovered that under the guidance of gradient descent, the parameters of a neural network converge to solution to an optimization problem aiming to maximize the classification boundary while concurrently minimizing the norm of the network parameters. Initial studies focused on linear logistic regression models [Rosset et al., 2003, Soudry et al., 2018], which were subsequently extended to homogeneous neural networks [Wei et al., 2019, Xu et al., 2018, Nacson et al., 2019, Lyu and Li, 2020b, Ji and Telgarsky, 2020, Le and Jegelka, 2022] and, more recently, to broader class of quasi-homogeneous neural networks [Kunin et al., 2023]. The theory Maximum- MMrriin Bias has also been leveraged by Haim et al. [2022] to recover training data. However, their objective was the restoration of data rather than training a generator, and their work was exclusively confined to binary classífication datasets and fully connected networks without bias terms.
3 Preliminary
To extract the information about the training dataset from the parameters of the pre-trained classifica- tion model, we leverage the theory of Maximum-Margih Bias.
Let (t: () Rd - y denote the classifier parameterized by Ç and trained on multi-class classification dataset D = ( J with each (æx, v:) E Rd x y. let L(C)= : ΣN,, ((b(a,; C), 9:) denote the standard cross-entropy loss of & on D. To extend the applicability of Maximum-Margin Bias analysis to various neural network structures, the definition of the A l-quasi-homogenece
model is introduced. For a (non-zero) positive semi-definite diagonal matrix A, A, a model (-:; () is A--uasi-homogennen the model output scales as ((x; (C)) = ((x; <) for all a R and input æ, when the parameter scales as (C) := aa Ç. Many commonly used neural network architectures satisfy the definition of --qussi-homogeneouu model, such as convolutional neural networks and fully connected networks with biases, residual connections, and normalization layers. The seminorm of Ç corresponding to is defined as ||<] == AÇ. Let Amax := max, Aij denote the maximal element in A, A is the matrix setting all elements less than Amaz to 0, i.e., Ajz = Amaz if Anj Amaa. otherwise, ر O. Accordingly, the seminorm of Ç corresponding to A is defined as ||<|1 := ÇTAG.. The
3

---

normalized parameters is defined as <= ()((),, such that 13l1 = 1. The Quasi-Homogeneous Maximum- MMraiin Theorem states as follows.
Theorem 1 (Paraphrased from [Kunin al., 2023]). Let ((:;C) denote a A-quasi-tomoggeeouus
classifier trained on D with cross-ent rass-entropy loss L. Assume that: for any fixed æ. ((x; () is locally
Lipschitz and admits a chain rule [Davis et al., 2020, Lyu and Li, 2020a]: J: (2) the learning dynamic
described by a gradient flow [Lyu and Li, 2020a]: (3) lime- {(t) exists; (4) 3k > 0 such that only
with A z K. separates the training data; and (5) 3ig such that L(((fo)) N log 2. 3a R.
such that := tha (lime {()) a first-order stationary point of the following maximum-marge
problem
min <'
(1a)
s.
min py (2x: $c(xsi() Vie [M] <e)/(an)
(1b)
where PolciC") is the prediction of 4 for the class c V
Method
Theorem I implies that the neural network parameters converge to the first-order stationary point (or the Karush-Kuhn- point (KKT) point) of the optimization problem in Eq. (1). Let (M)c)e)))).)))/1 denote the set of KKT multipliers, the KKT condition can be written as follows.
NC
Σ
Σ
Pio (.
V (1: {]]
(2a)
[e[N] cEM/HUL
for all [NV] and C D/(y1)
By, (z & Fc > 1,
(2b)
Min 7 0,
(2c)
Miel1 ba by (I;; =0
(2d)
where the Eqs. (2a) (2d) known as the stationarity condition, primal feasibility condition, dual feasibility condition, and the complementary slackness condition, respectively.
The intuition behind the stationarity condition be summarized as follows. According to the value of the corresponding element in A, the neural network parameters can be divided into two groups: the first group Z₁ == {šIlA; = Amar) includes all the parameters whose corresponding elements in A are equal to Amaz. and the second set Ze := {čylAІo # Amar) contains all the parameters whose corresponding elements in are not equal to Amars The stationarity condition in Eq. (2a) evaluates the linear combination of the derivatives of the neural network output corresponding to the parameters, with KKT multipliers (jSs))ee[V).oe (N) as coefficients of the combination. Such a linear combination of the derivatives corresponding the parameters in Zı is equal to the parameters in Z, In contrast, such a linear combination of the derivatives corresponding to the parameters in Zz is equal to zero vector.
The intuition behind other conditions can be summarized as follows. According the complementary slackness condition in Eq. (2d), pie nonzero only when 2,, (zz; () 4 (x () 1. Two conditions are required for Pic to be nonzero. First, for a pair of (i,c), Mie can only be nonzero when c the class with the second largest predicted probability for the sample Fi, i.e., c E S,, where S, = {y) Bo
max (ay; ()). Second, according to primal feasibility in Eq. (2b), the minimum possible value of p ?) (a ?) is 1. Therefore, for pair of (i, c), Hic can only be nonzero when the margin between the true class ys and the class with the second largest predicted probability for the sample is minimum.
The evaluation of V ( () and ( requires to first scale all parameters of a pre-trained network. For the convenience of practical implementation and following derivation, we transform Č back to using the definition of the quasi-homonn homaieihomoeneouss function. The KKT conditions in Eq. (2) are rewritten

---

as:
Aç Σ Σ Mic V (c(()))) v.
(3a)
ie[N] oE)/(u))
for all i [N]] and ce 1/(1it:
Py, (z): 22 () 2 e e
(3b)
Mic 20 O, Micle $c( () yy (3 {)]
(3c)
(3d)
where the new scaling parameter A == (2A
From KKT condition to loss function. Given only the pre-trained neural network Φ(-: (), the undetermined parts in the KKT conditions in Eq. (3) include the KKT multipliers, a set of (xr y) pairs, and constant a. Regarding the KKT multiplier predictable objective, We use neu- ral network to learn it. Given an approximate distribution of the discrete random variable y. we sample y directly. Our goal is to train conditional generator parameterized by 0 to generate input x = g(, y; 0) given the corre- sponding label and random noise We also treat a as a learnable parameter, which will be discussed in detail later. In the following para- graphs, we first discuss how to design the loss function for learning these parameters and training the generator.
0.4
0.1
1,6=03
0.3
controls the positioo
1(2,0, 0)
0.2
controls the wieth
0.1
0.0
0.0
0.4
0.6
1.0
Figure 2: The U-shaped duality loss 1(x; az, 6) :=
max (x (x-e 6, 0) min (x-e-n .0).
First, we discuss how to ensure that the generated samples satisfy the stationary condition in Eq. (3a). The evaluation of the right-hand side of Eq. (3a) requires generating dataset with fixed number of samples every time to calculate the summation. Alternatively, we divide both sides Eq. (3a) by the number of training samples N, which converts the sum on the right-hand side to an expectation:
Aç E. N
Σ (c (E)/(8)
Thus, according to the law large numbers, the expectation can be estimated by the empirical average over batch of M random samples, where M is a hyper-parameter. To ensure that the rescaled stationary condition Eq. (4) can be satisfied, we use the following Isstationarity to minimize the norm of the difference between both sides of Eq. (4).
Isstationaiity (0, =- 11 Aç
1
Σ ME[M] cE)/(us)
Σ
Mie V T on V D ( (1::
(5)
M
Conditions in Eqs. (3b) to (3d) constrain the valid values of KKT multipliers. Accordingly, in order to satisfy the positivity condition of the KKT multipliers in dual feasibility, instead of directly optimizing the KKT multipliers, we use the proxy variables /1/-- define jiic Re ())() For each generated sample z,, we set up μ E R! with p/e is its c-th element. We learn 14द = h(x,, Mi; 77) by network h parameterized by n7. To approximate primal feasibility and complementary slackness, we require each generated sample x; to satisfy & 4 (x+: - (e(xx;) - e em & ó, where 0 < 8 & 1 is hyper-parameter to ensure numerical stability. We minimize Lduality approximate this constraint. Fig. 2 illustrates the shape of Lavality-
(l,alit((,.)))
Σ xe[M] c
Σ
max (4y, (Z Fc(Ix: ()) - e 6.0)
M
min (x +c(1x; - -000) (6)
5

---

Final loss is the combination of Llogrange and diuntity balanced by hyper-parametee 3::
100
Line Style Grownd Trath 0 4
L Listationarityl (0. + Plduaaiiy (O,
50
Estimntion (
Value
Determine A. To compute Lstationarity. it necessary to determine the A of the quasi- homogeneous function (): Here, we will introduce two properties of quuge--hoooeenouu functions and demonstrate how to construct a sys- tem of linear equations based on these properties to efficiently solve for A. First, taking the deriva- tive of & (S)) corresponding to Ç, have
Different
-100
01
5
10
15 20
25
Figure 3: The plot of ((a;; e () and (a (), with Å as the estimation of A. Color distinction signifies diverse input æ. The coincidence of solid and dotted lines validates the precise esti- mation of A, because, by definition, an accurate estimation fulfills (xx; ea () e <' ().
V (C) (x;; Va(()) = ca(I-Ai pa(1-A)V - + x ().
(8)
This indicates that, for any parameter Ç whose
corresponding Aa * 1, the derivative of ob ((x;; () corresponding to Çi denoted by Vc, (a:; (),
A' -qquaii-homogeeouus function with A' 1-A,;
A. Second, taking the derivative V. ((;; (a())
0, we have:
(x:
(derivative equation of d at x))
(9)
For clear notation, we refer to the above Eq. (9) as the derivative equation of & at æ. which is a linear equation about and the coefficients can be calculated conveniently given the pre-trained model.
As these properties of quasi-hom homogenouss function are independent of input æ, we can establish a system of equations by evaluating the derivative equation of Φ and its higher order derivatives at set of random samples (e egg.. e.g.
T (amn: AV ((22; ()) ((x);(),
(
(derivative equation of O m))
(derivative equation of + 2:2)
(10)
An 5" AV () Va, (( C) (derivative equation of Vc, & at wx)
Then, the A can be calculated from it. As a proof of concept, the estimated results of two-layer fully-connected network with ReLU activation are shown in Fig. 3. The architecture of the network is [2nnear(2, 10)
Re - [Linear(10, 1))
To compute Lowatity is necessary to determine the a. Shown in the proof of Theorem [Kunin et al., 2023], the value of a depends on the minimum classification margin Jwin of the (normalized) neural network on the training dataset, the case of a multi-class problem:
Determine a.
Jmnn
min min ie[M] [e[N]cee[C]] ce(C)/(ur)
(Py 2) ( {)
(11)
Therefore, the value of depends the entire training dataset, which is not accessible. To avoid the accumulation of errors in estimating Jmin using generated samples during training, we directly optimize trainable parameter.
Summary of the method. short, the optimization problem used to train the generator g(:::0) can be written as:
min L(O, 11, o) min Isstationaiity (0, + B (0, a). 8.1.a
(12)
0.7.a
The training procedure is summarized in Alg. I in the Appendix.
An extension to multiple classifiers. We have previously discussed a method using a single classifier to train generator. Here, present extension of our approach employ multiple classifiers for training a single generator. Given T classifiers (0+1)) encode the training

---

10
90
60
7
180
0
180
8.0
0
180
0.0
0.5
3.5
270
270
(a) Training Data
(b) Prediction Landscape
Generated Data
Figure 4: An illustrative 2D example showcases proposed method. Figs. 4a to 4c are the training data employed for the classifier, the classifier's learned prediction landscape, and the generated samples using the learned generator, respectively.
data information from distinct classifiers into one generator by incorporating the classifier index t input of the generator. By providing random noise c, label y, and classifier index t as inputs, x is generated as x = ( y, 8). During the training process, we compute the loss for each classifier, (0, ny a (1) and aggregate them as the final optimization objective. For each classifier, we optimized a distinct a and use (*) to denote the a for the t-th classifier. We utilize a unified network Rd x V [T] - R! to compute the KKT multipliers, which also includes the classifier index input. The optimization objective for the training process can be formulated as:
min 0. (E
Σ [E[T]
L n. ()).
(13)
Upon completion of training, generating samples requires additional step sample suitable classifier index. Let Ty denote the set of classifier indices, where the indices in the set correspond to classifiers whose training set contains samples with label y. To generate sample æ with label y, first sample a classifier index from Ty, and then utilize I, along with and random noise as inputs to the generator to produce the sample.
The algorithm for training a generator using multiple classifiers is summarized in Alg. 2 in the Appendix.
5 Experiments
5.1 An Example of 2D Synthetic Data
In this subsection, we employ two-dimensional example to analyze the proposed approach. We evenly generate 18 data points on unit circle, and categorize them into three classes to create our training dataset D. The distribution of D is plotted in Fig. 4a, with distinct colors representing different categories. Utilizing this data, we train a three-layer fully connected network & as the classifier. The prediction landscape of D is displayed in Fig. 4b. The color of the region represents the category predicted by the classifier for samples within that region. The white areas represent classifier's decision boundary. Given + we train generator g using the proposed method. The samples generated by y are plotted in Fig. 4c. Despite the presence of certain noise, the distribution of the generated data aligns consistently with that of the classifier's 's training data.
Leveraging this two-dimensional example, we further analyze our proposed method of training a generator using multiple classifiers. We evenly split the previous training dataset into D, and Dz. plotted in Figs. 5a and 5f, respectively. Using these two subsets of data, we train two classifiers, Φ and 22 and subsequently train two generators, 91 and 92- Figs. 5c and 5h display the data generated by 91 and g2, respectively. Here, during the training of we solely use Φ1, and during the training 92, we solely use Φ2. Obviously in Figs. 4b and 5g, owing to the generalization ability of neural networks, & and $ learn additional categorization capabilities beyond the training data, with prediction areas with the same color surpassing the region covered by the training data. Contrarily, the samples generated by training 91 and g2 congregate around the actual data points, effectively
7

---

(a)
(b)
(c)
(e)
(d)
(f)
(h)
Figure 5: An illustrative 2D example showcases the process of using two pre-trained classifiers to train a generator. Figs. 5a to 5c and Figs. 5f to 5h are two groups of training data, classifier's learned prediction landscape, and generator's 's generated samples. Fig. 5d shows the generated samples of the generator trained using two classifiers. Figs. 5e and Si are the generated samples of the generator in Fig. 5d with fixed classifier index.
3
3
3
4
4
I
2
2
2
3
3
3
4
4
4 9
6
7
7
7
f
q
9
5 5 5
6
6
6
7
7
7
8
8
8
9
9
9
Figure 6: cennerator-producee samples. the MNIST dataset.
The generator is trained using a single classifier trained on
recovering the original distribution of the training data. Subsequently, by employing the extension of the proposed method, train generator 91+2 using Φ and Φ together. As shown in Fig. 5d, 91-2 possesses the generative capabilities of both 91 and 92, capable of generating samples on the entire training data D. We further fix the classifier index of 9142 either 1 or 2, randomly sample noise and category labels, and observe the data generated by 4142. As depicted in Figs. Se and 5h, 41 is also capable of independently generating data belonging to either D D2.
5.2 Image Generation
In this subsection, we showcase the experimental results on the MNIST [Lecun et al., 1998] and CelebA [Liu et al., 2015] datasets. More results and implementation details are left in the appendix. For the MNIST dataset, we set up a classification task corresponding to the digits 0-9 with 500 training data (50 images per class) randomly sampled from the original training set. For the CelebA dataset, we utilized various binary attributes to construct binary classification tasks on facial images, for example, distinguishing between males and females. For each task, we randomly sampled 100 images (50 images per class) from the original training dataset and resize them to 32x32 to be our training dataset.
We employed three-layer fully-connected network with a ReLU activation function and batch normalization as the classifier for the aforementioned classification tasks. The networks were trained until the classification loss converges using full batch gradient descent, which ensures the parameters close to the convergence point required in the theory of Maximum- Margin Bias.

---

(a) Gender
(b) Mouth Opened
(c) Pale Skin
Figure 7: Generator-produu samples. The generator trained using a single classifier trained on the CelebA dataset. The captions of the subtigures indicate the attributes used as the label.
3
3
2
7
2
3
3
3
4
4
4
7
7
7
8
8
9
9
9
S
S 5
6
6
6
7
7
7
8
8
8
9
9
9
Figure 8: Generator cenerator-produced samples. MNIST dataset.
The generator is trained using two classifiers trained on the
We use generators composed of three fully cully-connected layers followed by three transposed convolution layers, with ReLU activation function and batch normalization. Network parameters were initialized using Kaiming initialization [He al., 2015] and trained for 50, 000 epochs. The batch size and learning rate were set as hyperparameters and optimized via random search. In order to control the noise in generated images, we also utilized total variation loss in pixel space as regularization term.
The generated results on MNIST and CelebA are shown in Figs. and 7. Odd rows present the generated images, and even rows present the images from the training dataset that are closest the generated images. The distance between images is measured by the SSIM metric. As shown by the results, the generator trained using our method is capable of generating digits and facial images, even though it has never been exposed to images of digits or faces.
To validate the extended method proposed for multiple classifiers, partitioned the aforemen- tioned digit classification dataset into two subsets including digits 0 - 4 and digits - 9, respectively, and trained two classifiers separately. We then employed our method train a single generator using both classifiers. Fig. showcases the final images generated. The trained generator successfully integrates information from both classifiers, being capable of generating all digits from 0-9.
6 Conclusion
In this research, we investigate pioneering task: training a generator directly utilizing a pre- trained classifier, devoid of training data. Based on the maximum margin bias theory, we present the relationship between pre-trained neural network parameters and the training data distribution. Consequently, we devise an innovative loss function enable the generator's ''s training. Essentially, our loss function requires the generator to guarantee the optimality of the parameters of the pre-trained classifier under its generated data distribution. From a broader perspective, the reuse and revision of pre-trained neural networks have been a widely studied direction. [Ma et al., 2023, Fang et al., 2023a,b, al., 2023, Yang al., 2022a,b] Our method offers a novel direction for leveraging pre-trained models.
Acknowledgment
This project is supported by the National Research Foundation, Singapore under its Al Singapore Programme (AISG Award No: AISG2-RP-2021-022 and the Singapore Ministry of Education Academic Research Fund Tier (WBS: A0009440-01-00),

---

References
Martin Arjovsky, Soumith Chintala, and Lćon Bottou, Wasserstein generative adversarial networks. In International ConfreetonnCCoffeeeeeee on Machine Learning (ICML), 2017. 2
Devansh Arpit, Stanislaw Justrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj. Asja Fischer, Aaron C. Courville. Yoshua Bengio, and Simon Lacoste-Julien. A closer look at memorization deep networks. In International Conference on Machine Learning (ICML), 2017.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference Learning Representations (ICLR), 2019.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Conference on Neural Information Processing Systems (NeurIPS), 2016. 2
Amit Daniely. Neural networks learning and memorization with (almost) no over-parameterization. In Conference on Neural Information Processing Systems (NeurlPS), 2020.
Damek Davis, Dmitriy Drusvyatskiy, Sham M. Kakade, and Jason D. Lee. Stochastic subgradient method converges tame functions. Found. Comput. Math., 20(1):119 154, 2020. 4
Emily L. Denton, Soumith Chintala, Arthur Szlam, and Rob Fergus, Deep generative image models using laplacian pyramid of adversarial networks. In Conference Neural Information Processing Systems (NeuriPS), 2015. 2
Jeff Donahue. Philipp Krähenbühl, and Trevor Darrell. Adversarial feature learning. In International Conference on Learning Representations (ICLR), 2017. 2
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition scale. In International Conference on Learning Representations (ICLR), 2021.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Aleksander Madry. Adversarial robustness as prior for learned representations, 2019.
Gongfan Fang, Xinyin Ma. Mingli Song, Michael Bi Mi, and Xinchao Wang. Depgraph: Towards any structural pruning. In IEEE/ / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023a. 9
Gongfan Fang, Xinyin Ma. and Xinchao Wang. Structural pruning for diffusion models. In Conference on Neural Information Processing Systems (NeurlPS), 2023b. 9
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. In Conference Neural Information Processing Systems (NeurlPS), 2020. 1
Matt Fredrikson, Somesh Jha, and Thomas Ristempart. Model inversion attacks that exploit confidence in- formation and basic countermeasures. In Conference on Computer and Communications Security, 2015.
Ian J. Goodfellow, Jean Pouget-Abadic Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In Conference on Neural Information Processing Systems (NeuriPS), 2014. 1,2
Will Grathwohl, Kuan- Chich Wang.
Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier secretly energy based model and you should treat like one. In International Conference on Learning Representations (ICLR), 2020. 3
Qiushan Guo, Chuofan Ma, Yi Jiang, Zehuan Yuan, Yizhou Yu, and Ping Luo. Ege: Image generation and classification via single energy-based model. arXiv,, 2023. 3
Niv Haim, Gal Vardi, Gilad Ychudai, Ohad Shamir, and Michal Irani. Reconstructing training data from trained neural networks. In Conference Neural Information Processing Systems (NeurIPS), 2022. 3
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level
performance imagenet classification. 2015. 9
Zecheng He, Tianwei Zhang, and Ruby B. Lee. Model inversion attacks against collaborative inference. In David Balenson, editor, Proceedings of the 35th Annual Computer Security Applications Conference, ACSAC, 2019. 3
10

---

Phillip Isola. Jun- Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to-image translation with conditional adversarial networks. In IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR). 2017.
Jinwoo Jeon, Jaechang Kim, Kangwook Lee, Sewoong Oh, and Jungseul Ok. Gradient inversion with generative image prior. In Conference on Neural Information Processing Systems (NeurIPS), 2021.
Ziwei and Matus Telgarsky. Directional convergence and alignment deep learning. Conference on Neural Information Processing Systems (NeurlPS), 2020, 3
Yongcheng Jing, Chongbin Yuan, Ju, Yiding Yang, Xinchao Wang, and Dacheng Tao. Deep graph reprogram- ming. In CVPR, 2023.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In International Conference on Learning Representations (ICLR), 2018.
Daniel Kunin, Atsushi Yamamura, Chao Ma, and Surya Ganguli. The asymmetric maximum margin bias of humogeoomogsssuss neural networks. International Conference Learning Representations (ICLR), 2023. 3, 6
Thien Le and Stefanie Jegelka. Training invariances and the low-rank phenomenon: beyond linear networks. In International Conference on Learning Representations (ICLR), 2022.
Lecun, L. Bottou, Bengio, and P. Haffner. Gradient-basee learning applied to document recognition. Proceedings of the IEEE, 1998. 8
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. A tutorial on energy- based learning. Predicting structured data, 100, 2006. 3
Christian I Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew P. Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo-realistic single image super- resolution using generative adversarial network. In IEEE/ CVF Computer Vision and Pattern Recognition Conference (CVPR), 2017.
Tran Duy Linh, Son Minh Nguyen, Masayuki Arai. Gan-based noise model for denoising real images. In Computer Vision ACCV 2020 15th Asian Conference on Computer Vision, Kyoto, Japan, November 30 December 4, 2020, Revised Selected Papers, Part IV, 2020. 2
Ziwci Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In International Conference on Computer Vision (ICCV), December 2015. 8
Kaifeng Lyu and Jian Li. Gradient descent maximizes the margin of homogeneous neural networks. In International Conference on Learning Representations (ICLR), 2020a. 4
Kaifeng Lyu and Jian Li. Gradient descent maximizes the margin of homogeneous neural networks. In International Conference on Learning Representations (ICLR), 2020b. 3
Xinyin Ma, Gongfan Fang, and Xinchao Wang. LLM-Pruner: the Structural Pruning of Large Language Models. In Conference on Neural Information Processing Systems (NewriPS), 2023. 9
Mchdi Mirza and Simon Osindero. Conditional generative adversarial nets. CoRR, 2014.
Mor Shpigel Nacson, Suriya Gunasekar, Jason D. Lee, Nathan Srebro, and Daniel Soudry. Lexicographic and depth-sensitive margins in homogeneous and non-homogeneous deep models. In International Conference on Machine Learning (ICML), 2019. 3
Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, and Jason Yosinski. Plug & play generative networks: Conditional iterative generation of images in latent space. In IEEE/ CVF Computer Vision and Pattern Recognition Conference (CVPR), 2017.
Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Multifaceted feature visualization: Uncovering the different types of features learned by each neuron deep neural networks. CoRR, 2016.
Sebastian Nowozin, Botond Cseke and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In Conference on Neural Information Processing Systems (NeurlPS),
2016.7 2
Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. In International Conference Machine Learning (ICML), 2017. 2

---

Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11):e7. 2017. 3
Saharon Rosset, Ji Zhu, and Trevor Hastie. Margin maximizing loss functions. In Conference on Neural Information Processing Systems (NeurlPS), 2003. 3
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasckar, and Nathan Srebro. The implicit bias of gradient descent on separable data. J. Mach. Learn. Res., 2018.
Sergey Tulyakov, Ming-Yis Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. In IEEE/ / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. 1
Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with seene dynamics. In Conference on Neural Information Processing Systems (NeuriPS), 2016. 2
Colin Wei, Jason D. Lee, Qiang Liu, and Tengyu Ma. Regularization matters: Generalization and optimization of neural nets v.s. their induced kernel. In Conference on Neural Information Processing Systems (NeurIPS), 2019. 3
Xiang Wei, Boqing Gong, Zixia Liu, Wei Lu, and Liqiang Wang. Improving the improved training of wasserstein gans: consistency term and its dual effect. International Conference on Learning Representations (ICLR), 2018. 2
Huikai Wu, Shuai Zheng, Junge Zhang, and Kaiqi Huang. GP-GAN: towards realistic high-resolution image blending, CoRR, 2017. 2
Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenhaum. Learning a probabilistic latent space of object shapes via 3d generative-aadveraaiall modeling. In Conference Neural Information Processing Systems (NeurlPS), 2016.
Tengyu Xu, Yi Zhou, Kaiyi Ji, and Yingbin Liang. Convergence of SGD in learning relu models with separable data. CoRR, 2018. 3
Chao Yang. Xin Lu, Zhe Lin, Shechtman, Oliver Wang, and Hao Li. High-resolution image inpainting using multi-scale neural patch synthesis. In IEEF/ / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2017a. 2
Jianwei Yang, Anitha Kannan, Dhruv Batra, and Devi Parikh. LR-GAN: layered recursive generative adversarial networks for image generation. In International Conference Learning Representations (ICLR), 2017b. 2
Xingyi Yang, Jingwen Ye, and Xinchao Wang. Factorizing knowledge in neural networks. In European Conference on Computer Vision (ECCV), 2022a. 9
Xingyi Yang, Daquan Zhou, Songhua Liu, Jingwen Ye, and Xinchao Wang. Deep model reassembly. In Conference on Neural Information Processing Systems (NewrIPS), 2022b.
Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. Neural network inversion adversarial setting via background knowledge alignment. Conference on Computer and Communications Security, CCS, 2019.
Raymond A. Yeh, Chen Chen, Teck- Yian Lim, Alexander G. Schwing, Mark Hasegawa-Johnnon, and Minh Do. Semantic image inpainting with deep generative models. In IEEE/ / CVF Computer Vision and Patter Recognition Conference (CVPR), 2017. 2
Hongxu Yin. Pavlo Molchanov, Jose M. Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem. Niraj K. Jha, and Jan Kautz. Dreaming to distill: Data-free knowledge transfer via deepinversion. In IEEE/ CVF Computer Vision and Pattern Recognition Conference (CVPR), 2020.
Jiahui Yu, Zhe Lin, Jimei Yang. Xiaohui Shen, Xin La, and Thomas S. Huang. Generative image inpainting with contextual attention. JEEE/ CVF Computer Vision and Pattern Recognition Conference (CVPR), 2018.
Runpeng Yu, Songhua Liu, Xingyi Yang, and Xinchao Wang. Distribution shift inversion for -uitt--distiiuution prediction. In IEEE/ / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023. 9
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei. Wenxiao Wang. Bo Li, and Dawn Song. The secret revealer: Gener- ative model-inversioon attacks against deep neural networks. In IEEE/ / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2020. 3
12

---

Xuejun Zhao, Wencan Zhang, Xiaokui Xiao, and Brian Y. Lim. Exploiting explanations for model inversion attacks. In International Conference on Computer Vision (ICCV), 2021.
Jun-Yan Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image translation using
cycle- cycssscon adversarial networks. In International Conference Computer Vision (ICCV), 2017. 2
13


Reviews:
[{"review_id": "D7jyYKMOLo", "reviewer": "Reviewer_gUqt", "summary": "The work extends the dataset reconstruction (DR) method [1] of reconstructing the dataset from a classifier to the generative scenario. It provides several extensions to it:\n- Instead of reconstructing particular data points, it aims to build a generator for the original dataset.\n- Extends to the multi-class setting (from a binary classifier). \n- Moves from homogeneous to $\\Lambda$-quasi-homogenous assumptions for the NN classifier.\n\nThe method is tested on MNIST and CelebA and is able to generate samples which resemble the real ones.\n\n[1] Haim et al. \"Reconstructing Training Data from Trained Neural Networks\"", "questions": "My two biggest concerns are the writing quality and the scalability of the method, but I'm unsure how they could be addressed during the short rebuttal period. Another interesting direction would be exploring adversarially robust classifiers [3] (I guess, adversarially robust training should prevent the landscape from covering spurious minima and hence help the generation quality).\n", "limitations": "Limitations are not discussed at all.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "- The overall idea of obtaining a generator from a classifier is very interesting and potentially very influential.\n- The work is genuine and open about its strengths and limitations. It feels like one of the rare works which is \"what you see is what you get\".\n- The method is quite novel and shows some promise to be working. I guess, the current ideas might be extendable to GAN training (or at least Auxilliary Classifier GAN training).\n- The source code was provided, which should improve the reproducibility.\n- Table 1 with the notations in the appendix was quite helpful.", "weaknesses": "- The paper is not self-sufficient and not possible to understand without reading the DR paper [1] first (at least, I failed to). It lacks intuitive exposition for the general audience who do not have vast background on the related work. Also, there are some confusing formulations on their own — e.g., Eq 5 and 6 define the loss term over $\\theta$ and $\\eta$, but they do not explicitly appear in the equation itself (only implicitly through $x_i$). And at the same, $x_i$ is not uniquely defined — it denotes both real and fake data. Also, sometimes it appears in bold (L123), and sometimes in normal font (e.g., Eq 1b). Also, Equation 12 is more or less a tautological repetition of Equation 7.\n- The experimental results do not seem to be strong.  E.g., checking the images in Figure 1b, they are hardly recognizable. While for [1], some restored images look quite reasonable, even considering it's CIFAR10 (a more difficult dataset than CelebA).\n- There should be a comparison with standard approaches of generating samples from a classifier (e.g., DeepDream or DeepDream from an adversarially robust classifier [3]). After checking some random DeepDream implementation on github [2] (probably untuned), it's hard to tell which samples are really better.\n- Limitations are not discussed\n\nTypos:\n- L181: \"objective, We\" => \"objective, we\"\n- L234: \"during the training\" => \"during training\"\n\n[2] https://github.com/ianscottknight/deep-dream-implementation-on-mnist-using-pytorch-hooks/blob/main/1.0-deep-dream.ipynb\n\n[3] https://arxiv.org/abs/1906.09453", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Scalability**\nWe thank the reviewer for pointing out the concern about the scalability of our method. To address the concern, we scale up the classifier and conduct the experiments. We consider two types of architectures: (1) a 10-layer fully-connected network and (2) a network with 3 convolutional layers and 3 fully-connected layers. The quantitative scores corresponding to different classifiers are shown in Tab. 2 in the attached PDF file. After scaling up the classifier the ISC increases. This indicates that the scale-up of the classifier helps to preserve more information about the training data and thus improves the performance of the generators.\n\n### **Writting**\nWe appreciate the reviewer's comment regarding the writing. We will enhance the clarity in the introduction and method sections by incorporating more intuitive explanations of our approach. The main reason for our current writing style is our desire to gradually derive our method from a theoretical standpoint, which necessitated the introduction of numerous mathematical symbols and concepts. In the revised version, we will provide more intuitive explanations for the mathematical symbols and analyses employed.\n\n### **Exploration of the Adversarially Robust Classifiers**\nWe are grateful for the reviewer's suggestion to explore the direction of adversarially robust classifiers. We have replaced the standard classifiers used in our method with adversarially robust classifiers and conducted experiments accordingly. Tab. 3 compares our method with an Empirical Risk Minimization (ERM) classifier, our method with adversarially robust classifiers, and the method mentioned in [3]. It is evident that our method, when combined with adversarially robust classifiers, indeed enhances the quality of the generated images. However, [3] outperforms our method with an ERM classifier. The reasons for this result are two-fold. First, [3] processes a different setting from ours. It optimizes each image individually, thus having a significantly larger optimization space than our method. We are aiming at learning the parameters of a generator and use it to generate all images. Second, [3] utilizes strong prior information, such as the mean and variance of the samples in each class. However, such prior is not used in our method.\n\n### **Self-Sufficiency and Prerequisite Knowledge**\nWe extend our sincere gratitude to the reviewer for advising us to enhance the self-sufficiency of our paper and to incorporate prerequisite knowledge. We will add a section in the supplementary material providing an intuitive introduction to the background and preliminary knowledge, thereby facilitating a more accessible understanding of our work for our readers.\n\n### **Further Explanation of Some Formulations**\nWe appreciate the reviewer's attention to detail in our manuscript. \n1. In Eq. 5 and 6, the generated samples $x_i=g(\\epsilon, y_i;\\theta)$ and the Lagrange multipliers $\\mu_ic=ReLU(h(x_i,y_i;\\eta)[c])$ are produced by neural network $g$ and $h$, respectively. $\\theta$ and $\\eta$ represent the parameters of these networks. To avoid overcomplicating the equations, we omitted the specific expressions for $x_i$ and $\\mu_ic$ in Eq. 5 and 6. However, we will provide their complete expressions in the revised version.\n\n2. To distinguish between real data samples and generated samples, we will employ different symbols. \n\n3. The symbol $x_i$ should always be in bold, and we will ensure this is consistently applied in the revised version.\n\n4. Eq. 7 and 12 represent the loss function used for training and the optimization problem used for training, respectively. In the revised version, we will integrate Eq. 7 and 12 into a single equation.\n\n### **Performance and Comparison with [1] Haim et al. 2022**\nWe greatly appreciate the reviewer's comparison between our work and [1] Haim et al. We kindly direct the reviewer to the global response, where we systematically compare our approach with that of [1] Haim et al.\n\nAdmittedly, given the highly ambitious nature of the proposed task and being the first attempt along this line, the performance of our approach, as expected, is indeed not surprising, since our training does not rely on any data and training a generator is inherently challenging. \n\n### **Comparison with DeepDream**\nWe thank the reviewer for comparing our work with DeepDream. We kindly direct the reviewer to the global response, where we systematically compare our approach with that of DeepDream.\n\n### **Limitations**\nWe appreciate the reviewer's suggestion about the limitation section, which we will incorporate into the revised version. Our method has two primary limitations. \n(1) Estimating Lambda introduces additional computational overhead. Given that our approach requires the classifier to be a quasi-homogeneous model, we need to determine the classifier's Lambda before training the generator. The method we provide in our paper for calculating Lambda can be computationally intensive, especially when the classifier has a large number of parameters.\n(2) Our method exhibits class bias, with significant variations in the generation quality for different categories. We hypothesize that there might be two reasons leading to this phenomenon. First, different categories may have varying complexities in their data distributions, making it more challenging for the generator to produce certain categories. Second, the generator might inherit the classifier's bias. If the classifier does not fit well for certain categories, the information about that category in the classifier might be limited or inaccurate. This, in turn, would affect the generator's performance on that particular category.\n\n### **Typos**\nWe are grateful to the reviewer for pointing out the typos in our manuscript. We will rectify all such errors in the revised version."}, {"review_id": "TJrx02Mffk", "reviewer": "Reviewer_KxBC", "summary": "The paper propose a method to generate image samples from a trained neural classifier. It proposes a loss to train an image generator where the loss is based on recent results from the realm of the implicit bias of gradient descent of quasi-homogenous functions. It shows empirical results on 2D data and on models trained on MNIST and CELEBA datasets.", "questions": "Evaluation is lacking:\n- the main claim of the paper is the proposed generator that is being born from a classifier. But there are no evaluation whatsoever of generative models. Many common metrics for evaluating generative models exists like FID or IS (for GM that models a classified datasets), but non of these are provided. \n- There are only very few visual examples in the paper, and no further analysis\n- The method proposes creating a generative model -  what is the point in comparing the outputs of the model to their nearest neighbors (Fig. 6,7,8)? (there are established methods for evaluating generative models, see above)\n- No comparison to baselines: basically, the results look like a lesser version of the Haim et al. results. What happens if the paper uses Haim et al. reconstruction loss on their trained models? The only concern may be of the multiclass models - such an extension is shown in Buzaglo et al. 2023, which is somewhat concurrent to this submission. However, the current submission can show a comparison of their proposed method to that of Haim et al. only on binary classifiers.\n- Some other baselines could be other works that try to \"turn\" a classifier to generator:\n1) Use Classifier as Generator, Li 2022\n 2) Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANs, Wang and Torr, 2022\n\n\nPaper is poorly written. Main parts of the paper are difficult to follow and understand:\n- The main contribution in the paper is the training of a generator (as assumed from the title). However, the discussion of the generator itself is somewhat hidden in lines 185-186. It took me a while to figure out that this is the so called generator that the paper discusses. I suggest emphasising this element much more. \n- the discussion of determininig $\\Lambda$ (in lines 209-229) - I can't say that I understood the derivation. why is $\\Lambda' = \\frac{1}{1-\\Lambda_{ii}}\\Lambda$? I could not understand the two properties. I also did not understand the plot showni in Fig. 3 \n- discussion in lines 201-205 - why is this network h and ReLU is necessary? this part is very cryptic and not clear (not the mechanism itself, nor the motivation or intuition behind this design choice).\n- What is the difference between Eq. (7) and Eq. (12) - it seems like the exact same equation\n- Why is the extension to multiple classifiers (lines 239seqq.) necessary? what is the motivation for that? I also could not understand the mechanism in this paragraph.\n\n\nMany important technical details are missing from the paper:\n- what is the architectures of the trained models? said to be quasi-homogenous\n- what is the architecture of the generator g?\n- what is the architecture of the network h? (in line 202)\n\n\nSome comments in introduction and related section are unclear or irrelevant:\n- \"training process does not rely on any training data\" (line 44 and 56) is a bit misleading - it relies on the data on which the classifier was trained. (also it seems from eq.4 that the number of training samples N is assumed to be known).\n- \"GAN require an additional classifier\" (line 71) - what is the meaning here? unconditional GANs do not require additional classifiers\n- \"the classifier is trained concurrently with the generator\" (line 73) - is this referring to the discriminator? because this is not the classifiers in the sense that are used in the paper. The current paper submission discuss \"usual\" classifiers (that solve the classification problem of a supervised classification dataset) and discriminators are classifying between true and fake samples.\n- References in line 85 - \"model inversion... given the output and the trained model\" - Yin et al. 2020 do not assume the outputs but rather infer training samples from the trained batchnorm statistics. Gal et al. 2022 is really irrelevant here (textual \"inversion\" tries to find a token in the input of the text encoder that corresponds to a given set of images. It has nothing to do with training samples reconstruction).\n\n\nMinor:\n- line 207 and supplementary 68 - L_{lagrange} should be L_{stationarity}?\n\n\n", "limitations": "n/a", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "- Extension of reconstruction to quasi-homogenous neural networks\n\n- Trying to create a generator by using a fixed prior on top of the reconstruction scheme", "weaknesses": "I think that the main weakness is the novelty in the paper. The proposed paper seems like an extension of Haim et al. 2022 to multiclass quasi-homogenous network, using a \"generative\" prior. However:\n- Note that a multiclass extension to Haim et al was done in Buzaglo et al. 2023 ICLR workshops (https://openreview.net/forum?id=SBstNm4OajH), which is somewhat concurrent). \n- the extension of the reconstructoin loss to quasi-homogenous networks is very slim - technically speaking, the only difference is the introduction of $\\Lambda$ in the loss. If this is truly necessary it should be supported by some comparisons to Haim et al. which also showed results on non-homogenous networks. Such comparisons are not provided in the paper.\n- The generative prior is interesting, but there is no evaluation in the paper for a generative model. All in all, the results are quite similar to those of Haim et al. only it seems that they were not fully converged (what is the difference between a \"bad\" reconstruction and a \"generated sample\"?)\n\nThe paper is poorly written - terminology is cumbersome and hard to follow, which makes it difficult to understand key components of the paper. Many important technical details are missing from the paper.\n\nSome comments in introduction and related section are unclear or irrelevant.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Comparison with [1] Haim et al. 2022 and [2] Buzaglo et al. 2023**\n\nWe appreciate the reviewer's comparison of our work with [1] and [2]. We kindly direct the reviewer to the global response, where we systematically compare ours with [1] and [2].\n\n#### **The importance of Lambda**\nLambda's introduction is vital for complex classifiers. While [1] mandates homogeneous networks, we allow quasi-homogeneous ones. This necessitates solving for Lambda in each network and integrating it into the loss of generator training. \n\n### **Evaluation** \n#### **Comparisons with [3] Li, 2022 and [4] Wang and Torr, 2022**\nWe appreciate the reviewer's suggestion to compare our method with those in [3] and [4]. We kindly direct the reviewer to the global response, where we systematically compare ours with [3] and [4]. \n\n#### **Quantitative Metrics**\nThanks. We compared the quantitative metrics of our method with [1] and [3] and reported in Tab. 1 in the submitted PDF for the results. \n\nDue to the fact that [4] did not release their source code (the provided GitHub link directs to a non-existent repository), we attempted to implement their work. However, given the tight timeframe for the rebuttal, we were unable to fully reproduce the results reported in their paper. As a result, we have not included the results from [4] at this time.\n\n#### **Further Analysis**\nWe are grateful for the reviewer's suggestion about further analysis. We conducted an attribute editing experiment and show the results in Fig. 1 in the attached PDF file. By manipulating the input noise, we generated continuously rotating digits, which are not included in the original dataset, akin to a generator model trained directly on the data.\n\n#### **The reason for comparing with the nearest neighbors**\nWe appreciate the question posed by the reviewer. Our comparison with nearest neighbors serves merely as a reference to illustrate the perceptual quality of the images generated by our method. By comparing with real data, it's evident that our generator produces images with good quality.\n\n### **Writing**\nWe greatly appreciate the reviewer's suggestions and will make modifications accordingly.\n\n1. We will highlight the sections about the generator.\n\n2. We will further clarify our discussion on solving for Lambda.\n* Solving for Lambda involves two steps: (1) constructing a system of linear equations about Lambda and (2) solving the system.\n* Two properties used to construct the system are introduced in Section 3 of \"The Asymmetric Maximum Margin Bias Of Quasi-Homogeneous Neural Networks\".\n* We will include the derivative of Lambda' in the revised version.\n* The purpose of Fig. 3 in the manuscript is to verify whether the Lambda obtained by our method accurately estimates the true Lambda of the network. According to the definition, given a quasi-homogeneous model $\\Phi$, we have $\\Phi(x;e^{\\alpha \\Lambda}\\zeta) = e^{\\alpha}\\Phi(x;\\zeta)$. If the estimated Lambda is correct, then substituting it into the left-hand side of the equation, the equation should still hold. In Fig 3, we plotted the values of the right-hand (solid line) and the left side of the equation (dotted line) with estimated Lambda for different $x$ and $a$. The overlap of the two lines indicates that our estimate is accurate.\n\n3. The introduction of $h$ and ReLU in lines 201-205 is for the estimation of the KKT multiplier $\\mu$. Given a pair of generated data $x$ and label $y$, $\\mu'=h(x,y;\\eta)$.\nSince the KKT multiplier must be non-negative, we let $\\mu=ReLU(\\mu')$.\n\n4. Eq. 7 and 12 represent the loss function used for training and the optimization problem used for training, respectively. In the revised version, we will consolidate Equations 7 and 12 into a single equation.\n\n5. The extension to multiple classifiers. \n* Integrating multiple classifiers for training a generator can lead to a more powerful generator. Firstly, different classifiers encounter different data (see the case in Fig 5 in our manuscript), and combining knowledge from multiple classifiers can enhance the diversity of generated samples. On the other hand, using multiple classifiers is also a means to improve the performance of the resulting generator. As shown in Tab 4 in the attached PDF, a generator obtained using two smaller classifiers performs better than one obtained using a lrger classifier. This is because the two classifiers each fit the data they encounter without interfering with each other, thereby preserving the information from the training dataset more effectively.\n* We provide an example of our mechanism: The input to the generator is (noise, label, classifier index). If there are two classifiers, trained on even and odd numbers from MNIST, respectively. To generate the first even digit '0', the input to the generator is (noise, 0, 0); to generate the second odd digit '3', the input is (noise, 1, 1).\n\n### **Technical Details** \nWe thank the reviewer. We indeed include the structure of the classifier and that of the generator in ln 291-294 and ln 295-299, respectively. Specific architectures of the classifier, generator, and net $h$ are shown in Fig. 2 in the submitted PDF. We will also release our code.\n\n### **Other Comments in Introduction and related work**\nWe appreciate the reviewer's detailed suggestions, and we will make modifications and add explanations accordingly.\n\n1. What we intend to convey here is that the training process of the generator does not require the use of data, but merely a pre-trained classifier and the number of training samples, which does not contain information about data distribution.\n\n2. In lines 71 and 73, the classifier we mention is the discriminator, as the discriminator is indeed a binary classifier. \n\n3. We will revise rigorously the discussion about model inversion in the related work based on your comments. \n\n### **Typos**\nWe thank the reviewer for pointing out our typos, and we will correct them.\n"}, {"review_id": "1K4oFxkhlQ", "reviewer": "Reviewer_5Ctm", "summary": "This paper addresses a novel and intriguing issue, namely, training a generator reliant on a pre-trained classifier, rather than extensive training data. The authors propose an innovative solution, fundamentally aimed at enabling the generator's training process to continuously extract and leverage information about the dataset distribution from the classifier's parameters. Based on the theory of Maximum-Margin Bias of gradient descent, the authors design stationarity loss and duality loss to ensure that the distribution of data generated by the trained generator guarantee the optimality of the pre-trained classifier. As an expansion, the paper also proposes an algorithm to collectively train a generator using information from multiple pre-trained classifiers. The proposed learning paradigm is empirically validated through experiments in various image generation tasks.", "questions": "1.\tLn 226 mentions that the construction of the system of equations depends on certain random samples. Are these random samples drawn from the original training dataset? If the solution for \\Lambda relies on samples from the training dataset, does it imply that the method proposed in the paper intrinsically presumes \\Lambda to be known?\n2.\tThe dataset used for training the classifier in the paper is relatively small, consisting of only 500 samples. What is the reasoning behind this setup? How would increasing the size of the training dataset impact the results?\n3.\tTypo\na.\tLn. 241, the set of classifiers is missing {}\n", "limitations": "Owing to the theory of Maximum-Margin Bias, the proposed method has limitations regarding the architecture of the pre-trained classifier and the pre-computation of \\Lambda. However, the authors have adequately addressed these challenges.\nThe paper also points out potential privacy risks that may arise from the application of the proposed method, as well as outlining some potential applications.\n", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "This paper investigates an intriguing and innovative task: training a generator using information from the parameters of pre-trained classifiers, rather than relying on training data. Given the vast number of pre-trained models our community has accumulated, I view this as a task of substantial value with potential impact.\n\nThe paper clearly outlines its motivation and problem definition, and offers a technically sound and concrete method, underpinned by the theory of Maximum-Margin Bias of gradient descent. The paper processes a clear logical structure and reasonable organization. The proposed method and the technical details is validated through several convincing proof-of-concept experiments, as shown in Fig. 3, 4 and 5. The experiments across multiple image generation tasks exhibit promising results.\n", "weaknesses": "Considering that this is a new attempt in this task, the experimental setup is understandably simple. Both the generator and classifier architecture utilized are elementary compared to the practice today. Besides this, I have some minor comments, which I have listed as questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **How the random samples are generated when calculating Lambda?**\nWe appreciate the reviewer's question. The definition of a quasi-homogeneous function is independent of the dataset; here, the random samples refer to random noise, not samples selected from the original dataset. For instance, if the classifier's input is an image of size 3xHxW, sampling random noise of size 3xHxW as the classifier's input is sufficient for determining Lambda. Therefore, the value of Lambda does not depend on the training dataset but solely on the pre-trained neural network. Given a neural network, Lambda can be determined using the method described in the 'Determine Lambda' section of the paper, so it does not need to be known in advance.\n\n### **Concern about the Number of Training Samples**\nWe appreciate the reviewer's inquiry about our experimental setup. Since the validity of the Maximum-Margin Bias theory depends on the convergence of the pre-trained network, we used a small dataset to ensure the classifier's convergence. We conducted experiments using larger datasets, and the results are presented in Tab. 5 in the attached PDF file. As the number of training data increases, it offers more information about the data distribution, leading to an enhancement in the quality of generated images.\n\n### **Typos**\nWe appreciate the identification of our typos, and we will correct them."}, {"review_id": "VxxSfdLwlC", "reviewer": "Reviewer_LZcP", "summary": "This paper tackles the problem of reconstructing an image generator, without relying on any data. The authors propose a learning paradigm in which the generator is trained to ensure that the convergence conditions of the network parameters are satisfied over the generated distribution of the samples.", "questions": "1. Why does the generator generate realistic-looking samples even though it could cheat by learning to generate adversarial examples?\n2. Are there any experimental validations/theoretical justifications to confirm if the model is not just reconstructing the training data examples?\n3. Can the generator capture all the modes of the training data distribution sufficiently well?", "limitations": "The authors haven't discussed the potential negative social impact of their work, in particular, these methods could be used to recover the training data which may cause privacy concerns.", "rating": 7, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The paper addresses an important and challenging problem.\n2. The method appears to be novel.", "weaknesses": "1. The evaluation seems to be weak. the authors have performed the qualitative evaluation alone. The experimental section seems to be incomplete without quantitative evaluation using metrics such as precision, recall, and FID.\n2. Related works on Data free knowledge distillation are missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### **Why does the generator generate realistic-looking samples even though it could cheat by learning to generate adversarial examples?**\nWe appreciate the reviewer's question. Indeed, our loss design inherently penalizes the generation of adversarial samples. Since our loss is derived from the KKT conditions of the optimization problem stated in Theorem 1, it mandates the training of a generator such that the samples it produces ensure the optimality of the classifier network parameters. An intuitive explanation is as follows:\n\nOur approach requires the generator to address this question: \"How should I generate data to ensure that when a classifier is trained using the data I generate, the resulting classifier parameters precisely match the given classifier's parameters?\" \n\nGiven that overparameterized neural networks tend to overfit their training dataset, the generated data distribution needs to closely resemble the real data distribution to guarantee the optimality of the classifier network parameters. As a result, our method is capable of producing realistic-looking samples that are close to the true training data distribution.\n\n### **Are there any experimental validations/theoretical justifications to confirm if the model is not just reconstructing the training data examples?**\nWe appreciate the reviewer's question. To demonstrate that we are not merely reconstructing the original data, we conducted an attribute editing experiment and show the results in Fig. 1 in the attached PDF file. By manipulating the input noise, we generated continuously rotating digits, which are not included in the original dataset. This experiment demonstrates that we can indeed generate samples not present in the original dataset, akin to a generator model trained directly on the data.\n\n### **Can the generator capture all the modes of the training data distribution sufficiently well?**\nWe appreciate the reviewer pointing out the issue with our method in terms of capturing all the modes equally. As shown in Fig. 4 in our manuscript, our method does not perform well in reconstructing certain modes (certain categories). We hypothesize that there might be two reasons leading to this phenomenon. First, different categories may have varying complexities in their data distributions, making it more challenging for the generator to produce certain categories. Second, the generator might inherit the classifier's bias. If the classifier does not fit well for certain categories, the information about that category in the classifier might be limited or inaccurate. This, in turn, would affect the generator's performance on that particular category.\n\n### **Evaluation**\nWe appreciate the reviewer's suggestion to include more evaluations. We have calculated the FID and the precision and recall. The results are shown in Tab. 1. \n\n### **Related Work**\nWe appreciate the reviewer's suggestion to include related work on Data-Free Knowledge Distillation. This will be incorporated into our revised version.\n\n### **Potential Social Impact**\nWe appreciate the reviewer pointing out the potential negative social impact of our method in terms of privacy leakage. We mentioned the risk our method poses to privacy protection in lines 319-320. In our revised version, we will further expand the discussion on this potential negative social impact."}]

Author rebuttal:
Esteemed Senior AC, AC, and Reviewers,

We deeply appreciate the reviewers' and ACs' dedication to reviewing and managing our submission. It is with great pleasure that the reviewers highlighted several strengths of our work, such as the importance and the potential impact of the idea/task/problem [gUqt, 5Ctm, and LZcP], the novelty of our method [gUqt and LZcP], the soundness of our method [5Ctm], the promise of our method to be working [gUqt] and the promising results [5Ctm]. 

**We have additionally uploaded a PDF file containing all supplementary tables and images for perusal.**

Thanks once again for the valuable comments and consideration.

Respectfully,

Authors of 'Generator Born from Classifier'

---

### **Comparison with Other Methods** 
Using the global response, we systematically draw comparisons between our approach and the methods mentioned by the reviewers.

We also provide Tab. 1 in the submitted PDF file as a summary of the comparison, which provides an overview of the differences and similarities among the methods and also includes the quantitative evaluation on the MNIST.

#### **Comparison with DeepDream**
We thank the reviewer gUqt for mentioning DeepDream. DeepDream aims to visualize patterns learned by neural networks, while our method aims to train a generator for the conditional sampling of images. Technically, to generate each image, DeepDream requires a gradient optimization process to maximize a certain activation or network output. By contrast, once our generator is trained, images can be produced by sampling random noise. Thus, (1) DeepDream has larger computational complexity and takes longer to generate each image compared to our method; (2) DeepDream has a larger parameter space. 

#### **Comparison with [1] Haim et al. 2022 and [2] Buzaglo et al. 2023** 
We thank the reviewer gUqt for comparing our work with [1], and reviewer KxBC for comparing our work with [1] and [2].
By the NeurIPS 2023 policy, our work and [2] are concurrent works. 
Herein, we group [1] and [2] together for a comparative analysis with our method.

1. **Task.** Our goal is to train a generator network, which is capable of transforming random noise into data of good perceptual quality. 
By contrast, [1] and [2] aim to recover training data. Our task is more challenging in two respects. (1) we aim to generate samples that are not present in the original dataset but have good perceptual quality. (2) our generation process is controllable and supports conditional sampling. 

2. **#Categories.**  [1] only supports binary classifiers, while our work and [2] support multi-class classifiers.

3. **Supported Networks.** [1] and [2] only support homogeneous networks. By contrast, our method supports quasi-homogeneous networks, which encompass a broader range of networks. The experiments about non-homogeneous models in [1] only involve fully connected networks with bias terms. The networks in our experiments are more complex and diverse.

4. **Technical Aspect.**: [1] and [2] optimize directly on the pixel space. Specifically, #optimizable parameters equals $BatchSize×ImageSize$. In [1], batch size is set to twice the size of the training set. 
For MNIST with 500 images, \#parameters is 1M. However, our optimizable parameters only include the parameters of the generator. In our implementation, it is only 0.18M, which is significantly lower than those in [1] and [2]. 

Therefore, we conclude that we are addressing a more challenging task than [1] and [2] do, and we are doing so with a more constrained budget in terms of #optimizable parameters.

#### **Comparisons with [3] Li, 2022 and [4] Wang and Torr, 2022** 
We thank the reviewer KxBC for mentioning [3] and [4]. 
[3] is to generate an image $x$ given a pre-trained classifier $\Phi$ and a category $y$ according to
$$
    x = \arg\min_{\hat{x}} L(\Phi(\hat{x}),y) + \beta||\hat{x} - \bar{x}||,
$$
where $L$ is the cross-entropy loss, and the regularization term is the norm between the generated sample $\hat{x}$ and training data mean $\bar{x}$,.

[4] is to generate N images $x^1, \cdots, x^N$ that belong to category $y$ given a pre-trained classifier $\Phi$ according to
$$
    x^1, \cdots, x^N = \arg\min_{\hat{x}^1, \cdots, \hat{x}^N} \sum_{i=1}^N L(\Phi(\hat{x}^i),y) + \beta_1 L_{div}(\hat{x}^1, \cdots, \hat{x}^N) + \beta_2 L_{dist}(\hat{x}^1, \cdots, \hat{x}^N),
$$
where $L$ is the cross-entropy loss, $L_{div}$ is a similarity penalty, and $L_{dist}$ constrains the generated samples and the training data to have the same mean and variance in the feature space.

1. **Technical Aspect.** To generate images, [3] and [4] require multiple gradient descent steps to directly optimize each pixel of every image. Each gradient optimization step entails one forward pass and one gradient propagation through the classifier. By contrast, once our generator is trained, images can be generated by only a single forward pass through the generator. This results in two differences: (1) for the generation of each image, [3] and [4] possess a higher computational complexity and take longer to process; (2) [3] and [4] have more optimizable parameters and more expansive parameter spaces.

2. **Prior Information.** [3] and [4] use prior information about the data for generation. Specifically, the mean used in [3] and the mean and variance used in [4]. However, our method solely relies on a pre-trained classifier and does not use the prior information of the data.

Therefore, we conclude that these methods have a different setting. We are dealing with a more challenging task, and our solution uses a lower complexity.

[1] Haim et al, 2022. "Reconstructing Training Data from Trained Neural Networks".

[2] Buzaglo et al, 2023. "Reconstructing Training Data from Multiclass Neural Networks".

[3] Li, 2022. "Use Classifier as Generator". 

[4] Wang and Torr, 2022. "Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANs". 

Meta-review: This paper introduces a method that trains an image generator from a pre-trained classifier, without using samples in the training set. The proposed training objective is based on the Maximum-Margin Bias theory of gradient descent. The paper received four reviews, with three positive reviews acknowledging the novelty of the problem formulation and the proposed solution. One reviewer (KxBC) raises several concerns against the paper including the limited extension of previous work (Haim et al. 2022) to the multi-class setting, limited evaluation of the generative model (the original draft didn’t compare with baselines nor report standard metrics such as FID), poor writing and presentation of the technical part, and missing details of the model architecture. The authors address most of the concerns during rebuttal, which includes additional comparison with the suggested baselines, additional results of the suggested experiments (e.g., performance when a robust classifier is used), explanation and clarification of the writing, and details of the model architecture. Although reviewer KxBC still expresses reservations regarding the novelty of the paper, given other consistent positive scores and the pioneering efforts in this direction (argued by reviewer gUqt), we recommend acceptance for the paper. However, if accepted, the authors need to address the negative reviews in the revision, add suggested evaluations, and improve the clarity of the writing.
Decision: Accept (poster)