Title: COOPERATIVE TRAINING OF DESCRIPTOR AND GEN-ERATOR NETWORKS

Abstract: This paper studies the cooperative training of two probabilistic models of signals such as images. Both models are parametrized by convolutional neural networks (ConvNets). The first network is a descriptor network, which is an exponential family model or an energy-based model, whose feature statistics or energy function are defined by a bottom-up ConvNet, which maps the observed signal to the feature statistics. The second network is a generator network, which is a nonlinear version of factor analysis. It is defined by a top-down ConvNet, which maps the latent factors to the observed signal. The maximum likelihood training algorithms of both the descriptor net and the generator net are in the form of alternating back-propagation, and both algorithms involve Langevin sampling. We observe that the two training algorithms can cooperate with each other by jumpstarting each other's Langevin sampling, and they can be seamlessly interwoven into a CoopNets algorithm that can train both nets simultaneously.

1. INTRODUCTION 1.TWO CONVNETS OF OPPOSITE DIRECTIONS:

We begin with a story that the reader of this paper can readily relate to. A student writes up an initial draft of a paper. His advisor then revises it. After that they submit the revised paper for review. The student then learns from his advisor's revision, while the advisor learns from the outside review. In this story, the advisor guides the student, but the student does most of the work.

This paper is about two probabilistic models of signals such as images, and they play the roles of student and advisor as described above. Both models are parametrized by convolutional neural networks (ConvNets or CNNs) (LeCun et al., 1998; Krizhevsky et al., 2012) . The two nets take two opposite directions. One is bottom-up, and the other is top-down, as illustrate by the following diagram:

Bottom-up ConvNet:

Top-down ConvNet features latent variables ⇑ ⇓ signal signal (a) Descriptor Net (b) Generator Net

(1)

The simultaneous training of such two nets was first studied by the recent work of Kim & Bengio (2016) . These two nets belong to two major classes of probabilistic models. (a) The exponential family models or the energy-based models (LeCun et al., 2006) or the Markov random field models (Zhu et al., 1997) , where the probability distribution is defined by feature statistics or energy function computed from the signal by a bottom-up process. (b) The latent variable models or the directed graphical models, where the signal is assumed to be a transformation of the latent factors that follow a known prior distribution. The latent factors generate the signal by a top-down process. A classical example is factor analysis.

The two classes of models have been contrasted by Zhu (2003) ; Teh et al. (2003) ; Ngiam et al. (2011) . Zhu (2003) called the two classes of models the descriptive models and the generative models respectively. Both classes of models can benefit from the high capacity of the multi-layer ConvNets. (a) In the exponential family models or the energy-based models, the feature statistics or the energy function can be defined by a bottom-up ConvNet that maps the signal to the features and the energy function (Ngiam et al., 2011; Xie et al., 2016) . We call the resulting model a descriptive network or a descriptor net following Zhu (2003) , because it is built on descriptive feature statistics. (b) In the latent variable models or the directed graphical models, the transformation from the latent factors to the signal can be defined by a top-down ConvNet (Dosovitskiy et al., 2015) , which maps the latent factors to the signal. We call the resulting model a generative network or generator net following Goodfellow et al. (2014) , who proposed such a model in their work on the generative adversarial networks (GAN). In this article, we propose to couple Algorithms D and G into a cooperative training algorithm that interweaves the steps of the two algorithms seamlessly. We call the resulting algorithm the CoopNets algorithm, and we show that it can train both nets simultaneously. The generator learns from how the descriptor revises the initial draft by reconstructing the revised draft in Step G2. For each synthesized example, the generator knows the latent factors that generate the initial draft, so that Step G1 can infer the latent factors by initializing its Langevin dynamics from their known values.

1.2. TWO TRAINING ALGORITHMS AND THEIR COOPERATION:

In the CoopNets algorithm, the generator fuels the MCMC of the descriptor by supplying initial synthesized examples, which can be obtained by direct ancestral sampling. The generator then learns from the revised synthesized examples with virtually known latent factors. The cooperation is thus beneficial to both nets.

2. RELATED WORK:

Our work is inspired by the generative adversarial networks (GAN) (Goodfellow et al., 2014; Denton et al., 2015; Radford et al., 2015) . In GAN, the generator net is paired with a discriminator net. The two nets play adversarial roles. In our work, the generator net and the descriptor net play cooperative roles, and they feed each other the initial, revised and reconstructed synthesized data. The learning of both nets is based on maximum likelihood, and the learning process is quite stable because of the cooperative nature and the consistent directions of the two maximum likelihood training algorithms.

Another method to train the generator network is variational auto-encoder (VAE) (Kingma & Welling, 2014; Rezende et al., 2014; Mnih & Gregor, 2014) , which learns an inferential or recognition network to approximate the posterior distribution of the latent factors.

The connection between the descriptor net and the discriminator net has been explored by Xie et al. (2016) , where the descriptor can be derived from the discriminator.

Our work is most similar to the recent work of Kim & Bengio (2016) . In fact, the settings of the two nets are the same. In their work, the generator learns from the descriptor by minimizing the Kullback-Leibler divergence from the generator to the descriptor, which can be decomposed into an energy term and an entropy term. In our work, the two nets interact with each other via synthesized data, and the generator learns from the descriptor by reconstructing the revised draft of synthesized examples. Our method does not need to approximate the intractable entropy term.

Our work is related to the contrastive divergence algorithm (Hinton, 2002) for training the descriptor net. The contrastive divergence initializes the MCMC sampling from the observed examples. The CoopNets algorithm initializes the MCMC sampling from the examples supplied by the generator.

3. TWO NETS AND TWO TRAINING ALGORITHMS:



3.1. DESCRIPTOR NET AND TRAINING ALGORITHM:

Let Y be the D-dimensional signal, such as an image. The descriptor model is in the form of exponential tilting of a reference distribution (Xie et al., 2016) :

EQUATION

where q(Y ) is the reference distribution such as Gaussian white noise 

q(Y ) ∝ exp -Y 2 /2s 2 , f (Y ; W D ) (f

Z(W D ) = exp [f (Y ; W D )] q(Y )dY = E q {exp[f (Y ; W D )]}

is the normalizing constant, where E q is the expectation with respect to q.

Suppose we observe training examples {Y i , i = 1, ..., n} from an unknown data distribution P data (Y ). The maximum likelihood training seeks to maximize the log-likelihood function

L D (W D ) = 1 n n i=1 log P D (Y i ; W D ).

If the sample size n is large, the maximum likelihood estimator minimizes KL(P data |P D ), the Kullback-Leibler divergence from the data distribution P data to the model distribution P D . The gradient of the

EQUATION

where E W D denotes the expectation with respect to P D (Y ; W D ).

The expectation in equation ( 3) is analytically intractable and has to be approximated by MCMC, such as Langevin dynamics, which iterates the following step:

EQUATION

where τ indexes the time steps of the Langevin dynamics, δ is the step size, and U τ ∼ N(0, I D ) is the Gaussian white noise term.

We can run ñ parallel chains of Langevin dynamics according to (4) to obtain the synthesized examples { Ỹi , i = 1, ..., ñ}. The Monte Carlo approximation to

EQUATION

which is used to update W D .

Algorithm D (Xie et al., 2016) iterates the following two steps after initializing W D and { Ỹi , i = 1, ..., ñ}.

Step D1: Run l D steps of Langevin from the current { Ỹ } according according to (4).

Step D2: update

W (t+1) D = W (t) D + γ t L D (W (t)

D ) with learning rate γ t . The convergence of such an algorithm follows Younes (1999) .

3.2. GENERATOR NET AND TRAINING ALGORITHM:

The generator net (Goodfellow et al., 2014) seeks to explain the signal Y of dimension D by a vector of latent factors X of dimension d, and usually d D. The model is of the following form: The joint density of model ( 6) is

EQUATION

EQUATION

EQUATION

where the constant term is independent of X, Y and W G . The marginal density is obtained by integrating out the latent factors X, i.e., P G (Y ;

W G ) = P G (X, Y ; W G )dX. The inference of X given Y is based on the posterior density P G (X|Y ; W G ) = P G (X, Y ; W G )/P G (Y ; W G ) ∝ P G (X, Y ; W G ) as a function of X.

For the training data {Y i , i = 1, ..., n}, the generator net can be trained by maximizing the loglikelihood

L G (W G ) = 1 n n i=1 log P G (Y i ; W G ).

For large sample, the learned W G minimizes the Kullback-Leibler divergence KL(P data |P G ) from the data distribution P data to the model distribution P G . The gradient of L G (W G ) is obtained according to the following identity

EQUATION

which underlies the EM algorithm. In general, the expectation in ( 8) is analytically intractable, and has to be approximated by MCMC that samples from the posterior P G (X|Y ; W G ), such as Langevin dynamics, which iterates

EQUATION

where

U τ ∼ N(0, I d ). With X i sampled from P G (X i | Y i , W G ) for each observation Y i , the Monte Carlo approximation to L G (W G ) is L G (W G ) ≈ 1 n n i=1 ∂ ∂W G log P G (X i , Y i ; W G ) = 1 n n i=1 1 σ 2 (Y i -g(X i ; W G )) ∂ ∂W G g(X i ; W G ). (10)

Algorithm G (Han et al., 2017) iterates the following two steps after initializing W G and {X i , i = 1, ..., n}.

Step G1: run l G steps of Langevin from the current {X i } according to (9).

Step G2: update

W (t+1) G = W (t) G + γ t L G (W (t)

G ) with learning rate γ t . The convergence of such an algorithm follows Younes (1999) .

4. COOPNETS ALGORITHM: RECONSTRUCTING THE REVISION:

In Algorithms D and G, both steps D1 and G1 are Langevin dynamics, which may be slow to converge. An interesting observation is that the two algorithms can cooperate with each other by jumpstarting each other's Langevin sampling.

Specifically, in:

Step D1, we can initialize the synthesized examples by generating examples from the generator net. We first generate Xi ∼ N(0, I d ), and then generate Ŷi = g( Xi ; W G ) + i , for i = 1, ..., ñ. If the current generator P G is close to the current descriptor P D , then the generated { Ŷi } should be a good initialization for sampling from the descriptor net, i.e., starting from the { Ŷi , i = 1, ..., ñ}, we run Langevin dynamics in Step D1 for l D steps to get { Ỹi , i = 1, ..., ñ}, which are revised versions of { Ŷi }. These { Ỹi } can be used as the synthesized examples from the descriptor net. We can then update W D according to Step D2 of Algorithm D.

In order to update W G of the generator net, we treat the { Ỹi , i = 1, ..., ñ} produced by the above Step D1 as the training data for the generator. Since these { Ỹi } are obtained by the Langevin dynamics initialized from the { Ŷi , i = 1, ..., ñ} produced by the generator net with known latent factors { Xi , i = 1, ..., ñ}, we can update W G by learning from {( Ỹi , Xi ), i = 1, ..., ñ}, which is a supervised learning problem, or more specifically, a non-linear regression of Ỹi on Xi . At W (t) G , the latent factors Xi generates and thus reconstructs the initial example Ŷi . After updating W G , we want Xi to reconstruct the revised example Ỹi . That is, we revise W G to absorb the revision from Ŷi to Ỹi , so that the generator shifts its density from { Ŷi } to { Ỹi }. The reconstruction error can tell us whether the generator has caught up with the descriptor by fully absorbing the revision.

The left diagram in (11) illustrates the basic idea.

Xi Ŷi:

Ỹi See Appendix for a theoretical understanding of the convergence of the CoopNets algorithm. Step G0: For i = 1, ..., ñ, generate Xi ∼ N(0, I d ), and generate Ŷi = g( Xi ;

EQUATION

W (t) G ) + i . 4:

Step D1: For i = 1, ..., ñ, starting from Ŷi , Run l D steps of Langevin dynamics to obtain Ỹi , each step following equation (4).

5::

Step G1: Treat the current { Ỹi , i = 1, ..., ñ} as the training data, for each i, infer X i = Xi . Or more rigorously, starting from X i = Xi , run l G steps of Langevin dynamics to update X i , each step following equation ( 9).

6::

Step D2: Update W

(t+1) D = W (t) D + γ t L D (W (t) D ), where L D (W (t)

D ) is computed according to (5).

7::

Step G2: Update W

(t+1) G = W (t) G + γ t L G (W (t) G )

, where L G (W G ) is computed according to (10), except that Y i is replaced by Ỹi , and n by ñ.

8::

Let t ← t + 1 9: until t = T

5. EXPERIMENTS:

We use the MatConvNet of Vedaldi & Lenc (2015) for coding. For the descriptor net, we adopt the structure of Xie et al. (2016) , where the bottom-up network consists of multiple layers of convolution by linear filtering, ReLU non-linearity, and down-sampling. We adopt the structure of the generator network of Radford et al. (2015) ; Dosovitskiy et al. (2015) , where the top-down network consists of multiple layers of deconvolution by linear superposition, ReLU non-linearity, and up-sampling, with tanh non-linearity at the bottom-layer (Radford et al., 2015) to make the signals fall within [-1, 1].

5.1. QUANTITATIVE EXPERIMENT ON FACE COMPLETION:

We conduct an experiment on learning from complete training images of human faces, and then testing the learned model on completing the occluded testing images. The structure of the generator network is the same as in (Radford et al., 2015; Dosovitskiy et al., 2015) . We adopt a 4-layer descriptor net. The first layer has 96 5 × 5 filters with sub-sampling of 2, the second layers has 128 5 × 5 filters with sub-sampling of 2, the third layer has 256 5 × 5 filters with sub-sampling of 2, and the final layer is a fully connected layer with 50 channels as output. We use L=10 steps of Langevin revision dynamics within each learning iteration, and the Langevin step size is set at 0.002. The learning rate is 0.07. The training data are 10, 000 human faces randomly selected from CelebA dataset (Liu et al., 2015) . We run 600 cooperative learning iterations. Figure 2 displays 144 synthesized human faces by the descriptor net.

To quantitatively test whether we have learned a good generator net g(X; W G ) even though it has never seen the training images directly in the training stage, we apply it to the task of recovering the occluded pixels of testing images. For each occluded testing image Y , we use Step G1 of Algorithm G to infer the latent factors X. The only change is with respect to the term Y -g(X; W G ) 2 , where the sum of squares is over all the observed pixels of Y in back-propagation computation. We run 1000 Langevin steps, initializing X from N(0, I d ). After inferring X, the completed image g(X; W G ) is automatically obtained. We design 3 experiments, where we randomly place a 20 × 20, 30 × 30, or 40 × 40 mask on each 64 × 64 testing image. These 3 experiments are denoted by M20 M30, and M40 respectively (M for mask). We report the recovery errors and compare our method with 8 different image inpainting methods as well as the DCGAN of Radford et al. (2015) . For DCGAN, we use the parameter setting in Radford et al. (2015) except changing the number of learning iterations to 600. We use the same 10, 000 training images to learn DCGAN. After the model is learned, we keep the generator and use the same method as ours to infer latent factors X, and recover the unobserved pixels. In 8 inpainting methods, Methods 1 and 2 are based on Markov random field prior where the nearest neighbor potential terms are 2 and 1 differences respectively. Methods 3 to 8 are interpolation methods. Please refer to D'Errico ( 2004) for more details. Table 1 displays the recovery errors of the 3 experiments, where the error is measured by per pixel difference (relative to the range of pixel values) between the original image and the recovered image on the occluded region, averaged over 100 testing images. Fig. 3 displays some recovery results by our method. The first row shows the original images as the ground truth. The second row displays the testing images with occluded pixels. The third row displays the recovered images by the generator net trained by the CoopNets algorithm on the 10,000 training images. .0966 .2535 .1545 .1506 .1277 .1123 .2493 .1123 .1126 .1277 M30 .1112 .2606 .1820 .1792 .1679 .1321 .3367 .1310 .1312 .1679 M40 .1184 .2618 .2055 .2032 .1894 .1544 .3809 .1525 .1526 .1894 Figure 3 : Row 1: ground-truth images. Row 2: testing images with occluded pixels. Row 3: recovered images by our method.

5.2. QUALITATIVE EXPERIMENT ON SYNTHESIS:

We conduct an experiment on synthesizing images of categories from Imagenet ILSVRC2012 dataset (Deng et al., 2009) and MIT places205 dataset (Zhou et al., 2014) . We adopt a 4-layer descriptor net. The first layer has 64 5 × 5 filters with sub-sampling of 2, the second layers has 128 3 × 3 filters with sub-sampling of 2, the third layer has 256 3 × 3 filters with sub-sampling of 1, and the final layer is a fully connected layer with 100 channels as output. We set the number of Langevin dynamics steps in each learning iteration to 10 and the step size to 0.002. The learning rate is 0.07. For each category, we randomly choose 1,000 images as training data and resize the images to 64 × 64. We run 1, 000 cooperative learning iterations to train the model. Figures 4 and 5 display the results for two categories, where for each category, we show 144 original images sampled from the training set, and 144 synthesized images generated by our method. The appendix contains more synthesis results.

As a comparison, we apply the Algorithm G alone and GAN code on the same 1,000 hotel room training images to learn the generator of the same structure as in CoopNets. Figure 6 displays the synthesis results.

We also try to synthesize images at high resolution (224 × 224). We adopt a 4-layer descriptor net. The first layer has 128 15 × 15 filters with sub-sampling of 3, the second layer has 256 3 × 3 filters with sub-sampling of 2, the third layer has 512 3 × 3 filters with sub-sampling of 1, and the final layer is a fully connected layer with 100 channels as output. We enlarge the filters of the final layer of generator net to 14 × 14 to generate 224 × 224 images. The learning rate is 0.05. We run 1000 cooperative learning iterations to train the model. Figures 7 and 8 show the synthesized images of two categories from MIT places205 dataset.

6. CONCLUSION:

The most unique feature of our work is that the two networks feed each other the synthesized data in the learning process, including initial, revised, and reconstructed synthesized data.

Another unique feature of our work is that the learning process interweaves the existing maximum likelihood learning algorithms for the two networks.

A third unique feature of our work is that the MCMC for the descriptor keeps rejuvenating the chains by refreshing the samples by independent replacements supplied by the generator, so that a single chain effectively amounts to an infinite number of chains or the evolution of the whole marginal distribution modeled by the generator.

CODE AND DATA:

http://www.stat.ucla.edu/ ˜ywu/CoopNets/main.html 7 APPENDIX: CONVERGENCE

7.1. GENERATOR OF INFINITE CAPACITY:

In the CoopNets algorithm, the descriptor learns from the observed examples, while the generator learns from the descriptor through the synthesized examples. Therefore, the descriptor is the driving force in terms of learning, although the generator is the driving force in terms of synthesis. In order to understand the convergence of learning, we can start from Algorithm D for learning the descriptor.

Algorithm D is a stochastic approximation algorithm (Robbins & Monro, 1951) , except that the samples are generated by finite step MCMC transitions. According to Younes (1999) , Algorithm D converges to the maximum likelihood estimate under suitable regularity conditions on the mixing of the transition kernel of the MCMC and the schedule of the learning rate γ t , even if the number of Langevin steps l D is finite or small (e.g., l D = 1), and even if the number of parallel chains ñ is finite or small (e.g., ñ = 1). The reason is that the random fluctuations caused by the finite number of chains, ñ, and the limited mixing caused by the finite steps of MCMC, l D , are mitigated if the learning rate γ t is sufficiently small. At learning iteration t, let W 

= P D (Y ; W (t) D ) if l D → ∞), we still have W (t)

D → ŴD in probability according to Younes (1999) , where ŴD is the maximum likelihood estimate of W D .

The efficiency of Algorithm D increases if the number of parallel chains ñ is large because it leads to more accurate estimation of the expectation in the gradient L D (W D ) of equation ( 3), so that we can afford to use larger learning rate γ t for faster convergence. Now let us come back to the CoopNets algorithm. In order to understand how the descriptor net helps the training of the generator net, let us consider the idealized scenario where the number of parallel chains ñ → ∞, and the generator has infinite capacity, and in each iteration it estimates W G by maximum likelihood using the synthesized data from P (t+1) D

. In this idealized scenario, the learned generator P G (Y ; W (t+1) G

) will reproduce P serving as its data distribution. Then eventually the learned generator P G (Y, ŴG ) will reproduce P D (Y ; ŴD ). Thus the cooperative training helps the learning of the generator. Note that the learned generator P G (Y, ŴG ) will not reproduce the distribution of the observed data P data , unless the descriptor is of infinite capacity too.

Conversely, the generator net also helps the learning of the descriptor net in the CoopNets algorithm. In Algorithm D, it is impractical to make the number of parallel chains ñ too large. On the other hand, it would be difficult for a small number of chains { Ỹi , i = 1, ..., ñ} to explore the state space. In the CoopNets algorithm, because P G (Y ; W We assume both the observed data size n and the synthesized data size ñ are large enough so that we shall work on distributions or populations instead of finite samples. As explained above, assuming ñ → ∞ is sound because the generator net can supply unlimited number of examples.

The Langevin revision dynamics runs a Markov chain from W 

EQUATION

where the notation L•P denotes the marginal distribution obtained by running the Markov transition L from P . The distribution P (13)

The learning process alternates between Markov transition in (12) and projection in ( 13), as illustrated by Figure 9 . 

EQUATION

That is, we first project P data onto D, and from there continue to project onto G. Therefore, W D converges to the maximum likelihood estimate with P data being the data distribution, while W G converges to the maximum likelihood estimate with ŴD serving as the data distribution.

For finite l D , the algorithm may converge to the following fixed points. The fixed point for the generator satisfies

EQUATION

The fixed point for the descriptor satisfies

EQUATION

which is similar to contrastive divergence (Hinton, 2002) , except that ŴG takes the place of P data in the second Kullback-Leibler divergence. Because ŴG is supposed to be close to ŴD , the second Kullback-Leibler divergence is supposed to be small, hence our algorithm is closer to maximum likelihood learning than contrastive divergence. Kim & Bengio (2016) 

D ) = E W G [log P G (Y ; W G )] -E W G [log P D (Y ; W (t) D )],

where the first term is the negative entropy that is intractable, and the second term is the expected energy that is tractable. Our learning method for the generator is consistent with the learning objective KL(W G |W Kim & Bengio (2016) . But the Monte Carlo implementation of L in our work avoids the need to approximate the intractable entropy term.

7.3. MORE SYNTHESIS RESULTS:

We display more synthesis results at the resolution of 64 × 64. 

:



Figures and Tables:

Figure 1: (a) Algorithm D involves sampling from the current model by Langevin dynamics. (b) Algorithm G involves sampling from the posterior distribution of the latent factors by Langevin dynamics. (c) CoopNets algorithm. The part of the flowchart for training the descriptor is similar to Algorithm D, except that the D1 Langevin sampling is initialized from the initial synthesized examples supplied by the generator. The part of the flowchart for training the generator can also be mapped to Algorithm G, except that the revised synthesized examples play the role of the observed data, and the known generated latent factors can be used as inferred latent factors (or be used to initialize the G1 Langevin sampling of the latent factors).

Fig. 1(a) and (b) display the flowcharts of the maximum likelihood learning algorithms for training the descriptor and generator nets . We call the two algorithms Algorithm D and Algorithm G respectively. Algorithm D (Xie et al., 2016) iterates two steps: Step D1 synthesizes examples by sampling from the current model by Langevin dynamics. Step D2 updates the parameters to shift the density from the synthesized examples towards the observed examples. Algorithm G (Han et al., 2017) also iterates two steps. Step G1 infers latent factors for each observed example by sampling from their posterior distribution by Langevin dynamics. Step G2 updates the parameters by a non-linear regression of the observed examples on their corresponding latent factors. We use Langevin dynamics for Markov chain Monte Carlo (MCMC) sampling because the gradient term of Langevin dynamics can be readily computed via back-propagation. Thus all the steps D1, D2 and G1, G2 are powered by back-propagation, and both Algorithms D and G are alternating back-propagation algorithms.

Figure 1(c) displays the flowchart of the CoopNets algorithm. The generator is like the student. It generates the initial draft of the synthesized examples. The descriptor is like the advisor. It revises the initial draft by initializing its Langevin dynamics from the initial draft in Step D1, which produces the revised draft of the synthesized examples. The descriptor learns from the outside review in Step D2, which is in the form of the difference between the observed examples and the revised

stands for features) is the feature statistics or energy function, defined by a ConvNet whose parameters are denoted by W D . This ConvNet is bottom-up because it maps the signal Y to a number. See the diagram in (1).

g(X; W G ) (g stands for generator) is a top-down ConvNet defined by the parameters W G . The ConvNet g maps the latent factors X to the signal Y . See the diagram in (1).

11) In the two diagrams in (11), the double-line arrows indicate generation and reconstruction by the generator net, while the dashed-line arrows indicate Langevin dynamics for revision and inference in the two nets. The diagram on the right in (11) illustrates a more rigorous method, where we initialize the Langevin inference of {X i , i = 1, ..., ñ} in Step G1 from { Xi }, and then update W G in Step G2 based on {( Ỹi , X i ), i = 1, ..., ñ}. The diagram on the right shows how the two nets jumpstart each other's Langevin dynamics. Algorithm 1 describes the cooperative training that interweaves Algorithm D and Algorithm G. See Figure 1(c) for the flowchart of the CoopNets algorithm. In our experiments, we set l G = 0 and infer X i = Xi for simplicity, i.e., we follow the left diagram in (11).

Figure 2: Generating human face pattern. The synthesized images are generated by the CoopNets algorithm that learns from 10, 000 images.

Figure 4: Generating forest road images. The category is from MIT places205 dataset.

Figure 5: Generating hotel room images. The category is from MIT places205 dataset.

(a) Generated by Algorithm G alone. (b) Generated by DCGAN code.

Figure 6: Generating hotel room images by Algorithm G alone and by GAN.

Figure 7: Generating forest road images at high resolution (224 × 224).

Figure 8: Generating hotel room images at high resolution (224 × 224).

distribution of { Ỹi }. Even though P

, we can generate a completely new batch of independent samples { Ŷi } from P G (Y ; W (t) G ), and revise { Ŷi } to { Ỹi } by Langevin dynamics, instead of running Langevin dynamics from the same old batch of { Ỹi } as in the original Algorithm D. This is like implementing an infinite number of parallel chains, because each iteration evolves a fresh batch of examples, as if each iteration evolves a new set of chains. By updating the generator W G , it is like we are updating the infinite number of parallel chains, because W G memorizes the whole distribution. Even if ñ in the CoopNets algorithm is small, e.g., ñ = 1, viewed from the perspective of Algorithm D, it is as if ñ → ∞. Thus the above idealization ñ → ∞ is sound.7.2 GENERATOR OF FINITE CAPACITYFrom an information geometry point of view, let D = {P D (Y ; W D ), ∀W D } be the manifold of the descriptor models, where each distribution P D (Y ; W D ) is a point on this manifold. Then the maximum likelihood estimate of W D is a projection of the data distribution P data onto the manifold D. Let G = {P G (Y ; W G ), ∀W G } be the manifold of the generator models, where each distribution P G (Y ; W G ) is a point on this manifold. Then the maximum likelihood estimate of W G is a projection of the data distribution P data onto the manifold G.From now on, for notational simplicity and with a slight abuse of notation, we use W D to denote the descriptor distribution P D (Y ; W D ), and use W G to denote the generator distribution P G (Y ; W G ).

L W D be the Markov transition kernel of l D steps of Langevin revisions towards W D . The distribution of the revised synthesized data is

is in the middle between the two nets W (t) G and W (t) D , and it serves as the data distribution to train the generator, i.e., we project this distribution onto the manifold G = {P G (Y ; W G ), ∀W G } = {W G } (recall we use W G to denote the distribution P G (Y ; W G )) in the information geometry picture, so that W

Figure 9: The learning of the generator alternates between Markov transition and projection. The family of the generator models G is illustrated by the black curve. Each distribution is illustrated by a point.

) → 0 monotonically as l D → ∞ due to the second law of thermodynamics. The reduction of the Kullback-Leibler divergence in (18) and the projection in (13) in our learning of the generator are consistent with the learning objective of reducing KL(W G |W (t) D ) in

Figure 10: Generating swimming pool images. The category is from MIT places205 dataset.

Figure 11: Generating volcano images. The category is from MIT places205 dataset.

Figure 12: Generating rock images. The category is from MIT places205 dataset.

Figure 13: Generating desert images. The category is from MIT places205 dataset.

Figure 14: Generating schoolbus images. The category is from Imagenet ILSVRC2012 1000 object categories.

Figure 15: Generating lifeboat images. The category is from Imagenet ILSVRC2012 1000 object categories.

Figure 16: Generating zebra images. The category is from Imagenet ILSVRC2012 1000 object categories.

Figure 17: Generating strawberry images. The category is from Imagenet ILSVRC2012 1000 object categories.

Figure 18: Generating lemon images. The category is from Imagenet ILSVRC2012 1000 object categories.

Figure 19: Generating apartment building images. The category is from Imagenet ILSVRC2012 1000 object categories.

Figure 20: Generating dinning table images. The category is from Imagenet ILSVRC2012 1000 object categories.

Figure 21: Generating balloon images. The category is from Imagenet ILSVRC2012 1000 object categories.

Algorithm 1 CoopNets Algorithm Input:(1) training examples {Y i , i = 1, ..., n} (2) numbers of Langevin steps l D ad l G (3) number of learning iterations T Output:(1) estimated parameters W D and W G (2) synthesized examples { Ŷi , Ỹi , i = 1, ..., ñ} 1: Let t ← 0, initialize W D and W G .

Table content: <table/>

Comparison of recovery errors among different inpainting methods in 3 experiments

Table content: <table><tr><td>Exp</td><td>Ours</td><td>GAN</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td><td>6</td><td>7</td><td>8</td></tr><tr><td>M20</td><td/><td/><td/><td/><td/><td/><td/><td/><td/><td/></tr></table>

learned the generator by gradient descent on KL(W G |W The objective function is KL(W G |W

Table content: <table><tr><td>(t)</td></tr><tr><td>(t)</td></tr></table>

Peer review (iclr2017):
This paper introduces CoopNets, an algorithm which trains a Deep-Energy Model (DEM, the “descriptor”) with the help of an auxiliary directed bayes net, e.g. “the generator”. The descriptor is trained via standard maximum likelihood, with Langevin MCMC for sampling. The generator is trained to generate likely samples under the DEM in a single, feed-forward ancestral sampling step. It can thus be used to shortcut expensive MCMC sampling, hence the reference to “cooperative training”. The above idea is interesting and novel, but unfortunately is not sufficiently validated by the experimental results. First and foremost, two out of the three experiments do not feature a train /test split, and ignore standard training and evaluation protocols for texture generation (see [R1]). Datasets are also much too small. As such these experiments only seem to confirm the ability of the model to overfit. On the third in-painting tasks, baselines are almost non-existent: no VAEs, RBMs, DEM, etc which makes it difficult to evaluate the benefits of the proposed approach. In a future revision, I would also encourage the authors to answer the following questions experimentally. What is the impact of the missing rejection step in Langevin MCMC (train with, without ?). What is the impact of the generator on the burn-in process of the Markov chain (show sample auto-correlation). How bad is approximation of training the generator from ({	ilde{Y}, hat{X}) instead of ({	ilde{Y}, 	ilde{X}) ? Run comparative experiments. The paper would also greatly benefit from a rewrite focusing on clarity, instead of hyperbole (“pioneering work” in reference to closely related, but non-peer reviewed work) and prose (“tale of two nets”). For example, the authors fail to specify the exact form of the energy function: this seems like a glaring omission. PROS: + Interesting and novel idea CONS: - Improper experimental protocols - Missing baselines - Missing diagnostic experiments [R1] Heess, N., Williams, C. K. I., and Hinton, G. E. (2009). Learning generative texture models with extended fields of-experts.