Efficient generation of structured objects with Constrained Adversarial Networks

1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) have shown impressive performance on challenging tasks – like image generation ( Karras et al. , 2018 ) , text-to-image ( Zhang et al. , 2017 ) , and style transfer ( Zhu et al. , 2017 ) – where the goal is to produce believable configurations . A number of important applications , however , require to generate objects that are both credible and feasible with respect to hard structural constraints . Examples include generating drug molecules , which must satisfy chemical validity requirements , and game levels , where the goal must be reachable from the starting position . Recent studies ( Guimaraes et al. , 2017 ; De Cao & Kipf , 2018 ; Xue & van Hoeve , 2019 ) observed that GANs struggle in these tasks . The main reason is that the training examples alone are insufficient to capture the feasibility constraints and thus to guide the model toward producing valid objects . As a remedy , we propose Constrained Adversarial Networks ( CANs ) , a class of generative models that extend GANs to structured domains . Given a set of examples drawn from a latent distribution and a set of structural constraints , CANs learn to output valid structured with high probability . This is achieved by augmenting the standard GAN loss with a penalty term that discourages the model from producing infeasible structures . Since CANs inject the constraints directly into the learned model , valid structures can be obtained with high probability by performing forward inference on the generator , avoiding the need for costly sampling or optimization steps ( Volz et al. , 2018 ) . The penalty term is implemented using the semantic loss ( SL ) ( Xu et al. , 2018 ) . The SL leverages knowledge compilation ( Darwiche , 2011 ) to represent arbitrary Boolean constraint as a circuit and uses the latter to measure the mass allocated by the generator to infeasible objects . The procedure is probabilistically sound , exact , and does not require sampling . Notably , the circuit , which can be quite large depending on the complexity of the constraints , can be thrown away after learning . In addition , we show how to extend the SL complex constraints that would normally lead to intractably large circuits . This is accomplished by first using a neural network to map configurations to a different space in which the structural constraints can be compactly encoded , and then applying the SL to the latter . This enables us to deal with reachability on a graph , which is beyond the reach of standard SL . Finally , we show how the validity constraints baked into the generator can be enabled or disabled during inference using ideas from InfoGANs Chen et al . ( 2016 ) . In summary , we contribute : • Constrained Adversarial Networks ( CANs ) , a generalization of GANs in which the generator is encouraged ( at training time ) to output valid structures with high probability via forward inference . • An extensive empirical analysis on structured objects like constrained images , molecules , and video game levels showing that CANs generate structures that are stylistically coherent with the training data . • A decomposition of intractably complex constraints into a neural and logical components , showcased on graph reachability for video game level synthesis . 2 UNCONSTRAINED GANS . Let X be the object space and Z some latent space . GANs ( Goodfellow et al. , 2014 ) are composed of a generator g : Z → X and a discriminator d : X → [ 0 , 1 ] , both implemented as neural networks . The discriminator is trained to recognize “ real ” objects x taken from the data distribution Pr , while the generator is trained to map random latent vectors z ∈ Rm to “ fake ” objects that fool the discriminator . Learning amounts to solving the minimax game : min g max d fGAN ( g , d ) fGAN ( g , d ) : = Ex∼Pr [ logPd ( x ) ] + Ex∼Pg [ log ( 1− Pd ( x ) ) ] ( 1 ) Here Pg ( x ) is the distribution induced by the generator and Pd ( x ) : = Pd ( real |x ) by the discriminator . It is worth noting that the value of the game defines ( a constrained form of ) the Jensen-Shannon entropy and that Eq . 1 can be viewed as divergence minimization ( Nowozin et al. , 2016 ; Mescheder et al. , 2017 ) . After training , new objects can be generated by sampling random vectors z and mapping them to object space with the generator x = g ( z ) . Goodfellow et al . ( 2014 ) have shown that , under idealized assumptions , the learned generator g matches the data distribution , that is Pg = Pr . We report their theorem here for completeness : Theorem 1 . So long as ( i ) g and d are non-parametric , and ( ii ) the leftmost expectation in Eq . 1 is approximated arbitrarily well by the examples alone , any global equilibrium ( g∗ , d∗ ) of Eq . 1 satisfies Pd∗ ≡ 12 and Pg∗ ≡ Pr . Finding Nash equilibria of non-convex games like Eq . 1 is notoriously hard , rendering GAN training very challenging . The most common optimization algorithm is alternating gradient descent , whereby d and g are optimized sequentially , or the more well-understood simultaneous gradient descent ( Mescheder et al. , 2017 ) . A common failure state is mode collapse , whereby the generator outputs objects concentrated in a tiny portion of the object space . A plethora of theoretical and empirical remedies have been proposed , cf . ( Salimans et al. , 2016 ; Mescheder et al. , 2018 ) , including using alternative divergences ( Nowozin et al. , 2016 ; Arjovsky et al. , 2017 ) and encouraging smoothness of the discriminator by , e.g. , controlling the spectral norm of its parameters ( Miyato et al. , 2018 ) . In our experiments , we apply some of these strategies to stabilize training . In structured tasks , the objects of interest are usually discrete . GANs can be adapted to such settings by having the generator output a categorical distribution θ ( z ) overX and sampling objects from the latter . Below , we will focus on stochastic generators of this kind , although alternatives do exist ( Gulrajani et al. , 2017 ) . In this setting Pg ( x ) = ∫ Pg ( x|z ) p ( z ) dz = ∫ θ ( z ) p ( z ) dz = Ez [ θ ( z ) ] . 3 GENERATING STRUCTURES WITH CANS . Our goal is to learn to generate structures x consistent with respect to some validity constraint ψ according to some unobserved distribution Pr . Throughout , we will make the following assumptions : 1 . A single validity constraint ψ is provided as input . This is without loss of generality : if multiple constraints are necessary , then ψ can be taken to be their conjunction . 2 . The constraint ψ is compatible with the data distribution , that is , the support of Pr falls entirely within the feasible region determined by ψ . These assumptions hold in many tasks of interest , including all generative problems with known well-formedness requirements . For the sake of simplicity , we restrict our study to binary ( i.e . 0– 1 ) variables and logical constraints ( aka formulas ) only . This is a very general setup : any discrete structured space can be encoded using binary variables and formulas , at the cost of a larger model . Limitations of GANs Generators learned with the standard GAN training rule can output invalid structures , for two main reasons . First , for non-trivial constraints ψ , any finite set of examples that satisfy the constraint is insufficient to capture the full semantics of ψ . Second , in many cases of interest the examples are not even consistent with ψ . This more challenging case shows that regular GANs are easily lured into learning not to satisfy the constraint . More formally : Corollary 1 . Under the assumptions of Theorem 1 , given a target distribution Pr , a constraint ψ consistent with it , and a dataset of examples x sampled i.i.d . from a corrupted distribution P̃r 6= Pr inconsistent with ψ , GANs associate non-zero mass to infeasible objects . Indeed , by Theorem 1 , the optimal generator satisfies Pg ≡ P̃r , which is inconsistent with ψ. Ergo , ∑ x 1 { x 6|= ψ } Pg ( x ) > 0 . We stress that Theorem 1 captures the intent of GAN training , and thus this simple corollary shows that GANs are by design incapable of handling invalid examples . Constrained adversarial networks In order to avoid these issues , constrained adversarial networks ( CANs ) take both the examples and the validity constraint into account during learning . More specifically , the CAN value function is designed so that the generator maximizes the probability of generating valid structures Pg ( ψ ) : = Ex∼Pg [ 1 { x |= ψ } ] , that is : fCAN ( g , d ) : = fGAN ( g , d ) − λ logPg ( ψ ) ( 2 ) Here λ > 0 is a hyper-parameter controlling the importance of the constraint . Since the second term is always non-negative , the CAN value function upper bounds the GAN one ( Eq . 1 ) . The second term is the so-called semantic loss ( SL ) , proposed in ( Xu et al. , 2018 ) to inject knowledge into neural networks , and it is formally defined as SLψ ( g ) ∝ − logPg ( ψ ) , where : Pg ( ψ ) = ∑ x′ 1 { x′ |= ψ } Pg ( x′ ) = Ez [ ∑ x′∈X : x′|=ψ ∏ i : x′i=1 θi ( z ) ∏ i : x′i=0 ( 1− θi ( z ) ) ] ( 3 ) Recall that θ ( z ) is the categorical distribution output by the generator g for z , while x′ |= ψ means that x′ satisfies constraint ψ . Hence the sum runs over all configurations x′ consistent with ψ . The SL can also be viewed as the negative log-likelihood of ψ w.r.t . the generator . This shows that , in Eq . 2 , the SL rewards the generator g proportionally to the mass it allocates to valid structures . Since the SL is the negative logarithm of a polynomial in θ , it is fully differentiable ( so long as Pψ ( g ) 6= 0 , which is always the case in practice ) . If the SL is given large enough weight , CANs are strongly encouraged to generate valid structures in expectation . Under the preconditions of Theorem 1 , it can be shown that or λ→∞ CANs generate valid structures only : Proposition 1 . Under the assumptions of Corollary 1 , CANs associate zero mass to infeasible objects , irrespective of the discrepancy between Pr and P̃r . Intuitively , this holds because with λ =∞ any global equilibrium ( g∗ , d∗ ) of mingmaxd fCAN ( g , d ) must minimize the second term . If g is non-parametric , then the minimum is attained with logPg∗ ( ψ ) = 0 , which is equivalent to Pg∗ ( ψ ) = 1 . This in turn implies that Pg∗ ( ¬ψ ) = 0 , proving the claim . Of course , as with standard GANs , the prerequisites are often violated in practice . Regardless , Proposition 1 works as a sanity check , and shows that – in contrast to GANs – CANs are appropriate for constrained generative tasks . Computing the semantic loss Naı̈ve evaluation of the SL ( Eq . 3 ) involves summing over all ( exponentially many ) possible configurations x′ , which is intractable . More generally , computing the SL amounts to weighted model counting , which is # P-complete ( Chavira & Darwiche , 2008 ) . For many constraints of interest , however , the polynomial in Eq . 3 can be factorized into a much more compact form and evaluated very efficiently . Following Xu et al . ( 2018 ) , we make use of knowledge compilation to automatically factorize the polynomial by compiling it into an arithmetic circuit ( more specifically , into a sentential decision diagram ( SDD ) ( Darwiche , 2011 ) ) . When the resulting circuit is small enough , evaluation of the SL and of its gradient are extremely efficient , as shown in ( Xu et al. , 2018 ) . The main downside of knowledge compilation is that , depending on the complexity of the constraint ψ at hand , the compiled circuit may be very large . This is less of a problem during training , which is often performed on powerful machines , but it can be an issue for inference – especially on embedded devices . However , in CANs the circuit is not required for inference ( as it consists of a simple forward pass over the generator ) , and thus it can be thrown away after training . This means that CANs incur no space penalty during inference when compared to GANs . Dealing with intractable constraints When fed a particularly complex constraint , knowledge compilation may produce a circuit too large even for the training stage . In this case , we approximate the semantic loss by first mapping the objects from X to an application-specific space where ψ can be expressed in compact form , and then use the semantic loss on top of the transformed objects . We successfully employed this technique to synthesize mario levels where the goal tile is reachable from the starting tile ; all details are provided below . The same technique can be exploited for dealing with very complex logical formulas beyond the reach of state-of-the-art knowledge compilation .

Summary: In this paper the authors present a Generative Adversarial Neural Networks with Xu et al.’s semantic loss applied to the generator. They call this GAN a Constrained Adversarial Network or (CAN) and identify it as a new class of GAN. The authors present three different problem domains for their experiments focused on the generation of constrained images, chunks of Super Mario Bros.-style levels, and molecules. For each domain they include particular constraints for the semantic loss, which biases the generator towards creating valid content according to these constraints.