Efficient generation of structured objects with Constrained Adversarial Networks

1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) have shown impressive performance on challenging tasks – like image generation ( Karras et al. , 2018 ) , text-to-image ( Zhang et al. , 2017 ) , and style transfer ( Zhu et al. , 2017 ) – where the goal is to produce believable configurations . A number of important applications , however , require to generate objects that are both credible and feasible with respect to hard structural constraints . Examples include generating drug molecules , which must satisfy chemical validity requirements , and game levels , where the goal must be reachable from the starting position . Recent studies ( Guimaraes et al. , 2017 ; De Cao & Kipf , 2018 ; Xue & van Hoeve , 2019 ) observed that GANs struggle in these tasks . The main reason is that the training examples alone are insufficient to capture the feasibility constraints and thus to guide the model toward producing valid objects . As a remedy , we propose Constrained Adversarial Networks ( CANs ) , a class of generative models that extend GANs to structured domains . Given a set of examples drawn from a latent distribution and a set of structural constraints , CANs learn to output valid structured with high probability . This is achieved by augmenting the standard GAN loss with a penalty term that discourages the model from producing infeasible structures . Since CANs inject the constraints directly into the learned model , valid structures can be obtained with high probability by performing forward inference on the generator , avoiding the need for costly sampling or optimization steps ( Volz et al. , 2018 ) . The penalty term is implemented using the semantic loss ( SL ) ( Xu et al. , 2018 ) . The SL leverages knowledge compilation ( Darwiche , 2011 ) to represent arbitrary Boolean constraint as a circuit and uses the latter to measure the mass allocated by the generator to infeasible objects . The procedure is probabilistically sound , exact , and does not require sampling . Notably , the circuit , which can be quite large depending on the complexity of the constraints , can be thrown away after learning . In addition , we show how to extend the SL complex constraints that would normally lead to intractably large circuits . This is accomplished by first using a neural network to map configurations to a different space in which the structural constraints can be compactly encoded , and then applying the SL to the latter . This enables us to deal with reachability on a graph , which is beyond the reach of standard SL . Finally , we show how the validity constraints baked into the generator can be enabled or disabled during inference using ideas from InfoGANs Chen et al . ( 2016 ) . In summary , we contribute : • Constrained Adversarial Networks ( CANs ) , a generalization of GANs in which the generator is encouraged ( at training time ) to output valid structures with high probability via forward inference . • An extensive empirical analysis on structured objects like constrained images , molecules , and video game levels showing that CANs generate structures that are stylistically coherent with the training data . • A decomposition of intractably complex constraints into a neural and logical components , showcased on graph reachability for video game level synthesis . 2 UNCONSTRAINED GANS . Let X be the object space and Z some latent space . GANs ( Goodfellow et al. , 2014 ) are composed of a generator g : Z → X and a discriminator d : X → [ 0 , 1 ] , both implemented as neural networks . The discriminator is trained to recognize “ real ” objects x taken from the data distribution Pr , while the generator is trained to map random latent vectors z ∈ Rm to “ fake ” objects that fool the discriminator . Learning amounts to solving the minimax game : min g max d fGAN ( g , d ) fGAN ( g , d ) : = Ex∼Pr [ logPd ( x ) ] + Ex∼Pg [ log ( 1− Pd ( x ) ) ] ( 1 ) Here Pg ( x ) is the distribution induced by the generator and Pd ( x ) : = Pd ( real |x ) by the discriminator . It is worth noting that the value of the game defines ( a constrained form of ) the Jensen-Shannon entropy and that Eq . 1 can be viewed as divergence minimization ( Nowozin et al. , 2016 ; Mescheder et al. , 2017 ) . After training , new objects can be generated by sampling random vectors z and mapping them to object space with the generator x = g ( z ) . Goodfellow et al . ( 2014 ) have shown that , under idealized assumptions , the learned generator g matches the data distribution , that is Pg = Pr . We report their theorem here for completeness : Theorem 1 . So long as ( i ) g and d are non-parametric , and ( ii ) the leftmost expectation in Eq . 1 is approximated arbitrarily well by the examples alone , any global equilibrium ( g∗ , d∗ ) of Eq . 1 satisfies Pd∗ ≡ 12 and Pg∗ ≡ Pr . Finding Nash equilibria of non-convex games like Eq . 1 is notoriously hard , rendering GAN training very challenging . The most common optimization algorithm is alternating gradient descent , whereby d and g are optimized sequentially , or the more well-understood simultaneous gradient descent ( Mescheder et al. , 2017 ) . A common failure state is mode collapse , whereby the generator outputs objects concentrated in a tiny portion of the object space . A plethora of theoretical and empirical remedies have been proposed , cf . ( Salimans et al. , 2016 ; Mescheder et al. , 2018 ) , including using alternative divergences ( Nowozin et al. , 2016 ; Arjovsky et al. , 2017 ) and encouraging smoothness of the discriminator by , e.g. , controlling the spectral norm of its parameters ( Miyato et al. , 2018 ) . In our experiments , we apply some of these strategies to stabilize training . In structured tasks , the objects of interest are usually discrete . GANs can be adapted to such settings by having the generator output a categorical distribution θ ( z ) overX and sampling objects from the latter . Below , we will focus on stochastic generators of this kind , although alternatives do exist ( Gulrajani et al. , 2017 ) . In this setting Pg ( x ) = ∫ Pg ( x|z ) p ( z ) dz = ∫ θ ( z ) p ( z ) dz = Ez [ θ ( z ) ] . 3 GENERATING STRUCTURES WITH CANS . Our goal is to learn to generate structures x consistent with respect to some validity constraint ψ according to some unobserved distribution Pr . Throughout , we will make the following assumptions : 1 . A single validity constraint ψ is provided as input . This is without loss of generality : if multiple constraints are necessary , then ψ can be taken to be their conjunction . 2 . The constraint ψ is compatible with the data distribution , that is , the support of Pr falls entirely within the feasible region determined by ψ . These assumptions hold in many tasks of interest , including all generative problems with known well-formedness requirements . For the sake of simplicity , we restrict our study to binary ( i.e . 0– 1 ) variables and logical constraints ( aka formulas ) only . This is a very general setup : any discrete structured space can be encoded using binary variables and formulas , at the cost of a larger model . Limitations of GANs Generators learned with the standard GAN training rule can output invalid structures , for two main reasons . First , for non-trivial constraints ψ , any finite set of examples that satisfy the constraint is insufficient to capture the full semantics of ψ . Second , in many cases of interest the examples are not even consistent with ψ . This more challenging case shows that regular GANs are easily lured into learning not to satisfy the constraint . More formally : Corollary 1 . Under the assumptions of Theorem 1 , given a target distribution Pr , a constraint ψ consistent with it , and a dataset of examples x sampled i.i.d . from a corrupted distribution P̃r 6= Pr inconsistent with ψ , GANs associate non-zero mass to infeasible objects . Indeed , by Theorem 1 , the optimal generator satisfies Pg ≡ P̃r , which is inconsistent with ψ. Ergo , ∑ x 1 { x 6|= ψ } Pg ( x ) > 0 . We stress that Theorem 1 captures the intent of GAN training , and thus this simple corollary shows that GANs are by design incapable of handling invalid examples . Constrained adversarial networks In order to avoid these issues , constrained adversarial networks ( CANs ) take both the examples and the validity constraint into account during learning . More specifically , the CAN value function is designed so that the generator maximizes the probability of generating valid structures Pg ( ψ ) : = Ex∼Pg [ 1 { x |= ψ } ] , that is : fCAN ( g , d ) : = fGAN ( g , d ) − λ logPg ( ψ ) ( 2 ) Here λ > 0 is a hyper-parameter controlling the importance of the constraint . Since the second term is always non-negative , the CAN value function upper bounds the GAN one ( Eq . 1 ) . The second term is the so-called semantic loss ( SL ) , proposed in ( Xu et al. , 2018 ) to inject knowledge into neural networks , and it is formally defined as SLψ ( g ) ∝ − logPg ( ψ ) , where : Pg ( ψ ) = ∑ x′ 1 { x′ |= ψ } Pg ( x′ ) = Ez [ ∑ x′∈X : x′|=ψ ∏ i : x′i=1 θi ( z ) ∏ i : x′i=0 ( 1− θi ( z ) ) ] ( 3 ) Recall that θ ( z ) is the categorical distribution output by the generator g for z , while x′ |= ψ means that x′ satisfies constraint ψ . Hence the sum runs over all configurations x′ consistent with ψ . The SL can also be viewed as the negative log-likelihood of ψ w.r.t . the generator . This shows that , in Eq . 2 , the SL rewards the generator g proportionally to the mass it allocates to valid structures . Since the SL is the negative logarithm of a polynomial in θ , it is fully differentiable ( so long as Pψ ( g ) 6= 0 , which is always the case in practice ) . If the SL is given large enough weight , CANs are strongly encouraged to generate valid structures in expectation . Under the preconditions of Theorem 1 , it can be shown that or λ→∞ CANs generate valid structures only : Proposition 1 . Under the assumptions of Corollary 1 , CANs associate zero mass to infeasible objects , irrespective of the discrepancy between Pr and P̃r . Intuitively , this holds because with λ =∞ any global equilibrium ( g∗ , d∗ ) of mingmaxd fCAN ( g , d ) must minimize the second term . If g is non-parametric , then the minimum is attained with logPg∗ ( ψ ) = 0 , which is equivalent to Pg∗ ( ψ ) = 1 . This in turn implies that Pg∗ ( ¬ψ ) = 0 , proving the claim . Of course , as with standard GANs , the prerequisites are often violated in practice . Regardless , Proposition 1 works as a sanity check , and shows that – in contrast to GANs – CANs are appropriate for constrained generative tasks . Computing the semantic loss Naı̈ve evaluation of the SL ( Eq . 3 ) involves summing over all ( exponentially many ) possible configurations x′ , which is intractable . More generally , computing the SL amounts to weighted model counting , which is # P-complete ( Chavira & Darwiche , 2008 ) . For many constraints of interest , however , the polynomial in Eq . 3 can be factorized into a much more compact form and evaluated very efficiently . Following Xu et al . ( 2018 ) , we make use of knowledge compilation to automatically factorize the polynomial by compiling it into an arithmetic circuit ( more specifically , into a sentential decision diagram ( SDD ) ( Darwiche , 2011 ) ) . When the resulting circuit is small enough , evaluation of the SL and of its gradient are extremely efficient , as shown in ( Xu et al. , 2018 ) . The main downside of knowledge compilation is that , depending on the complexity of the constraint ψ at hand , the compiled circuit may be very large . This is less of a problem during training , which is often performed on powerful machines , but it can be an issue for inference – especially on embedded devices . However , in CANs the circuit is not required for inference ( as it consists of a simple forward pass over the generator ) , and thus it can be thrown away after training . This means that CANs incur no space penalty during inference when compared to GANs . Dealing with intractable constraints When fed a particularly complex constraint , knowledge compilation may produce a circuit too large even for the training stage . In this case , we approximate the semantic loss by first mapping the objects from X to an application-specific space where ψ can be expressed in compact form , and then use the semantic loss on top of the transformed objects . We successfully employed this technique to synthesize mario levels where the goal tile is reachable from the starting tile ; all details are provided below . The same technique can be exploited for dealing with very complex logical formulas beyond the reach of state-of-the-art knowledge compilation .

Summary: The authors describe a method to improve the performance of generative adversarial networks in the task of generating structured objectives that have to satisfy complicated constraints. The proposed solution involves using an additional term in the GAN objective that penalizes the generation of invalid samples. This term, called the semantic loss, is given by a multiple of the log probability of the model generating valid samples.