Original: E FFICIENT G ENERATION OF S TRUCTURED O BJECTS WITH C ONSTRAINED A DVERSARIAL N ETWORKS
Revised: E FFICIENT G ENERATION OF S TRUCTURED O BJECTS WITH C ONSTRAINED A DVERSARIAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Despite their success, generative adversarial networks (GANs) cannot easily generate structured objects like molecules or game maps.
Revised: Despite their success, generative adversarial networks (GANs) cannot easily generate structured objects like molecules or game maps.

Original: The issue is that such objects must satisfy structural requirements (e.g., molecules must be chemically valid, game maps must guarantee reachability of the end goal) that are difﬁcult to capture with examples alone.
Revised: The issue is that such objects must satisfy structural requirements (e.g., molecules must be chemically valid, game maps must guarantee reachability of the end goal) that are difﬁcult to capture with examples alone.

Original: As a remedy, we propose constrained adversarial networks (CANs), which embed the constraints into the model during training by penalizing the generator whenever it outputs invalid structures.
Revised: As a remedy, we propose constrained adversarial networks (CANs), which embed the constraints into the model during training by penalizing the generator whenever it outputs invalid structures.

Original: As in unconstrained GANs, new objects can be sampled straightforwardly from the generator, but in addition they satisfy the constraints with high probability.
Revised: As in unconstrained GANs, new objects can be sampled straightforwardly from the generator, but in addition they satisfy the constraints with high probability.

Original: Our approach handles arbitrary logical constraints and leverages knowledge compilation techniques to efﬁciently evaluate the expected disagreement between the model and the constraints.
Revised: Our approach handles arbitrary logical constraints and leverages knowledge compilation techniques to efﬁciently evaluate the expected disagreement between the model and the constraints.

Original: This setup is further extended to hybrid logical-neural constraints for capturing complex requirements like graph reachability.
Revised: This setup is further extended to hybrid logical-neural constraints for capturing complex requirements like graph reachability.

Original: An extensive empirical analysis on constrained images, molecules, and video game levels shows that
Revised: An extensive empirical analysis on constrained images, molecules, and video game levels shows that

Original: CANs efﬁciently generate valid structures that are both high-quality and novel.
Revised: CANs efﬁciently generate valid structures that are both high-quality and novel.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) have shown impressive performance on challenging tasks – like image generation (Karras et al., 2018), text-to-image (Zhang et al., 2017), and style transfer (Zhu et al., 2017) – where the goal is to produce believable conﬁgurations.
Revised: Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) have shown impressive performance on challenging tasks – like image generation (Karras et al., 2018), text-to-image (Zhang et al., 2017), and style transfer (Zhu et al., 2017) – where the goal is to produce believable conﬁgurations.

Original: A number of important applications, however, require to generate objects that are both credible and feasible with respect to hard structural constraints.
Revised: A number of important applications, however, require to generate objects that are both credible and feasible with respect to hard structural constraints.

Original: Examples include generating drug molecules, which must satisfy chemical validity requirements, and game levels, where the goal must be reachable from the starting position.
Revised: Examples include generating drug molecules, which must satisfy chemical validity requirements, and game levels, where the goal must be reachable from the starting position.

Original: Recent studies (Guimaraes et al., 2017; De Cao & Kipf, 2018; Xue & van Hoeve, 2019) observed that GANs struggle in these tasks.
Revised: Recent studies (Guimaraes et al., 2017; De Cao & Kipf, 2018; Xue & van Hoeve, 2019) observed that GANs struggle in these tasks.

Original: The main reason is that the training examples alone are insufﬁcient to capture the feasibility constraints and thus to guide the model toward producing valid objects.
Revised: The main reason is that the training examples alone are insufﬁcient to capture the feasibility constraints and thus to guide the model toward producing valid objects.

Original: As a remedy, we propose Constrained Adversarial Networks (CANs), a class of generative models that extend GANs to structured domains.
Revised: As a remedy, we propose Constrained Adversarial Networks (CANs), a class of generative models that extend GANs to structured domains.

Original: Given a set of examples drawn from a latent distribution and a set of structural constraints, CANs learn to output valid structured with high probability.
Revised: Given a set of examples drawn from a latent distribution and a set of structural constraints, CANs learn to output valid structured with high probability.

Original: This is achieved by augmenting the standard GAN loss with a penalty term that discourages the model from producing infeasible structures.
Revised: This is achieved by augmenting the standard GAN loss with a penalty term that discourages the model from producing infeasible structures.

Original: Since CANs inject the constraints directly into the learned model, valid structures can be obtained with high probability by performing forward inference on the generator, avoiding the need for costly sampling or optimization steps (Volz et al., 2018).
Revised: Since CANs inject the constraints directly into the learned model, valid structures can be obtained with high probability by performing forward inference on the generator, avoiding the need for costly sampling or optimization steps (Volz et al., 2018).

Original: The penalty term is implemented using the semantic loss (SL) (Xu et al., 2018).
Revised: The penalty term is implemented using the semantic loss (SL) (Xu et al., 2018).

Original: The SL leverages knowledge compilation (Darwiche, 2011) to represent arbitrary Boolean constraint as a circuit and uses the latter to measure the mass allocated by the generator to infeasible objects.
Revised: The SL leverages knowledge compilation (Darwiche, 2011) to represent arbitrary Boolean constraint as a circuit and uses the latter to measure the mass allocated by the generator to infeasible objects.

Original: The procedure is probabilistically sound, exact, and does not require sampling.
Revised: The procedure is probabilistically sound, exact, and does not require sampling.

Original: Notably, the circuit, which can be quite large depending on the complexity of the constraints, can be thrown away after learning.
Revised: Notably, the circuit, which can be quite large depending on the complexity of the constraints, can be thrown away after learning.

Original: In addition, we show how to extend the SL complex constraints that would normally lead to intractably large circuits.
Revised: In addition, we show how to extend the SL complex constraints that would normally lead to intractably large circuits.

Original: This is accomplished by ﬁrst using a neural network to map conﬁgurations to a different space in which the structural constraints can be compactly encoded, and then applying the SL to the latter.
Revised: This is accomplished by ﬁrst using a neural network to map conﬁgurations to a different space in which the structural constraints can be compactly encoded, and then applying the SL to the latter.

Original: This enables us to deal with reachability on a graph, which is beyond the reach of standard SL.
Revised: This enables us to deal with reachability on a graph, which is beyond the reach of standard SL.

Original: Finally, we show how the validity constraints baked into the generator can be enabled or disabled during inference using ideas from InfoGANs Chen et al.
Revised: Finally, we show how the validity constraints baked into the generator can be enabled or disabled during inference using ideas from InfoGANs Chen et al.

Original: In summary, we contribute:
Revised: In summary, we contribute:

Original: • Constrained Adversarial Networks (CANs), a new class of GANs in which the generator is encouraged (at training time) to output valid structures with high probability via forward inference.
Revised: • Constrained Adversarial Networks (CANs), a generalization of GANs in which the generator is encouraged (at training time) to output valid structures with high probability via forward inference.

Original: • An extensive empirical analysis on structured objects like constrained images, molecules, and video game levels showing that CANs generate structures that are stylistically coherent with the training data.
Revised: • An extensive empirical analysis on structured objects like constrained images, molecules, and video game levels showing that CANs generate structures that are stylistically coherent with the training data.

Original: • A decomposition of intractably complex constraints into a neural and logical components, showcased on graph reachability for video game level synthesis.
Revised: • A decomposition of intractably complex constraints into a neural and logical components, showcased on graph reachability for video game level synthesis.

Original:  U NCONSTRAINED GAN S
Revised:  U NCONSTRAINED GAN S

Original: Let X be the object space and Z some latent space.
Revised: Let X be the object space and Z some latent space.

Original: GANs (Goodfellow et al., 2014) are composed of a generator g : Z → X and a discriminator d : X → [0 , 1] , both implemented as neural networks.
Revised: GANs (Goodfellow et al., 2014) are composed of a generator g : Z → X and a discriminator d : X → [0 , 1] , both implemented as neural networks.

Original: The discriminator is trained to recognize “real” objects x taken from the data distribution
Revised: The discriminator is trained to recognize “real” objects x taken from the data distribution

Original: P r , while the generator is trained to map random latent vectors z ∈ R m to “fake” objects that fool the discriminator.
Revised: P r , while the generator is trained to map random latent vectors z ∈ R m to “fake” objects that fool the discriminator.

Original: Learning amounts to solving the minimax game:
Revised: Learning amounts to solving the minimax game:

Original: [Equation]
Revised: [Equation]

Original:  Here P g ( x ) is the distribution induced by the generator and P d ( x ) :
Revised: [Equation] Here P g ( x ) is the distribution induced by the generator and P d ( x ) :

Original: = P d ( real | x ) by the discriminator.
Revised: = P d ( real | x ) by the discriminator.

Original: It is worth noting that the value of the game deﬁnes (a constrained form of) the Jensen-Shannon entropy and that Eq. 1 can be viewed as divergence minimization (Nowozin et al., 2016; Mescheder et al., 2017).
Revised: It is worth noting that the value of the game deﬁnes (a constrained form of) the Jensen-Shannon entropy and that Eq. 1 can be viewed as divergence minimization (Nowozin et al., 2016; Mescheder et al., 2017).

Original: After training, new objects can be generated by sampling random vectors z and mapping them to object space with the generator x = g ( z ) .
Revised: After training, new objects can be generated by sampling random vectors z and mapping them to object space with the generator x = g ( z ) .

Original: Goodfellow et al.
Revised: Goodfellow et al.

Original: (2014) have shown that, under idealized assumptions, the learned generator g matches the data distribution, that is P g
Revised: (2014) have shown that, under idealized assumptions, the learned generator g matches the data distribution, that is P g

Original: = P r .
Revised: = P r .

Original: We report their theorem here for completeness:
Revised: We report their theorem here for completeness:

Original: Theorem 1.
Revised: Theorem 1.

Original: So long as (i) g and d are non-parametric, and (ii) the leftmost expectation in Eq.is approximated arbitrarily well by the examples alone, any global equilibrium ( g ∗ , d ∗ ) of Eq.satisﬁes P d ∗ ≡ 12 and P g ∗ ≡ P r .
Revised: So long as (i) g and d are non-parametric, and (ii) the leftmost expectation in Eq.is approximated arbitrarily well by the examples alone, any global equilibrium ( g ∗ , d ∗ ) of Eq.satisﬁes P d ∗ ≡ 12 and P g ∗ ≡ P r .

Original: Finding Nash equilibria of non-convex games like Eq.
Revised: Finding Nash equilibria of non-convex games like Eq.

Original: 1 is notoriously hard, rendering GAN training very challenging.
Revised: 1 is notoriously hard, rendering GAN training very challenging.

Original: The most common optimization algorithm is alternating gradient descent, whereby d and g are optimized sequentially, or the more well-understood simultaneous gradient descent (Mescheder et al., 2017).
Revised: The most common optimization algorithm is alternating gradient descent, whereby d and g are optimized sequentially, or the more well-understood simultaneous gradient descent (Mescheder et al., 2017).

Original: A common failure state is mode collapse, whereby the generator outputs objects concentrated in a tiny portion of the object space.
Revised: A common failure state is mode collapse, whereby the generator outputs objects concentrated in a tiny portion of the object space.

Original: A plethora of theoretical and empirical remedies have been proposed, cf.
Revised: A plethora of theoretical and empirical remedies have been proposed, cf.

Original: (Salimans et al., 2016; Mescheder et al., 2018), including using alternative divergences (Nowozin et al., 2016;
Revised: (Salimans et al., 2016; Mescheder et al., 2018), including using alternative divergences (Nowozin et al., 2016;

Original: Arjovsky et al., 2017) and encouraging smoothness of the discriminator by, e.g., controlling the spectral norm of its parameters (Miyato et al., 2018).
Revised: Arjovsky et al., 2017) and encouraging smoothness of the discriminator by, e.g., controlling the spectral norm of its parameters (Miyato et al., 2018).

Original: In our experiments, we apply some of these strategies to stabilize training.
Revised: In our experiments, we apply some of these strategies to stabilize training.

Original: In structured tasks, the objects of interest are usually discrete.
Revised: In structured tasks, the objects of interest are usually discrete.

Original: GANs can be adapted to such settings by having the generator output a categorical distribution θ ( z ) over X and sampling objects from the latter.
Revised: GANs can be adapted to such settings by having the generator output a categorical distribution θ ( z ) over X and sampling objects from the latter.

Original: Below, we will focus on stochastic generators of this kind, although alternatives do exist (Gulrajani et al., 2017).
Revised: Below, we will focus on stochastic generators of this kind, although alternatives do exist (Gulrajani et al., 2017).

Original: In this setting P g ( x ) = (cid:82) P g ( x | z ) p ( z ) d z = (cid:82) θ ( z ) p ( z ) d z = E z [ θ ( z )] .
Revised: In this setting P g ( x ) = (cid:82) P g [Equation] | [Equation]

Revised: p

Revised: [Equation]

Original: G ENERATING S TRUCTURES WITH CAN S
Revised: (cid:82) [Equation] p [Equation]

Original:  Our goal is to learn to generate structures x consistent with respect to some validity constraint ψ according to some unobserved distribution P r .
Revised: z [ [Equation]  G ENERATING S TRUCTURES WITH CAN S Our goal is to learn to generate structures x consistent with respect to some validity constraint ψ according to some unobserved distribution P r .

Original: Throughout, we will make the following assumptions:
Revised: Throughout, we will make the following assumptions:

Original: A single validity constraint ψ is provided as input.
Revised: A single validity constraint ψ is provided as input.

Original: This is without loss of generality: if multiple constraints are necessary, then ψ can be taken to be their conjunction.
Revised: This is without loss of generality: if multiple constraints are necessary, then ψ can be taken to be their conjunction.

Original: The constraint ψ is compatible with the data distribution, that is, the support of P r falls entirely within the feasible region determined by ψ.
Revised: The constraint ψ is compatible with the data distribution, that is, the support of P r falls entirely within the feasible region determined by ψ

Original: These assumptions hold in many tasks of interest, including all generative problems with known well-formedness requirements.
Revised: These assumptions hold in many tasks of interest, including all generative problems with known well-formedness requirements.

Original: For the sake of simplicity, wealso restrict our study to binary (i.e. 0–1) variables and logical constraints (aka formulas) only.
Revised: For the sake of simplicity, we restrict our study to binary (i.e. 0– 1) variables and logical constraints (aka formulas) only.

Original: This is a very general setup: any discrete structured space can be encoded using binary variables and formulas, at the cost of a larger model.
Revised: This is a very general setup: any discrete structured space can be encoded using binary variables and formulas, at the cost of a larger model.

Original: Limitations of GANs Generators learned with the standard GAN training rule can output invalid structures, for two main reasons.
Revised: Limitations of GANs Generators learned with the standard GAN training rule can output invalid structures, for two main reasons.

Original: First, for non-trivial constraints ψ , any ﬁnite set of examples that satisfy the constraint is insufﬁcient to capture the full semantics of ψ .
Revised: First, for non-trivial constraints ψ , any ﬁnite set of examples that satisfy the constraint is insufﬁcient to capture the full semantics of ψ .

Original: Second, in many cases of interest the examples are not even consistent with ψ .
Revised: Second, in many cases of interest the examples are not even consistent with ψ .

Original: This more challenging case shows that regular
Revised: This more challenging case shows that regular

Original: GANs are easily lured into learning not to satisfy the constraint.
Revised: GANs are easily lured into learning not to satisfy the constraint.

Original: More formally:
Revised: More formally:

Original: Corollary 1.
Revised: Corollary 1.

Original: Under the assumptions of Theorem 1, given a target distribution P r , a constraint ψ consistent with it, and a dataset of examples x sampled i.i.d.
Revised: Under the assumptions of Theorem 1, given a target distribution P r , a constraint ψ consistent with it, and a dataset of examples x sampled i.i.d.

Original: from a corrupted distribution ˜ P
Revised: from a corrupted distribution ˜ P

Original: r (cid:54) =
Revised: r (cid:54) =

Original: P r inconsistent with ψ , GANs associate non-zero mass to infeasible objects.
Revised: P r inconsistent with ψ , GANs associate non-zero mass to infeasible objects.

Original: Indeed, by Theorem 1, the optimal generator satisﬁes P g ≡ ˜ P r , which is inconsistent with ψ .
Revised: Indeed, by Theorem 1, the optimal generator satisﬁes P g ≡ ˜ P r , which is inconsistent with ψ .

Original: Ergo, (cid:80) x 1 { x (cid:54)| = ψ } P g ( x ) > 0 .
Revised: Ergo, (cid:80) x 1 { x (cid:54)| = ψ } P g ( x ) > 0 .

Original: We stress that Theorem 1 captures the intent of GAN training, and thus this simple corollary shows that GANs are by design incapable of handling invalid examples.
Revised: We stress that Theorem 1 captures the intent of GAN training, and thus this simple corollary shows that GANs are by design incapable of handling invalid examples.

Original: Constrained adversarial networks In order to avoid these issues, constrained adversarial networks (CANs) take both the examples and the validity constraint into account during learning.
Revised: Constrained adversarial networks In order to avoid these issues, constrained adversarial networks (CANs) take both the examples and the validity constraint into account during learning.

Original: More speciﬁcally, the CAN value function is designed so that the generator maximizes the probability of generating valid structures P g ( ψ ) :
Revised: More speciﬁcally, the CAN value function is designed so that the generator maximizes the probability of generating valid structures P g ( ψ ) :

Original: = E x ∼ P g [ 1 { x | = ψ } ] , that is:
Revised: = E x ∼ P g [ 1 { x | = ψ } ] , that is:

Original: [Equation]
Revised: [Equation]

Original: Here λ > 0 is a hyper-parameter controlling the importance of the constraint.
Revised: Here λ > 0 is a hyper-parameter controlling the importance of the constraint.

Original: Since the second term is always non-negative, the CAN value function upper bounds the GAN one (Eq. 1).
Revised: Since the second term is always non-negative, the CAN value function upper bounds the GAN one (Eq. 1).

Original: The second term is the so-called semantic loss (SL), proposed in (Xu et al., 2018) to inject knowledge into neural networks, and it is formally deﬁned as SL ψ ( g ) ∝ − log P g ( ψ ) , where:
Revised: The second term is the so-called semantic loss (SL), proposed in (Xu et al., 2018) to inject knowledge into neural networks, and it is formally deﬁned as SL ψ ( g ) ∝ − log P g ( ψ ) , where:

Original: [Equation]
Revised: [Equation]

Original: Here x (cid:48) | = ψ means that x (cid:48) satisﬁes constraint ψ and the sum runs over all conﬁgurations x (cid:48) consistent with ψ .
Revised: Recall that θ ( z ) is the categorical distribution output by the generator g for z , while x (cid:48) | = ψ means that x (cid:48) satisﬁes constraint ψ . Hence the sum runs over all conﬁgurations x (cid:48) consistent with ψ .

Original: The SL can also be viewed as the negative log-likelihood of ψ w.r.t.
Revised: The SL can also be viewed as the negative log-likelihood of ψ w.r.t.

Original: the generator.
Revised: the generator.

Original: This shows that, in Eq. 2, the SL rewards the generator g proportionally to the mass it allocates to valid structures.
Revised: This shows that, in Eq. 2, the SL rewards the generator g proportionally to the mass it allocates to valid structures.

Original: Since the SL is the negative logarithm of a polynomial in θ , it is fully differentiable (so long as P ψ ( g ) (cid:54) = 0 , which is always the case in practice).
Revised: Since the SL is the negative logarithm of a polynomial in θ , it is fully differentiable (so long as P ψ ( g ) (cid:54) = 0 , which is always the case in practice).

Original: If the SL is given large enough weight, CANs are strongly encouraged to generate valid structures in expectation.
Revised: If the SL is given large enough weight, CANs are strongly encouraged to generate valid structures in expectation.

Original: Under the preconditions of Theorem 1, it can be shown that or λ → ∞ CANs generate valid structures only :
Revised: Under the preconditions of Theorem 1, it can be shown that or λ → ∞ CANs generate valid structures only :

Original: Proposition 1.
Revised: Proposition 1.

Original: Under the assumptions of Corollary 1, CANs associate zero mass to infeasible objects, irrespective of the discrepancy between P r and ˜ P r .
Revised: Under the assumptions of Corollary 1, CANs associate zero mass to infeasible objects, irrespective of the discrepancy between P r and ˜ P r .

Original: Intuitively, this holds because with λ = ∞ any global equilibrium ( g ∗ , d ∗ ) of min g max d f CAN ( g, d ) must minimize the second term.
Revised: Intuitively, this holds because with λ = ∞ any global equilibrium ( g ∗ , d ∗ ) of min g max d f CAN ( g, d ) must minimize the second term.

Original: If g is non-parametric, then the minimum is attained with log P g ∗ ( ψ ) = 0 , which is equivalent to P g ∗ ( ψ )
Revised: If g is non-parametric, then the minimum is attained with log P g ∗ ( ψ ) = 0 , which is equivalent to P g ∗ ( ψ )

Original: = 1 .
Revised: = 1 .

Original: This in turn implies that P g ∗ ( ¬ ψ ) = 0 , proving the claim.
Revised: This in turn implies that P g ∗ ( ¬ ψ ) = 0 , proving the claim.

Original: Of course, as with standard GANs, the prerequisites are often violated in practice.
Revised: Of course, as with standard GANs, the prerequisites are often violated in practice.

Original: Regardless, Proposition 1 works as a sanity check, and shows that – in contrast to GANs –
Revised: Regardless, Proposition 1 works as a sanity check, and shows that – in contrast to GANs –

Original: CANs are appropriate for constrained generative tasks.
Revised: CANs are appropriate for constrained generative tasks.

Original: Computing the semantic loss Na¨ıve evaluation of the SL  involves summing over all (exponentially many) possible conﬁgurations x (cid:48) , which is intractable.
Revised: Computing the semantic loss Na¨ıve evaluation of the SL (Eq. 3) involves summing over all (exponentially many) possible conﬁgurations x (cid:48) , which is intractable.

Original: More generally, computing the SL amounts to weighted model counting, which is #P-complete (Chavira & Darwiche, 2008).
Revised: More generally, computing the SL amounts to weighted model counting, which is #P-complete (Chavira & Darwiche, 2008).

Original: For many constraints of interest, however, the polynomial in Eq.
Revised: For many constraints of interest, however, the polynomial in Eq.

Original: 3 can be factorized into a much more compact form and evaluated very efﬁciently.
Revised: 3 can be factorized into a much more compact form and evaluated very efﬁciently.

Original: Following Xu et al.
Revised: Following Xu et al.

Original: (2018), we make use of knowledge compilation to automatically factorize the polynomial by compiling it into an arithmetic circuit (more speciﬁcally, into a sentential decision diagram (SDD) (Darwiche, 2011)).
Revised: (2018), we make use of knowledge compilation to automatically factorize the polynomial by compiling it into an arithmetic circuit (more speciﬁcally, into a sentential decision diagram (SDD) (Darwiche, 2011)).

Original: When the resulting circuit is small enough, evaluation of the SL and of its gradient are extremely efﬁcient, as shown in (Xu et al., 2018).
Revised: When the resulting circuit is small enough, evaluation of the SL and of its gradient are extremely efﬁcient, as shown in (Xu et al., 2018).

Original: The main downside of knowledge compilation is that, depending on the complexity of the constraint ψ at hand, the compiled circuit may be very large.
Revised: The main downside of knowledge compilation is that, depending on the complexity of the constraint ψ at hand, the compiled circuit may be very large.

Original: This is less of a problem during training, which is often performed on powerful machines, but it can be an issue for inference – especially on embedded devices.
Revised: This is less of a problem during training, which is often performed on powerful machines, but it can be an issue for inference – especially on embedded devices.

Original: However, in CANs the circuit is not required for inference (as it consists of a simple forward pass over the generator), and thus it can be thrown away after training.
Revised: However, in CANs the circuit is not required for inference (as it consists of a simple forward pass over the generator), and thus it can be thrown away after training.

Original: This means that CANs incur no space penalty during inference when compared to GANs.
Revised: This means that CANs incur no space penalty during inference when compared to GANs.

Original: Dealing with intractable constraints When fed a particularly complex constraint, knowledge compilation may produce a circuit too large even for the training stage.
Revised: Dealing with intractable constraints When fed a particularly complex constraint, knowledge compilation may produce a circuit too large even for the training stage.

Original: In this case, we approximate the semantic loss by ﬁrst mapping the objects from X to an application-speciﬁc space where ψ can be expressed in compact form, and then use the semantic loss on top of the transformed objects.
Revised: In this case, we approximate the semantic loss by ﬁrst mapping the objects from X to an application-speciﬁc space where ψ can be expressed in compact form, and then use the semantic loss on top of the transformed objects.

Original: We successfully employed this technique to synthesize mario levels where the goal tile is reachable from the starting tile; all details are provided below.
Revised: We successfully employed this technique to synthesize mario levels where the goal tile is reachable from the starting tile; all details are provided below.

Original: The same technique can be exploited for dealing with very complex logical formulas beyond the reach of state-of-the-art knowledge compilation.
Revised: The same technique can be exploited for dealing with very complex logical formulas beyond the reach of state-of-the-art knowledge compilation.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We implemented CANs using Tensorﬂow and  PySDD 1 and tested them using different generator architectures on one synthetic and two real-world structured generative tasks.
Revised: We implemented CANs using Tensorﬂow and used PySDD 1 to compile the constraint into a compact circuit for computing the Semantic Loss. We tested CANs using different generator architectures on one synthetic and two real-world structured generative tasks 2 .

Original: In all cases, we compared the objects generated by CANs against those output by strong baselines using three metrics (adopted from Samanta et al.
Revised: In all cases, we evaluated the objects generated by CANs and those of the baselines using three metrics (adopted from Samanta et al.

Original: • validity is the proportion of sampled objects that are valid; • novelty is the proportion of valid sampled objects that are not present in the training data; • uniqueness is the proportion of valid unique (non-repeated) sampled objects;
Revised: • validity is the proportion of sampled objects that are valid; • novelty is the proportion of valid sampled objects that are not present in the training data; • uniqueness is the proportion of valid unique (non-repeated) sampled objects;

Original: Our experimental evaluation aims at answering the following questions:
Revised: Our experimental evaluation aims at answering the following questions:

Original: • Q1 Can CANs with tractable constraints achieve better results than GANs?
Revised: • Q1 Can CANs with tractable constraints achieve better results than GANs?

Original: • Q2 Can CANs with intractable constraints achieve better results than GANs?
Revised: • Q2 Can CANs with intractable constraints achieve better results than GANs?

Original: • Q3 Can constraints be combined with rewards to achieve better results than using rewards only?
Revised: • Q3 Can constraints be combined with rewards to achieve better results than using rewards only?

Original: 4. C ONSTRAINED IMAGES
Revised: 4. C ONSTRAINED IMAGES

Original: In this synthetic experiment, we used CANs to generate small, strongly constrained bitmap images.
Revised: In this synthetic experiment, we used CANs to generate small, strongly constrained bitmap images.

Original: The training set is composed of 23 , 040 black-and-white 20 × 20 images, each with a black background and two randomly placed random polygons – either a triangle ( 30% of the cases), a square ( 30% ), or a diamond ( 40% ).
Revised: The training set is composed of 23 , 040 black-and-white 20 × 20 images, each with a black background and two randomly placed random polygons – either a triangle ( 30% of the cases), a square ( 30% ), or a diamond ( 40% ).

Original: The two polygons are always different from each other, fully contained in the canvas, and do not overlap.
Revised: The two polygons are always different from each other, fully contained in the canvas, and do not overlap.

Original: The leftmost and rightmost columns are special, in that they contain parity information about the image.
Revised: The leftmost and rightmost columns are special, in that they contain parity information about the image.

Original: More speciﬁcally, row r of the leftmost column encodes the parity bit for all pixels in the left half of row r , see Fig.
Revised: More speciﬁcally, row r of the leftmost column encodes the parity bit for all pixels in the left half of row r , see Fig.

Original: 1 (left) for an illustration.
Revised: 1 (left) for an illustration.

Original: The rightmost column does the same for the right half of the image.
Revised: The rightmost column does the same for the right half of the image.

Original: Fig.
Revised: Fig.

Original: 1 (middle) shows a sample image.
Revised: 1 (middle) shows a sample image.

Original: As a baseline GAN, we implemented the generator and discriminator as deconvolutional and convolutional networks, respectively, sampling from the concrete distribution (Maddison et al., 2016) and using the vanilla loss.
Revised: As a baseline GAN, we implemented the generator and discriminator as deconvolutional and convolutional networks, respectively, sampling from the concrete distribution (Maddison et al., 2016) and using the vanilla loss.

Original: The CAN uses the same architecture, except for the extra semantic loss term which encodes the parity constraints using XOR formulas.
Revised: The CAN uses the same architecture, except for the extra semantic loss term which encodes the parity constraints using XOR formulas.

Original: Fig.
Revised: Fig.

Original: 1 (right) shows the results in terms of validity, novelty and uniqueness computed overobjects generated using CANs (with λ = 1 ) and Vanilla GANs, where novelty and uniqueness are computed over the subset of valid objects.
Revised: 1 (right) shows the results in terms of validity, novelty and uniqueness computed overobjects generated using CANs (with λ = 1 ) and Vanilla GANs, where novelty and uniqueness are computed over the subset of valid objects.

Original: CANs generate more valid objects, improving at the same time both novelty and uniqueness.
Revised: CANs generate more valid objects, improving at the same time both novelty and uniqueness.

Original: These results allow to answer Q1 afﬁrmatively.
Revised: These results allow to answer Q1 afﬁrmatively.

Original: Note that in this experiment, using CANs over Vanilla GANs results in a negligible training overhead.
Revised: Note that in this experiment, using CANs over Vanilla GANs results in a negligible training overhead.

Original: On average, training CANs takes 30 minutes against the 27 minutes taken by Vanilla GANs.
Revised: On average, training CANs takes 30 minutes against the 27 minutes taken by Vanilla GANs.

Original: 4. S UPER M ARIO B ROS LEVEL GENERATION
Revised: 4. S UPER M ARIO B ROS LEVEL GENERATION

Original: In the next experiment we show how CANs can help in the challenging task of learning to generate videogame levels from user-authored content.
Revised: In the next experiment we show how CANs can help in the challenging task of learning to generate videogame levels from user-authored content.

Original: While procedural approaches to videogame level generation have successfully been used for decades, the application of machine learning techniques in the creation of (functional) content is a relatively new area of research (Summerville et al., 2018).
Revised: While procedural approaches to videogame level generation have successfully been used for decades, the application of machine learning techniques in the creation of (functional) content is a relatively new area of research (Summerville et al., 2018).

Original: On the one hand, modern video game levels are characterized by aesthetical features that cannot be formally encoded and thus are difﬁcult to implement in a procedure, which motivates the use of ML techniques for task.
Revised: On the one hand, modern video game levels are characterized by aesthetical features that cannot be formally encoded and thus are difﬁcult to implement in a procedure, which motivates the use of ML techniques for task.

Original: On the other hand, the levels have often to satisfy a set of functional (hard) constraints that are easy to guarantee when the generator is hand-coded but pose challenges for current machine learning models.
Revised: On the other hand, the levels have often to satisfy a set of functional (hard) constraints that are easy to guarantee when the generator is hand-coded but pose challenges for current machine learning models.

Original: In the following, we show how the semantic loss can be used to encode useful hard constraints in the context of videogame level generation.
Revised: In the following, we show how the semantic loss can be used to encode useful hard constraints in the context of videogame level generation.

Original: These constraints might be functional requirements that apply to every generated object or might be contextually used to steer the generation towards objects with certain properties.
Revised: These constraints might be functional requirements that apply to every generated object or might be contextually used to steer the generation towards objects with certain properties.

Original: In our empirical analysis, we focus on Super Mario Bros (SMB), possibly one of the most studied video games in tile-based level generation.
Revised: In our empirical analysis, we focus on Super Mario Bros (SMB), possibly one of the most studied video games in tile-based level generation.

Original: See Section 5 for related work in SMB level generation.
Revised: See Section 5 for related work in SMB level generation.

Original: Recently, Volz et al.
Revised: Recently, Volz et al.

Original: applied Wasserstein GANs (WGANs)
Revised: applied Wasserstein GANs (WGANs)

Original: (Arjovsky et al., 2017) to SMB level generation.
Revised: (Arjovsky et al., 2017) to SMB level generation.

Original: The approach works by ﬁrst training a generator in the usual way, then using an evolutionary algorithm called Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search for the best latent vectors according to a user-deﬁned ﬁtness function on the corresponding levels.
Revised: The approach works by ﬁrst training a generator in the usual way, then using an evolutionary algorithm called Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search for the best latent vectors according to a user-deﬁned ﬁtness function on the corresponding levels.

Original: We stress that this technique is orthogonal to CANs and the two can be combined together.
Revised: We stress that this technique is orthogonal to CANs and the two can be combined together.

Original: Nonetheless, in the following we compare CANs with CMA-ES, as both techniques can be used to steer the network towards the generation of playable levels, i.e. having a feasible path 2 from the left-most to the right-most column of the level.
Revised: Nonetheless, in the following we compare CANs with CMA-ES, as both techniques can be used to steer the network towards the generation of playable levels, i.e. having a feasible path 3 from the left-most to the right-most column of the level.

Original: Pictures in 2 shows two SMB levels generated by our CAN using the semantic loss.
Revised: Pictures in 2 shows two SMB levels generated by our CAN using the semantic loss.

Original: In CMA-ES, the ﬁtness function doesn’t have to be differentiable and the playability can be computed on the output of an A* agent playing the level.
Revised: In CMA-ES, the ﬁtness function doesn’t have to be differentiable and the playability can be computed on the output of an A* agent playing the level.

Original: Having the SL to steer the generation towards playable levels is not trivial, since it requires a differentiable deﬁnition of playability.
Revised: Having the SL to steer the generation towards playable levels is not trivial, since it requires a differentiable deﬁnition of playability.

Original: Encoding the reachability of the right-most column with a propositional formula on the level variables x is unfeasible (consider the size of the formula resulting from unrolling and grounding a compact ﬁrst-order encoding).
Revised: Encoding the reachability of the right-most column with a propositional formula on the level variables x is unfeasible (consider the size of the formula resulting from unrolling and grounding a compact ﬁrst-order encoding).

Original: For this reason, our CAN applies the semantic loss to a different probability distribution θ over binary variables r , with the intended meaning that r i is true if and only if the tile x i is {+
 According to the game’s physics. [Figure] (2018)+} reachable (in expectation) from the ﬁrst column in the level x , with x sampled from the generator’s multinomial distribution γ over tiles.
Revised: For this reason, our CAN applies the semantic loss to a different probability distribution θ over binary variables r , with the intended meaning that r i is true if and only if the tile x i is {+
 According to the game’s physics. [Figure] (2018)+} reachable (in expectation) from the ﬁrst column in the level x , with x sampled from the generator’s multinomial distribution γ over tiles.

Original: Encoding the playability of a level is straightforward in r , although the problem becomes ﬁnding a fully differentiable transformation P R : Γ → Θ .[Figure]
Revised: Encoding the playability of a level is straightforward in r , although the problem becomes ﬁnding a fully differentiable transformation P R : Γ → Θ .

Original: We approximate P R with a feedforward neural network ˆ P R .
Revised: We approximate P R with a feedforward neural network ˆ P R .

Original: Our approach consists in pretraining ˆ
Revised: Our approach consists in pretraining ˆ

Original: P R with synthetic data { ( γ i , ˆ θ i ) } , consisting of syntetic distributions labelled using the tile-based pathﬁnder provided in Summerville et al. (2016).
Revised: P R with synthetic data { ( γ i , ˆ θ i ) } , consisting of syntetic distributions labelled using the tile-based pathﬁnder provided in Summerville et al. (2016).

Original: Inputs γ
Revised: Inputs γ

Original: i are collected from previous runs of the generator, while ˆ θ i are computed by sampling 100 levels from γ i and averaging the reachability of each tile using the pathﬁnder’s output.
Revised: i are collected from previous runs of the generator, while ˆ θ i are computed by sampling 100 levels from γ i and averaging the reachability of each tile using the pathﬁnder’s output.

Original: The data generation process is inexpensive, allowing to train ˆ
Revised: The data generation process is inexpensive, allowing to train ˆ

Original: P R to reasonable performance for the task.
Revised: P R to reasonable performance for the task.

Original: We adopt the same experimental setting, WGAN architecture and training procedure of Volz et al.
Revised: We adopt the same experimental setting, WGAN architecture and training procedure of Volz et al.

Original: The structured objects are 14 × 28 tile-based representations of SMB levels (e.g. Fig. 2) and the training data is obtained by sliding a 28 tiles window over levels from the Video game level corpus (Summerville et al., 2016).
Revised: The structured objects are 14 × 28 tile-based representations of SMB levels (e.g. Fig. 2) and the training data is obtained by sliding a 28 tiles window over levels from the Video game level corpus (Summerville et al., 2016).

Revised: In this experiment we train on individual levels, as in (Volz et al., 2018), to be consistent with MarioGAN, but we stress that CANs can be trained on multiple levels.

Original: Our semantic loss, applied on the output of the approximated reachability map ˆ P R , encodes that at least 4 tiles of the right-most column are reachable.
Revised: Our semantic loss, applied on the output of the approximated reachability map ˆ P R , encodes that at least 4 tiles of the right-most column are reachable.

Original: This choice is supported by two observations: if a tile is reachable, the same will probably be neighbours and the approximation given by ˆ P R may satisfy too easily a constraint asking for only 1 reachable tile in the rightmost column.
Revised: This choice is supported by two observations: if a tile is reachable, the same will probably be neighbours and the approximation given by ˆ P R may satisfy too easily a constraint asking for only 1 reachable tile in the rightmost column.

Original: Table 1 shows the playability, novelty and uniqueness of a batch of 200 levels generated respectively by the CAN with a forward pass (with λ = 0 . 01 , which validation experiments showed to be a reasonable trade-off between SL and discriminator loss) and by CMA-ES using the default parameters for the search.
Revised: Table 1 shows the playability, novelty and uniqueness of a batch of 1000 levels generated respectively by the CAN with a forward pass (with λ = 0 . 01 , which validation experiments showed to be a reasonable trade-off between SL and discriminator loss) and by CMA-ES using the default parameters for the search.

Original: Novelty and Uniqueness are really good because it is quite impossible to generate a level equal toan other in a  space [-with 
-]
Revised: Novelty and Uniqueness are notably higher compared to the syntetic case due to a much larger output space [-with 
-]4 .

Original: (14 × 28) solutions ( 13 is the number of possible types for each tile).

Original: Results shows that CANs achieve comparable results on the ﬁrst level, and substantial improvements on the second.
Revised: Results show that CANs achieve comparable results on the ﬁrst level, and substantial improvements on the second.

Original: Moreover, no signiﬁcant negative effects can be seen on the generated material with respect to the quality of the ones generated by the original GAN.
Revised: Moreover, no signiﬁcant negative effects can be seen on the generated material with respect to the quality of the ones generated by the original GAN.

Original: Moreover, at the cost of pretraining ˆ P R (1h35m on a 8-core machine with a GTX1080Ti), CANs avoid the execution of the A* agent during the generation, sampling high quality objects in milliseconds.
Revised: Moreover, at the cost of pretraining ˆ P R (1h35m on a 8-core machine with a GTX1080Ti), CANs avoid the execution of the A* agent during the generation, sampling high quality objects in milliseconds.

Original: On the other side, the training cost is slightly lower ( ∼ 20% less), but each run of CMA-ES, to ﬁnd a single best individual, takes between 20 and 25 minutes on the previously described machine. 
Revised: On the other side, the training cost is slightly lower ( ∼ 20% less), but each run of CMA-ES, to ﬁnd a single best individual, takes between 20 and 25 minutes on the previously described machine. [Table]

Original: Combining the two approaches and reﬁning the constraints for the generation of state-of-the-art
Revised: Combining the two approaches and reﬁning the constraints for the generation of state-of-the-art

Original: Super Mario levels is a promising research direction and is left for future work.
Revised: Super Mario levels is a promising research direction and is left for future work.

Original: With these results, we can answer Q2 afﬁrmatively.
Revised: With these results, we can answer Q2 afﬁrmatively.

Original: [Table]

Original: 4. M OLECULE GENERATION
Revised: 4. M OLECULE GENERATION

Original: In the next experiment we test how effective is SL in conjunction with different forms of supervision on the task of generating molecules with certain desirable chemical properties.
Revised: In the next experiment, we test how effective is SL in conjunction with different forms of supervision on the task of generating molecules with certain desirable chemical properties.

Original: Speciﬁcally, we consider MolGANs (De Cao & Kipf, 2018), a model that combines the adversarial loss with a reinforcement learning objective , used to maximize the druglikeness , sythesizability and solubility of the generated molecules.
Revised: Speciﬁcally, we consider MolGANs (De Cao & Kipf, 2018), a model that combines the adversarial loss with a reinforcement learning objective , used to maximize the druglikeness , sythesizability and solubility of the generated molecules.

Original: The reward is computed by a network that is trained to match the score provided by an external cheminformatics software.
Revised: The reward is computed by a network that is trained to match the score provided by an external cheminformatics software.

Original: In contrast with our previous experimental settings, here the structured objects are undirected graphs of bounded maximum size, represented by discrete tensors that encode the atom/node type ( padding atom (no atom), Carbon , Nitogren ,
Revised: In contrast with our previous experimental settings, here the structured objects are undirected graphs of bounded maximum size, represented by discrete tensors that encode the atom/node type ( padding atom (no atom), Carbon , Nitogren ,

Original: Oxygen , Fluorine ) and the bound/edge type ( padding bond (no bond), single , double , triple and aromatic bond).
Revised: Oxygen , Fluorine ) and the bound/edge type ( padding bond (no bond), single , double , triple and aromatic bond).

Original: As conﬁrmed by both the results reported by the authors of MolGANs and our previous experiments, providing an additional loss term might perturbate the already unstable adversarial game, possibly incurring in mode collapse.
Revised: As conﬁrmed by both the results reported by the authors of MolGANs and our previous experiments, providing an additional loss term might perturbate the already unstable adversarial game, possibly incurring in mode collapse.

Original: In the following, we explore the use of SL beyond the satisfaction of given constraints.
Revised: In the following, we explore the use of SL beyond the satisfaction of given constraints.

Original: Speciﬁcally, we observe that the formula can involve the latent variables too, and we show how this can be leveraged to increase the diversity of the generator’s output and to mitigate the mode collapse.
Revised: Speciﬁcally, we observe that the formula can involve the latent variables too, and we show how this can be leveraged to increase the diversity of the generator’s output and to mitigate the mode collapse.

Original: This can be achieved by using a portion of the latent vector (whose values are in the [0 , 1] domain) to trigger on and off SL terms that promote the presence of speciﬁc atoms in the molecule.
Revised: This can be achieved by using a portion of the latent vector (whose values are in the [0 , 1] domain) to trigger on and off SL terms that promote the presence of speciﬁc atoms in the molecule.

Original: Speciﬁcally, we apply the SL to MolGANs, making use of 5 latent dimensions to control the presence of one of the 5 types of atoms considered in the experiment.
Revised: Speciﬁcally, we apply the SL to MolGANs, making use of 4 latent dimensions to control the presence of one of the 4 types of atoms considered in the experiment.

Original: Each dimension represents the probability of having at least one atom of that type in the molecule, no matter the position.
Revised: Each dimension represents the probability of having at least one atom of that type in the molecule, no matter the position.

Original: Assigning SL terms to speciﬁc subregion of the latent space is a general approach that can potentially be used to achieve some level of controllable constrained generation via the latent codes.
Revised: Assigning SL terms to speciﬁc subregion of the latent space is a general approach that can potentially be used to achieve some level of controllable constrained generation via the latent codes.

Original: We defer this research direction to future work.
Revised: We defer this research direction to future work.

Original: In this experiment, we augment MolGAN with our constraints conditioned on the latent variables, starting from the ﬁrst epoch.
Revised: In this experiment, we augment MolGAN with our constraints conditioned on the latent variables, starting from the ﬁrst epoch.

Original: We consider two different variants of the reward network.
Revised: We consider two different variants of the reward network.

Original: In the ﬁrst setting, the network implicitly rewards validity and the maximization of the three chemical properties at once: QED (druglikeness), SA (synthesizability) and logP (solubility).
Revised: In the ﬁrst setting, the network implicitly rewards validity and the maximization of the three chemical properties at once: QED (druglikeness), SA (synthesizability) and logP (solubility).

Original: The experimental setting and evaluation metrics are identical to De Cao & Kipf (2018) except for the introduction of the SL, we thus report the same results for the baseline.
Revised: The experimental setting and evaluation metrics are identical to De Cao & Kipf (2018) except for the introduction of the SL, we thus report the same results for the baseline.

Original: In the second setting, the reward is proportional to the diversity of the generated batch, thus boosting the generator uniqueness .
Revised: In the second setting, the reward is proportional to the diversity of the generated batch, thus boosting the generator uniqueness .

Original: In this case, the training stops when the validity reaches 0 .
Revised: In this case, the training stops when the validity reaches 0 .

Original: The results, as shown in Table 2, indicate that the SL term is boosting the diversity of the generated molecules without negatively affecting the other metrics with both reward functions.
Revised: The results, as shown in Table 2, indicate that the SL term is boosting the diversity of the generated molecules without negatively affecting the other metrics with both reward functions.

Original: This preliminary results seem to suggest that CANs can be successfully coupled with a reinforcement learning objective, answering Q3 affermatively.
Revised: This preliminary results seem to suggest that CANs can be successfully coupled with a reinforcement learning objective, answering Q3 affermatively.

Original: In this setting, using CANs produced a negligible overhead during the training with respect to the original model, providing further evidence that the technique doesn’t heavily impact on the training.
Revised: In this setting, using CANs produced a negligible overhead during the training with respect to the original model, providing further evidence that the technique doesn’t heavily impact on the training.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Deep generative modeling has recently enjoyed substantial progress with the introduction of autoregressive models (Van Oord et al., 2016), variational autoencoders (VAEs) (Kingma & Welling, 2014; Rezende et al., 2014), ﬂow-based approaches (Dinh et al., 2014), and GANs (Goodfellow et al., 2014).
Revised: Deep generative modeling has recently enjoyed substantial progress with the introduction of autoregressive models (Van Oord et al., 2016), variational autoencoders (VAEs) (Kingma & Welling, 2014; Rezende et al., 2014), ﬂow-based approaches (Dinh et al., 2014), and GANs (Goodfellow et al., 2014).

Original: However, none of these approaches is designed to generate structures.
Revised: However, none of these approaches is designed to generate structures.

Original: Traditional approaches to such tasks, like graphical models (Koller & Friedman, 2009; Richardson & Domingos, 2006) and probabilistic grammars (Talton et al., 2012), are ill-suited for complex tasks like image generation and do not support efﬁcient inference under constraints.
Revised: Traditional approaches to such tasks, like graphical models (Koller & Friedman, 2009; Richardson & Domingos, 2006) and probabilistic grammars (Talton et al., 2012), are ill-suited for complex tasks like image generation and do not support efﬁcient inference under constraints.

Original: Tractable probabilistic circuits (e.g., probabilistic sentential decision diagrams (Kisa et al., 2014))
Revised: Tractable probabilistic circuits (e.g., probabilistic sentential decision diagrams (Kisa et al., 2014))

Original: leverage knowledge compilation techniques – like CANs – and sport high capacity, efﬁcient inference, and support for constraints, but they require the circuit even at inference time, meaning that inference can have large space requirements. 
Revised: leverage knowledge compilation techniques – like CANs – and sport high capacity, efﬁcient inference, and support for constraints, but they require the circuit even at inference time, meaning that inference can have large space requirements. Further, the runtime of inference depends on the size of the circuit.

Original: In contrast, inference in CANs boils down to a forward pass over the generator and does not rely on the circuit output by knowledge compilation.
Revised: In contrast, inference in CANs boils down to a forward pass over the generator and does not rely on the circuit output by knowledge compilation.

Revised: Finally, although solver-based approaches to constrained discrete sampling like (Ermon et al., 2013; Chakraborty et al., 2014) guarantee validity of the samples, they are also much slower than CANs and do not support learning the distribution from data.

Original: Injecting knowledge into learned models, and neural networks in particular, is a long-standing aim of machine learning.
Revised: Injecting knowledge into learned models, and neural networks in particular, is a long-standing aim of machine learning.

Original: Early approaches include grounding-speciﬁca Markov Logic Networks (Lippi & Frasconi, 2009), which combine Markov logic with neural networks.
Revised: Early approaches include grounding-speciﬁca Markov Logic Networks (Lippi & Frasconi, 2009), which combine Markov logic with neural networks.

Original: More recent models include target-speciﬁc architectures (Rockt¨aschel & Riedel, 2017; Donadello et al., 2017), as well as frameworks based on probabilistic logics (Manhaeve et al., 2018) and T-norms (Marra et al., 2019).
Revised: More recent models include target-speciﬁc architectures (Rockt ¨ aschel & Riedel, 2017; Donadello et al., 2017), as well as frameworks based on probabilistic logics (Manhaeve et al., 2018) and T-norms (Marra et al., 2019).

Original: Other recent approaches include also constraint learning component (Wang et al., 2019; Sourek et al., 2018).
Revised: Other recent approaches include also constraint learning component (Wang et al., 2019; Sourek et al., 2018).

Original: However, none of these models is at the same time generative and probabilistic.
Revised: However, none of these models is at the same time generative and probabilistic.

Revised: Posterior regularization (PR) has been used to introduce logic background knowledge into deep discriminative (Hu et al., 2016) and discriminative models (Hu et al., 2018).

Revised: Like the Semantic

Revised: Loss, PR can convert arbitrary discrete constraints into a loss.

Revised: In addition, it does not require the target model to have an explicit density.

Original:  A number of machine learning techniques were applied to Super Mario Bros level generation, e.g.
Revised: The major downside of the PR is that it is not semantically consistent: rewriting a constraint (by applying e.g. De Morgan’s laws) can lead to different loss functions. This never happens with the SL. Further, unlike the approach of (Hu et al., 2018), CANs do not need to deﬁne and train an approximate variational distribution: the SL is computed directly on the generator. A number of machine learning techniques were applied to Super Mario Bros level generation, e.g.

Original: LSTMs (Summerville & Mateas, 2016), probabilistic graphical models (Guzdial & Riedl, 2016) and multi-dimensional Markov Chain Monte Carlo (Snodgrass & Ontan ´ on, 2016). 
Revised: LSTMs (Summerville & Mateas, 2016), probabilistic graphical models (Guzdial & Riedl, 2016) and multi-dimensional Markov Chain Monte Carlo (Snodgrass & Ontan´on, 2016). Torrado et al.

Revised: leverages GANs, but differs from our work in that they consider level generation only and they constrain the mixture of tiles appearing in the level 5 .

Revised: This technique cannot be easily generalized to arbitrary constraints.

Original: Many approaches to molecule generation use VAEs (G ´ omez-Bombarelli et al., 2018; Kusner et al., 2017; Dai et al., 2018).
Revised: Many approaches to molecule generation use VAEs (G ´ omez-Bombarelli et al., 2018; Kusner et al., 2017; Dai et al., 2018).

Original: Closest to MolGANs are ORGANs (Guimaraes et al.,  2017), that used adifferent reinforcement learning objective with SeqGANs  to generate molecules as sequences (in SMILE encoding)
Revised: Closest to MolGANs are ORGANs (Guimaraes et al., 2017) and Sequence Tutor (Jaques et al., 2017), both using a reinforcement learning objective (with SeqGANs and RNNs respectively) to generate molecules as sequences in SMILE encoding. Finally, the approach of Seff et al. (2019) use MCMC to sample graph representations of valid molecules.

Original:   C ONCLUSION
Revised: Compared to CANs, their approach requires an expensive inference procedure based on Gibbs sampling, which must be additionally bootstrapped by providing a valid conﬁguration. When the space is constrained by an arbitrary logical formula, as in our case, this last step is NP-complete.  C ONCLUSION

Original: We presented Constrained Adversarial Networks (CANs), a new class of GANs in which the generator is encouraged during training to output valid structures.
Revised: We presented Constrained Adversarial Networks (CANs), a generalization of GANs in which the generator is encouraged during training to output valid structures.

Original: CANs make use of the semantic loss (Xu et al., 2018) to measure the mass allocated by the generator to invalid structures and penalize the latter accordingly.
Revised: CANs make use of the semantic loss (Xu et al., 2018) to measure the mass allocated by the generator to invalid structures and penalize the latter accordingly.

Original: As in GANs, generating (likely) valid structures then amounts to a simple forward pass of the generator.
Revised: As in GANs, generating (likely) valid structures then amounts to a simple forward pass of the generator.

Original: Importantly, the data structures used by the SL – which can be large if the structural constraints are very complex – can be discarded after training.
Revised: Importantly, the data structures used by the SL – which can be large if the structural constraints are very complex – can be discarded after training.

Original: Our framework was proven to be effective in different structured generative tasks, improving the validity of the generated structures (on average) while keeping the computational cost of training under control and substantially reducing inference runtimes.
Revised: Our framework was proven to be effective in different structured generative tasks, improving the validity of the generated structures (on average) while keeping the computational cost of training under control and substantially reducing inference runtimes.

Original: We also showed how the constraints to be turned on and off at inference time, suggesting different uses for the SL, namely promoting diversity of the generator’s outputs.
Revised: We also showed how the constraints to be turned on and off at inference time, suggesting different uses for the SL, namely promoting diversity of the generator’s outputs.

Original: Finally, our level generation experiment shows how to transform structures to a space in which the structural constraints are easier to encode and more compact
Revised: Finally, our level generation experiment shows how to transform structures to a space in which the structural constraints are easier to encode and more compact. A I MPLEMENTATION DETAILS A. C ONSTRAINED I MAGES

Revised: The deep neural network for this experiment is based on a Deep Convolutional Generative Adversarial Networks (DCGANs)

Revised: (Radford et al., 2015).

Revised: Batch Normalization (BN) and ReLU are applied between the layers of the generator G .

Revised: While Leaky ReLU with a slope of 0.2 has been used for the discriminator D .

Revised: In the last layer of the generator we apply a gumbel-sigmoid activation function (GS).

Revised: For the training, Adam optimizer has been used with β 1 = 0.5 and β 2 = 0.999.

Revised: The learning rate is constant throughout the training (250 epochs) and is equal to 0.0001.

Revised: The batch size used during the experiments has been set to 64.

Revised: Layers have been initialized using Xavier initialization for the generator and Orthogonal initialization for the discriminator.

Original: {+Finally, the size of the latent vector has been set to 64 and sampled from a normal distribution with mean = 0 . 
+}
Revised: {+Finally, the size of the latent vector has been set to 64 and sampled from a normal distribution with mean = 0 . 
+}

Revised: and std = 1 .

Revised: Table 3 shows the network architecture

Revised: [Table]

Revised: A. S UPER M ARIO B ROS L EVEL G ENERATION

Revised: The deep neural network for this experiment is based on the DCGANs used in (Volz et al., 2018).

Revised: Batch Normalization (BN) and ReLU are applied between the layers of the generator G .

Revised: While

Revised: Leaky ReLU with a slope of 0 .

Revised: 2 has been used for the discriminator D .

Revised: In the last layer of the generator we apply a Softmax activation function to obtain probabilities that are ﬁnally given as inputs to the Semantic Loss.

Revised: On the other hand, the generation of samples is done through the application, always on the generator, of a stretched Softmax function followed by a Tensorﬂow

Revised: OneHotCategorical layer, which allows to samples levels.

Revised: In our opinion, this is a best way of generating objects with respect to the simple argmax applied directly on the generator by (Volz et al., 2018).

Revised: The networks have been trained using the WGAN guidelines (Arjovsky et al., 2017).

Revised: Thus, the number of iterations on the discriminator has been set to 5 for each iteration on the generator.

Revised: RMSProp has been used as loss optimizer, with the learning rate constant throughout the training (epochs) and equal to 0 . 00005 .

Revised: The batch size used during the experiments has been set to 32 .

Revised: Layers have been initialized using Normal initializer for both the generator and the discriminator.

Revised: Moreover, weight clipping is applied on the layers of D with c equals to 0 .

Revised: Finally, the size of the latent vector has been set to 64 and sampled from a normal distribution N (0 , 1)

Revised: Table 4 shows the network architecture (as said, each layer is followed by batch normalization and by a ReLU/Leaky ReLU activation function).

Revised: The feedforward neural network ˆ P R has been implemented using a CNN with the architecture in

Revised: Table 5.

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: A. M OLECULE G ENERATION

Revised: The MolGAN architecture is composed of three networks, the generator G , the discriminator D and the reward network R .

Revised: D and R share the same architecture, but are trained with different objectives.

Revised: R must estimate either the product of the QED, SA, logP metrics, or the uniqueness of a batch of samples.

Revised: All these metrics are in [0 , 1] so that R should output 1 given a perfect sample (w.r.t.

Revised: the aforementioned metrics) or a batch made of different samples.

Revised: G is optimized to produce samples that maximize the output of R and are convincing to D , on top of that, the semantic loss is also applied.

Revised: While R is trained in parallel with D and G , the loss of G with respect to the output of R is activated only after 150 epochs, at which point the adversarial loss stops being used.

Revised: The semantic loss is applied to G from the start to the end of the training.

Revised: The weight of the semantic loss is equal to 0 .

Revised: 9 , whereas the adversarial loss of G has a weight of 0 . 1 .

Revised: Improved WGAN is used as the adversarial loss between G and D ( n critic = 5), whereas R is trained to estimate the desired target by mean squared error, and G is trained with deep deterministic policy gradient w.r.t.

Revised: the output of R , which is seen as a reward to maximize.

Revised: Training is ongoing until a stop condition is met, either the uniqueness of the batch falling below 0 .if

Revised: R is an estimator of the three chemical properties, or the validity reaching 0 .

Revised: 99 if R should output the uniqueness of the batch.

Revised: During training, the learning rate is set at a constant value of 0 .

Revised: 001 , the batch size is 32 and there is no dropout.

Revised: Adam with beta 1 = 0.9 and beta 2 = 0.999 is the optimizer of choice.

Revised: Batch discrimination is used.

Revised: Results are obtained by evaluating a batch of 5000 generated samples.

Revised: The input noise z has dimension 32 if the semantic loss is not applied, 36 if it is applied; with the ﬁrst 32 dimensions sampled from a standard normal distribution and the last four from a uniform distribution in [0 , 1] .

Revised: The maximum number of nodes for each molecule is 9, with 5 possible atom types and 5 bond types.

Revised: Each molecule is represented by two matrices, one mapping each node to a label, and one adjacency matrix informing about the presence or lack of edges between nodes, and their type.

Revised: The input noise is received by G and processed by three fully connected layers of 128 , 256 ,units each, while tanh acts as the activation function; a linear projection followed by a softmax is then applied to the output of the last layer to have it matching the size of the adjacency matrices.

Revised: D and R share the same architecture (no parameters are shared), based on two relational graph convolutions (De Cao & Kipf, 2018) of 64 , 32 hidden units, followed by an aggregation as in (De Cao & Kipf, 2018) to obtain a graph level representation of 128 features.

Revised: Two fully connected layers of 128 , 1 units then reduce the graph embedding to a single output value, with tanh as the activation for the hidden layer and with sigmoid being applied on the output in the case of R .