Efficient generation of structured objects with Constrained Adversarial Networks
Authors: Jacopo Gobbi
Luca Di Liello
Pierfrancesco Ardino
Paolo Morettin
Stefano Teso
Andrea Passerini
Venue:  (2020)
Keywords: deep generative models
generative adversarial networks
constraints

Abstract: Despite their success, generative adversarial networks (GANs) cannot easily generate structured objects like molecules or game maps. The issue is that such objects must satisfy structural requirements (e.g., molecules must be chemically valid, game maps must guarantee reachability of the end goal) that are difficult to capture with examples alone.  As a remedy, we     propose constrained adversarial networks (CANs), which embed the constraints into the model during training by penalizing the generator whenever it outputs invalid structures. As in unconstrained GANs, new objects can be sampled straightforwardly from the generator, but in addition they satisfy the constraints with high probability.  Our approach handles arbitrary logical constraints and leverages knowledge compilation techniques to efficiently evaluate the expected disagreement between the     model and the constraints.  This setup is further extended to hybrid logical-neural constraints for capturing complex requirements like graph reachability.  An extensive empirical analysis on constrained images, molecules, and video game levels shows that CANs efficiently generate valid structures that are both high-quality and novel.

Under review as a conference paper at ICLR 2020
EFFICIENT GENERATION OF STRUCTURED OBJECTS WITH CONSTRAINED ADVERSARIAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Despite their success, generative adversarial networks (GANs) cannot easily gen- erate structured objects like molecules or game maps. The issue is that such objects must satisfy structural requirements (e.g.. molecules must be chemically valid, game maps must guarantee reachability of the end goal) that are difficult to capture with examples alone. As remedy, we propose constrained adversar- ial networks (CANs), which embed the constraints into the model during training by penalizing the generator whenever it outputs invalid structures. As in uncon- strained GANs, new objects can sampled straightforwardly from the generator, but in addition they satisfy the constraints with high probability. Our approach handles arbitrary logical constraints and leverages knowledge compilation tech- niques efficiently evaluate the expected disagreement between the model and the constraints. This setup is further extended to hybrid logical-neural constraints for capturing complex requirements like graph reachability. An extensive empiri- cal analysis on constrained images, molecules, and video game levels shows that CANs efficiently generate valid structures that both high-quality and novel.
INTRODUCTION
Generative Adversarial Networks (GANs) (Goodfellow et al. (2014) have shown impressive perfor- mance on challenging tasks - like image generation (Karras et al. 2018), text-to-image (Zhang et al.) 2017). and style transfer (Zhu et al. 2017)- where the goal is to produce believable contigurations.
number of important applications, however, require to generate objects that are both credible and feasible with respect to hard structural constraints. Examples include generating drug molecules, which must satisfy chemical validity requirements, and game levels, where the goal must be reach- able from the starting position. Recent studies (Guimaraes et al.] 2017; De Cao & Kipf 2018 Xue & van Hoeve, 2019) observed that GANs struggle in these tasks. The main reason is that the train- ing examples alone are insufficient to capture the feasibility constraints and thus guide the model toward producing valid objects.
As remedy, we propose Constrained Adversarial Networks (CANs), class of generative models that extend GANs to structured domains. Given set of examples drawn from a latent distribution and set of structural constraints, CANs learn to output valid structured with high probability. This is achieved by augmenting the standard GAN loss with penalty term that discourages the model from producing infeasible structures. Since CANs inject the constraints directly into the learned model, valid structures can be obtained with high probability by performing forward inference on the generator, avoiding the need for costly sampling or optimization steps (Volz et al.. 2018).
The penalty term is implemented using the semantic loss (SL. (Xu et al. 2018). The SI. leverages knowledge compilation (Darwiche) (01T) to represent arbitrary Boolean constraint as a circuit and uses the latter to measure the mass allocated by the generator to infcasible objects. The procedure is probabilistically sound, exact, and does not require sampling. Notably, the circuit, which can be quite large depending the complexity of the constraints, can be thrown away after learning.
In addition, we show how to extend the SL complex constraints that would normally lead to in- tractably large circuits. This is accomplished by first using neural network to map configurations to different space in which the structural constraints can be compactly encoded, and then applying the SL. to the latter. This enables to deal with reachability graph, which is beyond the reach of

---

Under review as a conference paper at ICLR 2020
standard SL. Finally, show how the validity constraints baked into the generator can be enabled or disabled during inference using ideas from InfoGANs Chen et al. (2016).
In summary, we contribute:
Constrained Adversarial Networks (CANs), a generalization of GANs in which the gen- erator encouraged (at training time) to output valid structures with high probability via forward inference.
An extensive empirical analysis on structured objects like constrained images, molecules, and video game levels showing that CANs generate structures that are stylistically coherent with the training data.
A decomposition of intractably complex constraints into neural and logical components, showcased on graph reachability for video game level synthesis.
2 UNCONSTRAINED GANS
Let A' be the object space and 2 some latent space. GANs (Goodfellow et al., 2014) are composed of a generator g Z X' and discriminator d : X' - 1]. both implemented as neural networks. The discriminator is trained to recognize "real" objects æ taken from the data distribution Pr. while the generator trained map random latent vectors 2 E I to "fake" objects that fool the discriminator. Learning amounts to solving the minimax game:
min max (GAN(9, d) fGAN(g. d) := Ee [log Pa (a(x)) + Eaw [log Pa(x)) (1)
Here Pg (ax is the distribution induced by the generator and (a(x) := Pa (real x) by the discrimina-
It worth noting that the value of the game defines constrained form of) the Jensen-Shannon entropy and that Eq. can be viewed as divergence minimization (Nowozin et al.) 2016 Mescheder
al. 2017). After training, new objects can be generated by sampling random vectors and map- ping them to object space with the generator æ = =x)).
Goodfellow et al. (2014) have shown that, under idealized assumptions, the learned generator g matches the data distribution, that is Ps P,- We report their theorem here for completeness:
Theorem 1. So long as (i) and d are non-parametric, and (ii) the leftmost expectation in Eq. is approximated arbitrarily well by the examples alone, any global equilibrium (g" d²) of Eq. satisfies Pa. and Pg. Pr.
Finding Nash equilibria non non-convex games like Eq. Dis notoriously hard, rendering GAN train- ing very challenging. The most common optimization algorithm is alternating gradient descent, whereby and are optimized sequentially, or the more well-understood simultaneous gradient descent (Mescheder et al. (017). A common failure state is mode collapse, whereby the generator outputs objects concentrated in a tiny portion of the object space. A plethora of theoretical and em- pirical remedies have been proposed, cf. (Salimans al. 2016; Mescheder et al. 2018). including using alternative divergences (Nowozin et al.] 2 et al. (017]) and encouraging smooth- ness of the discriminator by, e.g., controlling the spectral norm of its parameters (Miyato et al.) 2018). In our experiments, we apply some of these strategies to stabilize training.
In structured tasks, the objects of interest are usually discrete. GANs can be adapted to such settings by having the generator output a categorical distribution ((z) over X' and sampling objects from the latter. Below, we will focus on stochastic generators of this kind, although alternatives do exist (Gul- rajani et al. 2017). In this setting Pa(2)
Ps ()()) (())p(z)dz E {()))..
GENERATING STRUCTURES WITH CANS
Our goal is learn to generate structures æ consistent with respect to some validity constraint & ac- cording to some unobserved distribution P... Throughout, we will make the following assumptions:
A single validity constraint v is provided as input. This is without loss of generality: multiple constraints are necessary, then d can be taken be their conjunction.
2

---

Under review as a conference paper at ICLR 2020
2. The constraint & is compatible with the data distribution, that is, the support of P. falls entirely within the feasible region determined by vb.
These assumptions hold in many tasks of interest, including all generative problems with known follenormedsess requirements. For the sake of simplicity, we restrict our study to binary (i.e. 0- 1) variables and logical constraints (aka formulas) only. This is a very general setup: any discrete structured space can be encoded using binary variables and formulas, at the cost of larger model.
Limitations of GANs Generators learned with the standard GAN training rule output invalid structures, for two main reasons. First, for non-trivial constraints v. any finite set of examples that satisfy the constraint is insufficient capture the full semantics of vb. Second, in many cases of interest the examples are not even consistent with v.. This more challenging case shows that regular GANs are easily lured into learning not satisfy the constraint. More formally:
Corollary 1. Under the assumptions of Theorem 0 given a target distribution Pr. constraint vi consistent with it, and a dataset of examples x sampled i.i.d. from corrupted distribution P,- + Pr inconsistent with vb GANs associate non-zero mass infeasible objects.
Indeed, by Theorem [ the optimal generator satisfies Pg Pr which is inconsistent with dh. Ergo, En 1 {a * 1b Pg ) > O. We stress that Theorem |
captures the intent of GAN training, and thus this simple corollary shows that GANs are by design incapable of handling invalid examples.
Constrained adversarial networks In order to avoid these issues, constrained adversarial net- works (CANs) take both the examples and the validity constraint into account during learning. More specifically, the CAN value function is designed so that the generator maximizes the probability of generating valid structures (1b)) == Ex [1 {ar - }. that is:
fCAN d) == fGAN (g. d) Aloo (1)
Here is hyper-parameter controlling the importance of the constraint. Since the second term is always non-ne negative, the CAN value function upper bounds the GAN one (Eq. D
The second term is the so-called semantic loss (SL), proposed in (Xu al. 2018 inject knowledge
into neural networks, and it is formally defined as SL (g) x log gg (,), where:
(v(v) Ex 1 {x' + (x = E.
I (z) II I1 - (1(2)) (3)
Recall that ((2) is the categorical distribution output by the generator g for z, while x' } s means that æ' satisfies constraint t.. Hence the sum runs over all configurations æ' consistent with vb. The SL can also be viewed as the negative log-like ligelikl of & w.r.t. the generator. This shows that, in Eq. 2 the SL rewards the generator g proportionally the mass it allocates to valid structures. Since the SL is the negative logarithm of polynomial in 0, it is fully differentiable (so long as (g(g) + 0, which is always the case in practice).
If the SL. is given large enough weight, CANs are strongly encouraged to generate valid structures in expectation. Under the preconditions of Theorem it can be shown that or X - oo CANs generate valid structures only:
Proposition Under the assumptions of Corollary ! CANs associate zero mass to infeasible ob- jects, irrespective of the discrepancy between P, and P,.
Intuitively, this holds because with À = oo any global equilibrium (g". dd') of ming max JCAN (g, d) must minimize the second term. If g is non-parametric, then the minimum is attained with log P3-((5) (15) 0, which is equivalent to (g)(*) proving the claim. Of course, as with standard GANs, the prerequisites are often violated prac- tice. Regardless, Proposition [ works sanity check, and shows that in contrast to GANs CANs are appropriate for constrained generative tasks.
1. This in turn implies that Pa- (-a) = O.
Computing the semantic loss
Naïve evaluation of the SL (Eq. involves summing over all (ex- ponentially many) possible configurations x', which intractable. More generally, computing the SL amounts to weighted model counting, which is #o-mleeeete (Chavira & Darwiche 2008).
For many constraints of interest, however, the polynomial in Eq. more compact form and evaluated very efficiently. Following Xu et al. (2018), we make of
can be factorized into a much
3

---

Under review as a conference paper at ICLR 2020
knowledge compilation to automatically factorize the polynomial by compiling it into an arithmetic circuit (more specifically, into a sentential decision diagram (SDD) (Darwiche) (01T)). When the resulting circuit is small enough, evaluation of the SL and of its gradient are extremely efficient, as shown in (Xu et al. 2018).
The main downside of knowledge compilation is that, depending the complexity of the constraint 3 at hand, the compiled circuit may be very large. This less of a problem during training, which is often performed on powerful machines, but it can be an issue for inference - especially on embedded devices. However, in CANs the circuit is not required for inference (as it consists of simple forward pass over the generator), and thus it can be thrown away after training. This means that CANs incur no space penalty during inference when compared to GANs.
Dealing with intractable constraints When fed particularly complex constraint, knowledge compilation may produce a circuit too large even for the training stage. In this case, we approximate the semantic loss by first mapping the objects from N' to application-speciiie space where & can be expressed in compact form, and then use the semantic loss on top of the transformed objects. We successfully employed this technique to synthesize mario levels where the goal tile is reachable from the starting tile; all details are provided below. The same technique can be exploited for dealing with very complex logical formulas beyond the reach of state-of-the-art knowledge compilation.
4 EXPERIMENTS
We implemented CANs using Tensorflow and used PySDE to compile the constraint into compact circuit for computing the Semantic Loss. We tested CANs using different generator architectures on synthetic and two real-world structured generative taske] In all cases, we evaluated the objects generated by CANs and those of the baselines using three metrics (adopted from Samanta et al. 2018
validity is the proportion of sampled objects that are valid;
novelty is the proportion of valid sampled objects that are not present the training data;
uniqueness is the proportion of valid unique (non-repeated) sampled objects;
Our experimental evaluation aims at answering the following questions:
Q1 Can CANs with tractable constraints achieve better results than GANs?
Q2 Can CANs with intractable constraints achieve better results than GANs?
Q3 Can constraints be combined with rewards to achieve better results than using rewards only?
4.1 CONSTRAINED IMAGES
In this synthetic experiment, we used CANs to generate small, strongly constrained bitmap images. The training set is composed of 23, 040 black-and-white 20 20 images, each with black back- ground and two randomly placed random polygons either a triangle (30% of the cases), a square (30%). or diamond (40%). The two polygons are always different from each other, fully contained in the canvas, and do not overlap. The leftmost and rightmost columns are special, in that they con- tain parity information about the image. More specifically, row of the leftmost column encodes the parity bit for all pixels in the left half of row see Fig. [[eft) for an illustration. The rightmost. column does the same for the right half of the image. Fig. T (middle) shows a sample image. As baseline GAN, we implemented the generator and discriminator as deconvolutional and convolu- tional networks, respectively, sampling from the concrete distribution (Maddison et al. 2016) and using the vanilla loss. The CAN uses the same architecture, except for the extra semantic loss term which encodes the parity constraints using XOR formulas.
' bttttttps... /pppipp..... http:/ojjetc//yypp/ Details can be found in the Appendix. Our code is publicly available at the URL ANONYM XED
4

---

Under review as a conference paper at ICLR 2020
280
CAN
216
200
188
Count
160
146
129
120
112
Valid
Novel
Unique
Figure 1: Left: illustration of the parity constraint the left border. Middle: example from the synthetic dataset. Right: comparison of validity, novelty and uniqueness between CAN and GAN.
Fig. [ (right) shows the results in terms of validity, novelty and uniqueness computed over 300 objects generated using CANs (with = 1) and Vanilla GANs, where novelty and uniqueness are computed over the subset of valid objects. CANs generate more valid objects, improving the same time both novelty and uniqueness. These results allow to answer Q1 affirmatively. Note that in this experiment, using CANs over Vanilla GANs results in negligible training overhead. On average, training CANs takes 30 minutes against the 27 minutes taken by Vanilla GANs.
4.2 SUPER MARIO BROS LEVEL GENERATION
In the next experiment we show how CANs can help in the challenging task of learning to generate videogame levels from user-authored content. While procedural approaches videogame level generation have successfully been used for decades, the application of machine learning techniques the creation of (functional) content is relatively new area of research (Summerville al. 2018)) On the one hand, modern video game levels are characterized by aesthetical features that cannot be formally encoded and thus are difficult to implement in a procedure, which motivates the use of ML techniques for task. On the other hand, the levels have often to satisfy set of functional (hard) constraints that are easy to guarantee when the generator is hand-coded but pose challenges for current machine learning models.
In the following, we show how the semantic loss can used to encode useful hard constraints in the context of videogame level generation. These constraints might be functional requirements that apply to every generated object or might be contextually used to steer the generation towards objects with certain properties. In our empirical analysis, we focus on Super Mario Brox (SMB), possibly one of the most studied video games in tile-based level generation. See Section for related work in SMB level generation.
Recently, Volz et al. (2018) applied Wasserstein GANs (WGANs) (Arjovsky et al. 2017) to SMB level generation. The approach works by first training generator in the usual way, then using an evolutionary algorithm called Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search for the best latent vectors according user-defined fitness function on the corresponding levels. We stress that this technique is orthogonal to CANs and the two can be combined together. Nonetheless, in the following we compare CANs with CMA-ES, as both techniques can be used to steer the network towards the generation of playable levels, i.e. having feasible pati from the left-most to the right-most column of the level. Pictures in 2 shows two SMB levels generated by CAN using the semantic loss.
In CMA-ES, the fitness function doesn't have to be differentiable and the playability can be com- puted on the output of an A agent playing the level. Having the SL to steer the generation towards playable levels is not trivial, since it requires differentiable definition of playability, Encoding the reachability of the right-most column with propositional formula on the level variables is unfea- sible (consider the size of the formula resulting from unrolling and grounding a compact first-order encoding). For this reason, CAN applies the semantic loss different probability distribu- tion 0 over binary variables with the intended meaning that ry is true if and only if the tile x; is
According the game's physies.
5

---

Under review as a conference paper at ICLR 2020
Figure 2: Illustration of two 14 x 28 tiles Super Mario Bros levels generated by CANs.
Network type
Mario Level
Tested samples
Validity 9,805
Uniquemess
Novely
Training time
Inforence time per sample
Haseline
1-
1010
100%
1007
1h min
MaioGAN CMA-ES
manio-1-3
100%
min
min
CAN
mano- 1-3
1000
71.00%
100%
10096
1 min
~ ms
Raseline
mani-3-3 3-3
1000
13,005
1008
100%
1h 11 min
ms
MarioAAN CMA-ES
mario-3-3
1000
62.305
100%
100%
min
<v mi
Table I: Results of using the semantic loss instead of post-processing with CMA-ES. Levels mario- 1-3 and mario-3-3 has been chosen due to their higher complexity in being solved. The table shows the validity, uniqueness and novelty of samples after a testing session with A A valid sample is one that can be completely solved by the A agent. The semantic loss has been activated from epoch 3000. Each run has lasted 5000 epochs with all the default hyper parameters defined in Volz et al. 2018
reachable (in expectation) from the first column in the level æ, with æ sampled from the generator's multinomial distribution 7 over tiles. Encoding playability of a level is straightforward r, although the problem becomes finding fully differentiable transformation Pn: O
We approximate PR with feedforward neural network PR. Our approach consists in pretraining PR with synthetic data {(v, 6.).. consisting of syntetic distributions labelled using the tile-based pathfinder provided in Summerville et al. (2016). Inputs "/s are collected from previous runs of the generator, while 0 are computed by sampling 100 levels from and averaging the reachability of each tile using the pathfinder's output. The data generation process inexpensive, allowing train PR reasonable performance for the task.
We adopt the same experimental setting, WGAN architecture and training procedure of Volz et al. (2018). The structured objects are 14 x tile-based representations of SMB levels (e.g. Fig. 2 and the training data is obtained by sliding a 28 tiles window over levels from the Video game level corpus (Summerville al. 2016). In this experiment we train on individual levels, as in (Volz al.) 2018). to be consistent with MarioGAN, but we stress that CANs can be trained on multiple levels.
Our semantic loss, applied the output of the approximated reachability map PR. encodes that at least 4 tiles the right-most column are reachable. This choice is supported by two observations: a tile is reachable, the same will probably be neighbours and the approximation given by PR may satisfy easily constraint asking for only reachable tile in the rightmost column.
Table shows the playability, novelty and uniqueness tively by the CAN with forward pass (with À = 0.01, which validation experiments showed to be reasonable trade-off between SL and discriminator loss) and by CMA-ES using the default parame- ters for the search. Novelty and Uniqueness a much larger output space") Results show that CANs achieve comparable results on the first level, and substantial improvements on the second. Moreover, no significant negative effects can be seen on the generated material with respect to the quality of the ones generated by the original GAN.
batch 1000 levels generated respec-
notably higher compared the syntetic case due
Moreover, at the cost of pretraining PR (1h35m on 8-core machine with a GTX1080Ti), CANs avoid the execution the * agent during the generation, sampling high quality objects in millisec- onds. On the other side, the training cost is slightly lower (~ 20% less), but each run of CMA-ES, to find a single best individual, takes between 20 and 25 minutes on the previously described machine.
'The size of the output space is (14x28) solutions (with 13 heing the number afdifferent tiles)

---

Under review as a conference paper at ICLR 2020
Reward for
Sennantic loss
validity
uniquesess
diversity
OLD
SA
loga
QED SA log!"
96.39 (2.22)
97.4
25
2.4
98.T (2.04)
91.0
11.75
(5.45)
84.0
73.62
Faise
99.19 (0.19)
4.35
66.87 (6.59)
S0.ED (1.00)
(9.55
$2.89 (4,55)
uniquamess
True
99.27 (0.16)
1X.1 (300)
91.28 (2.87)
(1.39
(8.35)
33.91 (2.102
Table 2: Results of using the semantic loss on the MolGAN architecture. The diversity score is obtained by comparing sub-structures of generated samples against a random subset of the dataset. A lower score indicates a higher amount of repetitions between the generated samples and the dataset. The first row refers to the results reported in the MolGAN paper. Experiments were run 8 times each (rows 2, 3, 4) to obtain mean and std values, the latter parentheses.
Combining the two approaches and refining the constraints for the generation of state-of-the-art Super Mario levels is a promising research direction and is left for future work.
With these results, we can answer Q2 affirmatively.
4.3 MOLECULE GENERATION
In the next experiment, we test how effective is SL. in conjunction with different forms of supervision on the task of generating molecules with certain desirable chemical properties. Specifically, we consider MolGANs (De Cao & Kipf, 2018). model that combines the adversarial loss with reinforcement learning objective, used to maximize the druglikeness, sythesizability and solubility of the generated molecules. The reward is computed by network that is trained to match the score provided by an external cheminformatics software. In contrast with our previous experimental settings, here the structured objects are undirected graphs of bounded maximum size, represented by discrete tensors that encode the atom/node type (padding atom (no atom), Carbon, Nitogren, Oxygen, Fluorine) and the bound/edge type (padding bond (no bond), single, double, triple and aromatic bond).
As confirmed by both the results reported by the authors of MolGANs and our previous experiments, providing an additional loss term might perturbate the already unstable adversarial game, possibly incurring in mode collapse. In the following, we explore the use of SL beyond the satisfaction of given constraints. Specifically, we observe that the formula can involve the latent variables too, and we show how this can be leveraged to increase the diversity of the generator's output and to mitigate the mode collapse. This can be achieved by using portion of the latent vector (whose values are in the 00, 1] domain) to trigger on and off SL terms that promote the presence of specific atoms in the molecule. Specifically, we apply the SL to MolGANs, making use of 4 latent dimensions to control the presence of one of the 4 types of atoms considered in the experiment. Each dimension represents the probability of having at least one atom of that type in the molecule, position. Assigning SL terms to specific subregion of the latent space is a general approach that can potentially be used to achieve some level of controllable constrained generation via the latent codes. We defer this research direction to future work.
matter the
In this experiment, we augment MolGAN with our constraints conditioned on the latent variables, starting from the first epoch. We consider two different variants of the reward network. In the first setting, the network implicitly rewards validity and the maximization of the three chemical prop- erties at once: QED (druglikeness), SA (synthesizability) and logP (solubility). The experimental setting and evaluation metrics are identical to De Cao & Kipf (2018) except for the introduction of the SL, we thus report the same results for the baseline. In the second setting, the reward is propor- tional to the diversity of the generated batch, thus boosting the generator uniqueness. In this case, the training stops when the validity reaches 0.99.
The results, as shown in Table 2 indicate that the SL term is boosting the diversity of the generated molecules without negatively affecting the other metrics with both reward functions. This prelimi- nary results seem to suggest that CANS can be successfully coupled with a reinforcement learning objective, answering Q3 affermatively. In this setting, using CANs produced negligible overhead during the training with respect to the original model, providing further evidence that the technique doesn't 't heavily impact on the training.
7

---

Under review as a conference paper at ICLR 2020
5
RELATED WORK
Deep generative modeling has recently enjoyed substantial progress with the introduction of au- toregressive models (Van Oord al., 2016). variational autoencoders (VAEs) (Kingma & Welling. 2014 Rezende et 2014), flow-based approaches (Dinh et al.) 2014), and GANs (Goodfellow)
ct al.
2014). However, none of these approaches is designed to generate structures.
Traditional approaches to such tasks, like graphical models (Koller & Friedman 2009 Richardson
& Domingos and probabilistic grammars (Tatton et al. 2012). are ill-suited for complex tasks like image generation and do not support efficient inference under constraints. Tractable probabilis- tie circuits (e.g.. probabilistic sentential decision diagrams (Kisa et al. 2014)) leverage knowledge compilation techniques like CANs and sport high capacity, efficient inference, and support for constraints, but they require the circuit even at inference time, meaning that inference can have large space requirements. Further, the runtime of inference depends on the size of the circuit. In contrast, inference in CANs boils down to a forward pass over the generator and does not rely on the circuit output by knowledge compilation. Finally, although solver-based approaches to constrained discrete sampling like (Ermon et al. 2013 Chakraborty et al. 2014) guarantee validity of the samples, they are also much slower than CANS and do not support learning the distribution from data.
Injecting knowledge into learned models, and neural networks in particular, is bong-standing aim of machine learning. Early approaches include crounding-speciiing Markov Logic Networks (Lippi & Frasconi 2009). which combine Markov logic with neural networks. More recent models include target-specific architectures (Rocktäschel & Riedel 2017: Donadeloo et al.| | [2017), as well as frame- works based on probabilistic logics ( (Manhaeve et al. (018)) and T-norms ( (Marra et al. 2019). Other
recent approaches include also constraint learning component (Wang et al. 2019 Sourek et al.) 2018 However, none of these models is at the same time generative and probabilistic.
Posterior regularization (PR) has been used to introduce logic background knowledge into deep discriminative (Hu et al. 2016) and discriminative models (Hu et al., 2018). Like the Semantic Loss, PR can convert arbitrary discrete constraints into a loss. In addition, it does not require the target model to have an explicit density. The major downside of the PR is that it is not semantically consistent: rewriting a constraint (by applying e.g. De Morgan's 's laws) can lead to different loss functions. This never happens with the SL. Further, unlike the approach of (Hu al., 2018). CANs do not need to define and train an approximate variational distribution: the SL is computed directly on the generator.
A number of machine learning techniques were applied to Super Mario Bros level generation, e.g. LSTMs (Summerville & Mateas, 2016). probabilistic graphical models (Guzdial & (016)) and multi-dimmensinal Markov Chain Monte Carlo (Snodgrass & Ontanón 2016). Torrado et al.((201) (2019) leverages GANs, but differs from our work in that they consider level generation only and they constrain the mixture of tiles appearing in the leve(] This technique cannot be easily generalized to arbitrary constraints. Many approaches to molecule generation use VAEs (Gomez-Bombarde 2018 Kusner et al.) [017 Dai et al. 2018). Closest to MolGANs are ORGANs (Guimaraes) et al. (with SeqGANs and RNNs respectively) to generate molecules sequences in SMILE encoding. Finally, the approach of [Seff' et al. (2019) use MCMC to sample graph representations of valid molecules. Compared to CANs, their approach requires an expensive inference procedure based on Gibbs sampling, which must be additionally bootstrapped by providing a valid configuration. When the space is constrained by an arbitrary logical formula, as in our case, this last step is NP-complete.
cet al.
2017) and Sequence Tutor (Jaques et al. 2017), both using a reinforcement learning objective
6 CONCLUSION
We presented Constrained Adversarial Networks (CANs), a generalization of GANs in which the generator encouraged during training to output valid structures. CANs make use of the semantic loss (Xu et al. 20188) to measure the mass allocated by the generator to invalid structures and penalize the latter accordingly. As GANs, generating (likely) valid structures then amounts to a simple forward pass of the generator. Importantly, the data structures used by the SL - which can be large the structural constraints are very complex can be discarded after training. Our framework
Thiis work is pre-print and the code is not publicly available yet, so no comparison could he made.
8

---

Under review as a conference paper at ICLR 2020
was proven to be effective in different structured generative tasks, improving the validity of the generated structures (on average) while keeping the computational cost of training under control and substantially reducing inference runtimes. We also showed how the constraints to be turned on and off at inference time, suggesting different uses for the SI., namely promoting diversity of the generator's outputs. Finally, level generation experiment shows how to transform structures to space in which the structural constraints are casier to encode and more compact.
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214-223, 2017.
Supratik Chakraborty, Daniel J Fremont, Kuldeep S Meel, Sanjit A Seshia, and Moshe Y Vardi. Distribution-awar sampling and weighted model counting for sat. In Twenty-Fighth AAAI Con- ference on Artificial Intelligence, 2014.
Mark Chavira and Adnan Darwiche. On probabilistic inference by weighted model counting. Arti- ficial Intelligence, 172(6-7):772-799, 2008.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in neural information processing systems, pp. 2172 2180, 2016.
Hanjun Dai, Yingtao Tian, Dai, Steven Skiena, and Le Song. Syntax-directee variational au- toencoder for molecule generation. In Proceedings of the International Conference on Learning Representations, 2018.
Adnan Darwiche. Sdd: A new canonical representation of propositional knowledge bases. In Twenty-Second International Joint Conference on Artificial Intelligence, 2011.
Nicola De Cao and Thomas Kipf. MolGAN: An implicit generative model for small molecular graphs. arXiv preprint 1rXiv 2018.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components esti- mation. arXiv preprint arXiv:1410.8516. 2014.
I Donadello, Serafini, and AS d' Avila Garcez. Logic tensor networks for semantic image inter- pretation. IJCAI, 2017.
Stefano Ermon, Carla P Gomes, Ashish Sabharwal, and Bart Selman. Embed and project: Discrete sampling with universal hashing. In Advances in Neural Information Processing Systems, pp- 2085-2093, 2013.
Rafael Gómez Bombarell,,,,, Jennifer Wei, David Duvenaud, José Miguel Hernández-Lobato Timothy D Hirzel,
Benjamín Sänchez-LLngellngg Dennis Sheberla, Jorge Ryan P Adams, and Alán Aspuru-Guzik. Automatic chemical design using data-driven contin- representation of molecules. ACS central science, 4(2):268-22 276, 2018.
Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzio Guzik. Objective-reinforred generative adversarial networks (ORGAN) for sequence generation models. arXiv preprint arXiv: 7 2017.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of Wasserstein GANs. In Advances in neural information processing systems, pp- 5767-5777, 2017.
Matthew Guzdial and Mark Riedl. Game level generation from gameplay videos. In Twelfth Artifi- cial Intelligence and Interactive Digital Entertainment Conference, 2016.

---

Under review as a conference paper at ICLR 2020
Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard H. Hovy, and Eric P. Xing. Hamnessing deep neural networks with logic rules. In ACL (1). The Association for Computer Linguistics, 2016. (tthtp:://db uni-trier de/ de/ htt//bbccoott http://///aa/aal11///5/ http//cc211166111 html 4HuMLHX
Zhiting Hu, Zichao Yang, Ruslan R Salakhutdinov, LIANHUI Qin, Xiaodan Liang, Haoye Dong, and Eric Xing. Deep generative models with learnable knowledge constraints. Advances in Neural Information Processing Systems, pp. 10501-10512 2018.
Natasha Jaques, Shixiang Gu, Dzmitry Bahdanau, José Miguel Hemández-Lobatt Richard E Turner, and Douglas Eck. Sequence tutor: Conservative fine-tuning sequence generation mod- els with kl-control. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pp. 1645-1654. JMLR. org, 2017.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. 2018.
Diederik Kingma and Max Welling. Auto-encoding variational Bayes. In International Conference on Learning Representations, 2014.
Doga Kisa, Guy Van den Broeck, Arthur Choi, and Adnan Darwiche. Probabilistic sentential de- cision diagrams. In Fourteenth International Conference the Principles of Knowledge Repre- sentation and Reasoning, 2014.
Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
Matt J Kusner, Brooks Paige, and José Miguel Hermández-Lobato. Grammar variational autoen- coder. In Proceedings of the 34th International Conference on Machine Learning-Volle Vollme 70, pp- 1945-1954. JMLR. org, 2017.
Marco Lippi and Paolo Frasconi. Prediction of protein 3-residue contacts by markov logic networks with grounding-specific weights. Bioinformatics, 25(18):3326-2335 2009.
Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous
relaxation of discrete random variables. arXiv preprint arXiv:1611.00712 2016.
Robin Manhaeve, Sebastijan Dumancie, Angelika Kimmig, Thomas Demeester, and Lue De Raedt. DeepProbLogg: Neural probabilistic logic programming. In Advances in Neural Information Pro- cessing Systems, pp. 3749-3759, 2018.
Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, and Marco Gori. LYRICS: General Interface Layer to Integrate Al and Deep Learning. arXiv preprint UrXiv:1903.073: 2019.
Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of gans. In Advances in Neural Information Processing Systems, pp. 1825-1835, 2017.
Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? arXiv preprint /rXiv:/801 04406, 2018.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint /802.v8800.05955 2018.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural sam- plers using variational divergence minimization. In Advances in neural information processing systems, pp. 271-279, 2016.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv: 1511.64434 2015.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap- proximate inference in deep generative models. In International Conference on Machine Learn- ing. pp. 1278-1286 2014.
Matthew Richardson and Pedro Domingos. Markov logic networks. Machine learning, 62(1-2): 107-136, 2006.
10

---

Under review as a conference paper at ICLR 2020
Tim Rocktäschel and Sebastian Riedel. End-to-end differentiable proving. In Advances in Neural Information Processing Systems, pp. 3788-380000 2017.
Tim Salimans, lan Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans, In Advances in neural information processing systems, pp. 2234-2242, 2016.
Bidisha Samanta, Abir De, Niloy Ganguly, and Manuel Gomez-Rodrigues Designing random graph models using variational autoencoders with applications to chemical design. arXiv preprint arXiv:/802.05288 2018.
Ari Seff, Wenda Zhou, Farhan Damani, Abigail Doyle, and Ryan P Adams. Discrete object genera- tion with reversible inductive construction. arXiv preprint arXiv: 1etXiv:9907.00 /ttp:/00700000 907.08268, 2019.
Sam Snodgrass and Santiago Ontanón. Controllable procedural content generation via constrained multi-dimensional markov chain sampling. IJCAL, pp. 780-786, 2016.
Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezny, Steven Schockaert, and Ondrej Kuzelka. Lifted relational neural networks: Efficient learning of latent relational structures. Journal of Artificial Intelligence Research, 62:69-100, 2018.
Adam Summerville and Michael Mateas. Super mario as a string: Platformer level generation via lstms. arXiv preprint arXiv:1603.00920 2016.
Adam Summerville, Sam Snodgrass, Matthew Guzdial, Christoffer Holmgård, Amy K Hoover, Aaron Isaksen, Andy Nealen, and Julian Togelius. Procedural content generation via machine learning (pegml). IEEE Transactions on Games, 10(3):257-277 2018.
Adam James Summerville, Sam Snodgrass, Michael Mateas, and Santiago Ontanón. The vgle: The video game level corpus. arXiv preprint arXiv: I 6606077477 2016.
Jerry Talton, Lingfeng Yang, Ranjitha Kumar, Maxine Lim, Noah Goodman, and Radomír Méch. Learning design patterns with bayesian grammar induction. In Proceedings of the 25th annual ACM symposium on User interface software and technology, pp. 63-74. ACM, 2012.
Ruben Rodriguez Torrado, Ahmed Khalifa, Michael Cerny Green, Niels Justesen, Sebastian Risi, and Julian Togelius. Bootstrapping conditional gans for video game level generation. arXiv preprint 1:Xiv:/900.0/e/ 2019.
Aaron Van Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In International Conference on Machine Learning, pp. 1747-1756, 2016.
Vanessa Volz, Jacob Schrum, Jialin Liu, Simon M Lucas, Adam Smith, and Sebastian Risi. Evolving mario levels in the latent space deep convolutional generative adversarial network. In Pro- ceedings of the Genetic and Evolutionary Computation Conference, pp. 221-228 ACM, 2018.
Po- Po-Wei Wang. Priya Donti, Bryan Wilder, and Zico Kolter. Satnet: Bridging deep learning and log- ical reasoning using a differentiable satisfiability solver. In International Conference on Machine Learning, pp. 6545-6555 2019.
Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Broeck. semantic loss function for deep learning with symbolic knowledge. In International Conference Machine Learning, pp. 1598-5507, 2018
Yexiang Xue and Willem-Jan van Hoeve. Embedding decision diagrams into generative adversar- ial networks. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pp. 616-632. Springer, 2019.
Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dim- itris N Metaxas. Stackgan: Text to photo-realistic image synthesis with stacked generative adver- sarial networks. In Proceedings of the IEEE International Conference on Computer Vision, pp- 5907-5915, 2017.
Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference computer vision, pp. 2223-2232, 2017.
11

---

Under review as a conference paper at ICLR 2020
A IMPLEMENTATION DETAILS
A.1 CONSTRAINED IMAGES
The deep neural network for this experiment is based on a Deep Convolutional Generative Adversar- ial Networks (DCGANs) (Radford al. 2013). Batch Normalization (BN) and ReLU are applied between the layers of the generator G. While Leaky ReLU with a slope of 0.2 has been used for the discriminator D. In the last layer of the generator we apply gumbel-sigmoid activation function (GS).
For the training, Adam optimizer has been used with 6 = 0.5 and 3n = 0.999. The learning rate is constant throughout the training (250 epochs) and is equal to 0.0001. The batch size used during the experiments has been set to 64. Layers have been initialized using Xavier initialization for the generator and Orthogonal initialization for the discriminator. Finally, the size of the latent vector has been set to 64 and sampled from a normal distribution with mean = 0.0 and std = 1.0 Table 3 shows the network architecture
Part
Input Shape - Output Shape
Layer Type
Kernel
Stride
(64) - (1024)
Fully Conn.
(1024) - (3200)
Fully Conn.
(3200) + (4. 7, 128)
Reshape
G
(4. a. 128) - ( 2. 64) ( w. 64) (h, w.
Deconv.
5 5 5
2 x2 5x5
Deconv.
x2
2
(h, w. 1) +
64)
Conv.
5
x2
( 7. 64) + (4 7. 128)
Conv.
5 5
x2
D
(E " 128) - (3200)
3.
Reshape
(3200) + (1024)
Fully Conn.
(1024) + (32) (32) (L)
Fully Conn.
Fully Conn.
Table 3: Constrained Images network architecture.
A.2 SUPER MARIO BROS LEVEL GENERATION
The deep neural network for this experiment is based on the DCGANs used in (Volz et al. 2018) Batch Normalization (BN) and ReLU applied between the layers of the generator G. While Leaky ReLU with a slope of 0.2 has been used for the discriminator D. In the last layer of the generator apply Softmax activation function obtain probabilities that are finally given as inputs to the Semantic Loss. On the other hand, the generation of samples is done through the application, always the generator, of stretched Softmax function followed by a Tensorflow OneHooCCategorica layer, which allows to samples levels. In our opinion, this is a best way of generating objects with respect to the simple argmax applied directly on the generator by (Volz et al. 2018).
The networks have been trained using the WGAN guidelines (Arjovsky al. 2017). Thus, the number of iterations on the discriminator has been set to 5 for each iteration on the generator. RM- SProp has been used loss optimizer, with the learning rate constant throughout the training (10000 epochs) and equal to 0.00005. The batch size used during the experiments has been set to 32. Layers have been initialized using Normal initializer for both the generator and the discriminator. More- over, weight clipping is applied latent vector has been set to 64 and sampled from a normal distribution / 1) Table 4 shows the network architecture (as said, each layer is followed by batch normalization and by a ReL./LLaaky ReLU activation function).
the layers of D with equals to 0.01. Finally, the size of the
The feedforward neural network Pn has been implemented using a CNN with the architecture in 3
12

---

256) Under review as a conference paper at ICLR 2020
Part
Input Shape + Output Shape
Layer Type
Kernel
Stride
(32) -> 1.32)
Reshape.
1, ( w 16)
Deconv.
4 4x
1
(4. g. 16) ( w.
Deconv.
4x
2
G
(2. g. 8) + *(. 2 2)
Deconv.
4
2
(2 22.4)
(/,, w. 13)
Deconv.
4 4
2
(h, w, 13) (4. x 64)
Conv. 4
2
2.
(4. 2. 64) + (7. T. 128)
Conv.
4x4
2 2
(h. w. 128) -
Conv.
4x4
2
4'
8 8 256) 7 (1, 1)
Conv.
4
1
Table Super Mario Bros Level Generation network architecture.
Input Shape - Output Shape
Layer Type
Kernel
Stride
(h, u. 13) - (h, w,
Conv.
3 x 3
1 x
(h, ur + (h, m. 16)
Conv.
x
1 1x
(h, w. 16) (/4, w. 24)
Conv.
7
1
(h, w. 24) > (h,, w. 32)
Conv.
9 x
1x
(h,, w. 32)
(J4, w. 64)
Conv.
11 x 11
1
(h, ws, 64) (h, w. 96)
Conv.
13 X 13
1
(h, w. 96) (h, w, 128)
Conv.
15
1
(h, w, 128) + +h, w, 192)
Conv.
17 X 17
1 1x
(h, w. 192) - (h, w, 32)
Fully Conv.
(h, uz 32) (h, w,
Fully Conv.
Table Reachability Network architecture.
A.3 MOLECULE GENERATION
The MolGAN architecture is composed of three networks, the generator G, the discriminator and the reward network R. D and R share the same architecture, but are trained with different objectives. R must estimate either the product of the QED, SA, logP metrics, or the uniqueness of a batch of samples. All these metrics are in [0, 1] so that R should output 1 given perfect sample (w.rt. the aforementioned metrics) or batch made of different samples. G is optimized to produce samples that maximize the output of Rt and are convincing D, on top of that, the semantic loss is also applied.
While R trained in parallel with D and G, the loss of G with respect to the output of R is activated only after 150 epochs, at which point the adversarial loss stops being used. The semantic loss applied G from the start to the end of the training. The weight of the semantic loss equal to 0.9, whereas the adversarial loss of G has a weight of 0...
Improved WGAN is used as the adversarial loss between and (Neritie 5), whereas R is trained to estimate the desired target by mean squared error, and G is trained with deep deterministic policy gradient w.r.t. the output of R, which is seen as a reward to maximize.
Training is ongoing until stop condition is met, either the uniqueness of the batch falling below 0.2 if is an estimator of the three chemical properties, or the validity reaching 0.99 if Rt should output the uniqueness of the batch. During training, the learning rate is set constant value of 0.001, the batch size is 32 and there is no dropout. Adam with betal = 0.9 and belo2 = 0.999 is the optimizer of choice. Batch discrimination is used.
Results are obtained by evaluating a batch of 5000 generated samples. The input noise has dimension if the semantie loss is not applied, 36 if applied; with the first 32 dimensions sampled from standard normal distribution and the last four from uniform distribution in [0, 1].
The maximum number of nodes for each molecule is 9, with 5 possible atom types and 5 bond types. Each molecule is represented by two matrices, one mapping each node to label, and one adjacency matrix informing about the presence lack of edges between nodes, and their type.
The input noise is received by G and processed by three fully connected layers 128, 256, 512 units each, while tanh acts as the activation function; linear projection followed by softmax is
13

---

Under review as a conference paper at ICLR 2020
then applied the output of the last layer to have it matching the size of the adjacency matrices. D and R. share the same architecture (no parameters are shared), based on two relational graph convolutions (De Cao & Kipf 2018) of 64, 32 hidden units, followed by aggregation in (De Cao & Kipf 2018) to obtain a graph level representation of 128 features. Two fully connected layers of T28, units then reduce the graph embedding to single output value, with tanh the activation for the hidden layer and with sigmoid being applied on the output in the case of R.
14


Reviews:
[{"review_id": "rkeHQhM3Kr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed Constrained Adversarial Networks (CAN), which incorporates structural constraints by augmenting a penalty term in the training object. The penalty term is formulated as the semantic loss proposed in [1] which can handle any logical constraints. Experiments are demonstrated to show the advantage of CAN over standard GAN in terms of whether the generated samples satisfy the hard constraints, and whether they are novel and unique.\n\nFirst, I'd like to thank the authors for making this paper easy to follow. I like the idea of encouraging constraints for generative models, which is useful and interesting. However, given the published paper [1], this work seems to be a bit incremental.\n\nThe semantic loss for incorporating constraints and the knowledge compilation techniques for efficient evaluation are both introduced and discussed in [1]. The novelty of this paper is to apply these techniques to generative models, which seem to be a bit straightforward. A similar idea is proposed in [2], where the authors also discussed logical constraints and generative models, but they call the augmented penalty as 'posterior regularization'. I will be interested in a comparison to their method in terms of both methodology level and experiment level.\n\nOverall the contribution of this paper does not seem to be strong enough. I would personally vote for weak rejection.\n\n[1] Xu, Jingyi, et al. \"A semantic loss function for deep learning with symbolic knowledge.\" arXiv preprint arXiv:1711.11157 (2017).\n[2] Hu, Zhiting, et al. \"Deep generative models with learnable knowledge constraints.\" Advances in Neural Information Processing Systems. 2018.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "SygPGB_3YB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors describe a method to improve the performance of generative adversarial networks in the task of generating structured objectives that have to satisfy complicated constraints. The proposed solution involves using an additional term in the GAN objective that penalizes the generation of invalid samples. This term, called the semantic loss, is given by a multiple of the log probability of the model generating valid samples.\n\nClarity:\n\nThe paper is not very well written and several parts need to be clarified. In particular, in equation 3. What is theta in this equation?  how is it obtained? The authors mention briefly how their method could be used to deal with intractable constraints, but they're almost no specific details or examples of how this is done in practice. The proposed approach relays on the knowledge compilation method, but they're very few details of it in the document. Is it used at all in the experiments?\n\nI am concern about the lack of reproducibility of the paper. I believe, from the paper as it is, it will be impossible to reproduce the results. There are no details about public code release, hyper-parameters settings, etc. For example,\nin section 4.3 the authors mention that they condition the constraint on 5 latent dimensions without giving details about which dimensions exactly.\n\nSignificance:\n\nIt is hard to quantify the significance of the contribution. The constrained images problem is very toy and simple and the experiments with molecules do not include any baseline (only the GAN model without the constraint). There have been\nmany recent contributions improving the validity of generative models for molecules and the authors do not compare with any of them.\n\nThe authors also fail to cite relevant work such as\n\nJaques, Natasha, et al. \"Sequence tutor: Conservative fine-tuning of sequence\ngeneration models with kl-control.\" Proceedings of the 34th International\nConference on Machine Learning-Volume 70. JMLR. org, 2017.\n\nSeff, Ari, et al. \"Discrete Object Generation with Reversible Inductive\nConstruction.\" arXiv preprint arXiv:1907.08268 (2019).\n\nNovelty:\n\nThe proposed approach is rather incremental and lacks novelty. It consists in just applying the semantic loss approach of Xu et al. 2018 to GAN training, with very limited new methodological or algorithm contributions.\n\nQuality:\n\nThe experiments performed are not strong enough to validate the proposed method. The authors do not consider strong baselines in their evaluations.\n\nSummary:\n\nI find that the problem addressed by the authors is highly relevant and the proposed approach has the potential to be useful in practice. However, the paper needs to be improved regarding its clarity, reproducibility and strength of experiments before it can be accepted for publication.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "BJgmAroiFB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper the authors present a Generative Adversarial Neural Networks with Xu et al.’s semantic loss applied to the generator. They call this GAN a Constrained Adversarial Network or (CAN) and identify it as a new class of GAN. The authors present three different problem domains for their experiments focused on the generation of constrained images, chunks of Super Mario Bros.-style levels, and molecules. For each domain they include particular constraints for the semantic loss, which biases the generator towards creating valid content according to these constraints. \n\nThe paper at present has a number of issues holding it back. First, I am not convinced by the author’s claims that the application of an existing loss function to the generator is sufficient to identify a new class of GAN. Second, there is a lack of technical detail in the experiments necessary to replicate them. Third, there is a lack of discussion of the experimental results to place them in context for readers. Finally following from the earlier points, there seems to be a lack of technical contributions in the paper. \n\nI certainly agree with the authors about the inability of GANs to learn structural constraints with insufficient training data, as this has been demonstrated in many examples of prior work. I also agree that particular problem domains, as identified by the authors, have stronger structural requirements. However, it is unclear to me why in these instances one would use GANs and not some alternative approach such as constraint-based solvers. Or even if one wanted to employ GANs, what the benefit of adapting the constraints into a loss function is compared to say constraining generated output in a post-hoc process.\n\nThe descriptions of the two of the three experiments do not include any discussion of the GAN architectures or hyperparameters. While this is not strictly necessary in the paper text some discussion in an appendix or a citation to a prior application of the architecture(s) would be appropriate. Without this, it is impossible for future researchers to replicate these results. Further, it is difficult for readers to place the results in context. For individual experiments, such as the Super Mario Bros. experiment, it is unclear why certain choices were made. For example, why train a GAN on just level 1-3 or 3-3, and not train a single model on multiple levels as is common in the field of procedural content generation via machine learning. \n\nThere is a lack of discussion in the paper on the results of each experiment. For example, the output of the GANs for all the experiments seems quite low, and the differences in terms of the results between the GAN and the CAN across the experiments do not seem to be substantial. Some discussion to put this into context for readers would be helpful.\n\nAs far as I can understand the primary technical contributions of the paper are: (1) the application of Xu et al.’s semantic loss to GANS, (2) the constraints developed for the three experiments, and (3) the results of the three experiments. I am unconvinced of the utility of these contributions to a general machine learning audience.\n\n---\n\nUpdated my review as the authors included extra detail regarding the experiments in a new draft, which helped with the reproducibility issue. However, I am still unconvinced in the contributions of the paper outside of what I previously listed. While I am also unfamiliar with any prior example demonstrating that GANs produce invalid structure, this is not a surprising result. Especially as validity can be defined in an arbitrary, domain-specific manner.\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper develops ideas for enabling the data generation with GANs in the presence of structured constraints on the data manifold. This problem is interesting and quite relevant to the ICLR community. The reviewers raised concerns about the similarity to prior work (Xu et al '17), and missing comparisons to previous approaches that study this problem (e.g. Hu et al '18) that make it difficult to judge the significance of the work. Overall, the paper is slightly below the bar for acceptance.
Decision: Reject