Neurosymbolic Deep Generative Models for Sequence Data with Relational Constraints
Authors: Halley Young
Maxwell Du
Osbert Bastani
Venue:  (2021)
Keywords: neurosymbolic
sequence
program synthesis
generative
constraint
music
poetry

Abstract: Recently, there has been significant progress designing deep generative models that generate realistic sequence data such as text or music. Nevertheless, it remains difficult to incorporate high-level structure to guide the generative process. We propose a novel approach for incorporating structure in the form of relational constraints between different subcomponents of an example (e.g., lines of a poem or measures of music).  Our generative model has two parts: (i) one model to generate a realistic set of relational constraints, and (ii) a second model to generate realistic data satisfying these constraints. To train model (i), we propose a novel program synthesis algorithm that infers the relational constraints present in the training data, and then train the models based on the resulting relational constraints.  In our experiments, we show that our approach significantly improves over state-of-the-art approaches in terms of capturing high-level structure in the data, while performing comparably or better in terms of low-level structure.

Under review as a conference paper at ICLR 2020
NEUROSYMBOLIC DEEP GENERATIVE MODELS FOR SEQUENCE DATA WITH RELATIONAL CONSTRAINTS
A Anonymous authors
Paper under double-blind review
ABSTRACT
Recently, there has been siguificant progress designing deep generative mod- els that generate realistic sequence dala such as lext or music. Nevertheless, it remains difficult to incorporate high-level structure to guide the gener- ative process. We propose a novel approach for incorporating structure in the form of relational constraints between different subcomponents of an example (e.g., lines of a poem or measures of music). Our generative model has two parts: (i) one model to generate a realistic set of relational con- straints, and (ii) a second model to generate realistic data satisfying these constraints. To train model (i), propose a novel program synthesis algo- rithm that infers the relational constraints present in the training data, and then train the models based the resulting relational constraints. In experiments, we show that our approach significantly improves over state- of-the-art approaches in terms of capturing high-level structure in the data, while performing comparably or better in terms of low-level structure.
INTRODUCTION
Over the past few years, there has been tremendous progress in designing deep generative models for generating sequence data such as natural language (Vaswaní et al., 2017) or music (Huang et al., 2019). These approaches leverage the vast quantitics of data available in conjunction with unsupervised and sule-vipd learning learn probabilistic models of the data; then, new examples can be generated by sampling from these models, with the possibility of conditioning on initial elements of the sequence.
Despite this progress, a key challenge facing deep generative models is the difficulty incor- porating high-level structure into the generated examples e.g., rhyming and meter across lines of poem, or repetition across measures of a piece of music. The ability to capture high-level structure is important for improving the quality of the generated data, especially in low-data regimes where only small numbers of examples are edge of the structure compresses the amount of information that the generative model has to learn. Furthermore, explicil representations of structure- i.e., in a symbolic way rather than implicitly in vector embedding can have the added benefit that users can modify the structure to guide the generative process.
intuuiiielly knowl-
Recently, Young et al. (2019) proposed a technique called neurosymbolic generative models for incorporating high-level structure into image generation, focusing on simple 2D repeating patterns in images of building facades (c.g., repeating windows). The basic idea is to leverage program synthesis extract structure from data-in particular, given an example image ar they devise algorithm A that extracts a program e A(x) that represents the set of 2D repeating patterns present in training examples x. Then, using the pairs (x, c), they train two generative models: (i) model Po(c) that generates program, and (ii) a model pa(x c) that generates an image that contains the structure represented by c.
However, their approach is heavily tailored to the image domain in several ways. First, their representation of structure is geared towards relatively simple patterns occurring in images of building facades. In addition, their algorithm A is specifically designed to extract this kind of program from an input image, as are their models pa(c) for generating programs and (=( c) for generating images conditioned the program.

---

Under review as a conference paper at ICLR 2020
We represent relational constraints C₂ present in example by relating each subcompo- nent w of a given example x with a prototype w, which can intuitively be thought of as the "original" subcomponent from which tween w and Ø is labeled with a set of relations R, which encodes the constraint that to and in should satisfy relation for each r R. Importantly, while each subcomponent is associated with a single prototype, each prototype may be associated with multiple subcomponents. As a consequence, different subcomponents associated with the same prototype are related in some way. This representation is compact, only requiring lincarly many constraints in the number of subcomponents in x (assuming the number of prototypes is constant). Intuitively, compactness ensures the representation both generalizes well and is easy to generate.
is constructed. In particular, the relationship be-
Then, we design a program synthesis algorithm that can extract an optimal representation of the structure present in training example x. We show how to express the synthesis problem as a constrained combinatorial optimization problem, which we solve using an SMT solver Z3 (De Moura & Bjorner, 2008). Next, we represent as sequence, and design Po(c) to be inferred through a specialized sequence VAE. Finally, we propose three possible designs of (0 | c) based on trying to identify example z that is realistic (c.g., according to pretrained generative model pz)) while simultaneously satisfies the constraints c.
We evaluate our approach two tasks: poetry generation, where the relational constraints include rhyming lines or lines with shared meter, and music generation, where the relational constraints include equality in terms of pitch or rhythm, that one measure is transposition of another (i.e., pitches shifted up or down by constant amount), etc. We show that our approaches outperform or perform similarly to SOTA models according to many metrics.
Finally, we also perform a qualitative evaluation where we show how the user can modify the high-level to generate examples that satisfy additional desired constraints. This abil- ity demonstrates an important feature of our approach-iac iier,ach-ie.., that the user can guide the generative process by modifying the relational constraints as desired.
Example. Figure 1 illustrates how our approach is applied to generate poetry. During training, our approach uses program synthesis to infer relational constraints ez present in the examples I, and uses both x and Cz to train the generative models. Here, Cr is a bipartite graph, where the LHS vertices are protolypes, and the RHS vertices correspond to lines of x. Each verlex on the right is connected to exactly one protolype, and is labeled with constraints on how it should relate to its prototype. To generate new examples, it first samples relational constraints c, and then samples an example that satisfies c i.e, need to choose a line to fill each RHS node in a way that the line satisfies the relations with its prototype. Furthermore, user modify the sampled constraint e to guide the generative process. Thus, our approach enables users to flexibly incorporate domain knowledge on the high-level structure of the data into the generative process, both in terms of the relational constraints included and by allowing them to modify the generated relational constraints.
Related work. There has been recent interest in leveraging program synthesis to improve machine learning. For instance, it has been applied to unsupervised learning of latent structure drawings (Ellis et al., 2015) and to reinforcement learning (Verma et al., 2018). These techniques have benefits such as improving interpretability Verma et al. (2018); Ellis et al. (2020), cnabling learning from fewer examples (Ellis et al., 2015), generalizing more robustly (Inala et al., 2019), and being easier to formally reason about (Bastani et al., 2018). More recently, there has been work leveraging program syuthesis in conjunction with deep learning, where the DNN handles perception and program synthesis handles high-level structure (Ellis et al., 2017), including work in the lifelong learning setting (Valkov et al., 2018). In contrast to these approaches, extend recent work leveraging these ideas in the setting of image generation to incorporating high-level relational structure into sequence generation tasks (Young et al., 2019).
focus is on generative models. In particular,
Early music gencration approaches were rulc-basod (Ovans & Davison, 1992) or used simple statistical models such as Markov models (Sandred et al., 2009; Cope, 1987) or probabilistic CFGs (Quick, 2016). Recent work has used deep learning to generate music (Huang et al., 2019; OpenAI, 2019) and poetry (Liao et al., 2019); our experiments show that these ap- proaches have difficulty generating realistic high-level structure. Approaches have incorpo-
2

---

Under review as a conference paper at ICLR 2020
"Futher," said, "Father, cannot play The horp
The terp that okde give slig
dayma
didst give me, and day Lisi in idleness, while to and fro About me thy serene, grave servants goi And I um weary of my lonely ease. Better a perilous journey overseus Away from thee, than this, the life lead, To sit all day in the sunshine like weed grows naught-
dhymo, mider
And anvwcary of leady cai,,
drymo
Mayy from thee taaithstte Mil
(hymos. mndel
fhymes, meter
rhymo,
fhymes
more they Who serve thee most; yet serve thee in no way
dhyms, mider
diyso, meter
Training Example
Relational Constrinnts Cz
Mudic to song, ns the city
Hyymes, miter
Music song, on the city air little music to live upon
fiee opin
Bymos
Sonp anea pour with sas,
dhymes, meler
Song, song, yes, yes, long, with sun, And all together if they are all ten,
0.5
Uni at resaly for remriggie mel
fhymos
the ciptall buil wita chaso.
dhymas, mila
Until all ready for men again and
And man ded anoend
fhymes,
To the capital hall ride with chase,
1.2.
Thonght sgos donae
thysess meter
And look like men tied full all around
dlyasss
You, and whisper that I shall found,
fhymos, miter
Though I'm not built upon a little dome,
fyymes
say, made home, make
Sample Latent Vector p()
Sample Relational Constraints e p2(-12)
Sample Exumple
Figure 1: Top: Process for training. For each training example 2, our algorithm uses program synthesis to infer the relational constraints O₂ = A(x) present in x. Then, it (i) uses Cr to train P²(c)= Ez-p(-) [po(c ( =))]( and (ii) uses (cz, x) train po c). Bottom: Process for generating a sample x from the learned models po(c 2) and po(x cc). Lines with the same prololype are shown in the same color; metrical constraints are represented
as purple and rhyme constraints as green edges.
rated structure into deep learning to generate music (Medeot et al., 2018) or poetry (Castro & Attarian, 2018), but they are domain specific; we find they do not perform at a human level on capturing global (and sometimes local) structure. Some approaches incorporate expert-provided constraints such rhyme and meter to generate poetry (Lau et al., 2018); unlike our approach, they cannot automatically learn and generate these constraints from data.
2 BACKGROUND ON NEUROSYMBOLIC GENERATIVE MODELS
Consider the problem of learning generative model given training data from the underlying distribution. Given training examples II,,.., Tg ~ p", our goal is to learn a generative model pe p" from which can draw additional samples x ~ pg. We consider sequence an example x e X is a sequence x = (W1,.., Wm) ) E W 1 For example, each subcomponent may be line of poem a measure of music, and x may be poem song.
We are interested in domains where likely examples satisfy latent relational constraints c E C over the subcomponents. For instance, may say that two measures w; and wy of start with the same series of pitches, or two lincs w; and wy of x rhyme. We assume given a set of relations R (e.g., r R might be "rhyme" or "equal"), and function ʃ W W R B (where B = {0, 13) such that [ww, w', indicates whether w and w' satisly relation r. Then, с is a compact representation of the relations present in an input æ. We describe the structure of in detail in Section 3.1; for now, the approach we describe works for any choice of In particular, we build on neurosymbolic generative models (Young et al., 2019),
Wee use fixed m simplify our exposition; our approach trivially extends variable
3

---

Under review as a conference paper at ICLR 2020
where is itself generated based latent value E 2 Z-ie.,
(x) = / /
po(æ c) pole) 2) p(z)dz.
dEC
Then, Young et al. (2019) considers the variational distribution
g3 x) {3(=10) g(c x)
where
g(c [(c12) ò c2).
Here, 8 is the Dirac delta function, and Cr is a single representative associated with x. In particular, Cr is generated from x using a program synthesis algorithm David & Krocning (2017) i.e, an algorithm A that takes as input an example x and outputs a program - A(x) encoding the relational constraints present in z. Next, Young et al. (2019) derive an evidence lower bound
log ( 2 log (x) + (agg(le x)] (2 (2) p(=)).
(1)
where DKL is the KL. divergence and H is the information entropy. The first term of (1) is the log-likelihood of a generative model predicting the probability of example æ given relational structure Cas and the second and third terms form the loss of variational au- toencoder (VAE) Po(c 2) and 92(2 approach separately learns (i) VAF to generate e given 2, and (ii) a generative model to generate x given e; the latter can be a second VAE or a generative adversarial network (GAN) (Goodfellow et al., 2014). This approach is called mmnthesis-guiddes generative models (SGM) since uses program synthesis to guide training.
(Kingma & Welling, 2019). In summary, this
To leverage this framework, have to instantiate (i) the space of relational constraints C, (ii) the syuthesis algorithm A X' + C used to extract a program encoding the structure of æ, and (iii) the architectures of (s(c) =), gz(z c), and po (o [c- In previous work, Young et al. (2019) used hcuristics specific to the the image domain to achieve these goals in particular, they used (i) simple equality constraints on sub-regions of the image designed to capture 2D repeating patterns, (ii) a custom synthesis algorithm that greedily adds constraints in the data to the prograi,, and (iii) a representation of &a generative model over images, and Po: 9g based on an encoding of as a fixed-length vector.
image, in which case po is
We design synthesis algorithm that expresses the synthesis problem as a constrained com- binatorial optimization problem, which it solves using Z3 (De Moura & Bjørner, 2008). In terms of (iii), our programs encode declarative constraints rather than imperative render- ings, the previous architectures of Pe, and 93 cannot be used. Instead, use expert domain-specific heuristics, transformers (Vaswani al., 2017), or graph neural networks (GNNs) (Kipf & Welling, 2017) for Ps and 93. For pa, we propose several methods for imposing the constraints encoded by e when generating an example x.
3 RELATIONAL CONSTRAINTS FOR SEQUENCE DATA
In this section, we describe how we represent relational constraints r, as well as our algorithm A for synthesizing the relational constraints C₂ A(x)) present in example sequence x.
3.1 GRAPH REPRESENTATION OF RELATIONAL CONSTRAINTS
Recall that our generative model operates by first generating a relational program c, and then generating an example that satisfies e. Thus, we need to design relational programs that encode constraints the structure of example z. Our programs encode a set of relational constraints, each of which imposes constraint that subcomponents of x should have certain kinds of relations. We begin by describing the structure of single relational constraint, and then describe how e encodes a set of relational constraints.
A relational constraint & & = x x R, where I {1, .......... 3 is tuple o == (w.i,r) we call t E W a prototype subcomponent. An example x satisfies o (denoted x - 6) f(ū, wi, 1, where w; is the ith subcomponent of a. That is, d says the ith subcomponent
4

---

Under review as a conference paper at ICLR 2020
w, of should have relation with prototype subcomponent w. Thus, we interpret as a function Φ : X + IB where <(z) = 1 if z satisfies @ and p(x) = 0 otherwise.
Next, relational program encodes collection of relational constraints examples an undirected labeled bipartite graph
We represent V and V and edges E S V V L, where L are the labels. The vertices i V are prototype subcomponents v e W; equivalently, they may be vector embeddings of prototype subcomponents. The vertices i V = (1, (1..... m) the indices of subeomponents in x. The edges e E E are tuples e = (i,, i, R), where R [ R. is a set of relations. We impose the constraint that each v E V is part of a single edge (w,, v, Rt) (though ū E V may be part of multiple edges). Finally, encodes the set of relational constraints
(V, VV E) with vertices
Φ. = {100
(w,i, R) E EAAERR
In other words, includes the relational constraint that each subcomponent wa of x should have all relations r E R with prototype w where v is connected to w.
As example, in Figure 1, the graph shown on the top right encodes relational constraint cx, and the top right shows an example x that satisfies all the constraints & E Pa.- The nodes on the left-hand side of Gr are protolype subcomponents w W, each of which is line of poetry. The nodes on the right-hand side correspond to indices i (from i= 1 on top to = m 10 on the bottom); each one is labeled with a set of relations R. Then, &e. contains constraints o = (w, Ra) for each edge ú + i in the graph, which says that line i of x should have relations r E R3 with w. For instance, the last (10th) node in &z has constraints R1o {rhyme, meter), and is connected to prototype line 인 = "Thee harp that thou..." ". Thus, this edge encodes a constraint & = (17, 10, Rio) saying that the last line of should rhyme and have the same meter as w. Indeed, the last line of is wio = "Whoo serve thee most..." which rhymes and has the same meter as "The harp that thou...".
Remark 3.1. We use prototypes rather than direct relationships between components to ensure the size of the graph is tractable. In particular, our approach ensures that the graph linear in the size of the imput (assuming the number of prototypes is constant). compact graph is both to synthesize (for training) or train a model to generate (for generation). Our approach can easily be generalized more complex representations.
program since it can be interpreted gram Ceri et al. (1989) (i.e., a relational logic program). At a high level, &e is a set of Datalog relations over examples x E X. Thus, e can be interpreted program e: X B such that c(z) = ((x)= = for all Φ E &. and c(x) = otherwise.
Remark 3.2. We refer to
Datalog pro-
3.2 SYNTHESIZING RELATIONAL CONSTRAINTS
Recall that when training our generative model, we need to design program synthesis algorithm A that synthesizes a relational program Gr = A(x) that best encodes the latent relational constraints present in each training example x. A key question where the prototypes come from. We simply choose the prototypes w to be actual subcomponents in x. Thus, Cr encodes that subcomponents of x are cach related of distinguished subcomponents of x. We formulate the problem of synthesizing as constrainod optimization problem, which we describe below.
one of a small number
Optimization variables. The variables are binary vector H E T and binary ma- trix K E Bmxm, Intuitively, H; indicates whether subcomponent w; x a prototype subcomponent in e, and Kij indicates whether w; is the prototype for subcomponent wj-
Constraints. Our optimization problem has the following three constraints:
m
1 =kmin& I
Σ H: H kmax
U22
A j=
Σ
KiJ =
=
KiJ S m H4.
First, 1/1 says that the number of prototype subcomponents is between kmin and kmax- Next, th says that every subcomponent w, corresponds to exactly prototype subcomponent wi Finally, thy says that for every i, if w; is the prototype subcomponent of wy according to K, then it must be a prototype subcomponent according to H as well.
5

---

Under review as a conference paper at ICLR 2020
Objective. The objective of our optimization problem is expressed in terms of precom- puted distance matrix D E Rmx", where Dij measures the similarity between components w; and wy; smaller values indicate greater degree of similarity. In particular, we define
Du
Σ FER
(ff(us wj, =),
R
i.e. Diy is the fraction of relations that are not satisfied by wa and wy. Then, our objective (which is be minimized) has the following three terms:
m Σ 6,=1
m
J1 Σ
Ki Diy
J2 J2
II K Kas) )
Dis.
= Σ 6J=1
M₁ M..M3
Dig-
First, J1 says that subcomponents should be similar to their protolypes. Second, J2 says that subcomponents should also be similar to other subcomponents that share the same prototype. Third, J/ says that different prototype subcomponents should be dissimilar.
Optimization problem. Our algorithm A uses Z3 to solve the optimization problem
"") = arg min (A - J1 + )2 J2 + X3 J3} subj. to v A th A 43, H.K
where d1, X2, A Rzu are hyperparamcters. Finally, to construct Cey A chooses
V = fw; H 1),
= mm,
E = {(w,, R.j) Kj = =],
where Rig- = E f(w;, wy. = i.e., V are the prototype subcomponents according to 11,, E the edges according to K*, and Ri are the relations satisfied by w, and wy-
Z3 is guaranteed to find the optimal solution; in the unlikely event that multiple such solutions exist, it chooses one nondeterministically. Intuitively, our approach should perform well when a handful of prototypes are sufficient to approximately capture the relational structure the data. Furthermore, since the user has the ability to define relations, they can adjust their definitions as needed to capture the desired structures.
4 NECROSYMBOLIC GENERATIVE MODELS WITH RELATIONAL CONSTRAINTS
In this section, we describe our deep generative model for generating examples x. Recall that our approach proceeds in two steps: (i) generate c, and (ii) generate x given $.. We describe each of these steps in detail below.
4.1 STEP 1: GENERATING RELATIONAL CONSTRAINTS
The first step of our generative model is to generate relational constraints P. using a VAE
ie., 2 ~ p(-) and ~ Po(- z), where Pa(c 2) VAE and p(2) = [222; 0, I) is a Gaussian distribution. The main choice is the architecture to use for the VAE. In particular, we consider a representation of as a sequence (s1,..,, 8m); where 8; (0, 1,..., m] for each i; intuitively, encodes that subcomponent w; should have the same prototype subcomponent as Wi-a, or if s; = 0, that w; corresponds to a new prototype subcomponent.
More precisely, initialize &. Ø. Then, generate the sequence E 1, m) and ri E (0, .... (where т; is represented as a binary vector of length n = [R]) using an LSTM-VAE, For each i, generate (w, R,) based on 8; and r₁. If 0, we generate prototype subcomponent ú using a domain-specific generative model, generate with using another LST,-VAF, and add di (w,, R,) to T... If s, > let Φi (wi R.).
4.2 STEP 2: GENERATING EXAMPLES GIVEN RELATIONAL CONSTRAINTS
Next, we describe how we implement the second step (x c) of our generative model. We propose three approaches for generating x given to; give details in Appendix A.

---

Under review as a conference paper at ICLR 2020
Approach 1: Constrained sampling. We sample values po(.) by sequentially
sampling W/ ~ p.(.) from a pretrained generative model po(w). We do so using rejection sampling at each step-ie.. we sample w, ~ po(-) until we find w, satisfying f(ü, w.rr) for each (w, i, r) E Φ.. In addition, to speed up sampling, at each step of sampling wa (e.g.> word in line or pitch in a measure), we eliminate choices that violate &c.
Approach 2: Constraint-aware embeddings. We train a conditional generative model po(W,,... Wm le) (in the form of graph convolutional network) that simultaneously gener- ates all m subcomponents in a way that satisfies e, and sample I = (W),.., wm) ) ~ po(- c).
A Approach 3: Combinatorial optimization. We sample x po(.) by sequentially gen- erating W/ by solving an optimization problem whose objective is to maximize adherence to le plus additional terms encoding domain-specific heuristics encouraging w, to be realistic.
5 EXPERIMENTS
We evaluate our approach on music and poctry generation; see Appendix B.6 for details.
Music generation. We evaluated our approach on a music generation task. In this setting, a is a song, and w are measures of music. We consider 20 relations including equality, same rhythm, same pitch progression; etc.; a full list is given in Appendix B.2. We used songs from the Essen folk song corpus (Schaffrath, 1995), using 2000 for training and 500 for testing. For this dataset, we used each of the three approaches A1, A2, and A3 described in Section 4 to sample I pa(- c). For A1, we use a pretrained transformer called MusicAutoBot (Shaw, 2020). For A2, we require a generative model that constructs vector embeddings of measures; we use the version of Magenta's Music which embeds single measures (Roberts et al., 2018). We finetune all models on our training examples.
We compare to MusicAutoBot,, an LSTM model with attention (AttentionRNN) (Waite, 2016), Magenta's 16-bar VAS, and an implementation of StructureNet, an approach that integrates structure into LSTM (Medeot et al., 2018). We also compare to con- straint generation approach called Motifate (Muhammad Faisal, 2017); see Appendix B.5. We compare performance in terms of both high-level and low-level structure. For high-level structure, given a generated (or human held-out) example 2, we use our program synthe- sis algorithm to synthesize its relational structure c. A(x). Then, given a collection Ogen = fcz x E X J of synthesized structure for generated examples, along with a collec- tion Chuman = {cz x ] of synthesized structure for the held-out human examples, we train graph convolutional neural network (GCN) to try and discriminate Cgen from Chuman- In addition, we also train a random forest (RF) over handcrafted features (de- scribed in B.4) to try and discriminate them. In both cases, we use a balanced dataset (i.e., 50% human held-out and 50% generated) so random predictions have accuracy 0.5. For low-level structure, we use the negative log likelihood (NLL) according to MusicAutoBot (pretrained and then fine-tuned on our dataset), Mus,cVAE, and our own GraphVAE (de- scribed in Section A.2). While these metrics are not perfect, they can be used to evaluate across all approaches. For models where NLL is available, we also compare their NLL on the held-out humand data.
We show results in Table 1. For each approach (as well as a held-out human dataset), we show the negative log-likelihood (NLL) assigned to that approach by one of the three models "MusicAutooott", "MusicVAE", and "GraphVAE", According to almost all metrics, our algorithm (using A2 for sampling pe(x | e)) outperforms or performs equally to all others, and achieves performance very similar to human. The one exception is our measure of low-level structure according to MusicAAtoBott however, this model rates StructureNet substantially better than human, indicating that it is not good measure of quality. Finally, our GraphVAL produces a lower NLL for the held-out human music than either MusicAutoBot or MusicVAF-116 which indicates that it models human data better than the others (the other approaches cannot be used to compute NLL).
and MusicVAE-LE
Poetry generation. We use Project Gutenberg's poetry collection (Parrish, 2018), filtered to focus on examples that contain rhymes and meter. We use 2700 10-line poems for training and 300 for testing. In this case, we were unable to apply A3 due to the large
7

---

Under review as a conference paper at ICLR 2020
Low-Level MusicVAB
HigelLevel
Model
MusicAutoBot
GraphVAE
RF Disc.
GCN Disc.
Ours (A1)
1518
1161
1637
0.89
0.54
Ours (A2)
1027
0.79
0.63
Ours (A3)
1240
1161
1029
0.91
0.43
MusieVAE-16
1093
1170
1070
0.85
0.50
MusicAutoBot
5452
1172
1100
0.95
0.51
AttentionRNN
1920
1271
1089
0.88
0.47
StructureNet
902
1162
1062
0.91
0.45
1764
1160
1028
0.51
0.69
Table 1: Results for the music domain. To evaluate low-level structure, we use negative log likelihood according to MusicAutoBot,, Magenta's verrsion of MusicVAE designed for hierarchical 16-bar melodies, and Graph VAF. For high-level structure, we use accuracy of the random forest ("RF Disc.") and GCN cross entropy loss ("GCN Disc."). The best (non-human) score in each column bolded; the human score italicized if best. We also bold the model that achieves the best NLL on the held-out human data.
Low-Level BERT
High-Level GCN Disc.
Diversity Entropy
Model
GPT2
Ours (A1)
3.66
6.51
0.69
4.64
Ours (Ablation)
0.59
GPT2
3.30
3.13
0.47
4.32
GPT2-Opt
3.63
3.87
0.56
4.35
BERT
3.90
5.76
0.50
2.59
RichLyrics
4.66
8.04
0.51
4.86
Human
4.02
4.98
0.70
4.86
Table 2: Results for the poetry domain. To evaluate low-level structure, log likelihood per token of a fine-tuned version of BERT and GPT2. For high-level structure, use cross-entropy loss of the GCN ("GCN Disc."). We also show the information entropy. The highest (non-human) score in each column is bolded.
the negative
size of the vocabulary, making constrained optimization infcasible. We were also unable to apply A2 since stato--ffthheart generative models such as BERT and GPT2 were unable to capture rhyming and meter, since they operate at the word level where this information is unavailable. In A1, rather than sample words going forward, we sample them backward warkwards, making it easier to sample lines that satisfy rhyming constraints; see Appendix A. Thus, we use BERT to sample (Devlin et al., 2018), since supports bidirectional sampling.
We compare to BERT and GPT2 (Radford et al., 2019), both finetuned on our dataset. We also consider a variant GPT2-Opt of GPT2 where we use beam search to choose line breaks in way that maximizes occurrences of rhyme and meter. We also tried variant of GPT2 that used const rained sampling to try and find poems that fit a given rhyme and meter scheme, but the search space was too large and it was unable to generate single poem even after several hours. We also compare to an implementation of RichLyrics (Castro & Attarian, 2018), where the consecutive parts of speech for each line given the previous line and the ability to fill in the correct word for the given part of speech were both learned scparately from the corpus. Finally, in addition to using BERT as a sequential generator, we considered an ablation where we perform constrained sampling, but with a uniformly random L. rather than sampling it from a learned distribution.
As before, we compare both high-level structure and low-level structure. For high-level structure, we again use a GCN discriminator. For low-level structure, we use the negative log likelihoods per token according to each of BERT and GPT2 (finetuned on our training dataset). In this case, because our approach uses constrained sampling from a pretrained generative model, we could not evaluate negative log-likelihood according to approach.
We show results in Table 2. Our approach significantly outperforms all basclines (including RichLyries) in terms of high-level structure. Furthermore, our approach produces a lower BERT NLL than either unmodified- or the BaRT-based implementation of Rich- Lyries. GPT2 and GPT2-Opt. produce more likely output than our technique according to
8

---

Under review as a conference paper at ICLR 2020
One was done. Another was done. And I wish you know the way. Full
One was done. Another was done.
OF nature and of nature nature the
And I wish know the way, this story pour
Only being able humon affairs to
story pour And know lot of things that were called a war See sokdee, fair far beautiful gmac
date
Combine with herself
And know a lot things that were called war See soldier, fair fair besutiful grace loward. Another race Together, married. Mach to see, the dead
Her will and therefore har existenes cunnot ever fail even
As nature huving desire create inself so soi00
race Together, married. Mach see, the dead Were gone. The who ascended to the head
tum't
Alone can nature produce any being
Human existence cannot
Ware gone. The who ascended the head
Then exist
Office retired, gave That doesa't 't tell name, but tells a face.
trace
and hair was littl But was old and not intended for bod.
With
only can
Exist because the nature only is
Po(+). Middle: user modified variant of where the last two lines share a prototype with the two lines before them. Right: A poem generated by GPT2 optimized to maximize rhyme and meter. The colors indicate the relations synthesized by our algorithm after the examples were gencrated.
Figure 2: Left: Poetry generated using relational constraints
the transformer probability metrics, most likely because they are better natural language generation than BERT. If we could instead build on GPT2 (i.e., perform backwards sam- pling with GPTT2, then our approach would likely achieve better performance; we leave this direction to future work. Furthermore, our approach achieved comparable BERT scores to GpT2-Opt, while significantly outperforming it in terms of structure.
Finally, we noticed that one way the baselines (except for the ablation study and RichLyries) tended to perform well was by being very repetitive. Thus, we additionally measured the information entropy of the different models. As can be seen, our approach (both with learned and random Φ.) by far closer human in terms of human entropy than GPT2-Opt and BERT, supporting our hypothesis that the baselines were overly repetitive. RichLyries avoided this lack of entropy, as consecutive words were constrained to belong to different part of speech groups. However, it did not receive a high likelihood score, the transformer model used to produce those parts of speech often resulted in unlikely output.
In Figure 2, we show example poem generated using our approach (left) along with one generated using GPT2-Opt (right). As can be seen, the GPT2-Opt poem does not capture structure in the same way human poems do do-e.g,, adjacent. lines are unrelated, lines have very unequal length, and the only rhymes are the word "the" in the brown lines and the words "to" and "too" in the green lines. There is even less structure in poems generated using vanilla GPT2. Thus, GPT2 is completely unable to capture high-level structure present in the real poctry provided as training data. In contrast, our poem captures structure very similar to the human poem shown in Figure 1, such as rhyming adjacent lines.
key benefit of our approach is that the user can modify the relational constraints (or construct their own from scratch) for use in the second step po(x c), giving the user a way to guide the generative process. Figure 2 demonstrates this process. We manually edited the part of the program corresponding to the last two lines so that they shared a prototype with the previous two lines. The example generated using the unmodified (sampled) constraints is shown on the left, and the example generated using the modified constraints is shown in the middle. We show in the supplement a similar process performed with music data.
User modifications.
6 CONCLUSION
We have presented a novel approach for representing and synthesizing relational constraints on sequence data, and for generating examples whose relational structure resembles that of the training data. Our experiments demonstrate that we outperform existing approaches in terms of achieving human-like structure, while performing comparably or better on widely- used metrics which do not explicitly account for structure. Equally importantly, our ap- proach gives the user a way to guide the generative process by modifying the relational constraints. Directions for future work include automatically discovering relational prim- itives, integrating our structure generator with more powerful language models such as GPT-3, and improving the ability to sample from generative models subject to constraints.
9

---

Under review as a conference paper at ICLR 2020
REFERENCES
Oshert Bastani, Yewen Pu, and Armando Solar-Lezama. Verifiable reinforcement learning via policy extraction. In Advances in neurol information processing systems, pp. 2494 2504, 2018
Pablo Samuel Castro and Maria Attarian. Combining learned lyrical structures and vocabulary for improved lyric generation. CoRR, abs/1811.04651, 2018. URL
http://arxiv.orgg httpp//..rraa///111111/1 04651.
Stefano Ceri, Georg Gottlob, Letizia Tanca, al. What you always wanted know about batalog(and never dared to ask). 1EEE transactions on knowledge and data engineering, 1(1):146-166, 1989.
David Cope. An expert system for computer-assisted composition. puter Music Journal, 11(4):30-46, 1987. ISSN 01489267, 15315169.
Com- URL
htttp://ww.jjsto./ org/stable/3680238
Zihang Dai. Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond a fixed-length context. CoRR, abs/ 19111.9911.00001 2019. URL
httg//bs/19000 028600.
Cristina David and Daniel Kroening. Program synthesis: challenges and opportunities. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineer- ing Sciences, 375(21044:00154003, 2017. doi: 10.1098/rstaa 2tt.s/tttt..1.000....
Leonardo De Moura and Nikolaj Bjorner. Z3: An efficient smt solver. Berlin, Heidelberg, 2008. Springer- Verlag. ISBN 3540787992.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810 fas/1810004805 2018. URI. http://arxiv. http://aaxxiv.oo/a//////// 04805.
Kevin Ellis, Armando Solar-Lezama, and Josh Tenenbaum. Unsupervised learning by pro- gram synthesis. In Advances neural information processing systems, pp. 973 981, 2015.
Kevin Ellis, Danicl Ritchic, Armando Solar-Lezama, and Joshua B. Tenenbaum. Learning to infer graphics programs from hand-drawn images. CoRR, abs/1707.09627, 2017. URL http:://axxvv.orgge
110//1777.00 096277.
Kevin Ellis, Catherine Wong, Maxwell I. Nye, Mathias Sablé-Meyer, Lue Cary, Lucas Morales, Luke B. Hewitt, Armando Solar-Lezams, and Joshua B. Tenenbaum. Dream- coder: Growing generalizable ble, interpretable knowledge with wake-sleep bayesian program learning. CoRR, abs/2006.00 08381,, 2020. URL https: / //arxiv...vv//////////ea//aaaav/ org/abs/2006 08381.
lan Goodfellow, Jean Pouget-Abadie Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahra- mani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 27, pp. 2672 2680. Curran Associates, Inc., 2014. URL http: /app://ppppeeeee www..cc/ppppp..ppptiittccc watss.ccpappaae 5423- /eeeearrrrree---eereeeetreeeeeee
C. Horton and L. Ritchey. Workbook for Harmony Through Melody; The Interaction of Melody, Counterpoint, and Harmony in Western Music. Scarecrow Press, 2000. ISBN 9781461664147, URL attps://books. wwwooooosoooolll...ooooooo.oooo. www.m//oooosiddd-opaaaaaoooo
Cheng-Zhi Anna Huang, Ashish Vaswani. Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew Dai, Matt Hoffman, Monica Dinculescu, and Douglas Eck. Music transformer: Generating music with long-term structure. 2019. URL attps://arxtts wwwrxiv.org/bbs... org/ 04281.
Jeevana Priya Inala, Osbert Bastani, Zenna Tavares, and Armando sizing programmatic policies that inductively generalize. In International Conference on Learning Representations, 2019.
Synthe-
10

---

Under review as a conference paper at ICLR 2020
Diederik P. Kingma and Max Welling. An introduction to variational autoencoders. CoRR, abs/1906.02691) 2019. URL http:://aaat org/abs/1906. 02691.
Thomas N. Kipf and Max Welling, Semi-Supervissed Classification with Graph Convolutional Networks. In Proceedings of the 5th International Conference Leorning Representa- tions, ICLR '17, 2017. URL https:/
Jey Han Lau, Trevor Cohn, Timothy Baldwin, Julian Brooke, and Adam Hammond. Deep- speare: A joint neural model of poctic language, meter and rhyme. CoRR, abs/1807.0382 03391, 2018. URL http://arxii www..org/gaaaa.aa harg/bbs//ee 1807 03491.
Yi Liao, Yasheng Wang, Qun Liu, and Xin Jiang. Gpt-based generation for classical chinese poetry. CoRR, abs/1907 1es/1907.00151, 2019. URL http:///rxixl arrg/abs//999 19t./19977000051...
Gabriele Medeot, Srikanth Cherla, Katerina Kosta, Matt McVicar, Samer Abdallah, Marco Selvi, Ed Newlon-Rex, and Kevin Webster. Structurenet: Inducing structure in gen- erated melodies. In Emilia Gómez, Xiao Hu, Eric Humphrey, and Emmanouil Benetos (eds.), Proceedings of the 19th International Society for Music Information Retrieval Con- ference, ISMIR 2018, Paris, France, September 23-27, 2018, pp. 725 725-731, 2018. URL http://ismir200 irriiraam...... ircam.fr/doc/ ht////oc//ffs./ 126 Paper.pd.
Islam Elgamal Muhammad Faisal, Islam Faisal. Generating random, yet, constrained music. 2017. URT.
OpenAL. Musenet, 2019. URL http:://pppna...c////////aaaaa
Russell Ovans and Rod Davison. An iterative constraint-based expert assistant for music composition. In Proceedings of the Biennial Conference-Canadian Society for Computa-
tional Studies of Intelligence, pp. 76-76. Citescer, 1992.
Allison Parrish, 2018. URL https: //gi htt.: cttp///pprrrii/tttteetteettoorrr
Donya Quick. Learning production probabilitics for musical grammars. Journal of New Music Research, 45:295 313, 10 2016. doi: 110.00/099992111111...222222.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Tlya Sutskever. Language models are unsupervised multitask learners. 2019.
Adam Roberts, Jesse H. Engel, Colin Raffel, Curtis Hawthorne, and Douglas Eck. A hierarchical latent vector model for learning long-term structure in music. CoRR, abs/11033.02 05428, 2018 URL
www.ai.oor/bbb///e//e/a. 05428.
Örjan Sandred, Mikael Laurson, and Mika Kuuskankare. Revisiting the illiac suite rule- based approach to stochastic processes. Sonic Ideas/Ideat Sonicas, 2:42 46, 01 2009.
H. Schaffrath. The essen folksong collection in the humdrum kern format. 1995.
Andrew Shaw, 2020. URL https://gitaii
Lazar Valkov, Dipak Chaudhari, Akash Srivastava, Charles Sutton, and Swarat Chaudhuri. Houdini: Lifelong learning program synthesis. In Advances in Neural Information Processing Systems, pp. 6687,8698, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762, 2017. URL http: pttt:///axxin htt/aaxxvv.rrgg. wrgg/bs/////eeeeeo/o/s 0ete/1666.0376...
Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmoct Kohli, and Swarat Chaudhuri. Programmatically interpretable reinforcement learning. In International Con- ference on Machine Learning, pp. 5045 5054, 2018.
Elliot Waite, 2016. URL URLhttts://magent tensorflow wtt.s//aaannaattttsroooooo
Halley Young, Osbert Bastani, and Mayur Naik. Learning neurosymbolic generative models via program synthesis. 2019.
11

---

Under review as a conference paper at ICLR 2020
A GENERATING EXAMPLES GIVEN RELATIONAL CONSTRAINTS
A.1 APPROACH 1: CONSTRAINED SAMPLING
In the music domain, we choose the pretrained generative model pe(w) to be a pretrained version of MusicAutoBot.. To generate z, we sequentially sample each measure wy condi- tioned on all prior measures W1, ... W;-1- Each measure is sampled by sequentially sampling sequence of pitch-duration pairs until the total duration is 16 beats (i.c., the length of measure). During sampling, we mask pitch-duration pairs that cannot satisfy & (i.e., we set their sampling probability to zero and rescale the remaining probabilities). For instance, if the "has similar interval" relation supposed to hold between the the prototype measure and measure i, and sampling the second note of measure then we mask any pitch j in measure i such that
(pitchh pitchh-1) - (pitch, pitchh-1)) 3,
where pitch, is pitch j in the prototype corresponding to w;. other words, we eliminate pitches that would cause sampling to violate this constraint.
In the the poetry domain, we finetune a pretrained BERT model on our dataset, by taking the pretrained models weights and then training the model our dataset with strong gradient weight decay. BERT has the ability to complete masked words in a sentence. We leverage this ability to sample lines that rhyme and have the same meter, which is challenging task since such lines are a tiny fraction of the search space. We describe how simultancously handle rhyming and equal meter; the cases where only one of these two constraints has to hold are similar. Given a protolype w, we work backwards on each step j, we sample from BERT word word, that has the same number of syllables the corresponding word wordy in the prototype. More prociscly, we feed BERT the sequence
wordz,..., Bordh...,wwodd,-1. word MASK, word
and ask it to fill in the masked word, setting the probability of any word with different number of syllables wordy to zero. In addition, also set the probability of any word too similar to the original word in terms of cosine similarity to zero. For the last word (which sample first), additionally restrict to words that rhyme with wordj. To increase diversity, we sample the remaining words twice (i) backwardssto-forwarr from word k to word 1, where & is the number of words, and (ii) we resample each of the k - words (i.e., except the last word) in a random order. We discard any lines which after being sampled are determined to be too unlikely according to BERT.
A.2 APPROACH 2: AAISTRAINT-AAREE EMBEDDINGS
In this approach, we start with a pretrained generative model pa (w) that ignores c; in particular, we assume that
po(w)= = //ooouu (w u) p(u)du,
where pe u) is the decoder network of a VAE over and p(u) = (), Now, rather than sample wy ~ po(+), we train another generative model
(u ... Um c) N(( um) [p)(c), (c)
to generate latent vectors u, U such that w, pe(- u:) are likely to satisfy o.. More precisely, Hy and Vg are the intermediate outputs of graph convolutional net- work (GCN) (Kipf & Welling, 2017) that takes as input the graph e (where edge attributes um)).
between nodes encode (b)) and ultimately outputs a sequence
Our approach be considered to be graph autoencoder in the sense that the objective function used during training rewards the reconstruction of the exact embeddings of the nodes and (implicitly, through the relationship consistency loss) their edge attribute. Our
12

---

Under review as a conference paper at ICLR 2020
graph incoder/decoder produces one latent vector per node, which are rewarded for close to i..d. Gaussian random variables with mean zero and variance one.
To train Pe, construct a training example (Cz, (u..... Um)) for each training example I = (Wi.... Wm), where u1,"., Um are obtained by the encoder network qø(u w) ie., U4 97(- wi). Then, train Pv using the objective
J(b) = Σ (Ku(W I ()) (c,0z)
_ 1a - (c)()) (ra(()(();c)
The first term enforces that the distribution of the latent vectors u should be Gaussian, and the second term enforces that each lalent vector u should be close its original value according to the VAE encoder (g(" u w). The third term is designed to enforce the sat- isfaction of the constraints Pe. In particular, we train a kind of "semantic discriminator" Pa (u w';r) r), that predicts whether w ~ pe(- u) and w' ~ pe(- u') satisfies relation
ie., [w, w', r) = 1. The network Pa is trained on data generated from the given training examples Then, given Pa: want (U1,.., Um) (c( to satisfy
Pa ū, x {o
if (w, R) ARER otherwise
where
u ~ qp( w).
In other words, want generate example that satisfies the relations in according to Pa- In particular, we use the loss
Jrot ΣΣ CE CE(pa ú, (
where
gp( w),
and where CE denotes the cross-entropy loss. Once we have trained pe: we generate se- quences by sampling ū p.(. ( c) and w. po(: u.), and constructing -w1,..., Wm)-
For the music domain, we use embeddings from a pretrained Magenta MAsi unlike the Music VAE used for evaluation, a 256-dimensional vector. Then,w e use the decoder portion this model to convert the embeddings u1,..., Um (u thm c) sampled from the GCN-VAE Pe into mcasures. The graphs in the training set vary in size depending on the number of prototype measures.
finetuned it to decode only 1-2 measures of music from
A.3 APPROACH 3: COMBINATORIAL OPTIMIZATION
Given sampled program e, this approach attempts to generate values x = (Wo,... Wm) such that x += $ by solving a system of constraint solving problem. However, the relational constraints ø E &e are not always consistent with one another, so we relax the constraint x |== 4. as an objective i.e.,
VIA
arg max JEN
Σ TE
1(R(ū, 4 (0,i,) &.).
Encoding this optimization problem as one Z3 can solve depends on the domain and rela- lions. For this approach to work, we need to include additional, handcrafted terms in the objective that encourage the generated example æ is realistic.
For the music domain, the optimization variables are the optimal sequence of pitches and their durations. The objective function a linear combination of the degree to which æ satisfies c, along with domain-specific heuristics e.g., minimizing barge.jump in pitch values (i.c., |pitch;41 pitchh| 2 4), not having any intervals of length (i.c., ppitch;41 - = 6) due to the unpleasant harmonic nature of that interval, and not having two consecutive jumps in pitch (i.e., |pitch,42 - pitch, 2 5) A |pitch; - pitch,/ 2 5). These heuristics based standard concepts from music theory (Horton & Ritchey, 2000).
13

---

Under review as a conference paper at ICLR 2020
B EVALUATION DETAILS
B.1 EXPERIMENTAL SETUP
Generating c. To generate c, we use an LSTM-VAE with LSTM layers and latent size of 200 in the music domain and 50 in the poctry domain. This model is trained to reproduce a given sequence (s),T) pairs which are given as imput, with an additional requirement that the distribution of their encodings should be roughly equivalent to a Gaussian normal distribution. Each (8). pair represented as (S + [[))dimmensoonl vector, where is the maximum distance between objects with the same prototype and R is the set of relations.
Evaluating low-level structure. We evaluate low-level structure by using negative log likelihood according to a deep generative model. the music domain, we use both Mu- sicAutoBot, transformer based the transformer-XL architecture (Dai et al., 2019), as well as the MusicVAE from Magenta, which is a hierarchical VAE that learns embeddings for each measure and then learns an LSTM-VAE lop of these embeddings. In the po- etry domain, we use BERT and GPT2, both finetuned on our dataset and solely pretrained on a non-poetry dataset. For the evaluation metrics that explicitly captured structure, we computed the optimal program A(x) for every example x in the held-out validation human dataset as well as all of the generated examples.
High-level structure. We evaluate high-level structure by using our algorithm to synthe- size the relational constraints in every generated example-i.c., Cgen [A(x) x E X ggnn where Xgen is the set of exaiples generated using model. Similarly, we construct Xhuman); ); where Xuman is the set of human-created examples held- out from the training dataset. Then, we evaluate high-level structure by training model to try and discriminate Cgen from Chumani if the model achieves lower performance, then the quality of high-level structure is higher. A general approach is to train graph neural network (e.g., graph convolutional network) to do so; this model takes as input the graph structure of relational constraints c, along with vector embeddings of the prototype subcom- ponents, and outputs whether e E Cgen or с E Chuman- We balance the data so it consists of 50% human data and 50% generated data. We report the cross-entropy (CE) loss; higher values correspond to better generative models. In the music domain, we additionally used random forest (RF) trained on manual featurization of c. We report the accuracy of the RF; lower values (i.e., closer to 50%) correspond to better generative models.
Chuman = (A(x))
B.2 MUSICAL RELATIONS USED
The following are the relations E R. used in the music domain:
1. Measures and j have the same pitch classes.
2. Measures and j have the same pitch class prefix.
3. Measures and j have the same pitch class suffix.
4. Measures and j's pitches have an edit distance 1.
5. Measures and j have approximately the same interval structure.
6. Measures and j have the same interval prefix.
7. Measures and j have the same interval suffix.
8. Measures and j have the same note (pitch + duration) prefix.
9. Measures and j have the same note (pitch + duration) suffix.
10. Measures and j have the same rhythm.
11. Measures and j's rhythm has an edit distance of < 2.
12. Either measure i's onsets are a subset of measure j's onsets, or measure j's onsets subset of measure i's onsets.
13. Measures and j have the same rhythmic and melodic contour.
14

---

Under review as a conference paper at ICLR 2020
14. Measures and have the same rhythmic and melodic contour prefix.
15. Measures and j have the same rhythmic and melodic contour sullix.
16. Either the first or second half of measures and are identical.
17. Either both or neither of measures and j have leaps.
18. Measures i and fit within the same diatonic scale.
19. Either both neither of mcasures and j have syncopation.
20. Either both or neither of measures and have consecutive notes shorter than an eighth note.
B.3 POETRY RELATIONS USED
The following are the relations R used in the poctry domain:
1. Lines i and have the same end rhyme.
2. Lines i and have the same meter.
B.4 RANDOM FOREST FEATURES
The following are the manually constructed features used in the random forest discriminator for the music domain:
1. Mean number of relations between prototype and sequence measures.
2. Variance of number of relations between prototype and sequence. measures
3. Variance in histogram prototype measure mappings.
4. Longest sequence é...j such that w; Wj all have the same prototype measure.
5. Number of pairs (i.j) such that w; ر and u Wj+1-
6. Mean distance between two measures with the same prototype.
7. Variance in distance between measures with the same prototype.
B.5 COMPARISON TO CONSTRAINT SOIVING
We also considered comparison to constraint-based implementation called Motifate, with explicit attention to development of musical material (Muhammad Faisal, 2017). This approach was designed with heuristics for 3-beat measures, while our evaluation models anticipated 4-beat measures, so could not obtain NLL scores. Nevertheless, that even the structure was insufficient our RF discriminator had accuracy 0.91, and our GCN discriminator had cross entropy loss 0.43, both of which are significantly worse than the other approaches.
found
B.6 QUALITATIVE RESULTS
Music domain. In addition to quantilative measurements, we evaluated the strengths and weaknesses of our approach using A2 (which was the best according quantitative metries). According to our observations, the strengths of A2 include clearer phrases with obvious resolutions, likely and plausibly repetitive rhythms, intervals between notes which scemed plausible but not overly repetitive, and less variance in quality. However, the results were not very rhythmically diverse, and certain idiomatic patterns of resolutions intervals between notes and at the end of phrases were not followed. Furthermore, AttentionRNN does better in terms creating realistic chord progressions (we did not explicitly consider chord progressions in model; doing so is promising direction for future work). Finally, while global structure is much better than the baselines, examples still relatively infrequently had the full four-bar repetitions characteristic of much folk music.
15

---

Under review as a conference paper at ICLR 2020
12
Figure 3: A song generated using approach A3 (top), and nearly identical song generated where part of the sampled relational constraints e were manually modified. These pieces were generated using A3, and the same reference measures w were used, but 4. was slightly perturbed (the similarity relations were changed).
Examples. Here show how modifications occur in the music setting, By explicitly modifying we are able to generate two pieces with similar internal patterns but with different structural characteristics.
We show an example of generated songs using our approach with each A1, A2, and A3 in Figure Figure 5, and Figure 6, respectively, and show an example generated using each of the baselines MusieVAEIl AutentionRNN, MusicAutoBol, and Structurenet in Figures 7, 8, 9, and 10, respectively. Qualitatively, the generated music and poetry appears plausible, exhibiting realistic high-level structure without sacrificing low-level structure.
Figure 4: An example of song generated using approach (A1). Measures that have the same prototype are shown in the same color. Note the existence of repeating four-bar phrases, found commonly in folk songs.
We also give examples of poetry generated using our baselines in particular, GPT2 fine- tuned and optimized for rhyme and meter in Figure 12, BERT finetuned as language generation model in Figure 14, RichLyrics, and our ablation (i.c., use BERT in conjunction with uniformly randomly sampled ..) in Figure 15.
16

---

Under review as a conference paper at ICLR 2020
Figure 5: An example of song generated using approach (A2). Measures that have the same prototype are shown in the same color. Note the existence of clear phrase endings marked by long notes rests, particularly the recurring pattern of fast notes resolving into long notes.
114
11
11
Figure 6: An example of a song generated using our approach (A3). Measures that have the same prototype are shown the same color. The existence of two-bar and three-bar phrases is apparent, but the close note and rhythm similarities among different prototypes weaken the overall clarity of the song's melody.
e
0
Figure 7: An example of a song generated using Magenta's hicrarchical MusicVAE model finetuned on our dataset. While the local structure extremely coherent, does not seem to possess the expected internal
17

---

Under review as a conference paper at ICLR 2020
11
14
Figure 8: An example of a song generated using AttentionRNN trained on our dataset. Note the existence of erratic rhythms and unclear structure, which common traits of custom-trained AttentionRNNN models.
too
Figure 9: An example of song generated using MusicAutoBot. Note the repetitive nature and stark contrast between the first half and second half of the song, which are common problems with transformer models.
Figure 10: An example of a song generated using StructureNet. While some degree inter- nal structure is apparent, and the local coherence is high, the pattern of internal repetition seems fairly arbitrary.
18

---

Under review as a conference paper at ICLR 2020
I know many things, and therefore I forgot, Though needed time to look ahead,
To understand something, time to let it fade away As though was yesterday as they
Were common things, free, freher-free, to go like the tide; But another is to make one, it does.
Perhaps you know it. A queen, her beautiful son, And another woman who has to go without one. The voices like their cries of war, They let us believe in good restore!
Figure 11: An example of poetry generated using our approach. Lines that have the same prototype shown in the same color.
Through the air and through the sky And through all the world
I saw the sun the moon star shine In the midst of the stars
The were shining in my eyes my heart Was throbbing with joy felt
My heart was beating with love I was
little child in a little town
Where the little boys play
Figure 12: A poem generated using GPT2-Opt. It is more plausible than BERT in terms of of global structure, which may be due to the fact that GPT2 better text generation tool than BERT, but it is still somewhat repetitive and its structure is not very human-like.
all all and and
and and and and and all all all all and and and and
o and and o and and
and of of of and o o o but and and of
and and a and and last last last of of and and
Figure 13: A poem generated using BERT. It is clearly overly repetitive and not very semantically coherent, and lacks high-level structure.
19

---

Under review as a conference paper at ICLR 2020
and after all text that all more appointment
make its room from sat and all district withour solf" she one is hundred first enjoy her
but her been two you shall be
above she leave enjoying the suffering usual for which more science this day sewing shall two houses for recent contributions and time which have left for self woman and morcover lot use been found called might where any boots accident
Figure 14: A poem generated using RichLyrics. While it is less repetitive than non- conditioned BERT, it is still not very semantically coherent, and lacks high-level structure.
the first independent, , like for the new songs for morning,
little world, they asked them for way she asked them for a night,
with two heds sometimes lying on light- bed. the first for women, with another, one band
with one paul simon never got play
on the subject, she bought a different dress for different tent,
and one dress for warning...
Figure 15: A poem generated using our ablation. While it is much more coherent, it lacks the idiomatic rhyme and meter structure of our approach.
20


Reviews:
[{"review_id": "wINdhSo3D1", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a neural-symbolic deep generative model to generate music and poems. Compared to other natural language sentence generation tasks, songs and poems usually require some structural constraints, e.g., rhythm and rhymes. Therefore, their approach first generates a program represented as a set of relational constraints, then generates sequences that satisfy the constraints. During the training time, given a training example, i.e., a sequence, they formulate the program synthesis task as a combinatorial optimization problem, and thus they can find the required constraints using the Z3 solver. They use the SOTA generative models for both of the two sequence generation domains. They propose several variants of the generation approach, to ensure that the generated sequences satisfy the constraints. They evaluate their approach in terms of both the low-level and high-level structures. To measure the low-level structure, they compute the negative log likelihood of the SOTA generative models. To measure the high-level structure, they apply the Z3 solver to solve the relational constraints of each generated sequence, and train some discriminators to distinguish the generated sequences from the human-written ones in the dataset. For the poetry domain, they also evaluate the diversity of the generated sequences. Generally, the neural-symbolic approach generates sequences with better high-level structures, and the generated samples are more diverse.\n\nI think this paper studies an interesting topic. The high-level framework design is kind of extended from Young et al.'s work on neural-symbolic generative models. However, as discussed in the paper, Young et al. focus on generating simple 2D images with repetitive patterns, and I think it is non-trivial to extend that work to sequence generation domains in this paper. Adding a program synthesis component is a good way of incorporating the structured constraints for generative models, and it is good to see that the approach indeed generates more natural songs and poems than using the pure neural generative models alone.\n\nI have a couple of questions listed below:\n\n1. How scalable is the proposed approach? For example, before the model generates a sequence that satisfies the constraints, typically how many tokens need to be generated during the sampling process, and how long does it take? I see that the authors mention the number of lines for the poetry domain and the number of beats for the music domain, but I would like to know more details about the token-level lengths. Also, how long does it take for the Z3 solver to find the constraints given a sequence?\n\n2. What are the entropy results for the music domain? Also, in Table 1, for the MusicAutoBot results, did you also present the negative log likelihood? If so, the lower should also be better, right? Why is the highest number highlighted?\n\n3. Have the authors conducted any user study? For example, the authors can present both the generated and the human-written songs and poems to people, and see whether they can identify the ones generated by neural models. I don't expect the authors to finish such a user study soon if they haven't done it, but would like to see a discussion if they already have some conclusions.\n\nWriting comments: On page 8, \"Figure 5\" should be \"Figure 2\".", "extra_scores": {}, "extra_text": {}}, {"review_id": "Vj3MDVsydGU", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# Overall summary\nThis paper proposes a generative model for sequential data structured which uses latent relational constraints and conditions upon them to generate the actual sequence.  The authors propose that such a model can be better suited for data like poetry or music where such constraints are an important feature of the data.\n\n# Strengths\n- The method provides a strong and useful inductive bias for modeling sequences which we can expect to have strong relational constraints, without having to learn them from scratch.\n- The method incorporates a discrete optimization-based component which can lead to much more interpretable results.\n\n# Weaknesses\n- The method requires significant hand design of the constraints in order to work well. This may be difficult or counter-effective when the constraints are not easily described for the domain at hand.\n- It is not entirely clear when the relational constraint optimization will produce good results. There is no theory presented about what it might find.\n- The empirical results presented are not especially strong.\n\n# Comments on the evaluation\nIt should be possible to estimate a lower bound on the log likelihood of the proposed model on a held-out test set, rather than using different models. After all, the goal is that the proposed method can model these sequential data with relational constraints better than the existing methods. Therefore, it's not clear that MusicAutoBot, MusicVAE, BERT, etc. putting higher or lower likelihood on samples from the proposed method should necessarily be a strong indication of the method's quality.\nTo provide a tighter lower bound, it's possible to sample z multiple times and average, as in IWAE: https://arxiv.org/abs/1509.00519\nFurthermore, if the priors about the data that the proposed method presupposes are correct, then it should be possible to show greater data efficiency than the baseline methods. \nI would be willing to revise the score if the authors can provide further explanation/data about this.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "L1zo_hA9zcx", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\n\nThis paper proposes an approach for generating sequences that possess high-level structure, and in particular structure that can be expressed using hand-crafted symbolic relations. Given a domain and a set of possible relations to consider, this approach first extracts a sequence representation of the relational constraints for each example, then trains a two-stage generative model that first generates relational constraints and then generates the final output conditioned on the constraints. The authors apply their approach to the generation of music and poetry, and show that, compared to simple baselines, the generated outputs are more consistent with human-generated examples (according to other learned models of high- and low-level structure).\n\nThe approach is quite general, and seems like an interesting way of imposing structure on sequences. However, I'm not convinced that their approach \"significantly improves over state-of-the-art approaches\" as the authors claim. The baselines are somewhat limited, and the evaluation metrics that the authors use are difficult to interpret. The authors also do not compare against existing approaches for constrained music and poetry generation, instead comparing only to neural sequence models. Additionally, the generated results still seem to lack global coherence despite having global relational structure.\n\n### Detailed comments\n\nThe authors present their approach as an extension of \"neurosymbolic generative modeling\", which uses program synthesis to extract structure and then trains one model to generate programs and another model to use the output of that program. In this work, the authors consider a different representation of structure, which associates each element of a sequence with a prototype and a set of relations that determine how the element relates to the prototype. The prototypes in this approach are subcomponents of the full output (for music, they are measures, and for poetry, they are lines), and the relations are hand-engineered (rhyme scheme and meter for poetry, and various music theory concepts for music). This relational graph is still referred to as a \"program\", and the extraction of the graph as \"program synthesis\", but I'm not sure that's the appropriate terminology; the graph isn't executable, it is just a symbolic object.\n\nThe generative process that the authors propose is to first generate constraints (expressed as a sequence), then generate prototypes, and finally generate output conditioned on the constraints and prototypes. Each of these components is trained separately. Given a dataset, a maximal set of constraints is extracted from each example by using an SMT solver. Afterward, a VAE is trained to produce constraints, and a pretrained model is used to generate prototypes. Finally, examples are sampled conditioned on these constraints, either by doing rejection sampling on a pretrained model or by training a new generative model to generate outputs conditioned on constraints (or, in some cases, by doing combinatorial search with heuristics).\n\nThe authors cite a large number of works in the realm of neurosymbolic machine learning, but are not as thorough regarding music and poetry generation, both of which are subfields of their own right. For music generation, there are existing approaches for trying to learn long-term structure, such as Music Transformer [1], hierarchical VAEs [2], and StructureNet [3]. For poetry, structure can be learned using methods like weighted transducers [4], and there are also previous works on using explicit constraints to generate poems [5]. The authors do not compare with any of these methods, and the ones they do compare against aren't necessarily representative of \"state of the art\".\n\nI'm also not convinced that the metrics the authors use are adequate for judging performance. For \"low level\" structure, the authors report the (negative) log-likelihood scores given by a pretrained sequence model, with the assumption that higher log-likelihood means better low-level structure. But it has been shown that log-likelihood of models does NOT directly correspond to what humans find \"good\" (see for instance [6]), and models will in some cases assign higher log-likelihood to data far outside of their training set (e.g. [7]), which means that higher log-likelihood isn't necessarily a good proxy for human-like structure. It would be more standard to report log-likelihood of the human test data under the proposed model, instead of log-likelihood of the proposed model's outputs under some pre-trained model.\n\nFor high-level structure, the authors extract relations using their method, then train a separate classifier to discriminate between human and generated examples based on those extracted relations as features. But this seems to be slightly unfair, since their model is trained directly to maximize this objective, whereas other models do not use this at all. The presentation of these results is also quite confusing (see below).\n\nIn a more qualitative sense, while it is true that the generated poems do have reasonable rhyme and meter, the semantic content of the poems is disjointed and does not have a sense of global coherence. I wonder if the proposed approach is satisfying the hand-engineered relations at the expense of losing the semantic structure that is harder to measure. \n\n### Questions and suggestions\n\nAbstract: Should \"generate\" be \"generation\"? Also, the sentence \"To train model (i), ... resulting relational constraints\" is a bit unclear. Is that whole sentence about model (i) or is the last part referring to model (ii)?\n\nIn figure 1, what do the colors of the lines represent? It seems that there are both green and purple relational constraints, but these colors aren't explained.\n\nThe notation in sections 2 and 3 is a bit unclear. The set $\\mathcal{C}$ is used before being defined, and it isn't clear what the difference is between $c$, $f$, and $\\Phi$; are those all just representations of the same thing? The sum in the equation at the bottom of page 3 is over $r \\in \\mathcal{R}$ but $r$ doesn't appear in the equation, only $c$ does.\n\nIt seems somewhat restrictive to assume that each line/measure has only one prototype, since it's possible for line C to share rhyme with line A but share meter with line B. Have you considered a more general formulation of constraints?\n\nI was surprised that the prototype subcomponents are represented as part of the constraints but are generated independently from the set of relations. It seems like those prototype subcomponents could just as easily be left out of the constraint sets and instead just be generated as part of the second step. Is there a reason that they need to be part of the constraints?\n\nThe optimization problem in 3.2 seems underdetermined. In particular, if X and Y have some relation between them, it doesn't seem like there's anything to determine which of the two should become the prototype. Is one of those simply chosen arbitrarily?\n\nIn section 4.1, $p_\\phi(c|z)$ is referred to as \"a VAE\", which seems slightly inaccurate; technically the VAE includes the prior, encoder, and decoder, not just the decoder.\n\nIn section 5, the description of how A3 is applied to the music domain seems to be missing from the first paragraph.\n\nThe results in table 1 are very difficult to understand, especially for the high-level classification task. Based on the appendix, it seems that \"RF Disc.\" is measuring classification accuracy, for which lower values are better (because more humanlike outputs are harder to classify). But \"CGN Disc.\" is measuring cross-entropy loss, for which higher values are better. This was very difficult to follow. I think it would be clearer to just report accuracy for both, or, at the very least, describe what each of the columns are measuring. I also don't understand the Human row of this table. Is this comparing one set of human examples to a different set of human examples? For RF, 50% seems like the expected chance accuracy, but distinguishing humans from humans is apparently possible 51% of the time? For GCN, the value 0.69 makes me think this must be measured with a natural logarithm, is that correct?\n\nThe statement \"our approach outperforms BERT as a language model in terms of its own score\" seems unfair due to the reasons described in the previous section. We would not expect a language model to assign the highest likelihood to all of its own predictions, since it is supposed to describe a full distribution of outputs. The likelihood assigned by a language model isn't a measure of goodness, just a measure of how predictable that sequence is by the model. At most, perhaps this indicates that the proposed approach is more predictable than real-world natural language, so BERT is able to predict it better. (Also, is the BERT used to score the models the same as the BERT used to generate? Or is one of them fine-tuned and the other left as-is?)\n\n### References\n\n[1] Huang, Cheng-Zhi Anna, et al. \"Music transformer: Generating music with long-term structure.\" International Conference on Learning Representations. 2018.\n\n[2] Roberts, Adam, Jesse Engel, and Douglas Eck. \"Hierarchical variational autoencoders for music.\" NIPS Workshop on Machine Learning for Creativity and Design. Vol. 3. 2017.\n\n[3] Medeot, Gabriele, et al. \"StructureNet: Inducing Structure in Generated Melodies.\" ISMIR. 2018.\n\n[4] Hopkins, Jack, and Douwe Kiela. \"Automatically generating rhythmic verse with neural networks.\" Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2017.\n\n[5] Toivanen, Jukka, Matti Järvisalo, and Hannu Toivonen. \"Harnessing constraint programming for poetry composition.\" The Fourth International Conference on Computational Creativity. The University of Sydney, 2013.\n\n[6] Meister, Clara, Tim Vieira, and Ryan Cotterell. \"If beam search is the answer, what was the question?.\" arXiv preprint arXiv:2010.02650 (2020).\n\n[7] Nalisnick, Eric, et al. \"Do deep generative models know what they don't know?.\" arXiv preprint arXiv:1810.09136 (2018).\n\n\n---\n\n## Post-revision update\n\nI cannot seem to add a new comment at this time, so I am editing this review instead. The updated paper seems to be an improvement, although some of my original concerns remain.\n\nImprovements:\n\n- Thank you for adding the additional baselines, which do give more context as to how this approach relates to prior work.\n- The section describing $c$, $f$, and $\\Phi$ is clearer in the revision, and is much easier to follow.\n- I also appreciate the clarification about nondeterministic choices in Z3.\n- The overly-strong claims about outperforming state-of-the-art approaches have been qualified appropriately.\n- Some of the confusing details regarding the evaluation method have been moved from the appendix to the main text, which makes them much easier to find.\n\nRemaining high-level concern:\n\n- I'm still not convinced that the results of this evaluation method are that meaningful.\n  + NLL under a pretrained model doesn't necessarily imply better low-level structure, for reasons stated in my initial review.\n  + Ease-of-discrimination of your extracted constraints doesn't necessarily imply better high-level structure, especially since the proposed model is trained on those constraints directly.\n\nOther remaining issues in the revision:\n\n- A few comments from my initial review have not been addressed:\n  + \"To train model (i), ... resulting relational constraints\" remains unclear\n  + In section 4.1, $p_\\phi(c|z)$ is referred to as \"a VAE\" but is just a small part of a VAE\n  + In section 5, the description of how A3 is applied to the music domain is missing\n  + Table 1 is still difficult to interpret, in particular regarding the higher-is-better vs lower-is-better columns, and the (new in this revision) bolding of the human data, which I don't quite follow.\n- Figure 1's caption now states that rhyme and meter constraints are the reason for the green and purple edges, but this is confusing because there are purple edges between poem lines that don't have meter constraints.\n- Regarding semantic content, the authors state in their response that \"The semantic content of the poems (or lack thereof) reflects defects in the underlying deep generative models\". But Section 4.2 approach 1 seems to imply that each of the lines are sampled independently of one another, which seems like a strong limitation that the underlying models do not have. Perhaps the notation is unclear, and the model does get the full input context; if this is the case, I would suggest revising the notation.\n\nI have raised my score from 5 to 6 in light of the improvements, and with the understanding that my concerns in \"Other remaining issues in the revision\" could be fixed in a final version of the paper. I'm still borderline on accepting the paper, however, due to my concern (shared with Reviewer 3) about how meaningful the evaluation results are and whether they match what humans mean by high-level and low-level structure.", "extra_scores": {}, "extra_text": {}}, {"review_id": "CWS1rIhHkFY", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**\n\nThis paper presents a novel method of incorporating structural and relational constraints into the generative process of sequences. This approach is explored in both the music and poetry domains using several generative approaches. An analysis of the resulting sequences finds that they successfully incorporate structure.\n\n**Strengths**\n\nNovel procedure for extracting constraints from training data\n\nResults explored in multiple domains\n\nExample of interactive user modifications, showing that the constraints are human-interpretable and useful for guiding generation toward a creative goal\n\n**Concerns**\nMy first big concern is a lack of comparison against other methods for using structural constraints in the generative process. A couple examples:\n\nFor music generation, I feel a comparison against “StructureNet: INDUCING STRUCTURE IN GENERATED MELODIES” by Medeot et al. (http://ismir2018.ircam.fr/doc/pdfs/126_Paper.pdf) is needed. The method for extracting structure is different, but the goals are very similar. I think it’s important to demonstrate that the method your paper proposes is more effective.\n\nFor poetry, I’d like to see a comparison against a method similar to the one described in “Combining Learned Lyrical Structures and Vocabulary for Improved Lyric Generation” by Castro and Attarian (https://arxiv.org/pdf/1811.04651.pdf).\n\nBoth of these methods have the advantage of using a relatively simple technique to extract structure from the training data, so I’d like to see evidence that the more complex method proposed in this paper provides an advantage.\n\nMy other big concern is the lack of rigour in the evaluation process. This paper both proposes a new method for generating sequences with improved structural characteristics and several new methods for evaluating high level structure. However, very little time is spent validating that these evaluation methods actually accomplish their goals. I would like to see something like a systematic human evaluation of outputs that shows human perception agrees with the automated evaluations.\n\nA few more concerns:\n\nSections 3 and 4 are very thorough in their descriptions, but I found it difficult to keep all the variables and their uses straight in my head as I was reading through. I think adding some diagrams and a few simplified concrete examples would greatly improve this section.\n\nSection 4.1: Either a citation or more architectural and training details are needed about the LSTM-VAE.\n\nI found Tables 1 and 2 confusing. The bolding doesn’t seem to match up with the best scores. Am I correct that in Table 1, the lowest Low-Level score should be best and the highest High-Level score should be best? For example, shouldn’t the MusicAutoBot row be bolded in the first column? Similar issues for Table 2. I think providing more visual separation between the different scoring mechanisms and explicitly saying “higher is better” or “lower is better” would help clear this up.\n\nLast 2 paragraphs of Section 5: There are a couple references to Figure 5, but there is no Figure 5 in the body of the paper.\n\nThe user modifications discussed in the final paragraph of Section 5 are very interesting, and show a real advantage to having interpretable structural constraints. I think this would be much more clear if you included a concrete example of what it looks like to modify the program to produce the new output. Even just a snippet of what the original program looked like, then the modification, then the new output. This is very much related to my comment on Sections 3 and 4: having some concrete examples would really help clarify exactly what’s happening.\n\n**Additional minor feedback**\n\nIn the abstract, second sentence, s/generate/generation/.\n\nSection 2, 2nd paragraph, “indicates whether w and w satisfy”. I think the second w was meant to be w’.\n\nSection 4.2, Approach 1. s/rejecting sampling/rejection sampling/.\n\n**Questions for the rebuttal period**\n\nPlease address the concerns above.\n\nI’d also like to hear your thoughts on how this can be expanded to music more complex than monophonic folk melodies.\n\n**Recommendation**\n\nMy recommendation is to reject due to the concerns listed above, in particular the lack of comparisons to existing structural constraint methods and rigour of the evaluations.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper was pretty borderline, but ultimately I am recommending rejection, for the following reason:

The two most negative reviewers (in terms of original score) were concerned about both the quality of the evaluation
and whether the evaluation metrics actually meant what the paper claimed they meant. 
The authors did make a good faith effort to update the paper to respond to some of the concerns about quality, 
but R4 (who has read the rebuttal) is still not convinced that the results of the evaluation are meaningful,
and I think I agree with their concern (and I don't see any attempt to address that concern in the rebuttal?). 
I view (maybe naively) the goal of this review process as being mostly a correctness check, 
and I don't think this paper has passed the correctness check to my satisfaction or to the satisfaction of the majority of reviewers. 

However! This is a fixable issue. The paper is definitely cool and interesting, and I would urge the authors to think harder
about what sort of evaluation makes sense here and resubmit to the next suitable machine learning conference.
Decision: Reject