Neurosymbolic Deep Generative Models for Sequence Data with Relational Constraints
Anonymous Author(s)
AffiliationAddress email
Abstract
There has been significant recent progress designing deep generative models thatgenerate realistic sequence data such as text or music.
Nevertheless, it remainsdifficult to incorporate high-level structure to guide the generative process, andmany such models perform well on local coherence, but less so on global coherence.
We propose a novel approach for incorporating global structure in the form ofrelational constraints between different subcomponents of an example (e.g., lines ofa poem or measures of music).
Our generative model has two parts: (i) one model togenerate a realistic set of relational constraints, and (ii) a second model to generaterealistic data satisfying these constraints.
For model (i), we propose a programsynthesis algorithm that infers the relational constraints present in the training data,and then learn a generative model based on the resulting constraint data.
In ourexperiments, we show that our approach significantly improves over state-of-the-artin terms of capturing high-level structure in the data, while performing comparablyor better in terms of low-level structure.
Introduction
There has been tremendous recent progress in designing deep generative models for generatingsequence data such as natural language (Vaswani et al., 2017) or music (Huang et al., 2019).
Theseapproaches leverage the vast quantities of data available in conjunction with unsupervised and selfsupervised learning to learn probabilistic models of the data; then, new examples can be generated bysampling from these models, with the possibility of conditioning on initial elements of the sequence.
A key challenge facing deep generative models is the difficulty incorporating high-level structureinto the generated examples—e g., rhyming and meter across lines of a poem, or repetition acrossmeasures of a piece of music.
Capturing high-level structure is important for improving the qualityof the generated data, especially in low-data regimes where only small numbers of examples areavailable; intuitively, knowledge of the structure compresses the amount of information the generativemodel has to learn.
Furthermore, explicit representations of structure (i.e., symbolically rather than asa vector embedding) has the benefit that users can modify the structure to guide generation.
Recently, Young et al.
(2019) proposed neurosymbolic generative models for incorporating high-levelstructure into image generation, focusing on simple 2D repeating patterns in images of buildingfacades (e.g., repeating windows).
The basic idea is to leverage program synthesis to extract structurefrom data—in particular, given an example image x , they devise an algorithm A that extracts aprogram c
= A ( x ) that represents the set of 2D repeating patterns present in training examples x .
Then, using the pairs ( x, c ) , they train two generative models: (i) a model p ϕ
( c ) that generates aprogram, and (ii) a model p θ ( x | c ) that generates an image that contains the structure represented byc .
However, their approach is heavily tailored to images in two ways.
First, their representation ofstructure is geared towards simple patterns occurring in images of building facades.
Second, their algorithm A is specifically designed to extract this kind of program from an image, as are theirmodels p ϕ
( c ) for generating programs c and p θ ( x | c ) for generating images x conditioned on c .
We represent the relational constraints c x present in an example x by relating each subcomponent wof a given example x with a prototype ˜ w , which can be thought of as the “original" subcomponentfrom which w is constructed.
In particular, the relationship between w and ˜ w is labeled with a setof relations R , which encodes the constraint that w and ˜ w should satisfy relation r for each r ∈ R .
Importantly, while each subcomponent is associated with a single prototype, each prototype may beassociated with multiple subcomponents.
As a consequence, different subcomponents associatedwith the same prototype are related in some way.
This representation is compact, only requiringlinearly many constraints in the number of subcomponents in x (assuming the number of prototypesis constant).
Compactness ensures the representation both generalizes well and is easy to generate.
Then, we design a synthesis algorithm that can extract an optimal representation of the structurepresent in a training example x (i.e., the relational constraints c x ).
We show how to express thesynthesis problem as a constrained combinatorial optimization problem, which we solve using an
SMT solver Z3 (De Moura & Bjørner, 2008).
Next, we represent relational constraints c as sequences,and design the model p ϕ
( c ) to be a specialized sequence VAE.
Finally, we propose three possibledesigns of p θ ( x | c ) , all of which try to identify an example x that is realistic (e.g., according to apretrained model p θ ( x ) ) while simultaneously satisfies the given constraints c .
We evaluate our approach on two tasks: poetry generation, where the relational constraints includerhyming lines or lines with shared meter, and music generation, where the relational constraintsinclude equality in terms of pitch or rhythm, that one measure is a transposition of another (i.e.,pitches shifted up or down by a constant amount), etc.
We show that our approaches outperformor perform similarly to state-of-the-art models in terms of low-level structure, while significantlyoutperforming them in terms of high-level structure.
We also perform a user study in the poetrydomain to determine user-perceived quality of the generated poetry along three dimensions (structure,lyricism, and coherence), and found that on average, our approach outperformed state-of-the-artbaselines including GPT-2.
Finally, we demonstrate how our approach allows users to guide thegeneration process without sacrificing overall realism by specifying values of constraints.
Example.
Figure 1 illustrates how our approach is applied to generate poetry.
During training, ourapproach uses program synthesis to infer relational constraints c x present in the examples x , and usesboth x and c x to train the generative models.
Here, c x is a bipartite graph, where the LHS verticesare prototypes , and the RHS vertices correspond to lines of x .
Each vertex on the right is connectedto exactly one prototype, and is labeled with constraints on how it should relate to its prototype.
Togenerate new examples, it first samples relational constraints c , and then samples an example x thatsatisfies c —i.e., we need to choose a line to fill each RHS node in a way that the line satisfies therelations with its prototype.
Furthermore, a user can modify the sampled constraint c to guide thegenerative process.
Thus, our approach enables users to flexibly incorporate domain knowledgeon the high-level structure of the data into the generative process, both in terms of the relationalconstraints included and by allowing them to modify the generated relational constraints.
Related work.
There has been recent work using program synthesis to improve machine learning.
For instance, it has been applied to unsupervised learning of latent structure in drawings (Elliset al., 2015) and to reinforcement learning (Verma et al., 2018).
These techniques have benefitssuch as improving interpretability (Verma et al., 2018;
Ellis et al., 2020), enabling learning fromfewer examples (Ellis et al., 2015), generalizing more robustly (Inala et al., 2019), and being easierto formally verify (Bastani et al., 2018).
More recently, there has been work leveraging programsynthesis in conjunction with deep learning, where the DNN handles perception and program synthesishandles high-level structure (Ellis et al., 2017), including work in the lifelong learning setting (Valkovet al., 2018).
In contrast to these approaches, our focus is on generative models.
In particular, weextend recent work leveraging these ideas for image generation to incorporating high-level relationalstructure into sequence generation tasks (Young et al., 2019).
Finally, much research over the pastfew decades has focused on music and poetry generation, and on using relational constraints in neuralmodels; we include a discussion of the most relevant such research in Appendix A.
[Figure]
[Figure] [Figure]
Background on Neurosymbolic Generative Models
Consider the problem of learning a generative model given training data from the underlying distribution.
Given training examples x 1 , ..., x k ∼ p ∗ , our goal is to learn a generative model p θ ≈ p ∗from which we can draw additional samples x ∼ p θ .
We consider sequence data—i.e., an examplex ∈
X is a sequence x = ( w 1 , ..., w m ) ∈ W m .
For example, each subcomponent w may be a lineof a poem or a measure of music, and x may be a poem or song.
We are interested in domains where likely examples satisfy latent relational constraints c ∈ C overthe subcomponents.
For instance, c may say that two measures w i and w j of x start with the sameseries of pitches, or that two lines w i and w j of x rhyme.
We assume given a set of relations R (e.g.,r ∈ R might be “rhyme” or “equal”), and a function f : W × W × R → [0 , 1] such that f ( w, w ′ , r )indicates to what extent w and w ′ satisfy relation r .
Then, c is a compact representation of therelations present in an input x .
We describe the structure of c in detail in Section 3.1; for now, theapproach we describe works for any choice of c .
In particular, we build on neurosymbolic generativemodels (Young et al., 2019), where c is itself generated based on a latent value z ∈ Z —i.e.,p θ,ϕ ( x ) =
(cid:90) (cid:88)
[Equation]
Then, Young et al.
(2019) considers the variational distribution
[Equation]
and
[Equation]
Here, δ is the Dirac delta function and c x is a single representative generated from x using a programsynthesis algorithm (David & Kroening, 2017)—i.e., an algorithm A that takes as input an examplex and outputs a program c
=
A ( x ) encoding the relational constraints present in x .
Next, Younget al.
(2019) derive an evidence lower bound
[Equation]
where D KL is the KL divergence and H is information entropy.
The first term is the log-likelihoodof a generative model predicting the probability of example x given relational constraints c x , andthe second and third terms form the loss of a variational autoencoder (VAE)
p ϕ
( c | z ) and q ˜ ϕ
( z |c ) (Kingma & Welling, 2019).
In summary, given training examples x ∈ X , this approach separatelylearns (i) a VAE to generate c , trained on the relational constraints c x synthesized from each trainingexample x , and (ii) a model to generate x given c x ; the latter can take multiple forms such as a second
VAE or a generative adversarial network (GAN) (Goodfellow et al., 2014).
This approach is calledsynthesis-guided generative models (SGM) since it uses program synthesis to guide training.
To leverage this framework, we have to instantiate (i) the space of relational constraints C , (ii)
thesynthesis algorithm A :
X → C used to extract a program encoding the structure of x , and (iii) thearchitectures of p ϕ ( c | z ) , q ˜ ϕ ( z | c ) , and p θ ( x | c ) .
In prior work, Young et al.
(2019) used heuristicsspecific to the the image domain to achieve these goals—in particular, they used (i) simple equalityconstraints on sub-regions of the image designed to capture 2D repeating patterns, (ii) a customsynthesis algorithm that greedily adds constraints in the data to the program, and (iii) a representationof c x as an image, in which case p θ is a generative model over images, and p ϕ , q ˜ ϕ
based on anencoding of c as a fixed-length vector.
We design a synthesis algorithm that expresses the synthesis problem as a constrained combinatorialoptimization problem, which it solves using an SMT solver called Z3 (De Moura & Bjørner, 2008).
In terms of (iii), our programs encode declarative constraints rather than imperative renderings, sothe previous architectures of p ϕ , and q ˜ ϕ cannot be used.
Instead, we use expert domain-specificheuristics, transformers (Vaswani et al., 2017), or graph neural networks (GNNs) (Kipf & Welling,2017) for p ϕ and q ˜ ϕ .
For p θ , we propose several methods for imposing the constraints encoded by cwhen generating an example x .
Relational Constraints for Sequence Data
We describe how we represent relational constraints r , as well as our algorithm A for synthesizingthe relational constraints
c x
= A ( x ) present in an example sequence x .
3.1 Graph Representation of Relational Constraints
Recall that our generative model operates by first generating a relational program c , and thengenerating an example x that satisfies c .
Thus, for each training example x , we need to design arelational program c that encode constraints on the structure of x .
A program c encodes a set ofrelational constraints , each of which imposes a constraint that subcomponents of x should havecertain kinds of relations.
We begin by describing the structure of a single relational constraint, andthen describe how c encodes a set of relational constraints.
A relational constraint ϕ ∈ Φ =
W × I × R , where I = { 1 , ..., m } , is a tuple ϕ
= ( ˜ w, i, r ) ; wecall ˜ w ∈ W a prototype subcomponent .
An example x satisfies ϕ to extent h (denoted x | = h ϕ ) iff ( ˜ w, w i , r ) = h , where w i is the i th subcomponent of x .
That is, ϕ says the i th subcomponent w iof x should have relation r with prototype subcomponent ˜ w .
Thus, we can interpret ϕ as a functionϕ : X → [0 , 1] , where ϕ ( x ) =
if x satisfies ϕ to the maximal extent and ϕ ( x ) =
if x does notsatisfy ϕ at all.
Next, a relational program c x encodes a set of relational constraints on examples x .
We represent c x asan undirected labeled bipartite graph c
=
( ˜ V , V, E ) with vertices ˜ V and V and edges E ⊆
˜ V
× V × 2 R ,where R is the set of relations and 2 R is the power set of R .
The vertices ˜ w ∈ ˜
V are prototypesubcomponents ˜ w ∈ W ; equivalently, they may be vector embeddings of prototype subcomponents.
The vertices i ∈ V
= { 1 , ..., m } are the indices of subcomponents in x .
The edges e ∈ E are tuplese =
( ˜ w, i, R ) , where R = [0 , 1] |R| .
For tractability of synthesis, we impose the constraint that eachv ∈ V is part of a single edge ( ˜ w, v, R ) (though ˜ v ∈
˜ V may be part of multiple edges).
Finally, cencodes the set of relational constraints
[Equation]
In other words, c includes the relational constraint that each subcomponent w i of x should have allrelations r ∈ R with prototype ˜ w to extent h , where v is connected to ˜ w .
In this paper, for most examples, we consider binary relationships that have 0 or 1 as values, andinformally state that a pair ˜ w, i does not have a relationship r if f ( ˜ w, i, r ) = 0.
However, as we showin our experiments, non-boolean functions with values between 0 and 1 can be used as well.
For example, in Figure 1, the graph shown on the top right encodes a relational constraint c x , and thetop right shows an example x that satisfies all the constraints
ϕ ∈
Φ c x with a value greater than 0.
The nodes on the left-hand side of c x are prototype subcomponents ˜ w ∈ W , each of which is a lineof poetry.
The nodes on the right-hand side correspond to indices i (from i = 1 on top to i = m =
10on the bottom)
; each one is labeled with a set of relations R i .
Then, Φ c x contains constraintsϕ =
( ˜ w, i, R i ) for each edge ˜
w → i in the graph, which says that line i of x should have relationsr ∈ R i with ˜ w .
For instance, the last (10th) node in c x has constraints R 10 = { rhyme , meter } , andis connected to prototype line ˜ w =
“The harp that thou...”.
Thus, this edge encodes a constraintϕ =
( ˜ w, 10 , R 10 ) saying that the last line of x should rhyme and have the same meter as ˜ w .
Indeed,the last line of x is w 10 =
“Who serve thee most...”, which satisfies this constraint.
Remark 3.1.
We use prototypes rather than direct relationships between components to ensure thesize of the graph is tractable—with this choice, the graph is linear in the size of the input (assuming thenumber of prototypes is constant) rather than quadratic.
A compact graph is both each to synthesize(for training) and train a model to generate (for generation).
In our experiments, we show that ourapproach significantly outperforms attempting to generate full graphs (i.e., adjacency tensors).
Remark 3.2.
We refer to c as a program since it can be interpreted as a Datalog program (Ceri et al.,1989)
(i.e., a relational logic program); in particular, Φ c is a set of Datalog relations over x ∈ X .
3.2 Synthesizing Relational Constraints
Recall that when training our generative model, we need to design a program synthesis algorithm Athat synthesizes a relational program c x
= A ( x ) that best encodes the latent relational constraintspresent in each training example x .
A key question is where the prototypes come from.
We simplychoose the prototypes ˜ w to be actual subcomponents in x .
Thus, c x encodes that subcomponents of xare each related to one of a small number of distinguished subcomponents of x .
As described below,we formulate the problem of synthesizing c x as a constrained optimization problem.
Optimization variables.
The variables are a binary vector H ∈ B m and a binary matrix K ∈ B m × m .
Intuitively, H i indicates whether subcomponent w i of x is a prototype subcomponent in c , and K ijindicates whether w i is the prototype for subcomponent w j .
Constraints.
Our optimization problem has the following three constraints:
[Equation]
First, ψ 1 says that the number of prototype subcomponents is between k min and k max .
Next, ψ 2 saysthat every subcomponent w j corresponds to exactly one prototype subcomponent w i .
Finally, ψ 3says that for every i , if w i is the prototype subcomponent of w j according to K , then it must be aprototype subcomponent according to H as well.
Objective.
The objective of our optimization problem is expressed in terms of a precomputed distancematrix D ∈ R m × m , where D ij measures the similarity between components w i and w j ; smallervalues indicate a greater degree of similarity.
In particular, we define
[Equation]
i.e., D ij is the extent to which each r ∈ R are not satisfied by w i and w j .
Then, our objective (whichis to be minimized) has the following three terms:
[Equation]
First, J 1 says that subcomponents should be similar to their prototypes.
Second, J 2 says thatsubcomponents should also be similar to other subcomponents that share the same prototype.
Third,
J 3 says that different prototype subcomponents should be dissimilar.
Optimization problem.
Our algorithm A uses Z3 to solve the optimization problem
[Equation]
to
[Equation]
where λ 1 , λ 2 , λ 3 ∈ R ≥ 0 are hyperparameters.
Finally, to construct c x , A chooses˜
[Equation] [Equation]
where R ij = { r ∈ R | f ( w i , w j , r ) = 1 } —i.e., ˜ V are the prototype subcomponents according to
H ∗ , E are the edges according to K ∗ , and R ij are the extent to which relations are satisfied by w iand w j . Z3 is guaranteed to find the optimal solution; in the event that multiple such solutions exist,it chooses one nondeterministically.
Intuitively, our approach should perform well when a handful ofprototypes are sufficient to approximately capture the relational structure in the data, which appearsto be true in the domains of rhyming poetry and melodies.
Also, the user can define relations in away that captures desired structure for any domain.
Relational Constraints in Neurosymbolic Generative Models
We describe our model for generating examples x .
Recall that our approach proceeds in two steps: (i)generate c , and (ii) generate x given Φ c .
We describe each step in detail below.
4.1 Step 1: Generating Relational Constraints
The first step of our generative model is to generate relational constraints Φ c using a VAE—i.e.,p ϕ ( c | z ) is a VAE with p ( z ) =
N ( z ; 0 , I ) being a Gaussian distribution.
The main choice isthe architecture to use for the VAE.
In particular, we consider a representation of c as a sequence( s 1 , ..., s m ) , where s i ∈ { 0 , 1 , ..., k } for each i ; intuitively, s i encodes that subcomponent w i shouldhave the same prototype subcomponent as w i − s i , or if s i ≤ 0 , that w i corresponds to a new prototypesubcomponent.
In practice, we found that in the music domain, the vast majority of examples couldbe described using k ≤ 6 , which decreased the number of possible values that could be predicted andsimplified the problem; however, it would be possible in other domains for k to be as large as m − 1 .
More precisely, we initialize Φ c = ∅ .
Then, we generate the sequence s i ∈ { 0 , 1 , ..., k } andr i ∈ { 0 , 1 , ..., m } (where r i is represented as a binary vector of length n = |R| ) using either: (i) an
LSTM-VAE, or (ii) a feedforward network whose output is iteratively sampled from as a categoricaldistribution and then used as input in the next step (see Appendix C.1 for details).
For each i , wegenerate ( ˜ w, R i ) based on s i and r i according to the following approach: If s i = 0 , we generate anew prototype subcomponent ˜
w using a domain-specific generative model, and add ϕ i =
( ˜ w i , i, R i )to Φ c .
If s i > 0 , we let ϕ i
=
( ˜ w i − s i , i, R i ) .
4.2 Step 2: Generating Examples Given Relational Constraints
Next, we describe how we implement the second step p θ ( x | c ) of our generative model.
We proposethree approaches for generating x given Φ c ; we give details in Appendix B.
Approach 1: Constrained sampling.
We sample values x ∼ p θ ( · ) by sequentially sampling w i ∼p θ ( · ) from a pretrained generative model p θ ( w ) .
We do so using rejection sampling at each step—i.e.,we sample w i ∼ p θ ( · ) until we find w i satisfying f ( ˜ w, w i , r )
≈ h for each ( ˜ w, i,
r, h ) ∈
Φ c .
Inaddition, to speed up sampling, at each step of sampling w i (e.g., a word in a line or a pitch in ameasure), we eliminate choices that violate Φ c .
Approach 2: Constraint-aware embeddings.
We train a conditional variational autoencoder (cVAE)p θ ( w 1 , ..., w m | c ) in the form of a graph convolutional network (GCN) that simultaneously generatesall m subcomponents in a way that satisfies c , and sample x = ( w 1 , ..., w m ) ∼ p θ ( · | c ) .
The GCNtakes as input embeddings of each prototype and subcomponent of x , and the adjacency matrix isgiven by the edges in c (where the relation is encoded as an edge attribute).
Then, the GCN-cVAEis trained using the standard VAE objective (Kingma & Welling, 2019), along with a semanticconsistency loss that encourages the generated examples to satisfy c .
Approach 3: Combinatorial optimization.
We sample x ∼ p θ ( · ) by sequentially generating w iby solving an optimization problem whose objective is to maximize adherence to Φ c plus additionalterms encoding domain-specific heuristics encouraging w i to be realistic.
[Table]
Experiments
We evaluate our approach on two domains: music and poetry generation.
We provide details onexperimental design and additional results in Appendix C.
5.1 Music Generation
We evaluated our approach on a music generation, where x is a song and w are measures of music.
We consider 20 relations including equality, same rhythm, etc.; see Appendix C.2.
Dataset.
We used songs from the Essen folk song corpus (Schaffrath, 1995), using 2223 fortraining and 555 for testing (after removing examples with less than 16 measures or that werenot in the standard 4/4 meter).
For this dataset, we used each of the three approaches A1, A2,and A3 described in Section 4 to sample x ∼ p θ ( · | c ) .
For A1, we use a pretrained transformercalled MusicAutoBot (Shaw, 2020).
For A2, we require a generative model that constructs vectorembeddings of measures; we use the pretrained version of Magenta’s MusicVAE which embedspairs of measures (Roberts et al., 2018) and adapted it to produce single-measure embeddings.
Wefinetune all models on our training examples.
Baselines.
We compare to MusicAutoBot, a pretrained and finetuned attention LSTM (AttentionRNN) (Waite, 2016), Magenta’s 16-bar MusicVAE (pretrained and finetuned), and StructureNet,which integrates structure into an LSTM (Medeot et al., 2018).
To show the importance of synthesis,we compare to an ablation that uses A2 but with full adjacency tensors instead of synthesizingcompact representations of relational constraints, and one that uses a greedy synthesizer—i.e., ateach step, greedily choose the single prototype and its relations that most increases (1).
Finally, weconsider using a continuous relation, namely, the cosine similarity of the MagentaVAE embeddings.
Metrics.
We compare performance in terms of both high-level and low-level structure.
For lowlevel structure, we use the negative log likelihood (NLL) on a held-out test set for MusicVAE,
MusicAutobot, and our approach with strategy A2.
The remaining approaches are not probabilistic(or estimating probabilities is intractable).
For these approaches, we use a variant of the standard
Fréchet distance (FD) score used to evaluate GANs (Borji, 2019)—i.e.
, the Fréchet distance betweenthe MusicVAE (16-bar) embeddings of the generated music and the held-out test set.
For high-level structure, given a generated (or human) example x , we use our synthesis algorithm tosynthesize its relational constraints
c x = A ( x ) .
Then, given a collection C gen = { c x | x ∈ X gen } ofsynthesized structure for generated examples, along with a collection C human = { c x | x ∈ X human } ofsynthesized structure for the held-out human examples, we train a graph convolutional neural network(GCN) to try and discriminate C gen from C human , as well as a random forest (RF) over handcraftedfeatures (see Appendix C.4).
Intuitively, higher discriminative power should indicate less realisticstructure.
In both cases, we use a balanced dataset (i.e., 50% human held-out and 50% generated) so
[Table]
based random predictions have accuracy 0 .
Recent work has shown that such discriminator-based metricsare valid for evaluating quality of generated examples (Lopez-Paz & Oquab, 2016).
Results.
In Table 1, we show results for models for which we can compute the test set NLL (left)and results for the remaining models (right).
As can be seen, our approach (SGM) with samplingstrategy A2 outperforms all other models in both tables, in terms of both high-level structure andlow-level structure.
In Table 1 (left), the closest alternative is MusicVAE, for which the NLL is nottoo much larger; however, it performs significantly worse than our approach in terms of high-levelstructure.
In Table 1 (right), we find that our other approaches also perform well (though not as wellas A2).
In particular, A1 performs well in terms of low-level structure, but is more mixed in termsof high-level structure.
In contrast, A3 performs well in terms of high-level structure, but is mixedin terms of low-level structure, most likely since it does not use a learning-based model to generatelow-level structure.
Our ablation where we perform no synthesis performs poorly, especially in termsof structure, as does the one using greedy synthesis, demonstrating the importance of using constraintsolving to synthesize compact representations of structure.
On the other hand, greedy synthesis canbe significantly more scalable than constrained optimization for large examples; thus, improving thisstrategy is an interesting direction for future work.
Finally, using a continuous relation performscompetitively in terms of FD score (though interestingly, it performs worse in terms of high-levelstructure), demonstrating that our approach can be applied with continuous relations.
5.2 Poetry Generation
Next, we apply our approach (SGM) to poetry generation; in this case, x is a poem, and w is a line.
We consider two relations, rhyming and equal meter; see Appendix C.3 for details.
Dataset.
We use from Project Gutenberg’s poetry collection (Parrish, 2018), focusing on 10-linepoems with rhymes and meter, with 2700 for training and 300 for testing.
Our approach.
In the rhyming domain, due to the lack of rhyme-aware line embeddings, we did notperform A2.
In applying A1, rather than sample words going forward, we sample them backwards,making it easier to sample lines that satisfy rhyming constraints; see
Appendix B. Thus, we use BERTto sample (Devlin et al., 2018), since it is bidirectional.
We apply A3 by performing constrainedoptimization to satisfy as many relations as possible while maintaining a low NLL.
Baselines.
We compare to generation using beam search for BERT and GPT2 (Radford et al., 2019;
Vaswani et al., 2017), both finetuned on our dataset.
We also consider a variant GPT2-Opt of GPT2where we use beam search to choose line breaks in a way that maximizes occurrences of rhyme andmeter.
We also tried a variant of GPT2 that used constrained sampling to try and find poems that fit agiven rhyme and meter, but the search space was too large and it failed to generate a single poemeven after several hours.
We also compare to an implementation of RichLyrics (Castro & Attarian,2018), where the consecutive parts of speech for each line given the previous line and the ability tofill in the correct word for the given part of speech were both learned separately from the corpus.
Finally, to show the importance of learning the distribution over constraints, we consider an ablationthat uses A1, but sampling Φ c uniformly randomly rather than from a learned distribution.
[Table]
Metrics.
For low-level structure, we use FD score on SentenceBert embeddings, which are unawareof rhyme and meter (Reimers & Gurevych, 2019); we cannot evaluate log-likelihood since we areusing constrained sampling.
For high-level structure, we train a GCN to discriminate synthesizedprograms for generated examples vs, test examples.
Results.
We show results in Table 2.
Our approach (SGM) with sampling strategy A3 significantlyoutperforms all baselines in terms of high-level programmatic structure, while also outperformingthem in terms of FD scores.
Approach A1 performs even better in terms of programmatic structure,but is not competitive with respect to FD scores, presumably due to the fact that GPT-2 is significantlybetter at natural language generation than BERT.
User study.
We also performed a user study, discussed in Appendix C.5, which further confirmedthis methods’ strength in the poetry domain.
in this domain, with 50 participants.
User modifications.
A key benefit of our approach is that the user can modify the relationalconstraints c (or construct their own from scratch) for use in the second step p θ ( x | c ) , giving theuser a way to guide the generative process.
An example in the poetry domain is shown in Figure 5.2,and musical examples are shown in Appendix 3.
Conclusion
We have presented a novel approach for representing and synthesizing relational constraints onsequence data, and for generating examples whose relational structure resembles that of the trainingdata.
Our experiments demonstrate that we outperform existing approaches in terms of achievinghuman-like structure, while performing comparably or better on both a user study and widely-usedquantitative metrics that do not explicitly account for structure.
Finally, our approach enables users toguide the generative process by modifying constraints.
A key direction for future work is to applyour approach to other applications such as dialog generation and summarization, which may requirenovel programmatic structure compared to the ones we study.
Limitations & ethical considerations.
We discuss limitations and ethical concerns in Appendix F.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[No]
• Did you include the license to the code and datasets?
[No] • Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] In theappendix(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes](b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[No](c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[No]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes](b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[Yes]
(b) Did you describe any potential participant risks, with links to Institutional ReviewBoard (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[Yes]
A Additional Related Work
Music and poetry generation.
Both early music generation and poetry generation approaches wererule-based (Ovans & Davison, 1992; Atanassova & Pulov, 2002) or used simple statistical modelssuch as Markov models (Sandred et al., 2009; Cope, 1987;
Suleiman et al., 2017) or probabilistic
CFGs (Quick, 2016; Thompson, 2009).
Recent work has used deep learning to generate music (Huanget al., 2019; OpenAI, 2019) and poetry (Liao et al., 2019a); our experiments show that theseapproaches have difficulty generating realistic high-level structure.
Music generation has been approached using many machine learning techniques, including convolutional neural networks (CNN’s)
(Yang et al., 2017a), graph convolutional networks (GCN’s) (Jeonget al., 2019), recurrent neural networks (RNN’s)
(Sübakan & Smaragdis, 2017)), and transformers (Huang et al., 2019).
In particular, we leverage attention-based RNN’s (Keerti et al., 2020) andtransformers for constrained sampling of program structure.
Poetry has been approached with a variety of techniques as well, including relying on finetunedtransformers (an approach we extend)
(Liao et al., 2019b), RNN-based conditional-VAEs (Yanget al., 2017b), RNN-based planning (Wang et al., 2016), and GANs with both transformer and LSTMbackends (Saeed et al., 2019).
Approaches have incorporated structure into deep learning to generate music (Medeot et al.
, 2018)or poetry (Castro & Attarian, 2018), but they are domain specific; we find they do not perform ata human level on capturing global (and sometimes local) structure.
Some approaches incorporateexpert-provided constraints such as rhyme and meter to generate poetry (Lau et al., 2018); unlike ourapproach, they cannot automatically learn and generate these constraints from data.
Constrained text generation.
There has been work on constraining language models to produceoutputs that satisfy a given decision function, or that maximize a given scoring function (Li & Rush,2020; Miao et al., 2018; Dathathri et al., 2019).
In contrast, our work focuses on settings whereconstraints are generated, and furthermore the distribution over constraints must itself be learned.
Relational constraints and neural models.
Several previous works have focused on learning datalogprograms, one component of our project.
Mei et al.
(2020b) learn Datalog programs which representcertain point processes, and use this to predict future events.
Similarly, Mei et al.
(2020a) also learn
Datalog programs, and is able to achieve great success in certain autoregressive domains; however,they do not frame their problem as a generative process to generate realistic human data from theiroutput.
B Generating Examples Given Relational ConstraintsB.1 Approach 1: Constrained Sampling
In the music domain, we choose the pretrained generative model p θ ( w ) to be a pretrained versionof MusicAutoBot.
To generate x , we sequentially sample each measure w i conditioned on all priormeasures w 1 , ..., w i − 1 .
Each measure is sampled by sequentially sampling a sequence of pitchduration pairs until the total duration is 16 beats (i.e., the length of a measure).
During sampling, wemask pitch-duration pairs that cannot satisfy Φ c (i.e., we set their sampling probability to zero andrescale the remaining probabilities).
For instance, if the “has similar interval” relation is supposedto hold between the the prototype measure and measure i , and we are sampling the second note ofmeasure
i , then we mask any pitch k in measure i such that
[Equation]
where (cid:103) pitch k is pitch k in the prototype corresponding to w i .
In other words, we eliminate pitchesthat would cause sampling to violate this constraint.
In the the poetry domain, we finetune a pretrained BERT model on our dataset, by taking thepretrained models weights and then training the model on our dataset with a strong gradient weightdecay.
BERT has the ability to complete masked words in a sentence.
We leverage this ability tosample lines that rhyme and have the same meter, which is a challenging task since such lines area tiny fraction of the search space.
We describe how we simultaneously handle rhyming and equalmeter; the cases where only one of these two constraints has to hold are similar.
Given a prototype ˜ w , we work backwards—on each step j , we sample from BERT a word word k that has the same numberof syllables as the corresponding word (cid:93) word k in the prototype. More precisely, we feed BERT thesequence
[Equation] and ask it to fill in the masked word, setting the probability of any word with different number ofsyllables as (cid:93) word k to zero. Here, word k is sampled according to the probabilities output by BERTfor filling in the mask in the sequence with a single word. In other words, we can sample from BERTsince we are always feeding it a complete sentence with a single masked word to be re-sampled.
To avoid producing a line which is similar to the original line, we also set the probability of any wordequal or too similar to the original word in terms of GloVe cosine similarity (Pennington et al., 2014)to zero, except for in the case of the last word, where we instead restrict to words that rhyme with(cid:93) word k .
To increase diversity, we sample the remaining words twice—(i) backwards-to-forwards fromword k − 1 to word 1 , where k is the number of words, and (ii) we resample each of the k − 1 words(i.e. , except the last word) in a random order.
We discard any lines which, according to BERT, afterbeing sampled are determined to be too unlikely when preceded by the previously generated lines.
B.2 Approach 2: Constraint-Aware Embeddings
This approach uses a graph convolutional network (GCN) conditional variational autoencoder (cVAE),or GCN-cVAE, which consists of a GCN encoder q θ ( z ′ | x, c x ) and a GCN decoder p θ ( x | z ′ , c x ) .
In more detail, we assume x = ( w 1 , ..., w m ) is represented as a sequence of vectors ( u 1 , ..., u m ) —e.g., in the music domain, we use MusicVAE p ψ to encode u ∼ q ψ ( · | w ) or decode w ∼ p ψ ( · | u ) .
Then, the latent encoding z ′ consists of an embedding vector for each subcomponent u of x .
Next, c x is incorporated into each GCN by converting it into a tensor with dimensions | ˆ w |×| ˆ w |×| R |used as the adjacency matrix of that GCN (the last dimension is the edge attribute).
Intuitively, theedges in x are relations in c between subcomponents and prototypes.
This GCN-cVAE it is trained using the usual VAE objective (Kingma & Welling, 2019): (i) a KLdivergence term encouraging the embeddings z ′ to be Gaussian, and (ii) a reconstruction loss interms of mean-squared error.
We also include a semantic consistency loss to enforce the satisfactionof the constraints Φ c .
In particular, we train a classifier p
α ( u, u ′ ; r ) that predicts whether twosubcomponents u, u ′ satisfy relation r (more precisely, when decoded by p ψ ).
The model p α istrained examples ( u, u ′ , r ) from the training data x .
Then, we include the loss(cid:88)
[Equation]
where ˜ u ∼ p ψ ( · | ˜ w ) is the encoding of ˜ w , u i ∼ p ψ ( · | w i ) is the encoding of w i , and r is a relation.
For the music domain, we use a pretrained MusicVAE for p ψ and q ψ ; unlike the MusicVAE we usefor evaluation, we finetune a model that decodes 1 measure of music from a 256-dimensional vector.
B.3 Approach 3: Combinatorial Optimization
Given sampled program c , this approach attempts to generate values x = ( w 0 , . . . ,
w m ) such thatx | =
Φ c by solving a system of constraints.
However, when generated using a neural network,relational constraints ϕ ∈
Φ c are not always consistent with one another, so we convert the constraintx | =
Φ c into an objective—i.e.,
[Equation]
The ability to encode this optimization problem as one that Z3 can solve depends on the domain andrelations.
For this approach to work, we may need to include additional, handcrafted terms in theobjective that encourage the generated example x is realistic.
For the music domain, the optimization variables are the optimal sequence of pitches and theirdurations.
The objective function is a linear combination of the degree to which x satisfies c , alongwith domain-specific heuristics—e.g., minimizing large jumps in pitch values (i.e., | pitch k +1 −pitch k | ≥ 4 ), not having any intervals of length 6 (i.e., | pitch k +1 − pitch k | = 6 ) due to the unpleasantharmonic nature of that interval, and not having two consecutive jumps in pitch (i.e., | pitch k +2 −pitch
k +1 | ≥ 5)
∧ | pitch k +1 − pitch k | ≥ 5 ).
These heuristics are based on standard concepts frommusic theory (Horton & Ritchey, 2000).
For the language domain, we use GPT-2 to sample a line except for the last word; then, the optimization variables are the last words in each line.
This strategy optimizes the relations between each lineand its prototype, while leveraging GPT-2 to maintain low NLL for the entire poem.
C Evaluation Details
C.1 Experimental Setup
Synthesizing programs.
The hyperparameters λ 1 , λ 2 , and λ 3 in the program synthesis task, asdescribed in the main section of this paper, regulate the degree to which the optimization favorssolutions which have high similarity between prototype and sequence measures, have high similaritybetween elements sharing a prototype, and have high difference between prototypes, respectively.
Their values were different with respect to the two different domains.
In the poetry domain, λ 1 = 1 ,λ 2 = 10 , λ 3 = 1 , k min = 1 , and k max = 6 ; in the music domain, λ 1 = 1 , λ = 5 , λ 3 = 1 , k min = 3 ,and k max = 6 .
These values were chosen by manually inspecting the quality of the programs producedfor 10 examples in the training set.
Generating c .
To generate c in the poetry domain, we use an LSTM-VAE with 6 LSTM layers and alatent size of 50.
This model is trained to reproduce a given sequence of ( s i , r i ) pairs which are givenas input, with an additional requirement that the distribution of their encodings should be roughlyequivalent to a Gaussian normal distribution.
In the music domain, while we experimented with usingan LSTM-VAE, empirically we had more success using a feedforward 3-layer network which tookthe previous n (usually n = 6) ( s i , r i ) pairs, and outputted a distribution over the following pair.
Thelearning rate used was 1.0 (arrived at by grid search), and included the KL and reconstruction loss.
Each ( s i , r i ) pair is represented as a ( S + | R | ) -dimensional
vector, where S is the maximum distancebetween objects with the same prototype and R is the set of relations.
High-level structure.
We evaluate high-level structure by using our algorithm to synthesize therelational constraints in every generated example—i.e., C gen = {A ( x ) | x ∈ X gen } , where X gen is theset of examples generated using a model.
Similarly, we can construct C human =
{A ( x ) | x ∈ X human } ,where X human is the set of human-created examples held-out from the training dataset.
Then, weevaluate high-level structure by training a model to try to discriminate C gen from C human ; if the modelachieves lower performance, then the quality of high-level structure is higher.
A general approach isto train a graph neural network—in particular, a graph convolutional network (GCN)—to do so; thismodel takes as input the graph structure of relational constraints c , along with vector embeddings ofthe prototype subcomponents, and outputs whether c ∈ C gen or c ∈ C human .
We balance the data so itconsists of 50% human data and 50% generated data.
We report the cross-entropy (CE) loss; highervalues correspond to better generative models.
In the music domain, we additionally used a randomforest (RF) trained on a manual featurization of c .
We report the accuracy of the RF; lower values(i.e., closer to 50%) correspond to better generative models.
We carefully tuned the hyperparameters of both the GCN and RF discriminators (as well as the
MusicVAE used to evaluate the NLL).
The main relevant parameters for the neural networks aretheir learning rates, which we tune using grid search.
For the random forest, we performed extensivefeature engineering to ensure good performance; the features we constructed are in Appendix C.4.
C.2 Musical Relations Used
The following are the relations r ∈ R used in the music domain:
Measures i and j have the same pitch classes.
Measures i and j have the same pitch class prefix. Measures i and j have the same pitch class suffix.
Measures i and j ’s pitches have an edit distance of 1.
Measures i and j have approximately the same interval structure.
Measures i and j have the same interval prefix. Measures i and j have the same interval suffix.
Measures i and j have the same note (pitch + duration) prefix.9. Measures i and j have the same note (pitch + duration) suffix.10.
Measures i and j have the same rhythm.
Measures i and j ’s rhythm has an edit distance of ≤ 2 .
Either measure i ’s onsets are a subset of measure j ’s onsets, or measure j ’s onsets are a
subset of measure i ’s onsets.
Measures i and j have the same rhythmic and melodic contour.
Measures i and j have the same rhythmic and melodic contour prefix.
Measures i and j have the same rhythmic and melodic contour suffix.
Either the first or second half of measures i and j are identical.
Either both or neither of measures i and j have leaps.
Measures i and j fit within the same diatonic scale.
Either both or neither of measures i and j have syncopation.
Either both or neither of measures i and j have consecutive notes shorter than an eighthnote.21.
(Continuous)
The cosine similarity between the Measure-VAE embeddings of measure iand measure j .
C.3 Poetry Relations Used
The following are the relations r ∈ R used in the poetry domain:
Lines i and j have the same end rhyme.2.
Lines i and j have the same meter.
C.4 Random Forest Features
The following are the manually constructed features used in the random forest discriminator for themusic domain:
Mean number of relations between prototype and sequence measures.2.
Variance of number of relations between prototype and sequence measures.
Variance in histogram of prototype measure mappings.
Longest sequence i . . .
j such that w i . . .
w j all have the same prototype measure.
Number of pairs ( i, j ) such that ˜ w i = ˜
w j and ˜ w i +1 =
˜ w j +1 .6.
Mean distance between two measures with the same prototype.7.
Variance in distance between two measures with the same prototype.
C.5 User Study Details
50 participants completed our study on Mechanical Turk.
Each was paid $5 to complete a surveywith 12 questions (3 poems each from four sources, Ours, GPT2-Finetune, BERT-Finetune, and
RichLyrics).
All poems were chosen automatically by taking the top 3 examples from the generateddatasets according to GPT2-log-likelihood.
The participants were asked to rank the following 3statements from "strongly disagree" to "strongly agree" (1-5) as follows: [Table]
It is obvious that this is a poem
This text is coherent
I notice that this text has rhyme and meter
We show the results in Table 4; note that each score in the table is based on 150 labels.
D Additional Results
D.1 Comparison to Constraint Solving
We also considered a comparison to a constraint-based implementation called Motifate, with explicitattention to development of musical material (Muhammad Faisal, 2017).
This approach was designedwith heuristics for 3-beat measures, while our evaluation models anticipated 4-beat measures, sowe could not obtain FD scores.
Nevertheless, we found that even the structure was insufficient—its
RF discriminator had accuracy 0.91, and its GCN discriminator had cross entropy loss 0.43, both ofwhich are significantly worse than the other approaches.
D.2 Conditioning on User-Provided Structures
Here we show how user modifications can occur in the music and poetry settings.
By explicitlymodifying c , we are able to generate two pieces of poetry or two tunes with similar internal patternsbut with different structural characteristics.
Furthermore, when we use constrained optimization (approach A3) to generate the music as well, wecan hold constant the structure while imposing additional constraints that affect other musical aspectsof the piece, such as harmony; see Figure 5.
E Regarding comparison of melody-based music models topolyphonic/multi-instrument models
Our model produced coherent monophonic (one note at a time) melodies.
In the past few years, largelanguage models accompanied with datasets of over 200,000 pieces of polyphonic music have madegeneration of such music feasible, and the results often appear to be coherent. We focused on themonophonic domain since (i) we could contribute a discriminator for structure in the monophonicdomain which could be extended to the polyphonic domain in future work, and (ii) because in doingso, we focused on one of the most promising areas for neurosymbolic research—i.e., the low-data domain. With only 2700 training examples, one of these large language models (musicautobot) whichhas demonstrated efficacy in the large-data domain still underperformed related to our model. E.1 Algorithm Running Time
Synthesis time.
Our algorithm uses the Z3 solver (De Moura & Bjørner, 2008) for synthesis.
Thissolver uses a worst-case exponential time algorithm to ensure completeness (i.e., given sufficientcompute time, it can always find the optimal answer), which is necessary given that the constraintsolving problem is in general NP complete.
In practice, Z3 is very efficient; in the poetry domain,all examples take less than 120 seconds to synthesize (without any special tuning), and often takemuch less time—e.g., only 3% took more than 10 seconds.
In the music domain, while there werea handful of instances where the model failed to produce the provably optimal solution in the 120seconds allotted, in all cases it produced a solution very close to the optimal one (by comparing itsobjective value to the optimal program obtained by running for longer than 120 seconds).
Overall,the program synthesis component of training took a comparable amount of time to network training.
Our generation algorithms in Section 4.2 are generally very efficient.
The constrained samplingstrategy also worked well in the poetry domain due to the local nature of our constraints (for instance,rhyming effectively constrains only a single word).
In our experiments, to find a valid sample, only6.4 samples were needed on average and fewer than 15 samples for 95% of examples.
Approach 3(combinatorial optimization) runs similarly to the program synthesis algorithm described above.
Forgeneral problems, Approach 2 (constraint-aware embeddings) always works well since it does notrequire any additional overhead during inference.
F Limitations & Ethical Considerations
Limitations.
Our work assumes that the relational primitives are given; the program synthesisalgorithm also introduces additional hyperparameters; we found these parameters easy to tune in ourdomains by manually inspecting the synthesized programs.
However, both of these choices requiredomain expertise; a key direction for future work is automatically discovering these primitives andtuning the hyperparameters.
In addition, our approach requires that this domain knowledge can be expressed in our language.
We note that the predicates are very general, enabling our constraint language to capture complexrelationships between subcomponents.
In addition, while we have used a fixed value for the numberof subcomponents m , we note that this assumption can be relaxed simply by allowing for “dummy”constraints that play the role of padding the example.
Finally, our sampling strategy can introduce additional overhead over existing approaches.
They weretractable in our domains; in general, sampling Approach 2 should always scale well since it does notrequire constraint solving.
We give a detailed discussion in Appendix E.1.
Ethical considerations.
Increasing automation of music can impact the livelihoods of artists.
In addition, we find that occasionally the poetry generation algorithm would produce somethingpotentially offensive.
Also, broadly speaking, our approach inherits the risks associated with largelanguage models, including the ability to generate spam or other harmful content.
In general, we do not believe it introduces any new ones (beyond the risks of improving performanceof these models); in fact, we believe our approach makes it easier for users to construct models thatavoid accidentally generate harmful content by imposing constraints on the generated output.
Inaddition, we plan on monitoring current research into removing toxicity from the output of largelanguage models, and will apply these methods as they develop.
G Qualitative AnalysesG.1 Qualitative Observations on the Music Domain
In addition to quantitative measurements, we evaluated the strengths and weaknesses of our approachusing A2 (which was the best according to quantitative metrics).
According to our observations,the strengths of A2 include clearer phrases with obvious resolutions, likely and plausibly repetitive rhythms, intervals between notes which seemed plausible but not overly repetitive, and less variancein quality.
However, the results were not very rhythmically diverse, and certain idiomatic patternsof resolutions of intervals between notes and at the end of phrases were not followed.
Furthermore,
AttentionRNN does better in terms of creating realistic chord progressions (we did not explicitlyconsider chord progressions in our model; doing so is a promising direction for future work).
Finally,while global structure is much better than the baselines, examples still relatively infrequently had thefull four-bar repetitions characteristic of much folk music.
G.2 Examples from the Music Domain
We show an example of generated songs using our approach with each A1, A2, and A3 in Figure 6,
Figure 7, and Figure 8, respectively, and show an example generated using each of the baselines
MusicVAE16, AttentionRNN, MusicAutoBot, and StructureNet in Figures 9, 10, 11, & 12, respectively.
Qualitatively, the generated music and poetry appears plausible, exhibiting realistic high-levelstructure without sacrificing low-level structure.
G.3 Examples from the Poetry Domain
In Figure G.3, we show an example poem generated using our approach (top) along with one generatedusing GPT2-Opt (bottom).
As can be seen, the GPT2-Opt poem does not capture structure in thesame way human poems do—e.g., adjacent lines are unrelated, lines have very unequal length, andthe only rhymes are the word “the” in the brown lines and the words “to” and “too” in the green lines.
There is even less structure in poems generated using vanilla GPT2.
Thus, GPT2 is completely unableto capture high-level structure in the real poetry provided as training data.
In contrast, our poemcaptures structure very similar to the human poem shown in Figure 1, such as rhyming adjacent lines.
We also give examples of poetry generated using our baselines—in particular, GPT2 finetuned andoptimized for rhyme and meter in Figure 16, BERT finetuned as a language generation model in
Figure 18, RichLyrics, and our ablation (i.e., use BERT in conjunction with a uniformly randomlysampled Φ c ) in Figure 19.
[Figure]