Neurosymbolic Deep Generative Models for Sequence Data with Relational Constraints

1 INTRODUCTION . There has been tremendous recent progress in designing deep generative models for generating sequence data such as natural language ( Vaswani et al. , 2017 ) or music ( Huang et al. , 2019 ) . These approaches leverage the vast quantities of data available in conjunction with unsupervised and selfsupervised learning to learn probabilistic models of the data ; then , new examples can be generated by sampling from these models , with the possibility of conditioning on initial elements of the sequence . A key challenge facing deep generative models is the difficulty incorporating high-level structure into the generated examples—e.g. , rhyming and meter across lines of a poem , or repetition across measures of a piece of music . Capturing high-level structure is important for improving the quality of the generated data , especially in low-data regimes where only small numbers of examples are available ; intuitively , knowledge of the structure compresses the amount of information the generative model has to learn . Furthermore , explicit representations of structure ( i.e. , symbolically rather than as a vector embedding ) has the benefit that users can modify the structure to guide generation . Recently , Young et al . ( 2019 ) proposed neurosymbolic generative models for incorporating high-level structure into image generation , focusing on simple 2D repeating patterns in images of building facades ( e.g. , repeating windows ) . The basic idea is to leverage program synthesis to extract structure from data—in particular , given an example image x , they devise an algorithm A that extracts a program c = A ( x ) that represents the set of 2D repeating patterns present in training examples x . Then , using the pairs ( x , c ) , they train two generative models : ( i ) a model pφ ( c ) that generates a program , and ( ii ) a model pθ ( x | c ) that generates an image that contains the structure represented by c. However , their approach is heavily tailored to images in two ways . First , their representation of structure is geared towards simple patterns occurring in images of building facades . Second , their algorithm A is specifically designed to extract this kind of program from an image , as are their models pφ ( c ) for generating programs c and pθ ( x | c ) for generating images x conditioned on c. We represent the relational constraints cx present in an example x by relating each subcomponent w of a given example x with a prototype w̃ , which can be thought of as the “ original '' subcomponent from which w is constructed . In particular , the relationship between w and w̃ is labeled with a set of relations R , which encodes the constraint that w and w̃ should satisfy relation r for each r ∈ R. Importantly , while each subcomponent is associated with a single prototype , each prototype may be associated with multiple subcomponents . As a consequence , different subcomponents associated with the same prototype are related in some way . This representation is compact , only requiring linearly many constraints in the number of subcomponents in x ( assuming the number of prototypes is constant ) . Compactness ensures the representation both generalizes well and is easy to generate . Then , we design a synthesis algorithm that can extract an optimal representation of the structure present in a training example x ( i.e. , the relational constraints cx ) . We show how to express the synthesis problem as a constrained combinatorial optimization problem , which we solve using an SMT solver Z3 ( De Moura & Bjørner , 2008 ) . Next , we represent relational constraints c as sequences , and design the model pφ ( c ) to be a specialized sequence VAE . Finally , we propose three possible designs of pθ ( x | c ) , all of which try to identify an example x that is realistic ( e.g. , according to a pretrained model pθ ( x ) ) while simultaneously satisfies the given constraints c. We evaluate our approach on two tasks : poetry generation , where the relational constraints include rhyming lines or lines with shared meter , and music generation , where the relational constraints include equality in terms of pitch or rhythm , that one measure is a transposition of another ( i.e. , pitches shifted up or down by a constant amount ) , etc . We show that our approaches outperform or perform similarly to state-of-the-art models in terms of low-level structure , while significantly outperforming them in terms of high-level structure . We also perform a user study in the poetry domain to determine user-perceived quality of the generated poetry along three dimensions ( structure , lyricism , and coherence ) , and found that on average , our approach outperformed state-of-the-art baselines including GPT-2 . Finally , we demonstrate how our approach allows users to guide the generation process without sacrificing overall realism by specifying values of constraints . Example . Figure 1 illustrates how our approach is applied to generate poetry . During training , our approach uses program synthesis to infer relational constraints cx present in the examples x , and uses both x and cx to train the generative models . Here , cx is a bipartite graph , where the LHS vertices are prototypes , and the RHS vertices correspond to lines of x . Each vertex on the right is connected to exactly one prototype , and is labeled with constraints on how it should relate to its prototype . To generate new examples , it first samples relational constraints c , and then samples an example x that satisfies c—i.e. , we need to choose a line to fill each RHS node in a way that the line satisfies the relations with its prototype . Furthermore , a user can modify the sampled constraint c to guide the generative process . Thus , our approach enables users to flexibly incorporate domain knowledge on the high-level structure of the data into the generative process , both in terms of the relational constraints included and by allowing them to modify the generated relational constraints . Related work . There has been recent work using program synthesis to improve machine learning . For instance , it has been applied to unsupervised learning of latent structure in drawings ( Ellis et al. , 2015 ) and to reinforcement learning ( Verma et al. , 2018 ) . These techniques have benefits such as improving interpretability ( Verma et al. , 2018 ; Ellis et al. , 2020 ) , enabling learning from fewer examples ( Ellis et al. , 2015 ) , generalizing more robustly ( Inala et al. , 2019 ) , and being easier to formally verify ( Bastani et al. , 2018 ) . More recently , there has been work leveraging program synthesis in conjunction with deep learning , where the DNN handles perception and program synthesis handles high-level structure ( Ellis et al. , 2017 ) , including work in the lifelong learning setting ( Valkov et al. , 2018 ) . In contrast to these approaches , our focus is on generative models . In particular , we extend recent work leveraging these ideas for image generation to incorporating high-level relational structure into sequence generation tasks ( Young et al. , 2019 ) . Finally , much research over the past few decades has focused on music and poetry generation , and on using relational constraints in neural models ; we include a discussion of the most relevant such research in Appendix A . 2 BACKGROUND ON NEUROSYMBOLIC GENERATIVE MODELS . Consider the problem of learning a generative model given training data from the underlying distribution . Given training examples x1 , ... , xk ∼ p∗ , our goal is to learn a generative model pθ ≈ p∗ from which we can draw additional samples x ∼ pθ . We consider sequence data—i.e. , an example x ∈ X is a sequence x = ( w1 , ... , wm ) ∈ Wm.1 For example , each subcomponent w may be a line of a poem or a measure of music , and x may be a poem or song . 1We use a fixed m to simplify our exposition ; our approach trivially extends to variable m. We are interested in domains where likely examples satisfy latent relational constraints c ∈ C over the subcomponents . For instance , c may say that two measures wi and wj of x start with the same series of pitches , or that two lines wi and wj of x rhyme . We assume given a set of relationsR ( e.g. , r ∈ R might be “ rhyme ” or “ equal ” ) , and a function f : W ×W ×R → [ 0 , 1 ] such that f ( w , w′ , r ) indicates to what extent w and w′ satisfy relation r. Then , c is a compact representation of the relations present in an input x . We describe the structure of c in detail in Section 3.1 ; for now , the approach we describe works for any choice of c. In particular , we build on neurosymbolic generative models ( Young et al. , 2019 ) , where c is itself generated based on a latent value z ∈ Z—i.e. , pθ , φ ( x ) = ∫ ∑ c∈C pθ ( x | c ) · pφ ( c | z ) · p ( z ) dz . Then , Young et al . ( 2019 ) considers the variational distribution qφ̃ ( c , z | x ) = qφ̃ ( z | c ) · q ( c | x ) and q ( c | x ) = δ ( c− cx ) . Here , δ is the Dirac delta function and cx is a single representative generated from x using a program synthesis algorithm ( David & Kroening , 2017 ) —i.e. , an algorithm A that takes as input an example x and outputs a program c = A ( x ) encoding the relational constraints present in x . Next , Young et al . ( 2019 ) derive an evidence lower bound log pθ , φ ( x ) ≥ log pθ ( x | cx ) + Eqφ̃ ( z|cx ) [ log pφ ( cx | z ) ] −DKL ( qφ̃ ( z | cx ) ‖ p ( z ) ) . where DKL is the KL divergence and H is information entropy . The first term is the log-likelihood of a generative model predicting the probability of example x given relational constraints cx , and the second and third terms form the loss of a variational autoencoder ( VAE ) pφ ( c | z ) and qφ̃ ( z | c ) ( Kingma & Welling , 2019 ) . In summary , given training examples x ∈ X , this approach separately learns ( i ) a VAE to generate c , trained on the relational constraints cx synthesized from each training example x , and ( ii ) a model to generate x given cx ; the latter can take multiple forms such as a second VAE or a generative adversarial network ( GAN ) ( Goodfellow et al. , 2014 ) . This approach is called synthesis-guided generative models ( SGM ) since it uses program synthesis to guide training . To leverage this framework , we have to instantiate ( i ) the space of relational constraints C , ( ii ) the synthesis algorithm A : X → C used to extract a program encoding the structure of x , and ( iii ) the architectures of pφ ( c | z ) , qφ̃ ( z | c ) , and pθ ( x | c ) . In prior work , Young et al . ( 2019 ) used heuristics specific to the the image domain to achieve these goals—in particular , they used ( i ) simple equality constraints on sub-regions of the image designed to capture 2D repeating patterns , ( ii ) a custom synthesis algorithm that greedily adds constraints in the data to the program , and ( iii ) a representation of cx as an image , in which case pθ is a generative model over images , and pφ , qφ̃ based on an encoding of c as a fixed-length vector . We design a synthesis algorithm that expresses the synthesis problem as a constrained combinatorial optimization problem , which it solves using an SMT solver called Z3 ( De Moura & Bjørner , 2008 ) . In terms of ( iii ) , our programs encode declarative constraints rather than imperative renderings , so the previous architectures of pφ , and qφ̃ can not be used . Instead , we use expert domain-specific heuristics , transformers ( Vaswani et al. , 2017 ) , or graph neural networks ( GNNs ) ( Kipf & Welling , 2017 ) for pφ and qφ̃ . For pθ , we propose several methods for imposing the constraints encoded by c when generating an example x .

Summary: This paper proposes a neural-symbolic deep generative model to generate music and poems. Compared to other natural language sentence generation tasks, songs and poems usually require some structural constraints, e.g., rhythm and rhymes. Therefore, their approach first generates a program represented as a set of relational constraints, then generates sequences that satisfy the constraints. During the training time, given a training example, i.e., a sequence, they formulate the program synthesis task as a combinatorial optimization problem, and thus they can find the required constraints using the Z3 solver. They use the SOTA generative models for both of the two sequence generation domains. They propose several variants of the generation approach, to ensure that the generated sequences satisfy the constraints. They evaluate their approach in terms of both the low-level and high-level structures. To measure the low-level structure, they compute the negative log likelihood of the SOTA generative models. To measure the high-level structure, they apply the Z3 solver to solve the relational constraints of each generated sequence, and train some discriminators to distinguish the generated sequences from the human-written ones in the dataset. For the poetry domain, they also evaluate the diversity of the generated sequences. Generally, the neural-symbolic approach generates sequences with better high-level structures, and the generated samples are more diverse.