Neurosymbolic Deep Generative Models for Sequence Data with Relational Constraints

Abstract: There has been significant recent progress designing deep generative models that generate realistic sequence data such as text or music. Nevertheless, it remains difficult to incorporate high-level structure to guide the generative process, and many such models perform well on local coherence at the cost of global coherence. We propose a novel approach for incorporating global structure in the form of relational constraints between different subcomponents of an example (e.g., lines of a poem or measures of music). Our generative model has two parts: (i) one model to generate a realistic set of relational constraints, and (ii) a second model to generate realistic data satisfying these constraints. For model (i), we propose a program synthesis algorithm that infers the relational constraints present in the training data, and then learn a generative model based on the resulting constraint data.  In our experiments, we show that our approach significantly improves over state-of-the-art in terms of capturing high-level structure in the data, while performing comparably or better in terms of low-level structure.

Reviews and author responses:

 Regarding applying our technique to other domains: Our experiments are tailored to tasks where there are syntactic relationships between different parts of the input, which occur naturally in music and poetry. Both these directions are important applications in their own right; for instance, there is a great deal of interest in using machine learning for music generation (e.g., Google's Magenta project).We agree that additional applications are an interesting direction, but it would require different kinds of programs than the ones we include. We have added a discussion for future work.

"However, important details of how to build the generative model are missing in the main part."
Thank you for this comment.  We have clarified this GCN architecture briefly in Section 4 and in more detail in Appendix B.2.

 "Moreover, it is worth to discuss the limitations of the proposed method: e.g., if a relation r is more conceptual (e.g., w and w′ are talking about the same topic), then would the method still work or what else is needed to make it work?"
* Thank you for this suggestion. To demonstrate this possibility, we have performed an additional experiment in which we used distance between measure-based VAE embeddings as a relation with a value between 0 and 1, and rewrote significant parts of the paper to generalize to a [0,1] instead of restricting to {0,1}.

"It also suffers some inconsistency between the method overview and detailed exposition."
* Thank you for pointing this out; we have tried to clarify it in our paper

"There is other recent work using relational constraints to configure neural sequential models."
Thank you for pointing this out; we have included a discussion in our related work section. Specifically, these works focus on  learning datalog programs, whereas our goal is to integrate Datalog programs into generative models of the data it describes.






 "The abstract mentioned that many recent works perform well on local coherence at the cost of global coherence. But I don't see this problem in the baseline experiment results though. Basically, while the method is interesting, I don't see what problem it is actually solving."
* To clarify, we are not claiming that current methods trade off global coherence for local coherence, but that they simply are unable to achieve global coherence.  This intuition is supported by our graph discriminator experiments; our approaches significantly outperform them in terms of global structure while performing similarly or better in terms of local structure.

"The process of training is confusing to me. It seems that Fig1 bottom shows the inference step of VAE starting with the sampling of latent variables. But I am not sure what is the input at training time. Is it c_x -> z -> c given z -> x given c ??? The paper uses (z given c) and (z given c_x) at different places. I find it confusing."
* We provide additional details on the training process. First, for each training example x, we use our synthesis algorithm to synthesize the relational structure c_x present in x. Then, we need to design two models:
  * p(c): This model generates an example c. In all our experiments, this model is an LSTM-VAE; in particular, it consists of an encoder q(z | c) and a decoder p(c | z), and is trained using the standard VAE objective on training examples c_x.
  * p(x | c): This model should sample x consistent with c, which is a constrained sampling problem. Section 4 provides for strategies for solving this problem.
Given these two models, the test process (i.e., how to generate an example x) is summarized in Figure 1; it first samples c ~ p(c) (as usual for VAEs, this step is achieved by sampling a latent vector z ~ p(z) and then sampling c ~ p(c | z)) and then samples x ~ p(x | c) (using a strategy in Section 4).
We have clarified Section 2 to better convey this structure.

"For the poem generation task, only A1 was used. This is less exciting since the constrained sampling could end up pretty slow and bad. The paper mentioned in the appendix B that typical causal language model doesn't fit for generating words with proper rhythm at the end of each sentence. While this makes sense, but there are apparently some heuristics that are easy to come up with to walk around that. Furthermore, for A3, while sequential generation via combinatorial optimization can be intractable, there are span-based generation approaches that can mimic that. I feel there are many options not explored in this task. A refuse of using A2 and A3 sounds like a drawback."
* We ran an experiment in which we used A3 for poetry generation by first sampling a line using GPT-2, and then using constrained optimization to choose the last word of each line to satisfy the constraints c. This strategy outperformed all of our baselines. We have added these results to our paper.

"I am not sure what story the loss based evaluations can tell. On one hand, the GCN discriminator metric is subject to artifacts in training data and thus potentially can be broken. On the other hand, the FD metric, as also said in this paper, is embedding based, thus does not evaluate rhyme and meter. Then why using them at all in Table 2?"
* For this benchmark, we do not have a single score to quantify performance; instead, we aim to measure local and global performance separately. Our results show that our approach achieves the best global structure (GCN discriminator) while outperforming most baselines at local structure (FD score). Only GPT-2 outperforms our approach in terms of local structure, and that is because we build on a worse generative model (i.e., BERT). Furthermore, we have added a variant of our approach (SGM, Ours, A3) that strictly outperforms all baselines in terms of both GCN discriminator and FD metric.




 "It is unclear to me how the prototype subcomponents are chosen. One concern is that when the prototype subcomponents are similar to each other, the resulting model might generate similar examples and does not generalize well. I guess the ideal scenario is to choose the prototype subcomponents to be as diverse as possible. But it is unclear how this is guaranteed from what is presented."
* The prototype subcomponents are learned from the training data using a VAE, where synthesis is used to create labeled examples. The synthesis process rewards diverse prototypes (since the objective is for the prototypes to cover as many subcomponents as possible), the VAE learns to generate diverse prototypes for each instance of generation.

"The other concern I have is on the efficiency of the synthesizing relational constraint process. The number of variables involved in the constrained optimization problem in Section 3.2 is quadratic in m, i.e., the number of subcomponents in example x. Further, the number of calls to solve this optimization problem is the size of training data. I wonder how efficient is this process."
* We agree that scalability is an important challenge; synthesis took about 20 hours for our entire training dataset in the music domain, which is comparable to the neural network training time. One potential strategy is to use scalable heuristics such as greedy synthesis. While it does not work as well as combinatorial optimization in our music domain (it performed comparably to Attention-RNN), we believe it is a promising direction for future work.

"Since this work follows the framework from [1], I wonder whether the generation processes of samples given relational constraints as described in Section 4.2 are original contributions and how are they compared with the generation process in [1]."
* Our first key contribution is our relational structure representation and our SMT based algorithm for synthesizing relational structure. Our second contribution is our sampling strategy, which is significantly more challenging than the one in [1] (they can simply “render” a program into a partial image and then decode it into a full image. In contrast, we need to perform constrained sampling to transform prototypes into sequences. The three approaches we present in Section 4 are designed to address this challenge.


 We thank all reviewers for their thoughtful comments. We note that we have added a new variant of our approach for the poetry domain, which uses our sampling approach A3 in conjunction with GPT-3; this approach outperforms all baselines in terms of both FD score (local structure) and GCN discriminator (global structure). We have also added experiments using non-binary relations and scalable greedy synthesis. We provide details in Section 5.


The aim of this work is to derive deep generative models that are able to incorporate relational constraints. The proposed model learns relational constraints from training data by solving constrained optimization problems and further generates examples given relational constraints. Empirical evaluations on music and poetry generation tasks are presented. [Strengths]

The paper is overall well-written and the method is clearly explained. The literature review is thorough. The idea of formulating the constraint synthesizing as a constrained optimization problem is novel to me.

[Weakness]
- It is unclear to me how the prototype subcomponents are chosen. One concern is that when the prototype subcomponents are similar to each other, the resulting model might generate similar examples and does not generalize well. I guess the ideal scenario is to choose the prototype subcomponents to be as diverse as possible. But it is unclear how this is guaranteed from what is presented.

- The other concern I have is on the efficiency of the synthesizing relational constraint process. The number of variables involved in the constrained optimization problem in Section 3.2 is quadratic in m, i.e., the number of subcomponents in example x. Further, the number of calls to solve this optimization problem is the size of training data. I wonder how efficient is this process.

- Since this work follows the framework from [1], I wonder whether the generation processes of samples given relational constraints as described in Section 4.2 are original contributions and how are they compared with the generation process in [1].


[1] Halley Young, Osbert Bastani, and Mayur Naik. Learning neurosymbolic generative models via program synthesis. 2019.
 The generative model for sequential data with relational constraints proposed in this work is interesting to me. Still, I have several concerns as mentioned in the main review.


This paper proposes a generative model for sequential data guided by relational constraint. It uses a specific type of high level structure represented by prototype alignment and a VAE framework to formulate the training. The paper experiments the proposed model on music generation and poem generation tasks, showing some substantial improvements given the proposed metrics. I think the constrained/guided generation problem is an important one. And the method proposed in this work is interesting. Although it's relatively weak on the neural-symbolic part (since it only touches on a very special type of constraints, and relations are deterministically optimized and this process does not participate in gradient-based learning, which is less exciting to me), but the construction of the model is reasonable and results are promising. I think the amount contribution is moderate. And besides the evaluation/analysis part is unconvincing, I don't see any other reason to reject.

1. The abstract mentioned that many recent works perform well on local coherence at the cost of global coherence. But I don't see this problem in the baseline experiment results though. Basically, while the method is interesting, I don't see what problem it is actually solving.

2. The process of training is confusing to me. It seems that Fig1 bottom shows the inference step of VAE starting with the sampling of latent variables. But I am not sure what is the input at training time. Is it c_x -> z -> c given z -> x given c ??? The paper uses (z given c) and (z given c_x) at different places. I find it confusing.

3. For the poem generation task, only A1 was used. This is less exciting since the constrained sampling could end up pretty slow and bad. The paper mentioned in the appendix B that typical causal language model doesn't fit for generating words with proper rhythm at the end of each sentence. While this makes sense, but there are apparently some heuristics that are easy to come up with to walk around that. Furthermore, for A3, while sequential generation via combinatorial optimization can be intractable, there are span-based generation approaches that can mimic that. I feel there are many options not explored in this task. A refuse of using A2 and A3 sounds like a drawback.

4. I am not sure what story the loss based evaluations can tell. On one hand, the GCN discriminator metric is subject to artifacts in training data and thus potentially can be broken. On the other hand, the FD metric, as also said in this paper, is embedding based, thus does not evaluate rhyme and meter. Then why using them at all in Table 2? I think the major contribution of this paper is the framework. While some evaluation metrics are questionable, but I can see the coverage is better than the last year's submission, and there are improvements over different metrics. Besides, evaluating generated sequence is a challenging task and is out of the scope of this paper.

The paper proposes a hierarchical approach for generating sequences with global structures (e.g., music and poems with repeating elements). The proposed method first draws a template with a few slots already filled and certain global relational constraints across the slots, and then fills the remaining slots with actual content. The global constraints are found out by solving a combinatorial optimization problem: for each training sequence, they select some parts of the sequence as prototypes and relate each of the other parts to one of the prototypes by minimizing the overall dissimilarity among associated parts. Music and poems generated this way turn out to have a higher global coherence, lyricism, and rhyme, compared to what are directly drawn from an autoregressive model. 

It is a novel algorithm, and achieves compelling empirical results in the domains of music and poetry generation. 
 The strengths of the paper is its novelty and empirical results; its weakness is its clarity and related work discussion. 

The overall framework is well-motivated for certain generation tasks such as music and poetry generation, and learning relational constraints from data is an interesting idea. 
It does perform as expected in real-world datasets. 

However, the paper sweeps under the rug some key foundations of the proposed approach. 
The most important is how to construct the function $f()$. According to appendices, it seems that $f(w,w',r)$ is assigned to 1 or 0 by checking whether $w$ and $w'$ follow a pre-defined relation $r$ and each possible relation $r$ can be automatically verified since they are superficial (e.g., same rhyme or same pitch class). This seems like the crucial premise that the proposed method could work, and it should be clearly stated in the main paper. Moreover, it is worth to discuss the limitations of the proposed method: e.g., if a relation $r$ is more conceptual (e.g., $w$ and $w'$ are talking about the same topic), then would the method still work or what else is needed to make it work? 

It also suffers some inconsistency between the method overview and detailed exposition. The subcomponents of sequence $x$ are defined on the utterance level (e.g., lines of poems) but the content generation is elaborated on the word level, with the same index symbol (i.e., $j$) as the lines, which really confused me. I believe this inconsistency is merely a presentation issue but the authors should make it clearer for the readers to follow the details. 

There is other recent work using relational constraints to configure neural sequential models. E.g., Li et al. uses temporal logic to constrain neural point processes: https://proceedings.mlr.press/v119/li20p.html; Mei et al. uses Datalog programs to compile human knowledge into the architecture of a neural sequence model: https://arxiv.org/abs/2006.16723. Relations with these papers should be discussed: e.g., they all condition sequence modeling on a set of global constraints; constraints of these papers are human-specified while those of this submission are drawn from a learned distribution (though its learning depends on human-specified relational features). 

 Based on the strengths and weaknesses explained above, I believe that this paper is at the borderline; I slightly tend to reject it since the clarity issues that I raise are important. However, if the authors could well address those concerns and we have high confidence in its camera-ready version, then I would consider elevating my rating. 


The paper presents a neurosymbolic generative model for sequence data. The model extends the one proposed by Young et al. (2019) in which a neurosymbolic generative model is initially proposed for images. Experiments are conducted on two datasets, one for music generation and the other for poem generation. Evaluation results indicate that the proposed model can effectively represent high-level structures in sequence generation.

Contributions:
1. a sequence generation model in which structure information is taken into account in a principled way.
2. effectiveness of the approach on music generation and poem generation. The authors follow the existing work (Young et al. (2019)) and extend it to handle sequence data. My understanding is that the model can take structure information into account while generating sequences, or more precisely, explicitly incorporate repetitive patterns in sequences into generation. I judge the work from the following perspectives:

1. Technical Novelty: the basic idea comes from the work of Young et al. with some adaptations to sequence data. I would rather view the work as an extension of the approach in Young et al. (2019) to sequence data. There are some new things for the adaptations (e.g., learning the constraints from data), but overall, the key building blocks (e.g., VAE, controllable text generation) have been proposed before and widely used in a lot of work. 

2. Empirical Novelty: as far as I know, there has been a lot of work on music generation and poem generation. Particularly, my experience is that it is not that difficult to make a generated poem follow rhyme and meter constraints as long as we have a relatively good language model, since such characteristics are very strong in poetry. Therefore, I would like to see more applications, or probably more important applications of the proposed approach, for example, can we apply it to dialogue generation or summarization? To do that, you may have to go beyond the repetitive patterns, but such applications, in my opinion, could generate more impact. 

3. Soundness: the authors pay a lot of effort to formalizing the constraints and describing the learning details of the constraints. However, important details of how to build the generative model are missing in the main part. Even in the appendix, some details are still missing, for example, how to build the GCN to learn \mu(c) and \Sigma(c) in B.2?  In Experiments, I see some metrics for structure consistency, but very few metrics for quality of the generated content in addition to NLL. How can we guarantee the quality of the generated content? If it is hard to find other metrics, at least human studies can also be done in music generation. In 5.2, the authors claim that they have to choose BERT because they need to do backward sampling. I do not think this is a strong reason, as BERT is basically not a generative model. I suggest the authors to think about how to adapt their approach to generative language models, or choose other PLMs as the base. 

4. Writing: there is a little mess in the organization. For example, Introduction and Related work are combined, and the explanation of the example in Figure 1 above the Related work just breaks the coherence of the Section.  Some descriptions about the related work are placed in appendix, and more seriously, some important details, such as the details of Section 4.2 are also placed in appendix. I think most of these content should be involved in the main part, even though there is page limit.  1. Technical innovation is not enough to meet the bar of the conference. 
2. Empirical studies do not fully convince me the effectiveness of the approach. 
3. There are some issues on organization of the content. 
4. I cannot think about broader applications of the proposed approach in sequence generation.

Meta-review: This work proposes an interesting approach for learning the relational constraints of a dataset and then generating according to those constraints. Learning the constraints via a constrained optimization problem is an interesting contribution. The application of constrained generation is also interesting and can be applied to several domains though only music and poetry is examined in this work. However, the music evaluation is unconvincing and the paper lacks clarity in the description of the approach such as building the GCN. Music evaluation could be improved with human evaluation since loss metrics don't paint a full picture. Finally, a more diverse set of experiments and datasets (rather than just one poetry collection and one folk song corpus) and more analysis on the learnt constraints could give a more complete story for this approach's effectiveness on sequence data.

Decision: Reject