Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: - Summary
This paper presents DialoGraph, which introduces graph structures to encode the relations of strategies and dialog acts among utterances in a dialog history. This paper empirically demonstrated the efficacy of DialoGraph on non-collaborative negotiation dialog tasks, and the model is evaluated automatically and human-evaluation.

- Strong points
	1. Training to control for the pragmatics of the negotiation dialogues has been less studied, and it is a crucial and interesting topic to build a logical dialog system.
	2. This paper is well structured and well written.
	3. Introducing GAT and ASAP to model strategies and dialog acts is novel, and the interpretation of their results is interesting.
	4. This paper also provides soundly experimental results and comparison with other SOTA models.

- Weak points
	1. The dialog task problem setting seems unrealistic. A tuple of utterance, dialog act, and strategy at turn $I$ is given, and based on the previous tuple sequence the model predicts the next response, dialog act, and strategy. However, in reality, the user’s dialog act and strategy are hidden. 
	2. Also, the sets of dialog acts and strategies are different depending on user and system. I agree this paper followed the task as previously defined, but it seems awkward.
	3. HED+Transformer vs. DialoGraph: Those two models show similar experimental results. According to the experiment configuration, the HED+Transformer used 6 decoder layers, whereas the DailoGraph used 2 graph layers. Those two models were fairly comparable with respect to the number of parameters? Or, the proposed method based on GAT+ASAP does not effective as much as Transformer, because both are basically based on attention mechanisms?
	4. In table 4, the average words per turn is required to be separately reported depending on users and bots.

- Questions
	1. How the bot decide the price to offer? Does it solely depend on the language model (i.e., the decoder)?
	2. How the bot encode the listed price?
	3. In table 4, why HED got a remarkably higher score on “natural” measure than other models?
	4. Please address and clarify the weak points above.
review_title: Well-written paper, Interesting topics and ideas.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes a end-to-end dialogue system that leverages Graph Attention Networks to model complex negotiation strategies. The main contributions that the author claims are to model negotiation strategies through a GNN and using these learned strategies to predict future strategies and generate a response leads to better negotiations.

The end to end model contains a traditional hierarchical encoder to obtain contextual representations along with a structure encoder that is designed to model strategies and dialog acts and obtain structural representations. The decoder is a simple GRU that produces the response by conditioning it on the contextual and structural representation along with the previous word.

Strength & Weakness:
1.  The aspect of generating a response based on the prediction of negotiation strategies and dialog acts is an interesting approach.
2. The results from Table 1 show that transformers perform comparatively or even better than DialoGRAPH. What is the overall gain on using the GNN if the transformers match their performance?
3. The results in Tables 2 & 3 are hard to interpret. Prior research has shown that BLEU and other automated metrics are not good enough to evaluate the performance of dialog systems. Why is the BERT F1 Score for Dialograph in Table 2 and 3 different? What is the major difference?
4. What were the definitions provided for persuasive, coherence, natural, and understandable? Why is there a huge dropoff on the metric for naturalness from HED to other models? How many participants were recruited?

Questions:
1. What is the rationale behind 5 negotiation classes? Was any ablation done with this to determine the optimal number or how this split affects the outcomes?
2. What is the overall size of the vocabulary during decoding?
3. How were the 4 outcome dialog acts labeled across the dataset? was this done through a human annotation process?
4. What decoding strategy was used?


Suggestion:
1. Section 2.4 has inconsistencies in the usage of notations. Please fix those issues.
2. Can the bolding of scores in Table 1 be made more consistent and highlight only those scores that are higher and lower based on the metrics being represented

=======================After reading the authors response ============================
I thank the authors for answering all the questions that been raised by the fellow reviewers. Looking at the responses and changes made to the paper, I have increased the score from 5 to 6 after the authors clarified the issues I had with the paper. Overall this paper demonstrates the effectiveness of using a GNN for negotiation dialogues. I feel that this approach can be applied for any non-collaborative dialog settings and the claims of interpretability make this approach better. 
review_title: dialogue system that leverages Graph Attention Networks to model complex negotiation strategies

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper deals with the problem of natural language generation for a dialogue system involved in complex communication tasks such as negotiation or persuasion. The proposed architecture consists of two encoders: one for the utterance and the other for dialogue acts and negotiation strategies. The decoder is an RNN that converts the encoded vectors to the output utterance. Each utterance is first passed through BERT to get an utterance-level encoding. The sequence of utterance encodings is then passed through an RNN to generate a conversation level encodings. The negotiation strategies and dialogue acts in a conversation are represented using a node-edge graph, where the nodes are one of the N different strategies/acts and there exists an edge from node a to node b if an utterance with strategy A precedes any utterance with strategy B. The entire architecture is trained in a multi-task setup where the loss function accounts for both the predictions of the model and generated language. The proposed architecture is evaluated on the CraigslistBargain dataset and compared against Zhou et al. 2020. 

The paper is very clearly written and the experimental work has sufficient detail to ensure reproducibility. The main contribution and the novelty of this paper is in the use of graph neural networks for encoding dialogue acts and negotiation strategies. This choice was mainly because it helps with better interpretability of predictions and this is demonstrated anecdotally in section 5. The proposed model shows better performance in three different metrics when compared to sota from Zhou et al: (1) prediction of dialogue acts and negotiation strategies, (2) on the downstream task of dialogue generation, and (3) human evaluation to quantify the quality of generated language. 

There are a few aspects of the paper unclear to me and could use more insight from the authors. (2) The input to the GNN is a node-edge graph where the edges exist between dialogue acts or negotiation strategies based on their precedence order in the conversation. It would be useful to explain why the authors chose this type of representation. What other types of representations were considered? (2) The authors use two different encoders for dialogue acts and negotiation strategies. Would it make sense to have a graphical representation that captures both dialogue acts and negotiation strategies simultaneously? (3) From my understanding, it seems the dialogue acts were annotated in the original work of (He He 2018) and strategies are obtained based on the models and rules published in (Zhou et al 2019). I am not sure if the model evaluation should entail predicting negotiation strategies which are in itself predictions of a different model. 

Minor comments/questions:
1. How were the train/test/dev splits done?
2. If I am not mistaken, it seems like the model uses predictions from time step (t) to predict and generate for time step (t+1). Does this mean errors in one of the earlier timesteps could lead to more errors in subsequent timesteps?
3. I think it would be clearer if you were to use a single-letter variable for strategies. It helps for better readability. 
4. References: Please use the peer-reviewed version of the paper as opposed to the arxiv version when available. Eg. He He et al 2018. 
5. How many conversations were used in human evaluations?


review_title: DialoGraph: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: * Summary:
This paper proposes to DialoGraph, a model that utilizes Graph Attention Networks to learn the pragmatic relations between strategy and dialogue-act in negotiation dialogues, and achieves better performances than previous baselines. The use of Graph Attention Networks also enables a better interpretation of the relation between strategies.

* Quality:
The motivation to introduce pragmatic information into negotiation dialogues is clear. The model is straightforward and effective. The experiments and analyzes are through. But the idea of directly applying Graph Attention Networks is not super exciting.

* Clarity: 
The paper is in general written clearly and easy to follow, with a few points to improve listed below. 
1. In 2.4, "$p_j$ is the positive weight associated with the particular strategy", is this $p_j$ the probability of the j-th strategy occurring in $u_{t+1}$ mentioned before?
2. In section 5, "We notice that as soon as the first propose at u5 happens, the strategies completely change and become independent of the strategies before the propose point. " 
It's not clear to me why the strategies are independent looking at Figure 3. And also, why is the weight in Figure 3 equal to 1 and the sum of outgoing edges bigger than 1? What's the max and min? And what's the cut-off threshold for the edges in Figure 3?
3. In section 4, "(several results are in bold if they have statistically insignificant differences)"
It's not clear which results are significant and which are not from the tables and the descriptions
4. I'd appreciate more dialogue examples in the appendix.
5. How hard is it to optimize the graph networks? Is the training stable?


* Originality:
The model is not very novel as Graph Networks have been used a lot in dialogue models.


* Significance:
DialoGraph could be potentially useful for other negotiation systems but it requires both strategy and dialogue-act annotations, which may be hard to obtain.

* Pros:
1. The interpretation of the learned strategy graph is useful, but that section needs a bit more work in clarity.
2. The use of Graph Attention Network improves the negotiation dialogue system.

* Cons:
1. The model is applied on one negotiation dataset only and requires both strategy and dialogue-act annotations, which may not be available in other datasets, making the model not so generalizable to other tasks.

2. From the results, it seems the major performance jump comes from using the embeddings from pretrained language models (transformer, BERT, etc)

3. It's not clear which results are significant and which are not.

4. It seems there is no connection between the "strategy structure encoder" and the "dialogue-act structure encoder"? Maybe adding a connection between these two modules would give a better performance since dialogue-act and strategies can be related? Also, are they related at all? If not, why do we need both labels?

 


* Typo: 
missing a period in Human Evaluation "effective negotiation system Our model also..."
review_title: This paper proposes to apply Graph Attention Networks to introduce pragmatic information into negotiation dialogues and achieves good results. The use of Graph networks also makes the relations between strategies more interpretable.

Meta-review: In the context of constructing negotiation dialogue strategies/policies, the authors explore the use of graph attention networks (GATs) for determining the sequence of negotiation dialogue acts -- specifically leading to a (1) hierarchical dialogue encoder via pooled BERT + GRU encoding -> (2) GAT over dialogue strategies/acts (many technical details around graph usage) -> (3) GRU decoder. While a relatively straightforward replacement relative to similar architectures with other 'structural' encoders, they provide a sound end-to-end training strategy that is shown to perform well on the buyer-seller negotiation task via CraigslistBargain dataset where they demonstrate SoTA performance.

== Pros ==
+ Studying the pragmatics component of negotiation dialogue strategies has received recent interest and this seems a good milepost that demonstrates mainstream methodological approaches for this task (i.e., this is a good baseline for future innovations)
+ The paper is well-written in that it is easy to understand intuitively while having sufficient detail to understand the details.
+ The empirical results appear promising and meet the standard within this sub-community -- showing improvements with automatic and human evaluation.

== Cons == 
- This builds on existing datasets, which are known to have undesirable properties (e.g., automatic evaluation, small number of dialogue datasets, use of explicit dialogues acts, etc.) While it still meets the standards of this sub-community, it still isn't a completely convincing task.
- While the use of GATs is novel in this setting and they get it to work within the overall architecture, this is something that many people are likely trying at this time -- so there isn't an exciting 'disruptive' step here.
- The empirical results, while satisfactory from a quantitative perspective, even in reading the Appendices, it isn't clear that these are significantly better from a planning perspective or if it is just 'pattern recognition' gains.

Evaluating along the requested dimensions:
- Quality: The underlying method is fairly straightforward and the authors incorporate up-to-date GAT-related methods to get this to work in this setting. The empirical results are sound if predicated on the general quality in this sub-community where you have the standard machine translation evaluation problem for meaning vs. lexical closeness. To mitigate, they use BERTScore and human evaluation -- which is at the higher end of what can be reasonably expected.
- Clarity: The paper is written clearly overall, especially if considering the appendices where there is significant detail. Related to empirical evaluation, it isn't easy to intuitively interpret the results, but this is again par for the course. Additionally, I believe the authors did a good job responding to reviewer concerns.
- Originality: While all of the reviewers agreed that the approach was novel in this setting, one of the reviewers explicitly pointed out that using GATs in negotiation dialogues isn't that exciting -- and I mostly agree. I view this as something that somebody would have done and will serve as a good baseline; although I think this sub-field is going to need more datasets to continue progressing.
- Significance: As stated above, it is a good baseline that I think many are likely thinking of (as the TOD community has been doing this for a bit now). However, it is done well.

Honestly, I agree with the reviewers that this is a somewhat borderline paper -- mostly due to it being a fairly 'obvious' idea and the nature of the subfield making it not entirely clear if the improvements are due to knowing the target performance while training or due to the methodological advance. Personally, I am convinced, but it isn't totally clear. That being said, it is a well-written paper and I think the reviewer issues were sufficiently addressed. Thus, I would prefer to see it accepted as I think it will be a strong methodological baseline for this problem (which hopefully will accumulate more convincing datasets and standard evaluation). 
Decision: Accept (Poster)