Venue: ICLR 2017 conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper proposes learning on the fly to represent a dialog as a graph (which acts as the memory), and is first demonstrated on the bAbI tasks. Graph learning is part of the inference process, though there is long term representation learning to learn graph transformation parameters and the encoding of sentences as input to the graph. This seems to be the first implementation of a differentiable memory as graph: it is much more complex than previous approaches like memory networks without significant gain in performance in bAbI tasks, but it is still very preliminary work, and the representation of memory as a graph seems much more powerful than a stack. Clarity is a major issue, but from an initial version that was constructive and better read by a computer than a human, the author proposed a hugely improved later version. This original, technically accurate (within what I understood) and thought provoking paper is worth publishing.

The preliminary results do not tell us yet if the highly complex graph-based differentiable memory has more learning or generalization capacity than other approaches. The performance on the bAbI task is comparable to the best memory networks, but still worse than more traditional rule induction (see http://www.public.asu.edu/~cbaral/papers/aaai2016-sub.pdf). This is still clearly promising.

 The sequence of transformation in algorithm 1 looks sensible, though the authors do not discuss any other operation ordering. In particular, it is not clear to me that you need the node state update step T_h if you have the direct reference update step T_h,direct. 

It is striking that the only trick that is essential for proper performance is the ‘direct reference’ , which actually has nothing to do with the graph building process, but is rather an attention mechanism for the graph input: attention is focused on words that are relevant to the node type rather than the whole sentence. So the question “how useful are all these graph operations” remain. A much simpler version of a similar trick may have been proposed in the context of memory networks, also for ICLR'17 (see match type in "LEARNING END-TO-END GOAL-ORIENTED DIALOG" by Bordes et al)


The authors also mention the time and size needed to train the model: is the issue arising for learning, inference or both? A description of the actual implementation would help  (no pointer to open source code is provide). The author mentions Theano in one of my questions: how are the transformations compiled in advance as units? How is the gradient back-propagated through the graph is this one is only described at runtime?


Typo: in the appendices B.2 and B.2.1, the right side of the equation that applies the update gate has h’_nu while it should be h_nu.

In the references, the author could mention the pioneering work  of Lee Giles on representing graphs with  RNNs.

Revision: I have improved my rating for the following reasons:
- Pointers to an highly readable and well structured Theano source is provided.
- The delta improvement of the paper has been impressive over the review process, and I am confident this will be an impactful paper.
- Much simpler alternatives approaches such as Memory Networks seem to be plateauing for problems such as dialog modeling, we need alternatives.
- The architecture is this work is still too complex, but this is often as we start with DNNs, and then find simplifications that actually improve performance

review_title: Complex implementation of a differentiable memory as a graph with promising preliminary results.

final_score: rating: null
review_content: 
Dear authors,

 First, I would like to say that the topic of the article is clearly interesting, but the paper is very hard to follow since the writing is a little bit rough. The paper could clearly benefit to some additional figures for example when presenting the model, and to separate the presentation of the different allowed operations with the “technical” computations.

 My first question is about the positioning of the paper and the links between your approach and other approaches like Relational Neural networks (or lifed relational neural networks)  for example or even differentiable neural computers. Particularly, having a discussion about the expressiveness of the different approaches would be very nice (if feasuible) 
Could you please discuss a little bit more the difference/advantages/drawbacks of your approach with existing works ? 

My second question concerns the way the structure is defined in the experimental part (Algorithm 1). It is not clear to me if other structures could be proposed. Could you explain a little bit more this point ? And particularly the advantage of the chosen structure w.r.t other possibilities and for the particular tasks studied here ? 
review_title: Hard paper...

final_score: rating: null
review_content: Dear Author,

Algorithm 1 applies to a single dialog with K sentences: the graph represents a memory of the dialog that is later used to answer a query.  As the graph starts from 0 at step 1, shall I assume that all the trainable parameters of the graph (U and W)  are only learnt from one sample dialog, basically only learnt to perform the inference? If this is true, what are the trainable parameters that encode knowledge from other dialogs? The parameters used to extract representation i(k)? If I am wrong, how are the U and W parameters passed from previous sample dialogs?
review_title: Inference vs. learning

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: The paper proposes an extension of the Gated Graph Sequence Neural Network by including in this model the ability to produce complex graph transformations. The underlying idea is to propose a method that will be able build/modify a graph-structure as an internal representation for solving a problem, and particularly for solving question-answering problems in this paper. The author proposes 5 different possible differentiable transformations that will be learned on a training set, typically in a supervised fashion where the state of the graph is given at each timestep. A particular occurence of the model is presented that takes a sequence as an input a iteratively update an internal graph state to a final prediction, and which can be applied for solving QA tasks (e.g BaBi) with interesting results.

The approach  in this paper is really interesting since the proposed model is able to maintain a representation of its current state as a complex graph, but still keeping the property of being differentiable and thus easily learnable through gradient-descent techniques. It can be seen as a succesfull attempt to mix continuous and symbolic representations. It moreover seems more general that the recent attempts made to add some 'symbolic' stuffs in differentiable models (Memory networks, NTM, etc...) since the shape of the state is not fixed here and can evolve. My main concerns is about the way the model is trained i.e by providing the state of the graph at each timestep which can be done for particular tasks (e.g Babi) only, and cannot be the solution for more complex problems. My other concern is about the whole content of the paper that would perhaps best fit a journal format and not a conference format, making the article still difficult to read due to its density. 
review_title: null

final_score: rating: 7: Good paper, accept
review_content: The main contribution of this paper seems to be an introduction of a set of differential graph transformations which will allow you to learn graph->graph classification tasks using gradient descent. This maps naturally to a task of learning a cellular automaton represented as sequence of graphs. In that task, the graph of nodes grows at each iteration, with nodes pointing to neighbors and special nodes 0/1 representing the values. Proposed architecture allows one to learn this sequence of graphs, although in the experiment, this task (Rule 30) was far from solved.

This idea is combined with ideas from previous papers (GGS-NN) to allow the model to produce textual output rather than graph output, and use graphs as intermediate representation, which allows it to beat state of the art on BaBi tasks. 
review_title: Architecture which allows to learn graph->graph tasks,  improves state of the art on babi

Meta-review: The idea of building a graph-based differentiable memory is very good. The proposed approach is quite complex, but it is likely to lead to future developments and extensions. The paper has been much improved since the original submission. The results could be strengthened, with more comparisons to existing results on bAbI and baselines on the experiments here. Exploring how it performs with less supervision, and different types of supervision, from entirely labeled graphs versus just node labels, would be valuable.
Decision: Accept (Oral)