Venue: AKBC 2019 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper presents an approach to cross-sentence relation extraction that combines LSTMs and convolutional neural network layers with word and position features.  Overall the choices made seem reasonable, and the paper includes some interesting analysis / variations (e.g., showing that an LSTM layer followed by a CNN is a better choice than the other way around).

Evaluation is performed on two datasets, Quirk and Poon (2016) and a chemical induced disease dataset.  The paper compares a number of model variations, but there don't appear to be any comparisons to state-of-the-art results on these datasets.  The paper could benefit from comparisons to SOTA on these or other datasets.
review_title: Review of Combining Long Short Term Memory and Convolutional Neural Network for Cross-Sentence n-ary Relation Extraction

final_score: rating: 7: Good paper, accept
review_content: The paper addresses cross-sentence n-ary relation extraction. The authors propose a model consisting of an LSTM layer followed by an CNN layer and show that it outperforms other model choices. The experiments are sound and complete and the presented results look convincing. The paper is well written and easy to follow. 
In total, it presents a nice experimental study.

Some unclear issues / questions for the authors:
- "The use of multiple filters facilitates selection of the most important feature for each feature map": What do you mean with this sentence? Don't you get another feature map for each filter? Isn't the use of multiple filters rather to capture different semantics within the sentence?
- "The task of predicting n-ary relations is modeled both as a binary and multi-class classification problem": How do you do that? Are there different softmax layers? And if yes, how do you decide which one to use?
- Table 1/2: How can you draw conclusions about the performance on binary and ternary relations from these tables? I can only see the distinction of single sentence and cross sentence there.
- Table 3: The numbers for short distance spans (mostly 20.0) look suspicious to me. What is the frequency of short/medium/long distance spans in the datasets? Are they big enough to be able to draw any conclusions from them?
- You say that CNN_LSTM does not work because after applying the CNN all sequential information is lost. But how can you apply an LSTM afterwards then? Is there any recurrence at all? (The sequential information would not be lost after the CNN if you didn't apply pooling. Have you tried that?)
- Your observation that more than two positional embeddings decrease the performance is interesting (and unexpected). Do you have any insights on this? Does the model pay attention at all to the second of three entities? What would happen if you simply deleted this entity or even some context around this entity (i.e., perform an adversarial attack on your model)?

Other things that should be improved:
- sometimes words are in the margin
- there are some typos, e.g., "mpdel", "the dimensions ... was set... and were initialised", "it interesting"
review_title: Nice experimental study

final_score: rating: 7: Good paper, accept
review_content: The paper presents a method for n-ary cross sentence relation extraction.
Given a list of entities, and a list of sentences the task is to identify which relation (from a predefined list) is described between the entities in the given sentences.
The proposed model stacks CNN on LSTM to get long range dependencies in the text, and shows to be effective, either beating or equalling the state-of-the-art on two datasets for the task.

Overall, I enjoyed reading the paper, and would like to see it appear in the conference.
While the proposed model is not very novel, and was shown effective on other tasks such as text classification or sentiment analysis, this is the first time it was applied for this specific task.
In addition, I appreciate the additional evaluations, which ablate the different parts of the model, analyze its performance by length between entities, compare it with many variations as baselines and against state-of-the-art for the task.

My main comments are mostly in terms of presentation - see below.


Detailed comments:

In general, I think that wording can be tighter, and some repetitive information can be omitted. For example, Section 4 could be condensed to highlight the main findings, instead of splitting them across subsections.
I think that Section 3.1.2 (“Position Features”) would benefit from an example showing an input encoding.
Table 5 shows up in the references. 

Minor comments and typos:

Text on P. 9 overflows the page margins.
I think that Table 3 would be a little easier to read if the best performance in each column were highlighted in some manner.
Section 2, p. 3: “mpdel” -> model.
Perhaps using “Figure 1” instead of “Listing 1” is more consistent with *ACL-like papers?

review_title: Good paper with small modeling improvements but thorough evaluations

Meta-review: metareview: The presented cross sentence relation extraction method is simple but well motivated. The experiments show that it works well, setting a new SOTA on the two relevant benchmarks. The  ablations and extended analyses are also well done and extensive. Overall, this is a clear paper with a solid contribution that we would all learn something by reading.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)