Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: The paper proposes an end-to-end joint model for named entity recognition (NER) and relation extraction (RE), using pre-trained language models. The model is very simple, with the key is to use BERT and take NER output as input to RE. The experimental results show the model, without the need for handcrafted features, get state-of-the-art results on five datasets. 

Although the paper is well written and shows good results, I would reject the paper because: 
- the idea is trivial and simple. I don't think there's significant novelty here: all the components are existing and combining them seems very trivial to me. 
- the good performance seems to be from BERT rather than the model's structure (table 2 suggests that). I thus think the contribution of the paper is pretty not significant. 

I think the paper does not fit this conference. It is better to be presented in a Demonstration section at a *ACL conference.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The paper proposes a new joint learning algorithm that works for two tasks, NER and RE. The model is based on a pre-trained BERT model, which provides the word vectors of the input word sequence. Then it solves two tasks with two network branches: the first branch minimizes the loss for NER, and the second branch minimizes the loss for RE. The second branch uses entity labels predicted by the first branch, so joint learning may benefit both tasks. 

The design of the architecture is novel, but it is also not groundbreaking. Each network branch is from known structures, but the combination is not proposed before. 

The submission has evaluated the proposed algorithms on four datasets and improved SOTA performances. The ablation study justifies the design details. 

The writing is generally clear. 

Now critics: 

Ablation study: 
1. As pointed by one public comment, the ablation study should show how much improvement is from BERT vectors. 

2. I'd like to see another ablation study of whether RE helps NER. If you remove the RE component, does the NER performance suffer? 


Writing: 
3. how are predicted labels embedded? Do you learn a vector of each tag of BIOES and then take a weighted sum of these vectors with predicted probabilities as weights?

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper presents an end-to-end methods for jointly training named entity recognition (NER) and relation extraction (RE). The model leverage pre-trained BERT language models, making it very fast to train. The methods is evaluated on 5 standard NER+RE datasets with good performances.

Pros:

- the paper is well written and very clear
- the proposed model has two main advantages: (1) it is very fast to train due to the use of pre-trained BERT representations and (2) it does not depends on any external NLP tool (such as dependency parser)

Cons: 

- I think the main source of improvement comes from the BERT representations used as input. As proposed in the comments, this should be assessed in the paper by replacing BERT representations by non-contextual representations such as GloVE.
- Without this ablation study, the contributions of this paper are to show that using BERT representations as input (1) leads to better performances for NER+RE  and (2) makes the model faster to train. This is not really surprising...
review_title: Official Blind Review #3

Meta-review: This paper presents an end-to-end technique for named entity recognition, that uses pre-trained models so as to avoid long training times, and evaluates it against several baselines. The paper was reviewed by three experts working in this area. R1 recommends Reject, giving the opinion that although the paper is well-written and results are good, they feel the technique itself has little novelty and that the main reason the technique works well is using BERT. R2 recommends Weak Reject based on similar reasoning, that the approach consists of existing components (albeit combined in a novel way) and suggest some ablation experiments to isolate the source of the good performance. R3 recommends Weak Accept but feels it is "unsurprising" that BERT allows for faster training and higher accuracy. In their response, authors emphasize that the application of pretraining to named entity recognition is new, and that theirs is a methodological advance, not purely a practical one (as R1 suggests and other reviews imply). They also argue it is not possible to do a fair ablation study that removes BERT, but make an attempt. The reviewers chose to keep their scores after the response. Given the split decision, the AC also read the paper. It is clear the paper has significant merit and significant practical value, as the reviews indicate. However, given that three expert reviewers -- all of whom are NLP researchers at top institutions -- feel that the contribution of the paper is weak (in the context of the expectations of ICLR) makes it not possible for us to recommend acceptance at this time. 
Decision: Reject