Venue: AKBC 2019 Conference

Reviews:

final_score: rating: null
review_content: I hope this paper was published.
review_title: Very interesting paper that describes a positive contribution to the state of the art in BioNLP

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper proposes a deep-learning-based method to solve the known BioNLP task of disease normalization on the NCBI disease benchmark (where disease named entities are normalized/disambiguated against the MeSH and OMIM disease controlled vocabularies and taxonomies). The best known methods (DNorm, TaggerOne) are based on a pipeline combination of sequence models (conditional random fields) for disease recognition, and (re)ranking models for linking/normalization.

The current paper proposes instead an end-to-end entity recognition and normalization system relying on word-embeddings, siamese architecture and recursive neural networks to improve significantly (4%, 84 vs 80% F1-score, T. 3). A key feature is the use of a GRU autoencoder to encode or represent the "context" (related entities of a given disease within the span of a sentence), as a way of approximating or simulating collective normalization (in graph-based entity linking methods), which they term "coherence model". This model is combined (weighted linear combination) with a model of the entity itself.
Finally, the complete model is trained to maximize similarity between MeSH/OMIM and this combined representation.
The model is enriched further with additional techniques (e.g., distant supervision). 

The paper is well written, generally speaking. The evaluation is exhaustive. In addition to the NCBI corpus, the BioCreative5 CDR (chemical-disease relationship) corpus is used. Ablation tests are carried out to test for the contribution of each module to global performance. Examples are discussed.

There are a few minor issues that it would help to clarify:

(1) Why GRU cells instead of LSTM cells?
(2) Could you please explain/recall why (as implied) traditional models are NP-hard? I didn't get it. Do you refer to the theoretical complexity of Markov random fields/probabilistic graphical models? Maybe you should speak of combinatorial explosion instead and give some combinatorial figure (and link this to traditional methods). My guess is that this is important, as the gain in runtime performance (e.g., training time - F. 4) might be linked to this.
(3) A link should be added to the GitHub repository archiving the model/code, to ensure reproducibility of results.
(4) Could you please check for *statistical significance* for T. 3, 5, 6 and 7? At least for the full system (before ablations). You could use cross-validation. 
review_title: Very interesting paper that describes a positive contribution to the state of the art in BioNLP

final_score: rating: 7: Good paper, accept
review_content: Summary:
The authors address the problem of disease normalization (i.e. linking). They propose a neural model with submodules for mention similarity and for entity coherence. They also propose methods for generating additional training data. Overall the paper is nicely written with nice results from simple, efficient methods.

Pros:
- Paper is nicely written with good coverage of related work
- LCA analysis is a useful metric for severity of errors
- strong results on the NCBI corpus
- methods are significantly faster and require far fewer parameters than TaggerOne while yielding comparable results

Cons:
- Results on BC5 are mixed. Why?
- Data augmentation not applied to baselines
- Methods are not very novel

Questions:
- Were the AwA results applied only at test time or were the models (including baselines) re-trained using un-resolved abbreviation training data?
review_title: Simple, fast method with decent results on disease normalization (linking)

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper presents a method for named entity disambiguation
tailored to the important case of medical entities,
specifically diseases with MeSH and OMIM
as the canonicalized entity repository.
The method, coined NormCo, is based on a cleverly designed
neural network with distant supervision from MeSH tags of
PubMed abstracts and an additional heuristic for estimating
co-occurrence frequencies for long-tail entities.

This is very competent work on an important and challenging
problem. The method is presented clearly, so it would be easy
to reproduce its findings and adopt the method for further
research in this area.
Overall a very good paper.

Some minor comments:

1) The paper's statement that coherence models have
been introduced only recently is exaggerated. 
For general-purpose named entity disambiguation, coherence
has been prominently used already by the works of
Ratinov et al. (ACL 2011), Hoffart et al. (EMNLP 2011)
and Ferragina et al. (CIKM 2010); and the classical
works of Cucerzan (EMNLP 2007) and Milne/Witten (CIKM 2008)
implicitly included considerations on coherence as well.
This does not reduce the merits of the current paper,
but should be properly stated when discussing prior works.

2) The experiments report only micro-F1. Why is macro-F1 
(averaged over all documents) not considered at all?
Wouldn't this better reflect particularly difficult cases
with texts that mention only long-tail entities,
or with unusual combinations of entities?

review_title: very competent work on an important problem

Meta-review: metareview: The reviewers all agree the paper is a clear accept.  The paper presents an end-to-end approach to biomedical concept normalization that supplants previous state of the art pipeline systems based on more conventional bio NLP methods.  Although the individual components of the solution are not novel, e.g.,  siamese networks, GRUs, and distant supervision, etc.,  they are combined together in highly appropriate ways to solve a difficult entity linking problem.  The authors did a commendable job addressing the reviewers comments, questions and concerns by running experiments, providing new results, updating related work to more accurately capture the fact that other entity linking approaches also capture coherence, and addressing a few minor clarity issues.
confidence: 5: The area chair is absolutely certain
Decision: Accept (Poster)