Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Comments by sections : 

Summary : 

The use of entity is not clear. Are you referring to named-entities ?

1 INTRODUCTION

"Due to the differences in language structure" : this paragraph is not clear. It should say explicitly that in Chinese word can be composed of one or a couple of characters and that word boundaries can not detected graphically. 

 "A common mitigation is to add external lexicon as a reference"   references are needed on how to includes words in embedding (except just training word embeddings)
 
 " an entity classification-assisted model" : not very clear : the classification is assisted ?
 
 " The first step is to form word embeddings of entities appearing in the trainning sentences through character embeddings and the second step is to aggregate entity embeddings by category and generate classification embeddings" : is it a multi-task training ?
 
 "After that, we designed a novel Attention mechanism to integrate entity " : is it the same model or two different propositions of the paper ? "After that" is not very clear as a transition.
 
 Section 2 :
 
 "more works Yang et al. (2016) Ruder12 et al. (2017) " : strange formulation
 
 "What’s more, the attention mechanism..." : odd expression.
 
 "In this paper, we revise the Scaled Dot-Product Attention to Classification Attention which would give a weighted representation of the input sentences through a series of entity classes." : maybe it should be moved to the introduction as a novelty proposed by the paper.
 
 
 3.2  EMBEDDING EXTRACTION FOR ENTITY CLASS
 
 " the smooth inverse frequency is abandoned" : why ? please explain this choice.
 "the weighted projection of the word embeddings on their first singular vector is removed." : explain. If it is a common practice, give a citation otherwise justify this choice.
 
 3.3 CLASSIFICATION ATTENTION 
 
 Here again, the proposed attention system is described but not justified : why would a class specific attention system be better ? What are the expected advantages ?
 
 4.1.3 EXPERIMENTAL RESULTS
 Experiments are conducted on 4 dataset and the proposed model is compared to a "standard" BERT-based model and several results form the litterature. The proposed model outperforms sometimes the other models, often by a small margin as it is usually the case in NER experiments. 
 But more insight on the strengths of the models should be given by conducting an ablation study. 
  
  
 In conclusion ,this paper present an incremental improvement over BERT-based NER for Chinese. The proposed approach is not sufficiently justified and experiments, even if showing improvements over state-of-the-art models or published results, does not sufficiently explore the benefits of the proposed model (with ablation study for example).  
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper tries to improve the performance of Chinese NER by developing a novel attention mechanism that leverages BERT pre-trained model which considers bi-directional context. Experiments on a number of tasks show that the proposed approach is effective.

Comments:
[1] A bunch of experiments are conducted
[2] Chinese NER is a hard problem, but it would be great to see the proposed approach generalizable to other tasks. So, the contribution of this paper is limited
[3] The proposed algorithm is simple and effective, but the novelty is a bit low

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: Summary:
      This paper discussed an approach to do named entity resolution (NER, the paper focuses only on Chinese NER but I think it could generalize to other languages as well). The idea is based on smart integration and extension of multiple existing building blocks: 1) BERT pre-trained model 2) a previous work to get document embedding by doing weighted average of word embedding (https://openreview.net/pdf?id=SyK00v5xx) and 3) Scaled dot-product attention mechanism applied directly to multi-label classification. The "Introduction", "Related work", and "Experiment Settings" sections are well written and covers many details and decent references. Especially, the "experiments" section is described in a great amount of details, which should be very helpful for reproducibility. 
      
Contributions:
      * The author found an interesting application of the original algorithm (https://openreview.net/pdf?id=SyK00v5xx) to represent the entity class embedding based on averaging "BERT" embeddings of all the component words. This could be implemented as a pre-processing step against any training dataset to derive "pre-learned" entity class embedding.
      * Instead of the common approach of connecting the BERT sequence outputs directly to CRF layer, the author added an intermediate layer to calculate the classification attention between a sentence (sequence of token embedding) and any entity class (based on the above pre-learned entity embedding). This result plus the original sentence embedding are concatenated.  The concatenation is further fed into a few additional layers to produce the final inputs into CRF layer.

Weakness:
     * The paper lacks novelty. As pointed above, I did not see that the contribution from the paper is sufficiently original. It is a good application of various existing methods though.

I also have a few suggestions/questions below:

* The ERNIE paper (https://arxiv.org/abs/1907.12412v1) is mentioned in the related work. Since ERNIE can potentially learn a good vocab for Chinese, did you ever compare your approach vs ERNIE+CRF? 
* There is one paper that I know which is pretty relevant to what you are doing here, which is probably worth a reference. https://arxiv.org/abs/1805.04174.  In that paper, the idea is to co-learn a class embedding and perform text classification. Their class attention is performed through dot-production attention though.
* The Table index seems wrong in your paper. (I think Table 2 is not mentioned in your paper, but all tables (3-6) is offset by 1). 
* There are some minor typos or places that need some clarifications.
   - in the abstract: "character-based" model. This is a little confusing. Because BERT is a word-piece based model. word-piece could across multiple characters for English. IIUC, You probably want to say "Chinese-character" instead of character.
   - in "Introduction", "providing greater weight to characters identical to each entity class", you might want to revise this sentence to clarify its meaning further.
   - In section 3.2, you might want to give some explanation to some notations (the first time you refer to it). For example, what is $L$, what is $m$ and $n$.  What is $S$?  Also why the denominator of Emb(Word) is not $n-m+1$? 

   - The last paragraph in section 3.3 needs more clarification as well. How do you merge the three tensors after attention stage? (a concatenation ?) . The last sentence mentioned "residential", I guess instead you want to say "residual".  You might also want to clarify where the "3 layers" of residual appear in your network.

  - In your experiment, (if I did not miss), did you freeze the BERT parameters and entity embeddings when finetuning your NER model?

  - in Table 2 and Table 3, why the 13-layer BERT + CRF performs significantly worse on Recall (Table 2) and significantly better on Recall (Table 3)? 


  
review_title: Official Blind Review #2

Meta-review: The paper is interested in Chinese Name Entity Recognition, building on a BERT pre-trained model. All reviewers agree that the contribution has limited novelty. Motivation leading to the chosen architecture is also missing. In addition, the writing of the paper should be improved.

Decision: Reject