Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Paper presents a bi-encoder framework for NER, which applies contrastive learning to map candidate text spans and entity types into the same vector representation space and make the representation of entity mentions be similar with corresponding entity type. It proposes span-based objectives that compare span and entity type, and position-based objectives that compare start and end tokens separately with entity type. It shows its effectiveness in nested NER with fast inference speed.
strength_and_weaknesses: Paper presents bi-encoder style NER framework with novel objectives suitable to NER and shows its effectiveness in nested NER problems. However, I have following major concerns:

(1) Lack of baseline. Span-based NER, which predicts over all the candidate spans in the sentence, has been widely explored in prompt-based learning (Ding et al., 2021, Cui et al., 2021, Ma et al., 2022… etc.) and . It should have baselines from other research lines for making this paper be more convincing.

(2) No details about candidate generation. Based on window-size, I think the number of span candidates will vary a lot. This ambiguity makes Table 7 be not convincing since the inference speed might be affected by the number of span candidates.

Prompt-Learning for Fine-grained Entity Typing,, Ding et al., 2021
Template-based Named Entity Recognition using BART, Cui et al., 2021
Template-free Prompt Tuning for Few-shot NER., Ma et al., 2022

clarity,_quality,_novelty_and_reproducibility: Writing is unclear.
- What is D(j-I). Span width embedding matrix?
- What is l_joint?

Not clear about the reproducibility.

summary_of_the_review: Paper presents bi-encoder style NER framework with novel objectives suitable to NER and shows its effectiveness in nested NER problems. However, I think paper needs to be improved a lot in terms of writing and experiments.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Most of existing works approach NER as sequence labeling, span classification, or sequence-to-sequence problems. These conventional methods rely on CRF to extract entity values which performs subpar on nested NER tasks. To address these limitations, this work formulate NER as a representation learning problem. Intuitively, their method associate entity type embeddings with entity name embeddings in a common subspace. To achieve the goal in a principle manner, they cast the problem into contrastive learning framework. Moreover, they proposed two contrastive objectives for model training and dynamic thresholding for adaptive model inference. Extensive experiment demonstrated the effectiveness of their approach and ablations conveyed insightful findings. 
strength_and_weaknesses: Strength
- This work is well-motivated. The idea of framing NER as representation learning is interesting. Although there are some works combine contrastive learning with NER, this paper proposed novel strategies to handle challenges specifically.
- The introduced position-based objective to treat entity spans differently. Specifically, they penalize more entity span that have completely no overlap with the gold span while penalize less partially correct span. Their experiment results (as shown in Table 8) implies this objective contribute performance gains consistently and substantially. 
- The authors conducted extensive experiment in various settings and benchmarks. Their methods beats SOTA baselines almost across the board. 
- Various ablation studies have been conducted to share insightful findings.

Major Concerns
- There are serval other works [1-3] which are quite relevant/similar with this paper as they solve NER under contrastive learning framework as well. I would appreciate the authors if they can compare their method with these works from methodology perspective.
- I am wondering how would the model handle entity name that could be a valid value in different entity types. For instance, Willson could be a person name, while it also could be a ORG(company) name. Could you share some thoughts on this case?
- Currently the method enumerates all candidate entity spans as negative samples for calculating contrastive losses. In the paper, the maximum token length is 256. However if we have even larger sequence length and/or large $L$, do we expect the computational cost would increase significantly? If so, is there any alternative and more efficient way to create negative samples?
- For span-based objective, what is the dimension of $h_i^T$? I guess it should be $\mathbb{R}^m$? Could you elaborate more about $D$? Is it learnable? and how to learn it? Is $D(j-i)$ refers to the token representations between $h_i^T$ and $h_j^T$? It seems the answer is not. So why do you using the token embeddings of $h_i^T$ and $h_j^T$ only instead of taking token embeddings between them into consideration?
- In eq(8), is $u_{[CLS]}=Linear_B^T(h_{[CLS]}^T)$? Do you think $u_{[CLS]}$ would collapse to $e_k^B$? why or why not?
- In eq(10), what is $\ell_{\text{joint}}^+$? Are you referring to $\ell_{span}^+$?

[1]. Fu, Yingwen, Nankai Lin, Ziyu Yang and Shengyi Jiang. “A Dual-Contrastive Framework for Low-Resource Cross-Lingual Named Entity Recognition.” ArXiv abs/2204.00796 (2022): n. pag.
[2]. Ye, Hongbin, Ningyu Zhang, Shumin Deng, Mosha Chen, Chuanqi Tan, Fei Huang and Huajun Chen. “Contrastive Triple Extraction with Generative Transformer.” AAAI (2021).
[3]. Lin, Bill Yuchen, Dong-Ho Lee, Minghan Shen, Ryan Rene Moreno, Xiao Huang, Prashant Shiralkar and Xiang Ren. “TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognition.” ACL (2020).
clarity,_quality,_novelty_and_reproducibility: Clarity: Good as the paper is easy to follow. The paper still can be further polished with proofreading. 
Quality: Good. The technique part is sound and solid. Extensive experiments and ablations are diverse and insightful.
Novelty: Good. The authors formulate the problem under contrastive learning with new methodology, though these exist some works are relevant to this work closely.
Reproducibility: No data point as the code is not shared.
summary_of_the_review: Overall the proposed method for NER is moderately novel. It can benefit the community in this filed. The experiments and ablations demonstrated the effectiveness of their method.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new framework that performs the Named Entity Recognition task from the semantic matching perspective (bi-encoder here). The idea is to map candidate text spans and entity types into the same vector representation space and perform the NER task using distance metric accordingly. One important issue is separating non-entity spans from desired entity mentions. To alleviate these problems, the paper proposes a position-based objective and a dynamic thresholding approach. Experiments on both supervised and distantly supervised NER tasks demonstrate the effectiveness of the proposed approaches.
strength_and_weaknesses: 
Strength:
- Utilizing Bi-encoder has never been considered, while this idea is very similar to the span-based ner. (see Weaknesses)
- The empirical results are extensive.

Weaknesses:
- The proposed approach is very similar to span-based NER. In span-based NER, the label type embeddings are randomly initialized (the final layer before softmax). However, the proposed approach utilizes a neural network to obtain that matrix. In the end, the novelty seems limited.
- Besides nested NER, I would also prefer adding some flat NER results (CoNLL, WNUT, and so on).
clarity,_quality,_novelty_and_reproducibility: 
- The overall writing is clear. 
- The quality meets the standard of ICLR. 
- The novelty is relatively limited. 
- As the paper has some detailed designs on special losses, the code is suggested to be public to reproduce empirical reesults.
summary_of_the_review: This paper proposes a new framework that performs the Named Entity Recognition task with the bi-encoder model. The paper shows strong results on nested NER and distant supervised NER tasks. While the results are promising, the proposed approach is similar to the span-based NER approach, which might have limited techniqual novelty.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a contrastive learning framework for NER tasks. The proposed model is flexible in predicting nested entities.  The idea is novel and experiments demonstrate the effectiveness of the proposed model. 
strength_and_weaknesses: Some important details are missed or not cleared. For example, how to set the maximum length L. What is the relation between L and the model performance (accuracy and speed). Experiments didn't include the most common dataset CoNLL 03. This framework setting is similar to the template-based NER (using prompt,"Template-based named entity recognition using BART") but this work didn't compare with this work. 
clarity,_quality,_novelty_and_reproducibility: Some important details are missed or not cleared. For example, how to set the maximum length L. What is the relation between L and the model performance (accuracy and speed).  I didn't see the open source statement in this paper, as the experiments require many details, without the source code, it is hard to reproduce the results. 
summary_of_the_review: Novel and strong results, but the details are not complete and the experiments/comparison are not complete.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a novel approach to the problem of named entity recognition by casting it as a representation learning task. Through contrastive learning, the model learns to distinguish spans that are valid entities from those that do not form entities.

The paper is making an exciting contribution to the field of named entity recognition, which may change many researchers' views of such a task. Backed by solid empirical results on both flat entity and nested entity recognition tasks, the work presents robust research findings worth sharing with the community.

Several reviewers requested additional experiments and raised questions on further details (such as the impact of L). The authors provided thorough information in the rebuttals.
justification_for_why_not_higher_score: While the work is solid, I think the topic is a little narrow – it is on named entity recognition only. Therefore, perhaps it is worthwhile to make it a poster presentation. This way, those who are particularly interested in the topic may be able to have closer interactions with the authors during the presentation.
justification_for_why_not_lower_score: This submission presents compelling findings that are worth sharing. I don't see a reason to reject it.
Decision: Accept: poster