Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper improves on the stat-of-the-art in entity linking.  Entity linking involves connecting entities to their mentions in test passages.
Current solutions rely on finding mentions first before running entity disambiguation stage which is difficult because mentions are being searched for unknown entities in the first stage.
The authors address this challenge by rearranging the standard QA pipeline by learning questions first (finding entities) and searching for answers (mentions) at a later stage.
This approach appears to be well-researched. The authors tested their model on GERBIL benchmarking platform and achived very positive results.

main_review: This paper addresses a problem of entity linking. The authors propose a reformulation of the problem as an inverted open-domain Question-Answers (QA) The authors used dense retriever in the first phase of their algorithm (retrieving top K entities) and then in the second phase authors trained reader mouse for ranking and extracting entity mentions from the output of the retriever.
The main advantages of the proposed method (besides outperforming previous strategies) is the data efficiency gained by eliminating dependency on hardcoded mention-candidate dictionary. Instead it finds entities from text passages and then finds lined mentions in test passages.
Authors work comes with strong backing from trying they model on GERBIL benchmark platform (trying it on various datasets) I especially appreciated the authors explaining the reason why their method did not do as well on OKE15/16 datasets compared to how it performed on other datasets.
Regarding the reproducibility of the results, the authors did not share their source code so it might take some effort to reproduce what they have done. On the other hand, they provided very detailed description of their training process including very helpful Model details section with lot's of relevant details parameters used with training and inference.
Authors provided very insightful discussion of their results. In addition to what's been provided I would be interested to learn how quick is the inference and how the time to run inference is impacted by the choice of K.
summary_of_the_review: Overall, I believe this paper is a worthy addition to this year's conference. It tackles an important problem and provides a better performing, simpler and more intuitive solution compare to previous work.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a new paradigm for entity linking. The proposed method EntQA retrieves candidate entities first, and then uses candidate entities as queries to find mention spans. 
EntQA addresses the issue of finding mentions without knowing their entities and offers many practical advantages. Experimental results show that EntQA can achieve good performance without mention-candidates dictionaries and large-scale task-specific weak supervision.

main_review: Strength:
* The novel entity linking paradigm proposed by this paper is insightful and the performance gain is significant.
* The proposed paradigm addresses a key issue in prior entity linking works. In EntQA, mention detection happens after entity detection, so that mention detection can utilize entity information. 
* The proposed QA paradigm allows EntQA to take advantage of existing question answering datasets. Pretraining on open domain question answering datasets may improve the generalizability of EntQA. So not surprisingly, EntQA performs better on out-of-domain datasets.
* The proposed method has several practical advantages. It costs less GPU hours and memory space in comparison with prior SOTA according to the paper. However, providing the exact number will be much better.

Weakness: 
* When discussing the difference between entity linking paradigms, this paper doesn’t compare with GENRE [1], although it is listed as a baseline. As the main contribution of this paper is the novel paradigm, I would like to see a section instead of a brief paragraph in introduction to discuss the theoretical and empirical difference between different paradigms. However, considering that Table 1 already shows the proposed paradigm performs better than other paradigms empirically, the theoretical analysis can be left for future work.
* The term ‘unknown entities’ is confusing, because it can also refer to entities not covered by KB. I suggest the authors clarify the definition. I think it is better to say ‘predict mentions when not knowing the corresponding entities’ than ‘predict mentions of unknown entities’. Besides, I wonder how mentions of entities not covered by KB will influence different paradigms, although it may be a minor issue. 
* The effectiveness of EntQA is only shown by overall performance. It would be better to have experiments to compare the mention detection results of different methods.

[1] De Cao, N., Izacard, G., Riedel, S., & Petroni, F. (2020, September). Autoregressive Entity Retrieval. In International Conference on Learning Representations.

summary_of_the_review: The insights and novel paradigm for entity linking proposed by this paper are attractive. Although the experiments can be more sufficient, the key claims are supported by the experimental results. I recommend accepting this paper.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Standard entity linking first does mention detection and then the actual linking of mentions to entities in the knowledge base, and the authors propose to flip this by using open domain QA. They first use the text to find candidate entities in the knowledge base. Then for each of these candidates, they look for possible mentions in the text. The model performs well on average, and has some nice advantages such as no mention-candidates dictionary and simplicity of generalization to out-of-domain tasks.
main_review: This paper is clearly written, has a working original idea, and provides a great new option for entity linking that has empirical promise. It's in good shape, and I don't have a lot of criticisms. Just a comments and questions below:

Page 1 - "End-to-end models alleviate the problem of error propagation, but the search is only approximate and the dilemma, albeit to a lesser degree, remains" - I think this sentence need some additional justification. Sure, search is approximate. That on its own doesn't mean that the dilemma remains. How would you establish that the dilemma remains? 

Page 3 - "In our experiments we simply set ψtopic(x) = x1 ∈V (i.e., the first token in the document)." That seems odd. The first token may not be very useful. It could be "The". Could you use the encoded first phrase?

Page 3 - "We write ⊕to denote the text concatenation (we insert a special symbol to represent the concatenation)" I don't understand the parenthetical. Can you elaborate?

The bad threshold suggest that the model is perhaps not calibrated in some key way. Could an explicit calibration step on the validation set help fix these errors? More broadly, the fixed threshold seems like the weakest part of the system. What ideas do you have about making it more flexible or adaptable? Did you try other approaches to thresholding and found them to not work? These findings would be good to include in the paper.
summary_of_the_review: Solid paper about an original approach to entity linking that performs pretty well.
review_title: null

Meta-review: This paper casts entity linking in a retrieve-then-read framework by first retrieving entity candidates and then finding their mentions via reading comprehension. All reviewers agree that the proposed approach is novel, well-motivated, and simple yet performant. The authors have done a good job of addressing all the concerns raised, and the reviewers are unanimous in their recommendation for accepting the paper. I hope the authors will also incorporate the feedback and their responses in the final version.
Decision: Accept (Spotlight)