BioEL: A Comprehensive Python Package for Biomedical Entity Linking
Authors: Prasanth Bathala, Batuhan Nursal, Christophe Ye, Shubham Lohiya, David Kartchner, Cassie S. Mitchell


Reviewer paper summaries: This paper presents a package for developing and evaluating biomedical entity linking systems in a consistent and unified manner. The package is built in a modular fashion enabling many different models, datasets, training strategies, and evaluation metrics. The authors detail the package and benchmark a suite of modern models on several standard datasets.
Summary of strengths: * Biomedical entity linking is an important problem with an active sub-community that would greatly benefit from such a package. This type of work is an act of service to the community
* The paper is well organized and very well-written.
Summary of weaknesses: * The paper does not seem to make enough of a contribution for a long paper in its current form. There is a lot of fluff throughout the paper and the content does not fill the 8 page limit (almost a whole page short). I think this paper could be improved by either submitting as a short paper and moving some of the code to the appendix or adding more evidence for the usefulness of the package (e.g. results, ease-of-implementation of complicated training or inference,...).
* The figures and tables could be improved (excluding Table 2 and Figure 4, those look great as is). The text is very small and they are difficult to understand in their current form, and thus the figures do not serve their purpose by providing a nice summary of a diagram of some central point of the paper.
* Not all of the results are reproduced and some results are missing. This is a major weakness where the main contribution is to make running these experiments more accessible. A single NVIDIA A40 should be plenty of GPU to run all of these models and all of the major differences in results should be resolved with the authors of those papers.
Comments, suggestions and typos: * Does the package have integration with tools such as wandb? This would be extremely beneficial for hyperparameter optimization and experiment tracking. 
* Is the package flexible enough to handle more extensive inference techniques such as clustering-based inference or RAG-based techniques?
* How easily can new transformer models be integrated into other systems? If a new transformer model is released, how easily can we use it with all available models?
* Maybe Figure 1 could be made into a larger figure that spans two columns placed at the top of the second page?
Overall assessment scores: 2