ProMiNER Russian BioNNE-L Dictionary-Pretrained Cross-Encoder

Cross-encoder pretrained on UMLS dictionary pseudo-pairs with compact candidate-context profiles.

This model is part of ProMiNER, a Russian-track biomedical entity-linking system for BioNNE-L. The system links mentions from NEREL-BIO/BioNNE-L texts to UMLS concepts by combining dense retrieval and cross-encoder reranking.

GitHub: https://github.com/bikingSolo/prominer
Model type: sentence-transformers-cross-encoder
Base/initializer: andorei/BERGAMOT-multilingual-GAT

Training

Pretrained from dictionary-derived pseudo-query/candidate pairs. Candidate concepts are represented by compact profiles containing representative names and selected aliases. This checkpoint is intended as the initializer for task-specific BioNNE-L reranking.

Hyperparameters:

reranker_model_name_or_path: andorei/BERGAMOT-multilingual-GAT
retriever_model_name_or_path: andorei/BERGAMOT-multilingual-GAT
loss_name: bce
epochs: 5
train_batch_size: 128
learning_rate: 2e-05
weight_decay: 0.01
warmup_ratio: 0.1
max_seq_length: 384
pretrain_num_negatives: 20
max_pseudo_queries_per_cui: 5
num_train_pairs: 5900264
num_eval_pairs: 317280
selection_metric: Acc@1

According to the Acc@1 on dev, the best epoch is 2.

Full local metadata exported from MLflow is included in prominer_metadata/.

Evaluation

Metrics below are copied from the local MLflow run artifacts.

split	num_queries	num_pairs	num_positive_pairs	num_negative_pairs	RetrieverHitRate	Acc@1	Acc@5	MRR
train	284890	5900264	284890	5615374	1.0
dev	15347	317280	15347	301933	1.0	0.9506092395907995	0.994200821007363	0.9696687373962294

Usage

from sentence_transformers import CrossEncoder

model = CrossEncoder("bikingSolo/prominer-ru-pretrained-cross-encoder", num_labels=1)
scores = model.predict([
    (
        "вестибулокохлеарный нерв",
        "слуховой нерв; вестибулокохлеарный нерв; nervus vestibulocochlearis [viii]",
    )
])

Intended Use

This checkpoint is intended for research and reproducibility of the ProMiNER BioNNE-L Russian entity-linking pipeline. For the full system, use:

prominer-ru-retriever to retrieve candidate UMLS concepts.
prominer-ru-reranker to rerank those candidates with candidate-context profiles.

The dictionary-pretrained cross-encoder is primarily an intermediate checkpoint used to initialize the final reranker.

Data and Citation

Training and evaluation use BioNNE-L/NEREL-BIO resources and UMLS-derived terminology available in this repository's data layout. Cite the relevant NEREL-BIO and BioNNE-L papers when using this model.

Check https://github.com/bikingSolo/prominer for more info.