monoT5 trained on MS-Marco
Implementation of
Nogueira, R., Jiang, Z., Lin, J., 2020. Document Ranking with a Pretrained Sequence-to-Sequence Model. arXiv:2003.06713 [cs].
This model has been trained on MsMarco v1, and uses the t5-base model
Parameters based on PyGaggle
Using the model
The model can be loaded with experimaestro IR
If you want to use the model in further experiments with XPMIR, use this code:
from xpmir.models import AutoModel
from xpmir.models import AutoModel
model, init_tasks = AutoModel.load_from_hf_hub("xpmir/monot5")
Use this code if you want to use the model in inference only:
from xpmir.models import AutoModel
from xpmir.models import AutoModel
model = AutoModel.load_from_hf_hub("xpmir/monot5", as_instance=True)
model.rsv("walgreens store sales average", "The average Walgreens salary ranges...")
Results
Dataset | AP | P@20 | RR | RR@10 | Success@5 | nDCG | nDCG@10 | nDCG@20 |
---|---|---|---|---|---|---|---|---|
msmarco_dev | 0.3797 | 0.0384 | 0.3851 | 0.3762 | 0.5497 | 0.4835 | 0.4382 | 0.4602 |
trec2019 | 0.4874 | 0.7209 | 0.9671 | 0.9671 | 1.0000 | 0.6918 | 0.7217 | 0.6939 |
trec2020 | 0.4605 | 0.6139 | 0.9396 | 0.9389 | 0.9815 | 0.6796 | 0.6925 | 0.6581 |
- Downloads last month
- 6