Hugging Face's logo

LilaBoualili
/
colbert-distilbert-margin_mse-T2-msmarco-encoder-only

Feature Extraction
Transformers
PyTorch
English
distilbert
dense-retrieval
knowledge-distillation
text-embeddings-inference
Inference Endpoints
Model card Files Community
colbert-distilbert-margin_mse-T2-msmarco-encoder-only / README.md
LilaBoualili's picture
LilaBoualili
Update README.md
ac9f6bf
preview code
|
raw
history blame
664 Bytes
metadata
language: en
tags:
  - dense-retrieval
  - knowledge-distillation
datasets:
  - ms_marco

Margin-MSE Trained ColBERT encoder model only

We provide the encoder model of the complete retrieval trained DistilBert-based ColBERT model. This model is trained with Margin-MSE using a 3 teacher BERT_Cat (concatenated BERT scoring) ensemble on MSMARCO-Passage, for more details check the full model card.

This encoder-only model is used as the oracle for distilling term topic embeddings in our ECIR'23 paper.