GPL
/

msmarco-distilbert-margin-mse

Feature Extraction

text-embeddings-inference

Inference Endpoints

Model card Files Files and versions Community

Edit model card

YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

This is the zero-shot baseline model in the paper "GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval"

The training setup:

Start from distilbert-base-uncased;
Mine 50 hard negatives for each query on MS MARCO with sentence-transformers/msmarco-distilbert-base-v3 and sentence-transformers/msmarco-MiniLM-L-6-v3;
Do Margin-MSE training on the tuples (including queries, gold relevant, and hard negatives) with the teacher model cross-encoder/ms-marco-MiniLM-L-6-v2 for 70K steps with batch size 75, max. sequence-length 350.

Downloads last month: 840

Feature Extraction

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.