Edit model card

Training

Details

The model is initialized from the dbmdz/bert-base-german-uncased checkpoint and fine-tuned on 10M triples via pairwise softmax cross-entropy loss over the computed scores of the positive and negative passages associated to a query. It was trained on a single Tesla A100 GPU with 40GBs of memory during 200k steps with 10% of warmup steps using a batch size of 96 and the AdamW optimizer with a constant learning rate of 3e-06. Total training time was around 12 hours.

Data

The model is fine-tuned on the German version of the mMARCO dataset, a multi-lingual machine-translated version of the MS MARCO dataset. The triples are sampled from the ~39.8M triples of triples.train.small.tsv

Evaluation

The model is evaluated on the smaller development set of mMARCO-es, which consists of 6,980 queries for a corpus of 8.8M candidate passages. We report the mean reciprocal rank (MRR) and recall at various cut-offs (R@k).

model Vocab. #Param. Size MRR@10 R@50 R@1000
ColBERTv1.0-german-mmarcoDE german 110M 440MB 26.62 63.66 68.32
Downloads last month
416
Safetensors
Model size
110M params
Tensor type
F32
·
Unable to determine this model’s pipeline type. Check the docs .

Dataset used to train AdrienB134/ColBERTv1.0-german-mmarcoDE

Collection including AdrienB134/ColBERTv1.0-german-mmarcoDE