File size: 1,561 Bytes
225472b faf8733 506766e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
---
license: cc-by-sa-3.0
datasets:
- natural_questions
language:
- en
tags:
- colbert
---
# ColBERT NQ Checkpoint
This trained model is based on the [ColBERT](https://github.com/stanford-futuredata/ColBERT) model, trained on the [Natural Questions](https://huggingface.co/datasets/natural_questions) dataset.
# Model Details
Model is based on ColBERT, which in turn is based around a BERT encoder. The model is trained for text retrieval using a contrastive loss; given a query there's a relevant and non relevant passages.
The corpus is based on [Wikipeida](https://huggingface.co/datasets/wiki_dpr).
# Uses
Model can be used by the [ColBERT](https://github.com/stanford-futuredata/ColBERT) codebase to initiate a retriever; one needs to build a vector index and then queries can be ran.
# Evaluation
Evaluation results on NQ dev:
<table>
<colgroup>
<col class="org-right">
<col class="org-right">
<col class="org-right">
</colgroup>
<thead>
<tr>
<th scope="col" class="org-right">NQ</th>
<th scope="col" class="org-right">Recall</th>
<th scope="col" class="org-right">MRR</th>
</tr>
</thead>
<tbody>
<tr>
<td class="org-right">10</td>
<td class="org-right">71.1</td>
<td class="org-right">52.0</td>
</tr>
<tr>
<td class="org-right">20</td>
<td class="org-right">76.3</td>
<td class="org-right">52.3</td>
</tr>
<tr>
<td class="org-right">50</td>
<td class="org-right">80.4</td>
<td class="org-right">52.5</td>
</tr>
<tr>
<td class="org-right">100</td>
<td class="org-right">82.7</td>
<td class="org-right">52.5</td>
</tr>
</tbody>
</table> |