danf commited on
Commit
506766e
1 Parent(s): 6eddc80

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -1
README.md CHANGED
@@ -4,4 +4,64 @@ datasets:
4
  - natural_questions
5
  language:
6
  - en
7
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  - natural_questions
5
  language:
6
  - en
7
+ tags:
8
+ - colbert
9
+ ---
10
+ # ColBERT NQ Checkpoint
11
+
12
+ This trained model is based on the [ColBERT](https://github.com/stanford-futuredata/ColBERT) model, trained on the [Natural Questions](https://huggingface.co/datasets/natural_questions) dataset.
13
+
14
+ # Model Details
15
+
16
+ Model is based on ColBERT, which in turn is based around a BERT encoder. The model is trained for text retrieval using a contrastive loss; given a query there's a relevant and non relevant passages.
17
+
18
+ The corpus is based on [Wikipeida](https://huggingface.co/datasets/wiki_dpr).
19
+
20
+ # Uses
21
+
22
+ Model can be used by the [ColBERT](https://github.com/stanford-futuredata/ColBERT) codebase to initiate a retriever; one needs to build a vector index and then queries can be ran.
23
+
24
+ # Evaluation
25
+
26
+ Evaluation results on NQ dev:
27
+
28
+ <table>
29
+ <colgroup>
30
+ <col class="org-right">
31
+ <col class="org-right">
32
+ <col class="org-right">
33
+ </colgroup>
34
+ <thead>
35
+ <tr>
36
+ <th scope="col" class="org-right">NQ</th>
37
+ <th scope="col" class="org-right">Recall</th>
38
+ <th scope="col" class="org-right">MRR</th>
39
+ </tr>
40
+ </thead>
41
+
42
+ <tbody>
43
+ <tr>
44
+ <td class="org-right">10</td>
45
+ <td class="org-right">71.1</td>
46
+ <td class="org-right">52.0</td>
47
+ </tr>
48
+
49
+ <tr>
50
+ <td class="org-right">20</td>
51
+ <td class="org-right">76.3</td>
52
+ <td class="org-right">52.3</td>
53
+ </tr>
54
+
55
+ <tr>
56
+ <td class="org-right">50</td>
57
+ <td class="org-right">80.4</td>
58
+ <td class="org-right">52.5</td>
59
+ </tr>
60
+
61
+ <tr>
62
+ <td class="org-right">100</td>
63
+ <td class="org-right">82.7</td>
64
+ <td class="org-right">52.5</td>
65
+ </tr>
66
+ </tbody>
67
+ </table>