ankur310794 commited on
Commit
8850fa3
1 Parent(s): 894e07f
Files changed (1) hide show
  1. README.md +15 -14
README.md CHANGED
@@ -1,6 +1,7 @@
1
  ---
2
  tags:
3
  - generated_from_keras_callback
 
4
  model-index:
5
  - name: dpr-question_encoder_bert_uncased_L-12_H-128_A-2
6
  results: []
@@ -11,23 +12,27 @@ probably proofread and complete it, then remove this comment. -->
11
 
12
  # dpr-question_encoder_bert_uncased_L-12_H-128_A-2
13
 
14
- This model was trained from scratch on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
 
17
 
18
- ## Model description
19
 
20
- More information needed
21
 
22
- ## Intended uses & limitations
 
 
 
23
 
24
- More information needed
25
 
26
- ## Training and evaluation data
 
 
 
27
 
28
- More information needed
29
-
30
- ## Training procedure
31
 
32
  ### Training hyperparameters
33
 
@@ -35,12 +40,8 @@ The following hyperparameters were used during training:
35
  - optimizer: None
36
  - training_precision: float32
37
 
38
- ### Training results
39
-
40
-
41
-
42
  ### Framework versions
43
 
44
  - Transformers 4.15.0
45
  - TensorFlow 2.7.0
46
- - Tokenizers 0.10.3
 
1
  ---
2
  tags:
3
  - generated_from_keras_callback
4
+ license: apache-2.0
5
  model-index:
6
  - name: dpr-question_encoder_bert_uncased_L-12_H-128_A-2
7
  results: []
 
12
 
13
  # dpr-question_encoder_bert_uncased_L-12_H-128_A-2
14
 
15
+ This model(google/bert_uncased_L-12_H-128_A-2) was trained from scratch on training data: data.retriever.nq-adv-hn-train(facebookresearch/DPR).
16
  It achieves the following results on the evaluation set:
17
 
18
 
19
+ ## Evaluation data
20
 
21
+ evaluation dataset: facebook-dpr-dev-dataset from official DPR github
22
 
23
+ |model_name|data_name|num of queries|num of passages|R@10|R@20|R@50|R@100|R@100|
24
+ |---|---|---|---|---|---|---|---|---|
25
+ |nlpconnect/dpr-ctx_encoder_bert_uncased_L-12_H-128_A-2(our)|nq-dev dataset|6445|199795|60.53%|68.28%|76.07%|80.98%|91.45%|
26
+ |*facebook/dpr-ctx_encoder-single-nq-base(hf/fb)|nq-dev dataset|6445|199795|40.94%|49.27%|59.05%|66.00%|82.00%|
27
 
28
+ evaluation dataset: UKPLab/beir test data but we have used first 2lac passage only.
29
 
30
+ |model_name|data_name|num of queries|num of passages|R@10|R@20|R@50|R@100|R@100|
31
+ |---|---|---|---|---|---|---|---|---|
32
+ |nlpconnect/dpr-ctx_encoder_bert_uncased_L-12_H-128_A-2(our)|nq-test dataset|3452|200001|49.68%|59.06%|69.40%|75.75%|89.28%|
33
+ |*facebook/dpr-ctx_encoder-single-nq-base(hf/fb)|nq-test dataset|3452|200001|32.93%|43.74%|56.95%|66.30%|83.92%|
34
 
35
+ Note: * means we have evaluated on same eval dataset.
 
 
36
 
37
  ### Training hyperparameters
38
 
 
40
  - optimizer: None
41
  - training_precision: float32
42
 
 
 
 
 
43
  ### Framework versions
44
 
45
  - Transformers 4.15.0
46
  - TensorFlow 2.7.0
47
+ - Tokenizers 0.10.3