basilevc commited on
Commit
1cbdf0f
1 Parent(s): 291f8e4

updated readme with training data/type info

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -57,11 +57,11 @@ can be around 0.5 to 1 GiB depending on the used GPU.
57
  - Base language model: [English BERT-Small](https://huggingface.co/google/bert_uncased_L-4_H-512_A-8)
58
  - Insensitive to casing and accents
59
  - Output dimensions: 256 (reduced with an additional dense layer)
60
- - Training procedure: TBD
61
 
62
  ### Training Data
63
 
64
- TBD
65
 
66
  ### Evaluation Metrics
67
 
 
57
  - Base language model: [English BERT-Small](https://huggingface.co/google/bert_uncased_L-4_H-512_A-8)
58
  - Insensitive to casing and accents
59
  - Output dimensions: 256 (reduced with an additional dense layer)
60
+ - Training procedure: A first model was trained with query-passage pairs, using the in-batch negative strategy with [this loss](https://www.sbert.net/docs/package_reference/losses.html#multiplenegativesrankingloss). A second model was then trained on query-passage-negative triplets with negatives mined from the previous model, like a variant of [ANCE](https://arxiv.org/pdf/2007.00808.pdf) but with different hyper parameters.
61
 
62
  ### Training Data
63
 
64
+ The model was trained on a Sinequa curated version of Google's [Natural Questions](https://ai.google.com/research/NaturalQuestions).
65
 
66
  ### Evaluation Metrics
67