Create README.md

README.md

@@ -0,0 +1,57 @@
+---
+language: multilingual
+datasets:
+- NQ
+- Trivia
+- SQuAD
+- MLQA
+- DRCD
+---
+
+# dpr-ctx_encoder-bert-base-multilingual
+
+## Description
+
+Multilingual DPR Model base on bert-base-multilingual-cased. 
+[DPR model](https://arxiv.org/abs/2004.04906)
+[DPR repo](https://github.com/facebookresearch/DPR)
+
+## Data
+1. [NQ](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
+2. [Trivia](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
+3. [SQuAD](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py)
+4. [DRCD*](https://github.com/DRCKnowledgeTeam/DRCD)
+5. [MLQA*](https://github.com/facebookresearch/MLQA)
+
+`question pairs for train`： 644,217  
+`question pairs for dev`： 73,710
+
+*DRCD and MLQA are converted using script from haystack [squad_to_dpr.py](https://github.com/deepset-ai/haystack/blob/master/haystack/retriever/squad_to_dpr.py)
+
+## Training Script
+I use the script from [haystack](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial9_DPR_training.ipynb)
+
+## Usage
+
+```python
+from transformers import DPRContextEncoder, DPRContextEncoderTokenizer
+tokenizer = DPRContextEncoderTokenizer.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual')
+model = DPRContextEncoder.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual')
+input_ids = tokenizer("Hello, is my dog cute ?", return_tensors='pt')["input_ids"]
+embeddings = model(input_ids).pooler_output
+```
+
+Follow the tutorial from `haystack`:
+[Better Retrievers via "Dense Passage Retrieval"](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial6_Better_Retrieval_via_DPR.ipynb)
+```
+from haystack.retriever.dense import DensePassageRetriever
+retriever = DensePassageRetriever(document_store=document_store,
+                                  query_embedding_model="voidful/dpr-question_encoder-bert-base-multilingual",
+                                  passage_embedding_model="voidful/dpr-ctx_encoder-bert-base-multilingual",
+                                  max_seq_len_query=64,
+                                  max_seq_len_passage=256,
+                                  batch_size=16,
+                                  use_gpu=True,
+                                  embed_title=True,
+                                  use_fast_tokenizers=True)
+```