Shitao commited on
Commit
2ab44ec
1 Parent(s): 3641912

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -13
README.md CHANGED
@@ -16,17 +16,9 @@ In this project, we introduce BGE-M3, which is distinguished for its versatility
16
  - Multi-Linguality: It can support more than 100 working languages.
17
  - Multi-Granularity: It is able to process inputs of different granularities, spanning from short sentences to long documents of up to 8192 tokens.
18
 
19
- **Some suggestions for retrieval pipeline in RAG:**
20
- We recommend to use following pipeline: hybrid retrieval + re-ranking.
21
- - Hybrid retrieval leverages the strengths of various methods, offering higher accuracy and stronger generalization capabilities.
22
- A classic example: using both embedding retrieval and the BM25 algorithm.
23
- Now, you can try to use BGE-M3, which supports both embedding and sparse retrieval.
24
- This allows you to obtain token weights (similar to the BM25) without any additional cost when generate dense embeddings.
25
- - As cross-encoder models, re-ranker demonstrates higher accuracy than bi-encoder embedding model.
26
- Utilizing the re-ranking model (e.g., [bge-reranker](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/reranker), [cohere-reranker](https://txt.cohere.com/rerank/)) after retrieval can further filter the selected text.
27
-
28
 
29
  ## News:
 
30
  - 2024/3/2: Release unified fine-tuning [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/unified_finetune) and [data](https://huggingface.co/datasets/Shitao/bge-m3-data)
31
  - 2024/2/6: We release the [MLDR](https://huggingface.co/datasets/Shitao/MLDR) (a long document retrieval dataset covering 13 languages) and [evaluation pipeline](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB/MLDR).
32
  - 2024/2/1: **Thanks for the excellent tool from Vespa.** You can easily use multiple modes of BGE-M3 following this [notebook](https://github.com/vespa-engine/pyvespa/blob/master/docs/sphinx/source/examples/mother-of-all-embedding-models-cloud.ipynb)
@@ -95,6 +87,15 @@ If you want to fine-tune all embedding function of m3, you can refer to the [uni
95
 
96
 
97
 
 
 
 
 
 
 
 
 
 
98
 
99
 
100
  ## Usage
@@ -217,9 +218,13 @@ print(model.compute_score(sentence_pairs,
217
 
218
  ## Evaluation
219
 
 
 
 
 
220
 
221
- We compare BGE-M3 with some popular methods, including BM25, openAI embedding, etc.
222
 
 
223
  - Multilingual (Miracl dataset)
224
 
225
  ![avatar](./imgs/miracl.jpg)
@@ -252,6 +257,7 @@ especially in long document retrieval.
252
  ![avatar](./imgs/bm25.jpg)
253
 
254
 
 
255
  ## Training
256
  - Self-knowledge Distillation: combining multiple outputs from different
257
  retrieval modes as reward signal to enhance the performance of single mode(especially for sparse retrieval and multi-vec(colbert) retrival)
@@ -266,8 +272,8 @@ Refer to our [report](https://arxiv.org/pdf/2402.03216.pdf) for more details.
266
 
267
  ## Acknowledgement
268
 
269
- Thanks the authors of open-sourced datasets, including Miracl, MKQA, NarritiveQA, etc.
270
- Thanks the open-sourced libraries like [Tevatron](https://github.com/texttron/tevatron), [Pyserini](https://github.com/castorini/pyserini).
271
 
272
 
273
 
@@ -284,4 +290,4 @@ If you find this repository useful, please consider giving a star :star: and cit
284
  archivePrefix={arXiv},
285
  primaryClass={cs.CL}
286
  }
287
- ```
 
16
  - Multi-Linguality: It can support more than 100 working languages.
17
  - Multi-Granularity: It is able to process inputs of different granularities, spanning from short sentences to long documents of up to 8192 tokens.
18
 
 
 
 
 
 
 
 
 
 
19
 
20
  ## News:
21
+ - 2024/3/8: **Thanks for the [experimental results](https://towardsdatascience.com/openai-vs-open-source-multilingual-embedding-models-e5ccb7c90f05) from @[Yannael](https://huggingface.co/Yannael). In this benchmark, BGE-M3 achieves top performance in both English and other languages, surpassing models such as OpenAI.**
22
  - 2024/3/2: Release unified fine-tuning [example](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/unified_finetune) and [data](https://huggingface.co/datasets/Shitao/bge-m3-data)
23
  - 2024/2/6: We release the [MLDR](https://huggingface.co/datasets/Shitao/MLDR) (a long document retrieval dataset covering 13 languages) and [evaluation pipeline](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB/MLDR).
24
  - 2024/2/1: **Thanks for the excellent tool from Vespa.** You can easily use multiple modes of BGE-M3 following this [notebook](https://github.com/vespa-engine/pyvespa/blob/master/docs/sphinx/source/examples/mother-of-all-embedding-models-cloud.ipynb)
 
87
 
88
 
89
 
90
+ **5. Some suggestions for retrieval pipeline in RAG**
91
+ We recommend to use following pipeline: hybrid retrieval + re-ranking.
92
+ - Hybrid retrieval leverages the strengths of various methods, offering higher accuracy and stronger generalization capabilities.
93
+ A classic example: using both embedding retrieval and the BM25 algorithm.
94
+ Now, you can try to use BGE-M3, which supports both embedding and sparse retrieval.
95
+ This allows you to obtain token weights (similar to the BM25) without any additional cost when generate dense embeddings.
96
+ - As cross-encoder models, re-ranker demonstrates higher accuracy than bi-encoder embedding model.
97
+ Utilizing the re-ranking model (e.g., [bge-reranker](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/reranker), [cohere-reranker](https://txt.cohere.com/rerank/)) after retrieval can further filter the selected text.
98
+
99
 
100
 
101
  ## Usage
 
218
 
219
  ## Evaluation
220
 
221
+ ### Benchmarks from the open-source community
222
+ ![avatar](./imgs/others.webp)
223
+ The BGE-M3 model emerged as the top performer on this benchmark (OAI is short for OpenAI).
224
+ For more details, please refer to the [article](https://towardsdatascience.com/openai-vs-open-source-multilingual-embedding-models-e5ccb7c90f05) and [Github Repo](https://github.com/Yannael/multilingual-embeddings)
225
 
 
226
 
227
+ ### Our results
228
  - Multilingual (Miracl dataset)
229
 
230
  ![avatar](./imgs/miracl.jpg)
 
257
  ![avatar](./imgs/bm25.jpg)
258
 
259
 
260
+
261
  ## Training
262
  - Self-knowledge Distillation: combining multiple outputs from different
263
  retrieval modes as reward signal to enhance the performance of single mode(especially for sparse retrieval and multi-vec(colbert) retrival)
 
272
 
273
  ## Acknowledgement
274
 
275
+ Thanks to the authors of open-sourced datasets, including Miracl, MKQA, NarritiveQA, etc.
276
+ Thanks to the open-sourced libraries like [Tevatron](https://github.com/texttron/tevatron), [Pyserini](https://github.com/castorini/pyserini).
277
 
278
 
279
 
 
290
  archivePrefix={arXiv},
291
  primaryClass={cs.CL}
292
  }
293
+ ```