Shitao commited on
Commit
db1bfad
1 Parent(s): 5f074dc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -34
README.md CHANGED
@@ -2614,22 +2614,21 @@ pipeline_tag: sentence-similarity
2614
  <a href=#usage>Usage</a> |
2615
  <a href="#evaluation">Evaluation</a> |
2616
  <a href="#train">Train</a> |
2617
- <a href="#contact">Contact</a> |
2618
  <a href="#license">License</a>
2619
  <p>
2620
  </h4>
2621
 
2622
- More details please refer to our github: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding).
2623
 
2624
- [English](README.md) | [中文](README_zh.md)
2625
 
2626
  FlagEmbedding can map any text to a low-dimensional dense vector which can be used for tasks like retrieval, classification, clustering, or semantic search.
2627
- And it also can be used in vector database for LLMs.
2628
 
2629
  ************* 🌟**Updates**🌟 *************
2630
  - 08/05/2023: Release base-scale and small-scale models, **best performance among the models of the same size 🤗**
2631
- - 08/02/2023: Release `bge-large-*`(short for BAAI General Embedding) Models, **rank 1st on MTEB and C-MTEB benchmark!**
2632
- - 08/01/2023: We release the Chinese Massive Text Embedding Benchmark (**C-MTEB**), consisting of 31 test dataset.
2633
 
2634
 
2635
  ## Model List
@@ -2638,12 +2637,12 @@ And it also can be used in vector database for LLMs.
2638
 
2639
  | Model | Language | Description | query instruction for retrieval |
2640
  |:-------------------------------|:--------:| :--------:| :--------:|
2641
- | [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en) | English | **rank 1st** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
2642
- | [BAAI/bge-base-en](https://huggingface.co/BAAI/bge-base-en) | English | **rank 2nd** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
2643
  | [BAAI/bge-small-en](https://huggingface.co/BAAI/bge-small-en) | English | a small-scale model but with competitive performance | `Represent this sentence for searching relevant passages: ` |
2644
- | [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) | Chinese | **rank 1st** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/benchmark) benchmark | `为这个句子生成表示以用于检索相关文章:` |
2645
- | [BAAI/bge-large-zh-noinstruct](https://huggingface.co/BAAI/bge-large-zh-noinstruct) | Chinese | This model is trained without instruction, and **rank 2nd** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/benchmark) benchmark | |
2646
- | [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) | Chinese | a base-scale model but with competitive performance | `为这个句子生成表示以用于检索相关文章:` |
2647
  | [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) | Chinese | a small-scale model but with competitive performance | `为这个句子生成表示以用于检索相关文章:` |
2648
 
2649
 
@@ -2652,10 +2651,12 @@ And it also can be used in vector database for LLMs.
2652
 
2653
  * **Using FlagEmbedding**
2654
  ```
2655
- pip install flag_embedding
2656
  ```
 
 
2657
  ```python
2658
- from flag_embedding import FlagModel
2659
  sentences = ["样例数据-1", "样例数据-2"]
2660
  model = FlagModel('BAAI/bge-large-zh', query_instruction_for_retrieval="为这个句子生成表示以用于检索相关文章:")
2661
  embeddings = model.encode(sentences)
@@ -2689,7 +2690,7 @@ embeddings = model.encode(sentences, normalize_embeddings=True)
2689
  print(embeddings)
2690
  ```
2691
  For retrieval task,
2692
- each query should start with a instruction (instructions see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list)).
2693
  ```python
2694
  from sentence_transformers import SentenceTransformer
2695
  queries = ["手机开不了机怎么办?"]
@@ -2718,7 +2719,7 @@ model = AutoModel.from_pretrained('BAAI/bge-large-zh')
2718
 
2719
  # Tokenize sentences
2720
  encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
2721
- # for retrieval task, add a instruction to query
2722
  # encoded_input = tokenizer([instruction + q for q in queries], padding=True, truncation=True, return_tensors='pt')
2723
 
2724
  # Compute token embeddings
@@ -2734,7 +2735,7 @@ print("Sentence embeddings:", sentence_embeddings)
2734
 
2735
  ## Evaluation
2736
  `baai-general-embedding` models achieve **state-of-the-art performance on both MTEB and C-MTEB leaderboard!**
2737
- More details and evaluation scripts see [benchemark](benchmark/README.md).
2738
 
2739
  - **MTEB**:
2740
 
@@ -2763,7 +2764,7 @@ More details and evaluation scripts see [benchemark](benchmark/README.md).
2763
 
2764
  - **C-MTEB**:
2765
  We create a benchmark C-MTEB for Chinese text embedding which consists of 31 datasets from 6 tasks.
2766
- Please refer to [benchemark](benchmark/README.md) for a detailed introduction.
2767
 
2768
  | Model | Embedding dimension | Avg | Retrieval | STS | PairClassification | Classification | Reranking | Clustering |
2769
  |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
@@ -2780,18 +2781,17 @@ Please refer to [benchemark](benchmark/README.md) for a detailed introduction.
2780
 
2781
 
2782
 
2783
-
2784
  ## Train
2785
  This section will introduce the way we used to train the general embedding.
2786
- The training scripts are in [flag_embedding](https://github.com/FlagOpen/FlagEmbedding/tree/master/flag_embedding/baai_general_embedding/),
2787
- and we provide some examples to do [pre-train](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/pretrain/) and [fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune).
2788
 
2789
 
2790
  **1. RetroMAE Pre-train**
2791
  We pre-train the model following the method [retromae](https://github.com/staoxiao/RetroMAE),
2792
  which shows promising improvement in retrieval task ([paper](https://aclanthology.org/2022.emnlp-main.35.pdf)).
2793
  The pre-training was conducted on 24 A100(40G) GPUs with a batch size of 720.
2794
- In retromae, the mask ratio of the encoder and decoder are 0.3, and 0.5 respectively.
2795
  We used the AdamW optimizer and the learning rate is 2e-5.
2796
 
2797
  **Pre-training data**:
@@ -2815,32 +2815,25 @@ We trained our model on 48 A100(40G) GPUs with a large batch size of 32,768 (so
2815
  We used the AdamW optimizer and the learning rate is 1e-5.
2816
  The temperature for contrastive loss is 0.01.
2817
 
2818
- For the version with `*-instrcution`, we add instruction to the query for the retrieval task in the training.
2819
- For English, the instruction is `Represent this sentence for searching relevant passages: `;
2820
- For Chinese, the instruction is `为这个句子生成表示以用于检索相关文章:`.
2821
  In the evaluation, the instruction should be added for sentence to passages retrieval task, not be added for other tasks.
2822
 
2823
 
2824
- The finetune script is accessible in this repository: [flag_embedding](https://github.com/FlagOpen/FlagEmbedding/tree/master/flag_embedding/baai_general_embedding/README.md).
2825
  You can easily finetune your model with it.
2826
 
2827
  **Training data**:
2828
 
2829
  - For English, we collect 230M text pairs from [wikipedia](https://huggingface.co/datasets/wikipedia), [cc-net](https://github.com/facebookresearch/cc_net), and so on.
2830
 
2831
- - For Chinese, we collect 120M text pairs from [wudao](https://github.com/BAAI-WuDao/Data), [simclue](https://github.com/CLUEbenchmark/SimCLUE) and so on.
2832
 
2833
  **The data collection is to be released in the future.**
2834
 
2835
  We will continually update the embedding models and training codes,
2836
  hoping to promote the development of the embedding model community.
2837
 
2838
-
2839
  ## License
2840
- FlagEmbedding is licensed under [MIT License](). The released models can be used for commercial purposes free of charge.
2841
-
2842
-
2843
-
2844
-
2845
-
2846
-
 
2614
  <a href=#usage>Usage</a> |
2615
  <a href="#evaluation">Evaluation</a> |
2616
  <a href="#train">Train</a> |
 
2617
  <a href="#license">License</a>
2618
  <p>
2619
  </h4>
2620
 
2621
+ For more details please refer to our GitHub repo: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding).
2622
 
2623
+ [English](README.md) | [中文](https://github.com/FlagOpen/FlagEmbedding/blob/master/README_zh.md)
2624
 
2625
  FlagEmbedding can map any text to a low-dimensional dense vector which can be used for tasks like retrieval, classification, clustering, or semantic search.
2626
+ And it also can be used in vector databases for LLMs.
2627
 
2628
  ************* 🌟**Updates**🌟 *************
2629
  - 08/05/2023: Release base-scale and small-scale models, **best performance among the models of the same size 🤗**
2630
+ - 08/02/2023: Release `bge-large-*`(short for BAAI General Embedding) Models, **rank 1st on MTEB and C-MTEB benchmark!** :tada: :tada:
2631
+ - 08/01/2023: We release the [Chinese Massive Text Embedding Benchmark](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB) (**C-MTEB**), consisting of 31 test dataset.
2632
 
2633
 
2634
  ## Model List
 
2637
 
2638
  | Model | Language | Description | query instruction for retrieval |
2639
  |:-------------------------------|:--------:| :--------:| :--------:|
2640
+ | [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en) | English | :trophy: rank **1st** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
2641
+ | [BAAI/bge-base-en](https://huggingface.co/BAAI/bge-base-en) | English | rank **2nd** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
2642
  | [BAAI/bge-small-en](https://huggingface.co/BAAI/bge-small-en) | English | a small-scale model but with competitive performance | `Represent this sentence for searching relevant passages: ` |
2643
+ | [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) | Chinese | :trophy: rank **1st** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB) benchmark | `为这个句子生成表示以用于检索相关文章:` |
2644
+ | [BAAI/bge-large-zh-noinstruct](https://huggingface.co/BAAI/bge-large-zh-noinstruct) | Chinese | This model is trained without instruction, and rank **2nd** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB) benchmark | |
2645
+ | [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) | Chinese | a base-scale model but has similar ability with `bge-large-zh` | `为这个句子生成表示以用于检索相关文章:` |
2646
  | [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) | Chinese | a small-scale model but with competitive performance | `为这个句子生成表示以用于检索相关文章:` |
2647
 
2648
 
 
2651
 
2652
  * **Using FlagEmbedding**
2653
  ```
2654
+ pip install FlagEmbedding
2655
  ```
2656
+ See [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md) for more methods to install FlagEmbedding.
2657
+
2658
  ```python
2659
+ from FlagEmbedding import FlagModel
2660
  sentences = ["样例数据-1", "样例数据-2"]
2661
  model = FlagModel('BAAI/bge-large-zh', query_instruction_for_retrieval="为这个句子生成表示以用于检索相关文章:")
2662
  embeddings = model.encode(sentences)
 
2690
  print(embeddings)
2691
  ```
2692
  For retrieval task,
2693
+ each query should start with an instruction (instructions see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list)).
2694
  ```python
2695
  from sentence_transformers import SentenceTransformer
2696
  queries = ["手机开不了机怎么办?"]
 
2719
 
2720
  # Tokenize sentences
2721
  encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
2722
+ # for retrieval task, add an instruction to query
2723
  # encoded_input = tokenizer([instruction + q for q in queries], padding=True, truncation=True, return_tensors='pt')
2724
 
2725
  # Compute token embeddings
 
2735
 
2736
  ## Evaluation
2737
  `baai-general-embedding` models achieve **state-of-the-art performance on both MTEB and C-MTEB leaderboard!**
2738
+ More details and evaluation tools see our [scripts](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md).
2739
 
2740
  - **MTEB**:
2741
 
 
2764
 
2765
  - **C-MTEB**:
2766
  We create a benchmark C-MTEB for Chinese text embedding which consists of 31 datasets from 6 tasks.
2767
+ Please refer to [C_MTEB](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md) for a detailed introduction.
2768
 
2769
  | Model | Embedding dimension | Avg | Retrieval | STS | PairClassification | Classification | Reranking | Clustering |
2770
  |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
 
2781
 
2782
 
2783
 
 
2784
  ## Train
2785
  This section will introduce the way we used to train the general embedding.
2786
+ The training scripts are in [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md),
2787
+ and we provide some examples to do [pre-train](https://github.com/FlagOpen/FlagEmbedding/blob/master/examples/pretrain/README.md) and [fine-tune](https://github.com/FlagOpen/FlagEmbedding/blob/master/examples/finetune/README.md).
2788
 
2789
 
2790
  **1. RetroMAE Pre-train**
2791
  We pre-train the model following the method [retromae](https://github.com/staoxiao/RetroMAE),
2792
  which shows promising improvement in retrieval task ([paper](https://aclanthology.org/2022.emnlp-main.35.pdf)).
2793
  The pre-training was conducted on 24 A100(40G) GPUs with a batch size of 720.
2794
+ In retromae, the mask ratio of encoder and decoder are 0.3, and 0.5 respectively.
2795
  We used the AdamW optimizer and the learning rate is 2e-5.
2796
 
2797
  **Pre-training data**:
 
2815
  We used the AdamW optimizer and the learning rate is 1e-5.
2816
  The temperature for contrastive loss is 0.01.
2817
 
2818
+ For the version with `*-instrcution`, we add instruction to the query for retrieval task in the training.
2819
+ For english, the instruction is `Represent this sentence for searching relevant passages: `;
2820
+ For chinese, the instruction is `为这个句子生成表示以用于检索相关文章:`.
2821
  In the evaluation, the instruction should be added for sentence to passages retrieval task, not be added for other tasks.
2822
 
2823
 
2824
+ The finetune script is accessible in this repository: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md).
2825
  You can easily finetune your model with it.
2826
 
2827
  **Training data**:
2828
 
2829
  - For English, we collect 230M text pairs from [wikipedia](https://huggingface.co/datasets/wikipedia), [cc-net](https://github.com/facebookresearch/cc_net), and so on.
2830
 
2831
+ - For chinese, we collect 120M text pairs from [wudao](https://github.com/BAAI-WuDao/Data), [simclue](https://github.com/CLUEbenchmark/SimCLUE) and so on.
2832
 
2833
  **The data collection is to be released in the future.**
2834
 
2835
  We will continually update the embedding models and training codes,
2836
  hoping to promote the development of the embedding model community.
2837
 
 
2838
  ## License
2839
+ FlagEmbedding is licensed under [MIT License](https://github.com/FlagOpen/FlagEmbedding/blob/master/LICENSE). The released models can be used for commercial purposes free of charge.