Shitao commited on
Commit
922064b
1 Parent(s): 787f6bc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -28
README.md CHANGED
@@ -2611,22 +2611,21 @@ pipeline_tag: sentence-similarity
2611
  <a href=#usage>Usage</a> |
2612
  <a href="#evaluation">Evaluation</a> |
2613
  <a href="#train">Train</a> |
2614
- <a href="#contact">Contact</a> |
2615
  <a href="#license">License</a>
2616
  <p>
2617
  </h4>
2618
 
2619
- More details please refer to our github: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding).
2620
 
2621
- [English](README.md) | [中文](README_zh.md)
2622
 
2623
  FlagEmbedding can map any text to a low-dimensional dense vector which can be used for tasks like retrieval, classification, clustering, or semantic search.
2624
- And it also can be used in vector database for LLMs.
2625
 
2626
  ************* 🌟**Updates**🌟 *************
2627
  - 08/05/2023: Release base-scale and small-scale models, **best performance among the models of the same size 🤗**
2628
- - 08/02/2023: Release `bge-large-*`(short for BAAI General Embedding) Models, **rank 1st on MTEB and C-MTEB benchmark!**
2629
- - 08/01/2023: We release the Chinese Massive Text Embedding Benchmark (**C-MTEB**), consisting of 31 test dataset.
2630
 
2631
 
2632
  ## Model List
@@ -2635,12 +2634,12 @@ And it also can be used in vector database for LLMs.
2635
 
2636
  | Model | Language | Description | query instruction for retrieval |
2637
  |:-------------------------------|:--------:| :--------:| :--------:|
2638
- | [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en) | English | **rank 1st** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
2639
- | [BAAI/bge-base-en](https://huggingface.co/BAAI/bge-base-en) | English | **rank 2nd** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
2640
  | [BAAI/bge-small-en](https://huggingface.co/BAAI/bge-small-en) | English | a small-scale model but with competitive performance | `Represent this sentence for searching relevant passages: ` |
2641
- | [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) | Chinese | **rank 1st** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/benchmark) benchmark | `为这个句子生成表示以用于检索相关文章:` |
2642
- | [BAAI/bge-large-zh-noinstruct](https://huggingface.co/BAAI/bge-large-zh-noinstruct) | Chinese | This model is trained without instruction, and **rank 2nd** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/benchmark) benchmark | |
2643
- | [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) | Chinese | a base-scale model but with competitive performance | `为这个句子生成表示以用于检索相关文章:` |
2644
  | [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) | Chinese | a small-scale model but with competitive performance | `为这个句子生成表示以用于检索相关文章:` |
2645
 
2646
 
@@ -2649,10 +2648,12 @@ And it also can be used in vector database for LLMs.
2649
 
2650
  * **Using FlagEmbedding**
2651
  ```
2652
- pip install flag_embedding
2653
  ```
 
 
2654
  ```python
2655
- from flag_embedding import FlagModel
2656
  sentences = ["样例数据-1", "样例数据-2"]
2657
  model = FlagModel('BAAI/bge-large-zh', query_instruction_for_retrieval="为这个句子生成表示以用于检索相关文章:")
2658
  embeddings = model.encode(sentences)
@@ -2686,7 +2687,7 @@ embeddings = model.encode(sentences, normalize_embeddings=True)
2686
  print(embeddings)
2687
  ```
2688
  For retrieval task,
2689
- each query should start with a instruction (instructions see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list)).
2690
  ```python
2691
  from sentence_transformers import SentenceTransformer
2692
  queries = ["手机开不了机怎么办?"]
@@ -2715,7 +2716,7 @@ model = AutoModel.from_pretrained('BAAI/bge-large-zh')
2715
 
2716
  # Tokenize sentences
2717
  encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
2718
- # for retrieval task, add a instruction to query
2719
  # encoded_input = tokenizer([instruction + q for q in queries], padding=True, truncation=True, return_tensors='pt')
2720
 
2721
  # Compute token embeddings
@@ -2731,7 +2732,7 @@ print("Sentence embeddings:", sentence_embeddings)
2731
 
2732
  ## Evaluation
2733
  `baai-general-embedding` models achieve **state-of-the-art performance on both MTEB and C-MTEB leaderboard!**
2734
- More details and evaluation scripts see [benchemark](benchmark/README.md).
2735
 
2736
  - **MTEB**:
2737
 
@@ -2760,7 +2761,7 @@ More details and evaluation scripts see [benchemark](benchmark/README.md).
2760
 
2761
  - **C-MTEB**:
2762
  We create a benchmark C-MTEB for Chinese text embedding which consists of 31 datasets from 6 tasks.
2763
- Please refer to [benchemark](benchmark/README.md) for a detailed introduction.
2764
 
2765
  | Model | Embedding dimension | Avg | Retrieval | STS | PairClassification | Classification | Reranking | Clustering |
2766
  |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
@@ -2777,18 +2778,17 @@ Please refer to [benchemark](benchmark/README.md) for a detailed introduction.
2777
 
2778
 
2779
 
2780
-
2781
  ## Train
2782
  This section will introduce the way we used to train the general embedding.
2783
- The training scripts are in [flag_embedding](https://github.com/FlagOpen/FlagEmbedding/tree/master/flag_embedding/baai_general_embedding/),
2784
- and we provide some examples to do [pre-train](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/pretrain/) and [fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune).
2785
 
2786
 
2787
  **1. RetroMAE Pre-train**
2788
  We pre-train the model following the method [retromae](https://github.com/staoxiao/RetroMAE),
2789
  which shows promising improvement in retrieval task ([paper](https://aclanthology.org/2022.emnlp-main.35.pdf)).
2790
  The pre-training was conducted on 24 A100(40G) GPUs with a batch size of 720.
2791
- In retromae, the mask ratio of the encoder and decoder are 0.3, and 0.5 respectively.
2792
  We used the AdamW optimizer and the learning rate is 2e-5.
2793
 
2794
  **Pre-training data**:
@@ -2812,26 +2812,25 @@ We trained our model on 48 A100(40G) GPUs with a large batch size of 32,768 (so
2812
  We used the AdamW optimizer and the learning rate is 1e-5.
2813
  The temperature for contrastive loss is 0.01.
2814
 
2815
- For the version with `*-instrcution`, we add instruction to the query for the retrieval task in the training.
2816
- For English, the instruction is `Represent this sentence for searching relevant passages: `;
2817
- For Chinese, the instruction is `为这个句子生成表示以用于检索相关文章:`.
2818
  In the evaluation, the instruction should be added for sentence to passages retrieval task, not be added for other tasks.
2819
 
2820
 
2821
- The finetune script is accessible in this repository: [flag_embedding](https://github.com/FlagOpen/FlagEmbedding/tree/master/flag_embedding/baai_general_embedding/README.md).
2822
  You can easily finetune your model with it.
2823
 
2824
  **Training data**:
2825
 
2826
  - For English, we collect 230M text pairs from [wikipedia](https://huggingface.co/datasets/wikipedia), [cc-net](https://github.com/facebookresearch/cc_net), and so on.
2827
 
2828
- - For Chinese, we collect 120M text pairs from [wudao](https://github.com/BAAI-WuDao/Data), [simclue](https://github.com/CLUEbenchmark/SimCLUE) and so on.
2829
 
2830
  **The data collection is to be released in the future.**
2831
 
2832
  We will continually update the embedding models and training codes,
2833
  hoping to promote the development of the embedding model community.
2834
 
2835
-
2836
  ## License
2837
- FlagEmbedding is licensed under [MIT License](). The released models can be used for commercial purposes free of charge.
 
2611
  <a href=#usage>Usage</a> |
2612
  <a href="#evaluation">Evaluation</a> |
2613
  <a href="#train">Train</a> |
 
2614
  <a href="#license">License</a>
2615
  <p>
2616
  </h4>
2617
 
2618
+ For more details please refer to our GitHub repo: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding).
2619
 
2620
+ [English](README.md) | [中文](https://github.com/FlagOpen/FlagEmbedding/blob/master/README_zh.md)
2621
 
2622
  FlagEmbedding can map any text to a low-dimensional dense vector which can be used for tasks like retrieval, classification, clustering, or semantic search.
2623
+ And it also can be used in vector databases for LLMs.
2624
 
2625
  ************* 🌟**Updates**🌟 *************
2626
  - 08/05/2023: Release base-scale and small-scale models, **best performance among the models of the same size 🤗**
2627
+ - 08/02/2023: Release `bge-large-*`(short for BAAI General Embedding) Models, **rank 1st on MTEB and C-MTEB benchmark!** :tada: :tada:
2628
+ - 08/01/2023: We release the [Chinese Massive Text Embedding Benchmark](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB) (**C-MTEB**), consisting of 31 test dataset.
2629
 
2630
 
2631
  ## Model List
 
2634
 
2635
  | Model | Language | Description | query instruction for retrieval |
2636
  |:-------------------------------|:--------:| :--------:| :--------:|
2637
+ | [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en) | English | :trophy: rank **1st** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
2638
+ | [BAAI/bge-base-en](https://huggingface.co/BAAI/bge-base-en) | English | rank **2nd** in [MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard | `Represent this sentence for searching relevant passages: ` |
2639
  | [BAAI/bge-small-en](https://huggingface.co/BAAI/bge-small-en) | English | a small-scale model but with competitive performance | `Represent this sentence for searching relevant passages: ` |
2640
+ | [BAAI/bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) | Chinese | :trophy: rank **1st** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB) benchmark | `为这个句子生成表示以用于检索相关文章:` |
2641
+ | [BAAI/bge-large-zh-noinstruct](https://huggingface.co/BAAI/bge-large-zh-noinstruct) | Chinese | This model is trained without instruction, and rank **2nd** in [C-MTEB](https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB) benchmark | |
2642
+ | [BAAI/bge-base-zh](https://huggingface.co/BAAI/bge-base-zh) | Chinese | a base-scale model but has similar ability with `bge-large-zh` | `为这个句子生成表示以用于检索相关文章:` |
2643
  | [BAAI/bge-small-zh](https://huggingface.co/BAAI/bge-small-zh) | Chinese | a small-scale model but with competitive performance | `为这个句子生成表示以用于检索相关文章:` |
2644
 
2645
 
 
2648
 
2649
  * **Using FlagEmbedding**
2650
  ```
2651
+ pip install FlagEmbedding
2652
  ```
2653
+ See [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md) for more methods to install FlagEmbedding.
2654
+
2655
  ```python
2656
+ from FlagEmbedding import FlagModel
2657
  sentences = ["样例数据-1", "样例数据-2"]
2658
  model = FlagModel('BAAI/bge-large-zh', query_instruction_for_retrieval="为这个句子生成表示以用于检索相关文章:")
2659
  embeddings = model.encode(sentences)
 
2687
  print(embeddings)
2688
  ```
2689
  For retrieval task,
2690
+ each query should start with an instruction (instructions see [Model List](https://github.com/FlagOpen/FlagEmbedding/tree/master#model-list)).
2691
  ```python
2692
  from sentence_transformers import SentenceTransformer
2693
  queries = ["手机开不了机怎么办?"]
 
2716
 
2717
  # Tokenize sentences
2718
  encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
2719
+ # for retrieval task, add an instruction to query
2720
  # encoded_input = tokenizer([instruction + q for q in queries], padding=True, truncation=True, return_tensors='pt')
2721
 
2722
  # Compute token embeddings
 
2732
 
2733
  ## Evaluation
2734
  `baai-general-embedding` models achieve **state-of-the-art performance on both MTEB and C-MTEB leaderboard!**
2735
+ More details and evaluation tools see our [scripts](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md).
2736
 
2737
  - **MTEB**:
2738
 
 
2761
 
2762
  - **C-MTEB**:
2763
  We create a benchmark C-MTEB for Chinese text embedding which consists of 31 datasets from 6 tasks.
2764
+ Please refer to [C_MTEB](https://github.com/FlagOpen/FlagEmbedding/blob/master/C_MTEB/README.md) for a detailed introduction.
2765
 
2766
  | Model | Embedding dimension | Avg | Retrieval | STS | PairClassification | Classification | Reranking | Clustering |
2767
  |:-------------------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
 
2778
 
2779
 
2780
 
 
2781
  ## Train
2782
  This section will introduce the way we used to train the general embedding.
2783
+ The training scripts are in [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md),
2784
+ and we provide some examples to do [pre-train](https://github.com/FlagOpen/FlagEmbedding/blob/master/examples/pretrain/README.md) and [fine-tune](https://github.com/FlagOpen/FlagEmbedding/blob/master/examples/finetune/README.md).
2785
 
2786
 
2787
  **1. RetroMAE Pre-train**
2788
  We pre-train the model following the method [retromae](https://github.com/staoxiao/RetroMAE),
2789
  which shows promising improvement in retrieval task ([paper](https://aclanthology.org/2022.emnlp-main.35.pdf)).
2790
  The pre-training was conducted on 24 A100(40G) GPUs with a batch size of 720.
2791
+ In retromae, the mask ratio of encoder and decoder are 0.3, and 0.5 respectively.
2792
  We used the AdamW optimizer and the learning rate is 2e-5.
2793
 
2794
  **Pre-training data**:
 
2812
  We used the AdamW optimizer and the learning rate is 1e-5.
2813
  The temperature for contrastive loss is 0.01.
2814
 
2815
+ For the version with `*-instrcution`, we add instruction to the query for retrieval task in the training.
2816
+ For english, the instruction is `Represent this sentence for searching relevant passages: `;
2817
+ For chinese, the instruction is `为这个句子生成表示以用于检索相关文章:`.
2818
  In the evaluation, the instruction should be added for sentence to passages retrieval task, not be added for other tasks.
2819
 
2820
 
2821
+ The finetune script is accessible in this repository: [FlagEmbedding](https://github.com/FlagOpen/FlagEmbedding/blob/master/FlagEmbedding/baai_general_embedding/README.md).
2822
  You can easily finetune your model with it.
2823
 
2824
  **Training data**:
2825
 
2826
  - For English, we collect 230M text pairs from [wikipedia](https://huggingface.co/datasets/wikipedia), [cc-net](https://github.com/facebookresearch/cc_net), and so on.
2827
 
2828
+ - For chinese, we collect 120M text pairs from [wudao](https://github.com/BAAI-WuDao/Data), [simclue](https://github.com/CLUEbenchmark/SimCLUE) and so on.
2829
 
2830
  **The data collection is to be released in the future.**
2831
 
2832
  We will continually update the embedding models and training codes,
2833
  hoping to promote the development of the embedding model community.
2834
 
 
2835
  ## License
2836
+ FlagEmbedding is licensed under [MIT License](https://github.com/FlagOpen/FlagEmbedding/blob/master/LICENSE). The released models can be used for commercial purposes free of charge.