BAAI
/

bge-small-en

@@ -2762,7 +2762,7 @@ More details and evaluation scripts see [benchemark](benchmark/README.md).
 - **C-MTEB**:
-We create a benchmark C-MTEB for chinese text embedding which consists of  31 datasets from 6 tasks.
 Please refer to [benchemark](benchmark/README.md) for a detailed introduction.
 | Model | Embedding dimension | Avg | Retrieval | STS | PairClassification | Classification | Reranking | Clustering |
@@ -2783,15 +2783,15 @@ Please refer to [benchemark](benchmark/README.md) for a detailed introduction.
 ## Train
 This section will introduce the way we used to train the general embedding.
-The training scripts are in [flag_embedding](./flag_embedding/baai_general_embedding/README.md),
-and we provide some examples to do [pre-train](examples/pretrain/README.md) and [fine-tune](examples/finetune/README.md).
 **1. RetroMAE Pre-train**
 We pre-train the model following the method [retromae](https://github.com/staoxiao/RetroMAE),
 which shows promising improvement in retrieval task ([paper](https://aclanthology.org/2022.emnlp-main.35.pdf)).
 The pre-training was conducted on 24 A100(40G) GPUs with a batch size of 720.
-In retromae, the mask ratio of encoder and decoder are 0.3, 0.5 respectively.
 We used the AdamW optimizer and the learning rate is 2e-5.
 **Pre-training data**:
@@ -2808,49 +2808,36 @@ We used the AdamW optimizer and the learning rate is 2e-5.
 We fine-tune the model using a contrastive objective.
 The format of input data is a triple`(query, positive, negative)`.
 Besides the negative in the triple, we also adopt in-batch negatives strategy.
-We employ the cross-device negatives sharing method to sharing negatives among different GPUs,
 which can dramatically **increase the number of negatives**.
 We trained our model on 48 A100(40G) GPUs with a large batch size of 32,768 (so there are **65,535** negatives for each query in a batch).
 We used the AdamW optimizer and the learning rate is 1e-5.
 The temperature for contrastive loss is 0.01.
-For the version with `*-instrcution`, we add instruction to the query for retrieval task in the training.
-For english, the instruction is `Represent this sentence for searching relevant passages: `;
-For chinese, the instruction is `为这个句子生成表示以用于检索相关文章：`.
 In the evaluation, the instruction should be added for sentence to passages retrieval task, not be added for other tasks.
-The finetune script is accessible in this repository: [flag_embedding](./flag_embedding/baai_general_embedding/README.md).
 You can easily finetune your model with it.
 **Training data**:
 - For English, we collect 230M text pairs from [wikipedia](https://huggingface.co/datasets/wikipedia), [cc-net](https://github.com/facebookresearch/cc_net), and so on.
-- For chinese, we collect 120M text pairs from [wudao](https://github.com/BAAI-WuDao/Data), [simclue](https://github.com/CLUEbenchmark/SimCLUE) and so on.
 **The data collection is to be released in the future.**
-## Schedule
-- [x] Chinese Massive Text Embedding Benchmark
-- [x] release baai-general-embedding models
-- [x] release codes for training
-- [ ] Training Datasets
-- [ ] Multilingual model
-- [ ] ...
 We will continually update the embedding models and training codes,
 hoping to promote the development of the embedding model community.
-## Contact
-If you have any question or suggestion related to this project, feel free to open an issue or pull a request.
-You also can email Shitao Xiao(stxiao@baai.ac.cn) and Zheng Liu(liuzheng@baai.ac.cn).
 ## License
-FlagEmbedding is licensed under [MIT License](LICENSE). The released models can be used for commercial purposes free of charge.

 - **C-MTEB**:
+We create a benchmark C-MTEB for Chinese text embedding which consists of  31 datasets from 6 tasks.
 Please refer to [benchemark](benchmark/README.md) for a detailed introduction.
 | Model | Embedding dimension | Avg | Retrieval | STS | PairClassification | Classification | Reranking | Clustering |
 ## Train
 This section will introduce the way we used to train the general embedding.
+The training scripts are in [flag_embedding](https://github.com/FlagOpen/FlagEmbedding/tree/master/flag_embedding/baai_general_embedding/),
+and we provide some examples to do [pre-train](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/pretrain/) and [fine-tune](https://github.com/FlagOpen/FlagEmbedding/tree/master/examples/finetune).
 **1. RetroMAE Pre-train**
 We pre-train the model following the method [retromae](https://github.com/staoxiao/RetroMAE),
 which shows promising improvement in retrieval task ([paper](https://aclanthology.org/2022.emnlp-main.35.pdf)).
 The pre-training was conducted on 24 A100(40G) GPUs with a batch size of 720.
+In retromae, the mask ratio of the encoder and decoder are 0.3, and 0.5 respectively.
 We used the AdamW optimizer and the learning rate is 2e-5.
 **Pre-training data**:
 We fine-tune the model using a contrastive objective.
 The format of input data is a triple`(query, positive, negative)`.
 Besides the negative in the triple, we also adopt in-batch negatives strategy.
+We employ the cross-device negatives sharing method to share negatives among different GPUs,
 which can dramatically **increase the number of negatives**.
 We trained our model on 48 A100(40G) GPUs with a large batch size of 32,768 (so there are **65,535** negatives for each query in a batch).
 We used the AdamW optimizer and the learning rate is 1e-5.
 The temperature for contrastive loss is 0.01.
+For the version with `*-instrcution`, we add instruction to the query for the retrieval task in the training.
+For English, the instruction is `Represent this sentence for searching relevant passages: `;
+For Chinese, the instruction is `为这个句子生成表示以用于检索相关文章：`.
 In the evaluation, the instruction should be added for sentence to passages retrieval task, not be added for other tasks.
+The finetune script is accessible in this repository: [flag_embedding](https://github.com/FlagOpen/FlagEmbedding/tree/master/flag_embedding/baai_general_embedding/README.md).
 You can easily finetune your model with it.
 **Training data**:
 - For English, we collect 230M text pairs from [wikipedia](https://huggingface.co/datasets/wikipedia), [cc-net](https://github.com/facebookresearch/cc_net), and so on.
+- For Chinese, we collect 120M text pairs from [wudao](https://github.com/BAAI-WuDao/Data), [simclue](https://github.com/CLUEbenchmark/SimCLUE) and so on.
 **The data collection is to be released in the future.**
 We will continually update the embedding models and training codes,
 hoping to promote the development of the embedding model community.
 ## License
+FlagEmbedding is licensed under [MIT License](). The released models can be used for commercial purposes free of charge.