Update README.md

README.md

@@ -27,7 +27,7 @@ With the approval of the original author, this model is re-implemented with the
 - **Language(s) (NLP):** English
 - **License:** MIT License
 
-### Model Sources [optional]
+### Model Sources
 
 <!-- Provide the basic links for the model. -->
 
@@ -43,37 +43,18 @@ With the approval of the original author, this model is re-implemented with the
 
 <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
-[More Information Needed]
+This model can be used directly to retrieve documents from a large corpus using text queries. The retrieval usage can be found in the [PreFLMR implementation](https://github.com/LinWeizheDragon/FLMR).
 
-### Downstream Use [optional]
+### Downstream Use
 
 <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
 
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+This model can be used combined with language models to create a retrieval-augmented language model. 
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
+For details of training, indexing, and performing retrieval, please refer to [here](https://github.com/LinWeizheDragon/FLMR).
 
-[More Information Needed]
 
 ## Training Details
 
@@ -81,26 +62,13 @@ Use the code below to get started with the model.
 
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
+The model was trained on [MS MARCO Passage Ranking](https://github.com/microsoft/MSMARCO-Passage-Ranking)
 
+More details can be found in the [official ColBERT repository](https://github.com/stanford-futuredata/ColBERT?tab=readme-ov-file)
 
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+### Training Procedure
 
-[More Information Needed]
+The training details can be found in the [official ColBERT repository](https://github.com/stanford-futuredata/ColBERT?tab=readme-ov-file) and the [ColBERTv2 paper](https://arxiv.org/abs/2112.01488)
 
 ## Evaluation
 
@@ -110,92 +78,91 @@ Use the code below to get started with the model.
 
 #### Testing Data
 
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+The model is evaluated on a range of retrieval tasks, such as:
+- [MS MARCO Passage Ranking](https://github.com/microsoft/MSMARCO-Passage-Ranking)
+- [BEIR](https://arxiv.org/abs/2104.08663)
+- [LoTTE](https://github.com/stanford-futuredata/ColBERT/blob/main/LoTTE.md)
+- [OOD Wikipedia Open QA](https://arxiv.org/abs/2112.01488)
 
 ### Results
 
-[More Information Needed]
-
-#### Summary
-
+Find the evaluation results in the [ColBERTv2 paper](https://arxiv.org/abs/2112.01488)
 
 
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
+## Citation
 
 <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 
 **BibTeX:**
 
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
+Please cite the original papers:
+```
+@inproceedings{10.1145/3397271.3401075,
+author = {Khattab, Omar and Zaharia, Matei},
+title = {ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT},
+year = {2020},
+isbn = {9781450380164},
+publisher = {Association for Computing Machinery},
+address = {New York, NY, USA},
+url = {https://doi.org/10.1145/3397271.3401075},
+doi = {10.1145/3397271.3401075},
+abstract = {Recent progress in Natural Language Understanding (NLU) is driving fast-paced advances in Information Retrieval (IR), largely owed to fine-tuning deep language models (LMs) for document ranking. While remarkably effective, the ranking models based on these LMs increase computational cost by orders of magnitude over prior approaches, particularly as they must feed each query-document pair through a massive neural network to compute a single relevance score. To tackle this, we present ColBERT, a novel ranking model that adapts deep LMs (in particular, BERT) for efficient retrieval. ColBERT introduces a late interaction architecture that independently encodes the query and the document using BERT and then employs a cheap yet powerful interaction step that models their fine-grained similarity. By delaying and yet retaining this fine-granular interaction, ColBERT can leverage the expressiveness of deep LMs while simultaneously gaining the ability to pre-compute document representations offline, considerably speeding up query processing. Crucially, ColBERT's pruning-friendly interaction mechanism enables leveraging vector-similarity indexes for end-to-end retrieval directly from millions of documents. We extensively evaluate ColBERT using two recent passage search datasets. Results show that ColBERT's effectiveness is competitive with existing BERT-based models (and outperforms every non-BERT baseline), while executing two orders-of-magnitude faster and requiring up to four orders-of-magnitude fewer FLOPs per query.},
+booktitle = {Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval},
+pages = {39–48},
+numpages = {10},
+keywords = {bert, deep language models, efficiency, neural ir},
+location = {Virtual Event, China},
+series = {SIGIR '20}
+}
+
+@inproceedings{santhanam-etal-2022-colbertv2,
+    title = "{C}ol{BERT}v2: Effective and Efficient Retrieval via Lightweight Late Interaction",
+    author = "Santhanam, Keshav  and
+      Khattab, Omar  and
+      Saad-Falcon, Jon  and
+      Potts, Christopher  and
+      Zaharia, Matei",
+    editor = "Carpuat, Marine  and
+      de Marneffe, Marie-Catherine  and
+      Meza Ruiz, Ivan Vladimir",
+    booktitle = "Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
+    month = jul,
+    year = "2022",
+    address = "Seattle, United States",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2022.naacl-main.272",
+    doi = "10.18653/v1/2022.naacl-main.272",
+    pages = "3715--3734",
+    abstract = "Neural information retrieval (IR) has greatly advanced search and other knowledge-intensive language tasks. While many neural IR methods encode queries and documents into single-vector representations, late interaction models produce multi-vector representations at the granularity of each token and decompose relevance modeling into scalable token-level computations. This decomposition has been shown to make late interaction more effective, but it inflates the space footprint of these models by an order of magnitude. In this work, we introduce ColBERTv2, a retriever that couples an aggressive residual compression mechanism with a denoised supervision strategy to simultaneously improve the quality and space footprint of late interaction. We evaluate ColBERTv2 across a wide range of benchmarks, establishing state-of-the-art quality within and outside the training domain while reducing the space footprint of late interaction models by 6{--}10x.",
+}
+
+@inproceedings{10.1145/3511808.3557325,
+author = {Santhanam, Keshav and Khattab, Omar and Potts, Christopher and Zaharia, Matei},
+title = {PLAID: An Efficient Engine for Late Interaction Retrieval},
+year = {2022},
+isbn = {9781450392365},
+publisher = {Association for Computing Machinery},
+address = {New York, NY, USA},
+url = {https://doi.org/10.1145/3511808.3557325},
+doi = {10.1145/3511808.3557325},
+abstract = {Pre-trained language models are increasingly important components across multiple information retrieval (IR) paradigms. Late interaction, introduced with the ColBERT model and recently refined in ColBERTv2, is a popular paradigm that holds state-of-the-art status across many benchmarks. To dramatically speed up the search latency of late interaction, we introduce the Performance-optimized Late Interaction Driver (PLAID) engine. Without impacting quality, PLAID swiftly eliminates low-scoring passages using a novel centroid interaction mechanism that treats every passage as a lightweight bag of centroids. PLAID uses centroid interaction as well as centroid pruning, a mechanism for sparsifying the bag of centroids, within a highly-optimized engine to reduce late interaction search latency by up to 7x on a GPU and 45x on a CPU against vanilla ColBERTv2, while continuing to deliver state-of-the-art retrieval quality. This allows the PLAID engine with ColBERTv2 to achieve latency of tens of milliseconds on a GPU and tens or just few hundreds of milliseconds on a CPU at large scale, even at the largest scales we evaluate with 140M passages.},
+booktitle = {Proceedings of the 31st ACM International Conference on Information \& Knowledge Management},
+pages = {1747–1756},
+numpages = {10},
+keywords = {colbert, dynamic pruning, efficient search, late interaction, neural information retrieval},
+location = {Atlanta, GA, USA},
+series = {CIKM '22}
+}
+```
+
+
+If the implementation helps your research, please cite:
+```
+@article{Lin_Mei_Chen_Byrne_2024, 
+        title={PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers}, 
+        url={http://arxiv.org/abs/2402.08327}, 
+        number={arXiv:2402.08327}, 
+        publisher={arXiv}, 
+        author={Lin, Weizhe and Mei, Jingbiao and Chen, Jinghong and Byrne, Bill}, 
+        year={2024}}
+```