Model: Areeb-02/bge-small-en-MultiplrRankingLoss-30-Rag-paper-dataset
Author: Areeb-02
Pipeline: sentence-similarity
Library: sentence-transformers
Tags: sentence-transformers, safetensors, bert, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:1010, loss:MultipleNegativesRankingLoss, arxiv:1908.10084, arxiv:1705.00652, base_model:BAAI/bge-small-en, base_model:finetune:BAAI/bge-small-en, model-index, text-embeddings-inference, endpoints_compatible, region:us
Base models (finetune): id: BAAI/bge-small-en
Card metadata: base_model: BAAI/bge-small-en
library_name: sentence-transformers
metrics: cosine_accuracy@1, cosine_accuracy@3, cosine_accuracy@5, cosine_accuracy@10, cosine_precision@1, cosine_precision@3, cosine_precision@5, cosine_precision@10, cosine_recall@1, cosine_recall@3, cosine_recall@5, cosine_recall@10, cosine_ndcg@10, cosine_mrr@10, cosine_map@100, dot_accuracy@1, dot_accuracy@3, dot_accuracy@5, dot_accuracy@10, dot_precision@1, dot_precision@3, dot_precision@5, dot_precision@10, dot_recall@1, dot_recall@3, dot_recall@5, dot_recall@10, dot_ndcg@10, dot_mrr@10, dot_map@100
pipeline_tag: sentence-similarity
tags: sentence-transformers, sentence-similarity, feature-extraction, generated_from_trainer, dataset_size:1010, loss:MultipleNegativesRankingLoss
widget: source_sentence: How does Prompt-RAG differ from traditional vector embedding-based methodologies?
sentences: Prompt-RAG differs from traditional vector embedding-based methodologies by adopting a more direct and flexible retrieval process based on natural language prompts, eliminating the need for a vector database or an algorithm for indexing and selecting vectors., By introducing a pre-aligned phrase prior to the standard SFT stage, LLMs are guided to concentrate on the aligned knowledge, thereby unlocking their internal alignment abilities and improving their performance., The accuracy of GPT 3.5 on 2500 overall TeleQnA questions related to 3GPP documents is 60.1, while the accuracy of GPT 3.5 + Telco-RAG is 6.9 points higher., source_sentence: Explain the concept of in-context learning as described in the paper 'An explanation of in-context learning as implicit Bayesian inference'.
sentences: The main theme of the paper is that language models can learn to perform many tasks in a zero-shot setting, without any explicit supervision., In-context learning, as explained in the paper, is a process where a language model uses the context provided in the input to make predictions or generate outputs without explicit training on the specific task. The paper argues that this process can be understood as an implicit form of Bayesian inference., The paper was presented in the 55th Annual Meeting of the Association for Computational Linguistics., source_sentence: What is the purpose of the survey conducted by Huang et al. (2023)?
sentences: The purpose of the survey conducted by Huang et al. (2023) is to provide a comprehensive overview of hallucination in large language models, including its principles, taxonomy, challenges, and open questions., The study of Human and American Translation Learning contributes to language development by understanding the cognitive processes involved in translating between languages, which can lead to improved teaching methods and translation technology., Using profile data, triplet examples are constructed in the format of (𝑥𝑖, 𝑥 𝑖−, 𝑥 𝑖+). The anchor example 𝑥𝑖 is constructed as the combination of the content 𝑐𝑖 and the corresponding label 𝑙𝑖., source_sentence: Who is the first author of the paper and what is their last name?
sentences: The key findings are that Vul-RAG achieves the highest accuracy and pairwise accuracy among all baselines, substantially outperforming the best baseline LLMAO. It also achieves the best trade-off between recall and precision., The first author of the paper is Nandan Thakur. Their last name is Thakur., The paper was presented at the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)., source_sentence: Compare the top-5 retrieval accuracy of BM25 + MQ and SERM + BF for the NQ Dataset and HotpotQA.
sentences: For the NQ Dataset, SERM + BF has a top-5 retrieval accuracy of 88.22, which is significantly higher than BM25 + MQ's accuracy of 25.19. For HotpotQA, SERM + BF was not tested, but BM25 + MQ has a top-5 retrieval accuracy of 49.52., The paper was presented at the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval., The proof for Equation 5 progresses from Equation 20 to Equation 22 by applying the transformation motivated by Xie et al. [2021] and introducing the term p(R, x1:i−1|z) to the equation.
model-index: name: SentenceTransformer based on BAAI/bge-small-en
results: task: type: information-retrieval
name: Information Retrieval
dataset: name: Unknown
type: unknown
metrics: type: cosine_accuracy@1
value: 0.01782178217821782
name: Cosine Accuracy@1
verified: no, type: cosine_accuracy@3
value: 0.04356435643564356
name: Cosine Accuracy@3
verified: no, type: cosine_accuracy@5
value: 0.06534653465346535
name: Cosine Accuracy@5
verified: no, type: cosine_accuracy@10
value: 0.12475247524752475
name: Cosine Accuracy@10
verified: no, type: cosine_precision@1
value: 0.01782178217821782
name: Cosine Precision@1
verified: no, type: cosine_precision@3
value: 0.015841584158415842
name: Cosine Precision@3
verified: no, type: cosine_precision@5
value: 0.016039603960396043
name: Cosine Precision@5
verified: no, type: cosine_precision@10
value: 0.015841584158415842
name: Cosine Precision@10
verified: no, type: cosine_recall@1
value: 1.839902956558168e-05
name: Cosine Recall@1
verified: no, type: cosine_recall@3
value: 4.498766525563503e-05
name: Cosine Recall@3
verified: no, type: cosine_recall@5
value: 7.262670252004521e-05
name: Cosine Recall@5
verified: no, type: cosine_recall@10
value: 0.00015079859335392304
name: Cosine Recall@10
verified: no, type: cosine_ndcg@10
value: 0.016300874257683427
name: Cosine Ndcg@10
verified: no, type: cosine_mrr@10
value: 0.04234598459845988
name: Cosine Mrr@10
verified: no, type: cosine_map@100
value: 0.0018766020656866668
name: Cosine Map@100
verified: no, type: dot_accuracy@1
value: 0.01782178217821782
name: Dot Accuracy@1
verified: no, type: dot_accuracy@3
value: 0.04356435643564356
name: Dot Accuracy@3
verified: no, type: dot_accuracy@5
value: 0.06534653465346535
name: Dot Accuracy@5
verified: no, type: dot_accuracy@10
value: 0.12475247524752475
name: Dot Accuracy@10
verified: no, type: dot_precision@1
value: 0.01782178217821782
name: Dot Precision@1
verified: no, type: dot_precision@3
value: 0.015841584158415842
name: Dot Precision@3
verified: no, type: dot_precision@5
value: 0.016039603960396043
name: Dot Precision@5
verified: no, type: dot_precision@10
value: 0.015841584158415842
name: Dot Precision@10
verified: no, type: dot_recall@1
value: 1.839902956558168e-05
name: Dot Recall@1
verified: no, type: dot_recall@3
value: 4.498766525563503e-05
name: Dot Recall@3
verified: no, type: dot_recall@5
value: 7.262670252004521e-05
name: Dot Recall@5
verified: no, type: dot_recall@10
value: 0.00015079859335392304
name: Dot Recall@10
verified: no, type: dot_ndcg@10
value: 0.016300874257683427
name: Dot Ndcg@10
verified: no, type: dot_mrr@10
value: 0.04234598459845988
name: Dot Mrr@10
verified: no, type: dot_map@100
value: 0.0018766020656866668
name: Dot Map@100
verified: no, type: cosine_accuracy@1
value: 0.019801980198019802
name: Cosine Accuracy@1
verified: no, type: cosine_accuracy@3
value: 0.040594059405940595
name: Cosine Accuracy@3
verified: no, type: cosine_accuracy@5
value: 0.06534653465346535
name: Cosine Accuracy@5
verified: no, type: cosine_accuracy@10
value: 0.12673267326732673
name: Cosine Accuracy@10
verified: no, type: cosine_precision@1
value: 0.019801980198019802
name: Cosine Precision@1
verified: no, type: cosine_precision@3
value: 0.01485148514851485
name: Cosine Precision@3
verified: no, type: cosine_precision@5
value: 0.014851485148514853
name: Cosine Precision@5
verified: no, type: cosine_precision@10
value: 0.016831683168316833
name: Cosine Precision@10
verified: no, type: cosine_recall@1
value: 1.9670857914229207e-05
name: Cosine Recall@1
verified: no, type: cosine_recall@3
value: 3.554268094376118e-05
name: Cosine Recall@3
verified: no, type: cosine_recall@5
value: 6.67664165823309e-05
name: Cosine Recall@5
verified: no, type: cosine_recall@10
value: 0.0001670844654494185
name: Cosine Recall@10
verified: no, type: cosine_ndcg@10
value: 0.01679069935920913
name: Cosine Ndcg@10
verified: no, type: cosine_mrr@10
value: 0.04252396668238257
name: Cosine Mrr@10
verified: no, type: cosine_map@100
value: 0.002057887757857092
name: Cosine Map@100
verified: no, type: dot_accuracy@1
value: 0.019801980198019802
name: Dot Accuracy@1
verified: no, type: dot_accuracy@3
value: 0.040594059405940595
name: Dot Accuracy@3
verified: no, type: dot_accuracy@5
value: 0.06534653465346535
name: Dot Accuracy@5
verified: no, type: dot_accuracy@10
value: 0.12673267326732673
name: Dot Accuracy@10
verified: no, type: dot_precision@1
value: 0.019801980198019802
name: Dot Precision@1
verified: no, type: dot_precision@3
value: 0.01485148514851485
name: Dot Precision@3
verified: no, type: dot_precision@5
value: 0.014851485148514853
name: Dot Precision@5
verified: no, type: dot_precision@10
value: 0.016831683168316833
name: Dot Precision@10
verified: no, type: dot_recall@1
value: 1.9670857914229207e-05
name: Dot Recall@1
verified: no, type: dot_recall@3
value: 3.554268094376118e-05
name: Dot Recall@3
verified: no, type: dot_recall@5
value: 6.67664165823309e-05
name: Dot Recall@5
verified: no, type: dot_recall@10
value: 0.0001670844654494185
name: Dot Recall@10
verified: no, type: dot_ndcg@10
value: 0.01679069935920913
name: Dot Ndcg@10
verified: no, type: dot_mrr@10
value: 0.04252396668238257
name: Dot Mrr@10
verified: no, type: dot_map@100
value: 0.002057887757857092
name: Dot Map@100
verified: no, type: cosine_accuracy@1
value: 0.01881188118811881
name: Cosine Accuracy@1
verified: no, type: cosine_accuracy@3
value: 0.03762376237623762
name: Cosine Accuracy@3
verified: no, type: cosine_accuracy@5
value: 0.06435643564356436
name: Cosine Accuracy@5
verified: no, type: cosine_accuracy@10
value: 0.1306930693069307
name: Cosine Accuracy@10
verified: no, type: cosine_precision@1
value: 0.01881188118811881
name: Cosine Precision@1
verified: no, type: cosine_precision@3
value: 0.013861386138613862
name: Cosine Precision@3
verified: no, type: cosine_precision@5
value: 0.015841584158415842
name: Cosine Precision@5
verified: no, type: cosine_precision@10
value: 0.01722772277227723
name: Cosine Precision@10
verified: no, type: cosine_recall@1
value: 1.8836739119030395e-05
name: Cosine Recall@1
verified: no, type: cosine_recall@3
value: 3.852282962664283e-05
name: Cosine Recall@3
verified: no, type: cosine_recall@5
value: 7.907232140954174e-05
name: Cosine Recall@5
verified: no, type: cosine_recall@10
value: 0.00018073758516299118
name: Cosine Recall@10
verified: no, type: cosine_ndcg@10
value: 0.01704492626324548
name: Cosine Ndcg@10
verified: no, type: cosine_mrr@10
value: 0.04188786735816444
name: Cosine Mrr@10
verified: no, type: cosine_map@100
value: 0.002251865468050825
name: Cosine Map@100
verified: no, type: dot_accuracy@1
value: 0.01881188118811881
name: Dot Accuracy@1
verified: no, type: dot_accuracy@3
value: 0.03762376237623762
name: Dot Accuracy@3
verified: no, type: dot_accuracy@5
value: 0.06435643564356436
name: Dot Accuracy@5
verified: no, type: dot_accuracy@10
value: 0.1306930693069307
name: Dot Accuracy@10
verified: no, type: dot_precision@1
value: 0.01881188118811881
name: Dot Precision@1
verified: no, type: dot_precision@3
value: 0.013861386138613862
name: Dot Precision@3
verified: no, type: dot_precision@5
value: 0.015841584158415842
name: Dot Precision@5
verified: no, type: dot_precision@10
value: 0.01722772277227723
name: Dot Precision@10
verified: no, type: dot_recall@1
value: 1.8836739119030395e-05
name: Dot Recall@1
verified: no, type: dot_recall@3
value: 3.852282962664283e-05
name: Dot Recall@3
verified: no, type: dot_recall@5
value: 7.907232140954174e-05
name: Dot Recall@5
verified: no, type: dot_recall@10
value: 0.00018073758516299118
name: Dot Recall@10
verified: no, type: dot_ndcg@10
value: 0.01704492626324548
name: Dot Ndcg@10
verified: no, type: dot_mrr@10
value: 0.04188786735816444
name: Dot Mrr@10
verified: no, type: dot_map@100
value: 0.002251865468050825
name: Dot Map@100
verified: no, type: cosine_accuracy@1
value: 0.01881188118811881
name: Cosine Accuracy@1
verified: no, type: cosine_accuracy@3
value: 0.03663366336633663
name: Cosine Accuracy@3
verified: no, type: cosine_accuracy@5
value: 0.06435643564356436
name: Cosine Accuracy@5
verified: no, type: cosine_accuracy@10
value: 0.1306930693069307
name: Cosine Accuracy@10
verified: no, type: cosine_precision@1
value: 0.01881188118811881
name: Cosine Precision@1
verified: no, type: cosine_precision@3
value: 0.013531353135313529
name: Cosine Precision@3
verified: no, type: cosine_precision@5
value: 0.015643564356435644
name: Cosine Precision@5
verified: no, type: cosine_precision@10
value: 0.01722772277227723
name: Cosine Precision@10
verified: no, type: cosine_recall@1
value: 1.8836739119030395e-05
name: Cosine Recall@1
verified: no, type: cosine_recall@3
value: 3.715905688573237e-05
name: Cosine Recall@3
verified: no, type: cosine_recall@5
value: 7.929088142504806e-05
name: Cosine Recall@5
verified: no, type: cosine_recall@10
value: 0.0001757722267344924
name: Cosine Recall@10
verified: no, type: cosine_ndcg@10
value: 0.01701867523723249
name: Cosine Ndcg@10
verified: no, type: cosine_mrr@10
value: 0.0418477919220494
name: Cosine Mrr@10
verified: no, type: cosine_map@100
value: 0.0022453604762727357
name: Cosine Map@100
verified: no, type: dot_accuracy@1
value: 0.01881188118811881
name: Dot Accuracy@1
verified: no, type: dot_accuracy@3
value: 0.03663366336633663
name: Dot Accuracy@3
verified: no, type: dot_accuracy@5
value: 0.06435643564356436
name: Dot Accuracy@5
verified: no, type: dot_accuracy@10
value: 0.1306930693069307
name: Dot Accuracy@10
verified: no, type: dot_precision@1
value: 0.01881188118811881
name: Dot Precision@1
verified: no, type: dot_precision@3
value: 0.013531353135313529
name: Dot Precision@3
verified: no, type: dot_precision@5
value: 0.015643564356435644
name: Dot Precision@5
verified: no, type: dot_precision@10
value: 0.01722772277227723
name: Dot Precision@10
verified: no, type: dot_recall@1
value: 1.8836739119030395e-05
name: Dot Recall@1
verified: no, type: dot_recall@3
value: 3.715905688573237e-05
name: Dot Recall@3
verified: no, type: dot_recall@5
value: 7.929088142504806e-05
name: Dot Recall@5
verified: no, type: dot_recall@10
value: 0.0001757722267344924
name: Dot Recall@10
verified: no, type: dot_ndcg@10
value: 0.01701867523723249
name: Dot Ndcg@10
verified: no, type: dot_mrr@10
value: 0.0418477919220494
name: Dot Mrr@10
verified: no, type: dot_map@100
value: 0.0022453604762727357
name: Dot Map@100
verified: no
Likes: 0 | All-time downloads: 8
Created: 2024-07-04 16:21:38