Note https://github.com/project-miracl/miracl?tab=readme-ov-file MTEB: https://github.com/embeddings-benchmark/mteb

Note https://aclanthology.org/2023.eacl-main.75/

Note Original "needle-in-haystack-test" for long context input - Passkey Retrieval More: https://arxiv.org/pdf/2402.13753.pdf

Note The hyperparameters, including temperature, top-k and top-p, of the LLMs’decoder determine the diversity of the generations. To evaluate the impact of those hyper parameters. We provide a sensitivity analysis in Figure 4. As observed, the performance is greatly influenced by temperature but shows little sensitivity to top-k. The performance of the consistency based methods (EigenScore and Lexical Similarity) drops significantly when the temperature is greater than 1.

Note As a substitute for G-Eval