G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Paper
•
2303.16634
•
Published
•
1
Evals & Monitoring
-
miracl/miracl-corpus
Viewer • Updated • 6.38k • 39Note https://github.com/project-miracl/miracl?tab=readme-ov-file MTEB: https://github.com/embeddings-benchmark/mteb
-
Judging LLM-as-a-judge with MT-Bench and Chatbot Arena
Paper • 2306.05685 • Published • 20 -
How is ChatGPT's behavior changing over time?
Paper • 2307.09009 • Published • 22 -
Evaluating Large Language Models: A Comprehensive Survey
Paper • 2310.19736 • Published • 2 -
Instruction-Following Evaluation for Large Language Models
Paper • 2311.07911 • Published • 17
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Paper • 2303.08896 • Published • 4
Landmark Attention: Random-Access Infinite Context Length for Transformers
Paper • 2305.16300 • PublishedNote Original "needle-in-haystack-test" for long context input - Passkey Retrieval More: https://arxiv.org/pdf/2402.13753.pdf
INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection
Paper • 2402.03744 • Published • 4Note The hyperparameters, including temperature, top-k and top-p, of the LLMs’decoder determine the diversity of the generations. To evaluate the impact of those hyper parameters. We provide a sensitivity analysis in Figure 4. As observed, the performance is greatly influenced by temperature but shows little sensitivity to top-k. The performance of the consistency based methods (EigenScore and Lexical Similarity) drops significantly when the temperature is greater than 1.
Chainpoll: A high efficacy method for LLM hallucination detection
Paper • 2310.18344 • Published • 1Note As a substitute for G-Eval
-
LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models
Paper • 2305.13711 • Published • 2 -
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Paper • 2012.07421 • Published • 1 -
Extending the WILDS Benchmark for Unsupervised Adaptation
Paper • 2112.05090 • Published • 1 -
vectara/hallucination_evaluation_model
Text Classification • Updated • 15.4k • 163 -
MMMU/MMMU
Viewer • Updated • 187k • 133 -
HuggingFaceH4/mt_bench_prompts
Viewer • Updated • 3.95k • 7 -
Running on CPU Upgrade40🥇
HHEM Leaderboard
-
Sleeping6✅
Transparency Self Assessment (FMTI)
-
JudgeLM: Fine-tuned Large Language Models are Scalable Judges
Paper • 2310.17631 • Published • 31 -
TRUE: Re-evaluating Factual Consistency Evaluation
Paper • 2204.04991 • Published • 1 -
Evaluating Very Long-Term Conversational Memory of LLM Agents
Paper • 2402.17753 • Published • 17 -
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
Paper • 2405.01535 • Published • 70