PromptBench: A Unified Library for Evaluation of Large Language Models
Authors: name: Kaijie Zhu; name: Qinlin Zhao; name: Hao Chen; name: Jindong Wang; name: Xing Xie
Published: 2023-12-13 05:58:34

Abstract: The evaluation of large language models (LLMs) is crucial to assess their
performance and mitigate potential security risks. In this paper, we introduce
PromptBench, a unified library to evaluate LLMs. It consists of several key
components that are easily used and extended by researchers: prompt
construction, prompt engineering, dataset and model loading, adversarial prompt
attack, dynamic evaluation protocols, and analysis tools. PromptBench is
designed to be an open, general, and flexible codebase for research purposes
that can facilitate original study in creating new benchmarks, deploying
downstream applications, and designing new evaluation protocols. The code is
available at: https://github.com/microsoft/promptbench and will be continuously
supported.