Evaluate documentation


Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started


Before you start, you will need to setup your environment and install the appropriate packages. πŸ€— Evaluate is tested on Python 3.7+.

Virtual environment

You should install πŸ€— Evaluate in a virtual environment to keep everything neat and tidy.

  1. Create and navigate to your project directory:

    mkdir ~/my-project
    cd ~/my-project
  2. Start a virtual environment inside the directory:

    python -m venv .env
  3. Activate and deactivate the virtual environment with the following commands:

    # Activate the virtual environment
    source .env/bin/activate
    # Deactivate the virtual environment
    source .env/bin/deactivate

Once you have created your virtual environment, you can install πŸ€— Evaluate in it.


The most straightforward way to install πŸ€— Evaluate is with pip:

pip install evaluate

Run the following command to check if πŸ€— Evaluate has been properly installed:

python -c "import evaluate; print(evaluate.load('exact_match').compute(references=['hello'], predictions=['hello']))"

This should return:

{'exact_match': 1.0}


Building πŸ€— Evaluate from source lets you make changes to the code base. To install from source, clone the repository and install with the following commands:

git clone https://github.com/huggingface/evaluate.git
cd evaluate
pip install -e .

Again, you can check if πŸ€— Evaluate has been properly installed with:

python -c "import evaluate; print(evaluate.load('exact_match').compute(references=['hello'], predictions=['hello']))"