Spaces:

evaluate-metric

/

bertscore

like 31

Running

1. pip install evaluate
2. pip install bert-score
3. download a metric model such as 'roberta-large' in facebookAI/roberta-large, you should download several files ('config.json', 'merges.txt', 'pytorch_model.bin', 'tokenizer.json', 'tokenizer_config.json', 'vocab.json') https://huggingface.co/FacebookAI/roberta-large/tree/main
4. download the bertscore.py' in this spaces.

then you can use bert score with code:

from evaluate import load
from bert_score.utils import model2layers

bertscore = load("pretrained_metric/bertscore/bertscore.py")
predictions = ["hello there", "general kenobi"]
references = ["hello there", "general kenobi"]
results = bertscore.compute(
    predictions=predictions,
    references=references,
    model_type="pretrained_weight/HF/roberta-large",
    num_layers=model2layers["roberta-large"],
)
print(results)

"pretrained_metric/bertscore/bertscore.py" and "pretrained_weight/HF/roberta-large" is the paths where I save bertscore.py and roberta-large.