Hugging Face Space: aynetdia/semscore
Author: aynetdia
SDK: gradio
title: SemScore
tags: evaluate
metric
description: SemScore measures semantic textual similarity between candidate and reference texts. It has been shown to strongly correlate with human judgment on a system-level when evaluating the instructing following capabilities of language models. Given a set of model-generated outputs and target completions, a pre-trained sentence-transformer is used to calculate cosine similarities between them.