We present ParaMetric, an automatic evaluation metric for data-driven approaches to paraphrasing. ParaMetric provides an objective measure of quality using a collection of multiple translations whose paraphrases have been manually annotated. ParaMetric calculates precision and recall scores by comparing the paraphrases discovered by automatic paraphrasing techniques against gold standard alignments of words and phrases within equivalent sentences. We report scores for several established paraphrasing techniques.
The only prior attempt to devise an automatic evaluation metric for paraphrases that we are aware of is ParaMetric REF , which compares the collection of paraphrases discovered by automatic paraphrasing algorithms against a manual gold standard collected over the same sentences.
ParaMetric: An Automatic Evaluation Metric for Paraphrasing
International Conference On Computational Linguistics
Computer Science