This paper studies the impact of paraphrases on the accuracy of automatic evaluation. Given a reference sentence and a machine-generated sentence, we seek to find a paraphrase of the reference sentence that is closer in wording to the machine output than the original reference. We apply our paraphrasing method in the context of machine translation evaluation. Our experiments show that the use of a paraphrased synthetic reference refines the accuracy of automatic evaluation. We also found a strong connection between the quality of automatic paraphrases as judged by humans and their contribution to automatic evaluation.
REF used paraphrases of the reference translations to improve automatic MT evaluation.
Paraphrasing For Automatic Evaluation
Human Language Technology Conference And Meeting Of The North American Association For Computational Linguistics
Computer Science