Question: How much better peformance is achieved in human evaluation when model is trained considering proposed metric?
Answer: Pearson correlation to human judgement - proposed vs next best metric
Sample level comparison:
- Story generation: 0.387 vs 0.148
- Dialogue: 0.472 vs 0.341
Model level comparison:
- Story generation:  0.631 vs 0.302
- Dialogue: 0.783 vs 0.553