Definition: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Input: Since the ground-truth reference captions in ShapeWorld are randomly sampled, we take the sampled captions accompanying the test images as a proxy for optimal caption diversity, and compare it with the empirical output diversity of the evaluated model on these test images. Practically, we look at language constructions used and compute the corresponding diversity score as the ratio of observed number versus optimal number
Question: How is diversity measured?
Output:
diversity score as the ratio of observed number versus optimal number