instruction:
In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.
question:
Experimental results show that our joint model outperforms the visual-only model in all cases, and the text-only model on Wikipedia and two subsets of arXiv.
answer:
Do the methods that work best on academic papers also work best on Wikipedia?


question:
Our goal was to generate questions without templates and with minimal human involvement using machine learning transformers that have been demonstrated to train faster and better than RNNs. Such a system would benefit educators by saving time to generate quizzes and tests.
answer:
What is the motivation behind the work? Why question generation is an important task?


question:
For task 1, we use F1-score as evaluation metric. We use manual evaluation for task 2. There are five evaluation metrics for task 2 as following.

Task completion ratio: The number of completed tasks divided by the number of total tasks.

User satisfaction degree: There are five scores -2, -1, 0, 1, 2, which denote very dissatisfied, dissatisfied, neutral, satisfied and very satisfied, respectively.

Response fluency: There are three scores -1, 0, 1, which indicate nonfluency, neutral, fluency.

Number of dialogue turns: The number of utterances in a task-completed dialogue.

Guidance ability for out of scope input: There are two scores 0, 1, which represent able to guide or unable to guide.
answer:

What metrics are used in the evaluation?