In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.
--------
Question: In all our experiments, we used the out-of-the-box BERT models without any task-specific fine-tuning.

Answer: How does their model differ from BERT?


Question: The retrieved pairs are compared to the gold standard and evaluated using precision at $k$ (P@$k$, evaluating how often the correct translation is within the $k$ retrieved nearest neighbours of the query). Throughout this work we report P@1, which is equivalent to accuracy, but we also provide results with P@5 and P@10 in the Appendix.

Answer: What evaluation metrics did they use?


Question: We collect human judgments on Amazon Mechanical Turk via ParlAI BIBREF18.

Answer:
Do they use crowdsourcing to collect human judgements?