In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

Ex Input:
 We opted for an automatic approach instead, that can be scaled arbitrarily and at little cost: we generate synthetic sentence pairs $(, \tilde{})$ by randomly perturbing 1.8 million segments $$ from Wikipedia. We use three techniques: mask-filling with BERT, backtranslation, and randomly dropping out words.  
 Question: How are the synthetic examples generated?

Ex Output:
Random perturbation of Wikipedia sentences using mask-filling with BERT, backtranslation and randomly drop out


Ex Input:
We recruited 100 crowdworkers on Amazon Mechanical Turk (AMT) and measured completion times and accuracies for typing randomly sampled sentences from the Yelp corpus.  
 Question: What user variations have been tested?

Ex Output:
completion times and accuracies 


Ex Input:
Evaluation is done using the development set, consisting of 22 documents and 1112 PIE candidates, and the test set, which consists of 23 documents and 1127 PIE candidates. For each method the best set of parameters and/or options is determined using the development set, after which the best variant by F1-score of each method is evaluated on the test set.

Since these documents in the corpus are exhaustively annotated for PIEs (see Section SECREF40), we can calculate true and false positives, and false negatives, and thus precision, recall and F1-score. 
 Question: Are PIEs extracted automatically subjected to human evaluation?

Ex Output:

Yes