In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

The whole dataset consists of 81,826 samples annotated by native English speakers. 80% of them are used as training set. 10% of them are used as validation set while the rest is used as test set. 3000 hard samples are selected from the test set.
What is the difference between the full test set and the hard test set?

The neural machine translation was trained using Nematus. For the NMT system as well as for the PreMT system, we used the default configuration.
Which NMT architecture do they use?

We performed experiments on synthetically generated dataset since it gives us a better control over the distribution of the data. Specifically we compared the gains obtained using our approach versus the variance of the distribution. We created dataset from the following generative process. [H] Generative Process [1] Generate data

Pick k points INLINEFORM0 as domain -1 means and a corresponding set of k points INLINEFORM1 as domain-2 means, and covariance matrices INLINEFORM2

iter INLINEFORM0 upto num INLINEFORM1 samples Sample class INLINEFORM2 Sample INLINEFORM3 Sample INLINEFORM4 Add q and a so sampled to the list of q,a pairs We generated the dataset from the above sampling process with means selected on a 2 dimensional grid of size INLINEFORM5 with variance set as INLINEFORM6 in each dimension.10000 sample points were generated. The parameter INLINEFORM7 of the above algorithm was set to 0.5 and k was set to 9 (since the points could be generated from one of the 9 gaussians with centroids on a INLINEFORM8 grid).

How do they generate the synthetic dataset?