In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

In Table TABREF15 we measure BLEU BIBREF19, NIST BIBREF20, METEOR BIBREF21, ROUGE-L BIBREF22 and CIDEr BIBREF23 metrics on the 2018 E2E NLG Challenge test data using the evaluation script provided by the organizers.  We use the gold standard selection during training and validation of the text generation model, as well as the automatic evaluation. As we deploy our text generation model for manual evaluation, we use a Conditional Random Field (CRF) model to predict which events to mention. The second human evaluation aimed at judging the acceptability of the output for production use in a news agency.  In the minimum edit evaluation, carried out by the annotator who created the news corpus, only factual mistakes and grammatical errors are corrected, resulting in text which may remain awkward or unfluent. The word error rate (WER) of the generated text compared to its corrected variant as a reference is 5.6% (6.2% disregarding punctuation).  The factual errors and their types are summarized in Table TABREF23. From the total of 510 game events generated by the system, 78 of these contained a factual error, i.e. 84.7% were generated without factual errors. Most fluency issues relate to the overall flow and structure of the report. Addressing these issues would require the model to take into account multiple events in a game, and combine the information more flexibly to avoid repetition.  The second human evaluation aimed at judging the acceptability of the output for production use in a news agency. The output is evaluated in terms of its usability for a news channel labelled as being machine-generated, i.e. not aiming at the level of a human journalist equipped with substantial background information. The evaluation was carried out by two journalists from the STT agency, who split the 59 games among themselves approximately evenly. 
 Question: What evaluation criteria and metrics were used to evaluate the generated text?
BLEU  NIST  METEOR  ROUGE-L CIDEr  evaluation script automatic evaluation human evaluation minimum edit evaluation word error rate (WER) factual errors and their types fluency issues acceptability of the output for production use in a news agency

We apply attention probes to the task of identifying the existence and type of dependency relation between two words. Our first experiment is an exploratory visualization of how word sense affects context embeddings.  
 Question: What linguistic features were probed for?
dependency relation between two words word sense

We investigate the following six idiom resources:

Wiktionary;

the Oxford Dictionary of English Idioms (ODEI, BIBREF31);

UsingEnglish.com (UE);

the Sporleder corpus BIBREF10;

the VNC dataset BIBREF9;

and the SemEval-2013 Task 5 dataset BIBREF15. 
 Question: What dictionaries are used for automatic extraction of PIEs?

Wiktionary Oxford Dictionary of English Idioms UsingEnglish.com (UE) Sporleder corpus VNC dataset SemEval-2013 Task 5 dataset