In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.

Q: In Table TABREF15 we measure BLEU BIBREF19, NIST BIBREF20, METEOR BIBREF21, ROUGE-L BIBREF22 and CIDEr BIBREF23 metrics on the 2018 E2E NLG Challenge test data using the evaluation script provided by the organizers.  We use the gold standard selection during training and validation of the text generation model, as well as the automatic evaluation. As we deploy our text generation model for manual evaluation, we use a Conditional Random Field (CRF) model to predict which events to mention. The second human evaluation aimed at judging the acceptability of the output for production use in a news agency.  In the minimum edit evaluation, carried out by the annotator who created the news corpus, only factual mistakes and grammatical errors are corrected, resulting in text which may remain awkward or unfluent. The word error rate (WER) of the generated text compared to its corrected variant as a reference is 5.6% (6.2% disregarding punctuation).  The factual errors and their types are summarized in Table TABREF23. From the total of 510 game events generated by the system, 78 of these contained a factual error, i.e. 84.7% were generated without factual errors. Most fluency issues relate to the overall flow and structure of the report. Addressing these issues would require the model to take into account multiple events in a game, and combine the information more flexibly to avoid repetition.  The second human evaluation aimed at judging the acceptability of the output for production use in a news agency. The output is evaluated in terms of its usability for a news channel labelled as being machine-generated, i.e. not aiming at the level of a human journalist equipped with substantial background information. The evaluation was carried out by two journalists from the STT agency, who split the 59 games among themselves approximately evenly. 
 Question: What evaluation criteria and metrics were used to evaluate the generated text?

A: BLEU  NIST  METEOR  ROUGE-L CIDEr  evaluation script automatic evaluation human evaluation minimum edit evaluation word error rate (WER) factual errors and their types fluency issues acceptability of the output for production use in a news agency
****
Q:  We opted for an automatic approach instead, that can be scaled arbitrarily and at little cost: we generate synthetic sentence pairs $(, \tilde{})$ by randomly perturbing 1.8 million segments $$ from Wikipedia. We use three techniques: mask-filling with BERT, backtranslation, and randomly dropping out words.  
 Question: How are the synthetic examples generated?

A: Random perturbation of Wikipedia sentences using mask-filling with BERT, backtranslation and randomly drop out
****
Q: We manually investigate the errors made by artificial neural networks for morphological inflection in a target language after pretraining on different source languages. For our qualitative analysis, we make use of the validation set. Therefore, we show validation set accuracies in Table TABREF19 for comparison. We manually annotate the outputs for the first 75 development examples for each source–target language combination. All found errors are categorized as belonging to one of the following categories. 
 Question: How is the performance on the task evaluated?

A:
Comparison of test accuracies of neural network models on an inflection task and qualitative analysis of the errors
****