In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
--------
Question: We design the word-level refine decoder because this process is similar to the cloze task in BERT's pre-train process, therefore by using the ability of the contextual language model the decoder can generate more fluent and natural sequences. 
 Question: Why masking words in the decoder is helpful?

Answer: ecause this process is similar to the cloze task in BERT's pre-train process, therefore by using the ability of the contextual language model the decoder can generate more fluent and natural sequences.


Question: We introduce a surrogate training objective that avoids these problems and as a result is fully continuous.  Specifically, we form a continuous function softLB that seeks to approximate the result of running our decoder on input INLINEFORM0 and then evaluating the result against INLINEFORM1 using INLINEFORM2 . 
 Question: Do they provide a framework for building a sub-differentiable for any final loss metric?

Answer: Yes


Question: We thus also conducted human studies on Amazon MTurk to evaluate the generated responses with pairwise comparison for dialogue quality. We compare our models with an advanced decoding algorithm MMI BIBREF2 and two models, namely LSTM BIBREF0 and VHRED BIBREF7, both with additive attention. To our best knowledge, LSTM and VHRED were the primary models with which F1's were reported on the Ubuntu dataset. Following BIBREF5 (BIBREF5), we employ two criteria: Plausibility and Content Richness. The first criterion measures whether the response is plausible given the context, while the second gauges whether the response is diverse and informative.  
 Question: How is human evaluation performed, what was the criteria?

Answer:
Through Amazon MTurk annotators to determine plausibility and content richness of the response