In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

[Q]: We conduct our annotation study on Amazon Mechanical Turk, presenting Turkers with Human Intelligence Tasks (henceforth, HITs) consisting of a single conversation between a customer and an agent. In each HIT, we present Turkers with a definition of each dialogue act, as well as a sample annotated dialogue for reference. For each turn in the conversation, we allow Turkers to select as many labels from our taxonomy as required to fully characterize the intent of the turn. Additionally, annotators are asked three questions at the end of each conversation HIT, to which they could respond that they agreed, disagreed, or could not tell:
[A]: How are customer satisfaction, customer frustration and overall problem resolution data collected?


[Q]: Recently, MADAMIRA BIBREF6 system has been evaluated using a blind testset (25K words for Modern Standard Arabic (MSA) selected from Penn Arabic Tree bank (PATB)), and the reported accuracy was 96.2% as the percentage of words where the chosen analysis (provided by SAMA morphological analyzer BIBREF7 ) has the correct lemma. As MSA is usually written without diacritics and IR systems normally remove all diacritics from search queries and indexed data as a basic preprocessing step, so another column for undiacritized lemma is added and it's used for evaluating our lemmatizer and comparing with state-of-the-art system for lemmatization; MADAMIRA.
[A]: What is the state of the art?


[Q]:  This baseline has an F1 of 41.59% and accuracy of 71.22%.
[A]:
what are their evaluation metrics?