TASK DEFINITION: In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.
PROBLEM: When testing across languages, we report accuracy for two setups: average accuracy over each single-language model (Avg), and accuracy obtained when training on the concatenation of all languages but the target one (All). The latter setting is also used for the embeddings model. We report accuracy for all experiments.

SOLUTION: What are the evaluation metrics used?

PROBLEM: First, workers were given a seed qualitative relation q+/-( INLINEFORM0 ) in the domain, expressed in English (e.g., "If a surface has more friction, then an object will travel slower"), and asked to enter two objects, people, or situations to compare. They then created a question, guided by a large number of examples, and were encouraged to be imaginative and use their own words. Second, the LFs were elicited using a novel technique of reverse-engineering them from a set of follow-up questions, without exposing workers to the underlying formalism. This is possible because of the constrained space of LFs. Referring to LF templates (1) and (2) earlier (Section SECREF13 ), these questions are as follows:

From this information, we can deduce the target LF ( INLINEFORM0 is the complement of INLINEFORM1 , INLINEFORM2 , we arbitrarily set INLINEFORM3 =world1, hence all other variables can be inferred).

SOLUTION: How do they obtain the logical forms of their questions in their dataset?

PROBLEM: Car-speak is abstract language that pertains to a car's physical attribute(s). In this instance the physical attributes that the term "fast" pertains to could be the horsepower, or it could be the car's form factor (how the car looks). However, we do not know exactly which attributes the term "fast" refers to.

SOLUTION:
How does car speak pertains to a car's physical attributes?