Task: Question Answering
Input (English): A recently published dataset called SCAN BIBREF2 (Simplified version of the CommAI Navigation tasks), tests compositional generalization in a sequence-to-sequence (seq2seq) setting by systematically holding out of the training set all inputs containing a basic primitive verb ("jump"), and testing on sequences containing that verb. 
 Question: How does the SCAN dataset evaluate compositional generalization?
Output (English): it systematically holds out inputs in the training set containing basic primitive verb, "jump", and tests on sequences containing that verb.