Original: C OMPOSITIONAL S EMANTIC P ARSING WITH L ARGE L ANGUAGE M ODELS
Revised: C OMPOSITIONAL S EMANTIC P ARSING WITH L ARGE L ANGUAGE M ODELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Humans can reason compositionally when presented with new tasks.
Revised: Humans can reason compositionally when presented with new tasks.

Original: Previous research shows that appropriate prompting techniques enable large language models (LLMs) to solve artiﬁcial compositional generalization tasks such as SCAN.
Revised: Previous research shows that appropriate prompting techniques enable large language models (LLMs) to solve artiﬁcial compositional generalization tasks such as SCAN.

Original: In this work, we identify additional challenges in more realistic semantic parsing tasks with larger vocabulary and reﬁne these prompting techniques to address them.
Revised: In this work, we identify additional challenges in more realistic semantic parsing tasks with larger vocabulary and reﬁne these prompting techniques to address them.

Original: Our best method is based on least-to-most prompting: it decomposes the problem using prompting-based syntactic parsing, then uses this decomposition to select appropriate exemplars and to sequentially generate the semantic parse.
Revised: Our best method is based on least-to-most prompting: it decomposes the problem using prompting-based syntactic parsing, then uses this decomposition to select appropriate exemplars and to sequentially generate the semantic parse.

Original: This method allows us to set a new state of the art for CFQ while requiring only 1% of the training data used by traditional approaches.
Revised: This method allows us to set a new state of the art for CFQ while requiring only 1% of the training data used by traditional approaches.

Original: Due to the general nature of our approach, we expect similar efforts will lead to new results in other tasks and domains, especially for knowledge-intensive applications.
Revised: Due to the general nature of our approach, we expect similar efforts will lead to new results in other tasks and domains, especially for knowledge-intensive applications.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Compositionality is a key part of human intelligence as it allows us to understand and produce a potentially inﬁnite number of novel combinations of known components (Chomsky, 1957; Montague, 1970; Lake et al., 2017).
Revised: Compositionality is a key part of human intelligence as it allows us to understand and produce a potentially inﬁnite number of novel combinations of known components (Chomsky, 1957; Montague, 1970; Lake et al., 2017).

Original: In contrast, standard neural sequence models, transformers and recurrent neural networks, often fail to capture the compositional structure of the problem domain and thus fail to generalize compositionally (Keysers et al., 2020;
Revised: In contrast, standard neural sequence models, transformers and recurrent neural networks, often fail to capture the compositional structure of the problem domain and thus fail to generalize compositionally (Keysers et al., 2020;

Original: Furrer et al., 2020).
Revised: Furrer et al., 2020).

Original: Prior efforts to improve compositional generalization primarily rely on specialized architectures or training procedures (Lake, 2019; Chen et al., 2020; Nye et al., 2020; Andreas, 2020; Conklin et al., 2021; Aky ¨ urek et al., 2021;
Revised: Prior efforts to improve compositional generalization primarily rely on specialized architectures or training procedures (Lake, 2019; Chen et al., 2020; Nye et al., 2020; Andreas, 2020; Conklin et al., 2021; Aky ¨ urek et al., 2021;

Original: Liu et al., 2021).
Revised: Liu et al., 2021).

Original: Although effective, these can be task-speciﬁc.
Revised: Although effective, these can be task-speciﬁc.

Original: Even more general purpose methods that rely on data augmentation are limited in the class of data it can support (Shaw et al., 2021; Qiu et al., 2022a).
Revised: Even more general purpose methods that rely on data augmentation are limited in the class of data it can support (Shaw et al., 2021; Qiu et al., 2022a).

Original: Prompting on the other hand is sufﬁciently ﬂexible and, with recent advancement of large-scale pretrained language models (LLMs), has become an effective and generic approach to address a wide range of language understanding problems (Brown et al., 2020).
Revised: Prompting on the other hand is sufﬁciently ﬂexible and, with recent advancement of large-scale pretrained language models (LLMs), has become an effective and generic approach to address a wide range of language understanding problems (Brown et al., 2020).

Original: Prompting now performs on-par or better than model ﬁnetuning in many cases (Wei et al., 2022b; Chowdhery et al., 2022; Wei et al., 2022a; Kojima et al., 2022; Ahn et al., 2022), and might be suitable for improving language model performance on compositional generalization.
Revised: Prompting now performs on-par or better than model ﬁnetuning in many cases (Wei et al., 2022b; Chowdhery et al., 2022; Wei et al., 2022a; Kojima et al., 2022; Ahn et al., 2022), and might be suitable for improving language model performance on compositional generalization.

Original: In particular, recent work (Zhou et al., 2022) found that least-to-most prompting shows a lot of potential for adapting LLMs for compositional generalization, achieving 99 .
Revised: In particular, recent work (Zhou et al., 2022) found that least-to-most prompting shows a lot of potential for adapting LLMs for compositional generalization, achieving 99 .

Original: 7% accuracy on SCAN, a commonly used compositional generalization benchmark.
Revised: 7% accuracy on SCAN, a commonly used compositional generalization benchmark.

Original: Least-to-most prompting decomposes each problem into a series of subproblems, then sequentially solves one after another.
Revised: Least-to-most prompting decomposes each problem into a series of subproblems, then sequentially solves one after another.

Original: However,
Revised: However,

Original: SCAN is an artiﬁcial task built upon a synthetic language with a tiny vocabulary and is generated from a small set of grammar rules, and it is unclear whether strong results transfer to more realistic tasks that are based on a larger vocabulary and more complicated grammars (Furrer et al., 2020).
Revised: SCAN is an artiﬁcial task built upon a synthetic language with a tiny vocabulary and is generated from a small set of grammar rules, and it is unclear whether strong results transfer to more realistic tasks that are based on a larger vocabulary and more complicated grammars (Furrer et al., 2020).

Original: Additional challenges arise when applying least-to-most prompting to more realistic semantic parsing benchmarks.
Revised: Additional challenges arise when applying least-to-most prompting to more realistic semantic parsing benchmarks.

Original: Among others, they may require information beyond what ﬁts in a single prompt.
Revised: Among others, they may require information beyond what ﬁts in a single prompt.

Original: Also, decomposing a problem is more difﬁcult than with SCAN, exacerbated by constituents that cannot be translated independent of their context.
Revised: Also, decomposing a problem is more difﬁcult than with SCAN, exacerbated by constituents that cannot be translated independent of their context.

Original: We address these challenges with dynamic leastto-most prompting , a generic reﬁnement of least-to-most prompting that involves the following steps:
Revised: We address these challenges with dynamic leastto-most prompting , a generic reﬁnement of least-to-most prompting that involves the following steps:

Original: (1) tree-structured decomposition of natural language inputs through LM-predicted syntactic parsing, (2) use the decomposition to dynamically select exemplars, and (3) linearize the decomposition tree and prompt the model to sequentially generate answers to subproblems.
Revised: (1) tree-structured decomposition of natural language inputs through LM-predicted syntactic parsing, (2) using the decomposition to dynamically select exemplars, and (3) linearizing the decomposition tree and prompting the model to sequentially generate answers to subproblems.

Original: We evaluate our approach on two realistic benchmarks that, like SCAN, are designed to measure compositional generalization: CFQ (Keysers et al., 2020) and COGS (Kim & Linzen, 2020).
Revised: We evaluate our approach on two realistic benchmarks that, like SCAN, are designed to measure compositional generalization: CFQ (Keysers et al., 2020) and COGS (Kim & Linzen, 2020).

Original: On
Revised: On

Original: CFQ, our best performing method outperforms previous fully supervised ﬁnetuning approaches and achieves a new state-of-the-art accuracy of 95% (averaged across MCD splits) and thereby reduced the error rate by about 45% compared to the previous best result while using about 1% of the training data as candidates for exemplars.
Revised: CFQ, our best performing method outperforms previous fully supervised ﬁnetuning approaches and achieves a new state-of-the-art accuracy of 95% (averaged across MCD splits) and thereby reduced the error rate by about 45% compared to the previous best result while using about 1% of the training data as candidates for exemplars.

Original: On COGS, our approach scores an accuracy of 99.2% when evaluated on the generalization test set, comparable with strong baselines.
Revised: On COGS, our approach scores an accuracy of 99.2% when evaluated on the generalization test set, comparable with strong baselines.

Original: We also demonstrate robustness of our approach to exemplar pool size, and when using less than 0.1% of the training data as exemplars, dynamic least-to-most prompting is still competitive with previous approaches.
Revised: We also demonstrate robustness of our approach to exemplar pool size, and when using less than 0.1% of the training data as exemplars, dynamic least-to-most prompting is still competitive with previous approaches.

Original:  B ACKGROUND AND M OTIVATION 2. C OMPOSITIONAL G ENERALIZATION
Revised:  B ACKGROUND AND M OTIVATION 2. C OMPOSITIONAL G ENERALIZATION

Original: Compositionality is the idea that the meanings of complex expressions are constructed from the meanings of the less complex expressions that are their constituents.
Revised: Compositionality is the idea that the meanings of complex expressions are constructed from the meanings of the less complex expressions that are their constituents.

Original: —Fodor & Lepore (2002)
Revised: —Fodor & Lepore (2002)

Original: Given the knowledge of conceptual primitives and a few combinations, compositional generalization is the capability to use and comprehend unseen combinations.
Revised: Given the knowledge of conceptual primitives and a few combinations, compositional generalization is the capability to use and comprehend unseen combinations.

Original: SCAN (Lake & Baroni, 2018;
Revised: SCAN (Lake & Baroni, 2018;

Original: Loula et al., 2018) is one of the earliest benchmarks that shows neural sequence models cannot systematically generalize to novel combinations of the primitive items of the language.
Revised: Loula et al., 2018) is one of the earliest benchmarks that shows neural sequence models cannot systematically generalize to novel combinations of the primitive items of the language.

Original: The benchmark requires the learner to translate simple commands to action sequences, where all commands are generated from a set of 20 grammar rules and use a a vocabulary consisting of about 20 words.
Revised: The benchmark requires the learner to translate simple commands to action sequences, where all commands are generated from a set of 20 grammar rules and use a a vocabulary consisting of about 20 words.

Original: Recent work has achieved perfect generalization accuracy on SCAN by inferring grammar rules in symbolic form (Chen et al., 2020; Nye et al., 2020; Liu et al., 2020; Shaw et al., 2021).
Revised: Recent work has achieved perfect generalization accuracy on SCAN by inferring grammar rules in symbolic form (Chen et al., 2020; Nye et al., 2020; Liu et al., 2020; Shaw et al., 2021).

Original: Most recently, Zhou et al.
Revised: Most recently, Zhou et al.

Original: (2022) demonstrate that SCAN can be solved by least-to-most prompting, which leverages a pretrained large language model (LLM) and a prompt consisting of only 14 exemplars, which is less than 0.1% of the training data used by previous approaches.
Revised: (2022) demonstrate that SCAN can be solved by least-to-most prompting, which leverages a pretrained large language model (LLM) and a prompt consisting of only 14 exemplars, which is less than 0.1% of the training data used by previous approaches.

Original: 2. L EAST - TO -M OST P ROMPTING E NABLES C OMPOSITIONAL G ENERALIZATION
Revised: 2. L EAST - TO -M OST P ROMPTING E NABLES C OMPOSITIONAL G ENERALIZATION

Original: Least-to-most prompting teaches a language model how to solve a complex problem by reducing it to a set of easier subproblems.
Revised: Least-to-most prompting teaches a language model how to solve a complex problem by reducing it to a set of easier subproblems.

Original: This is done by constructing two types of prompts.
Revised: This is done by constructing two types of prompts.

Original: The ﬁrst type of prompt tells the language model how to decompose a problem into a list of subproblems, while the second type of prompt describes how to sequentially solve the subproblems.
Revised: The ﬁrst type of prompt tells the language model how to decompose a problem into a list of subproblems, while the second type of prompt describes how to sequentially solve the subproblems.

Original: As an illustration, consider the application of least-to-most prompting to SCAN.
Revised: As an illustration, consider the application of least-to-most prompting to SCAN.

Original: The decomposition of the input “look around right thrice and walk twice” yields the following subproblems: “look right”, “look around right”, “look around right thrice”, and “walk twice”.
Revised: The decomposition of the input “look around right thrice and walk twice” yields the following subproblems: “look right”, “look around right”, “look around right thrice”, and “walk twice”.

Original: Since SCAN commands are generated by a simple grammar of only 20 rules, this decomposition task can be performed using a prompt consisting of only 8 decomposition exemplars.
Revised: Since SCAN commands are generated by a simple grammar of only 20 rules, this decomposition task can be performed using a prompt consisting of only 8 decomposition exemplars.

Original: This decomposition allows the translation of the original input to be produced sequentially rather than in one step (as would be the case with naive prompting).
Revised: This decomposition allows the translation of the original input to be produced sequentially rather than in one step (as would be the case with naive prompting).

Original: The ﬁrst subproblem is translated by passing the language model a prompt context consisting of 14 simple translation exemplars followed by the command “look right”.
Revised: The ﬁrst subproblem is translated by passing the language model a prompt context consisting of 14 simple translation exemplars followed by the command “look right”.

Original: The model’s answer is then appended to the prompt such that it is used as additional context when translating the next subproblem “look around right”, etc.
Revised: The model’s answer is then appended to the prompt such that it is used as additional context when translating the next subproblem “look around right”, etc.

Original: Did M1 star M2 , star M3 , and star a art director and editor of M0 SELECT count(
Revised: Did M1 star M2 , star M3 , and star a art director and editor of M0 SELECT count(

Original: *) WHERE { ?x0 edited M0 .
Revised: *) WHERE { ?x0 edited M0 .

Original: x0 art directed M0 .
Revised: x0 art directed M0 .

Original: M1 starred ?
Revised: M1 starred ?

Original: x0 .
Revised: x0 .

Original: M1 starred M2 .
Revised: M1 starred M2 .

Original: M1 starred M3 }
Revised: M1 starred M3 }

Original: What was produced by a art director that M1 and M2 employed SELECT DISTINCT WHERE { ?x0 produced by ?x1 .
Revised: What was produced by a art director that M1 and M2 employed SELECT DISTINCT WHERE { ?x0 produced by ?x1 .

Original: [Equation]
Revised: [Equation]

Original: director .
Revised: director .

Original: M0 employed ?
Revised: M0 employed ?

Original: x1 .
Revised: x1 .

Original: [Equation]
Revised: [Equation]

Original: 2. L EAST - TO -M OST P ROMPTING : L IMITATIONS AND C HALLENGES
Revised: 2. L EAST - TO -M OST P ROMPTING : L IMITATIONS AND C HALLENGES

Original: While the performance of least-to-most prompting on SCAN is impressive, it is not clear whether and how the same technique can be applied to compositional generalization problems that are based on a more realistic subset of natural language.
Revised: While the performance of least-to-most prompting on SCAN is impressive, it is not clear whether and how the same technique can be applied to compositional generalization problems that are based on a more realistic subset of natural language.

Original: In particular, we identiﬁed three challenges that are common for more realistic natural language tasks and need to be addressed in order to apply least-tomost prompting: (1) decomposition is more challenging, (2) the knowledge required for translation may be too large to ﬁt into a single prompt, and (3) translation of constituents is context-dependent.
Revised: In particular, we identiﬁed three challenges that are common for more realistic natural language tasks and need to be addressed in order to apply least-tomost prompting: (1) decomposition is more challenging, (2) the knowledge required for translation may be too large to ﬁt into a single prompt, and (3) translation of constituents is context-dependent.

Original: As we consider extending least-to-most to the more realistic data setting, we have two semantic parsing benchmarks in mind: CFQ (Keysers et al., 2020) and COGS (Kim & Linzen, 2020).
Revised: As we consider extending least-to-most to the more realistic data setting, we have two semantic parsing benchmarks in mind: CFQ (Keysers et al., 2020) and COGS (Kim & Linzen, 2020).

Original: CFQ is a compositional semantic parsing task where natural language questions need to be translated into SPARQL commands.
Revised: CFQ is a compositional semantic parsing task where natural language questions need to be translated into SPARQL commands.

Original: Compared to SCAN, CFQ is based on a much larger vocabulary as well as more complex linguistic structures produced by a uniﬁcation-based grammar (Shieber, 2003).
Revised: Compared to SCAN, CFQ is based on a much larger vocabulary as well as more complex linguistic structures produced by a uniﬁcation-based grammar (Shieber, 2003).

Original: As a result, CFQ has proven to be quite challenging for generic ML architectures such as transformers and LSTMs.
Revised: As a result, CFQ has proven to be quite challenging for generic ML architectures such as transformers and LSTMs.

Original: Even with custom architectures, the best generalization accuracy is less than 91%, achieved by specialized architectures for compositional grammar learning (Liu et al., 2021)
Revised: Even with custom architectures, the best generalization accuracy is less than 91%, achieved by specialized architectures for compositional grammar learning (Liu et al., 2021)

Original: COGS is another semantic parsing tasks where natural language sentences need to be translated into a formal representation.
Revised: COGS is another semantic parsing task where natural language sentences need to be translated into a formal representation.

Original: As for CFQ and SCAN, the training data for COGS contains multiple systematic gaps that can only be addressed by compositional generalization; these include new combinations of familiar syntactic structures and familiar structures.
Revised: As for CFQ and SCAN, the training data for COGS contains multiple systematic gaps that can only be addressed by compositional generalization; these include new combinations of familiar syntactic structures and familiar structures.

Original: While COGS proves to be quite challenging for generic ML architectures, the best specialized models achieve 99.7% accuracy (Qiu et al., 2022a).
Revised: While COGS proves to be quite challenging for generic ML architectures, the best specialized models achieve 99.7% accuracy (Qiu et al., 2022a).

Original: Natural Language is Challenging to Decompose SCAN commands are constructed from eight distinct symbols with a ﬁxed precedence (“left”, “right”, “twice”, “thrice”, “opposite”, “around”, “and”, and “after”).
Revised: Natural Language is Challenging to Decompose SCAN commands are constructed from eight distinct symbols with a ﬁxed precedence (“left”, “right”, “twice”, “thrice”, “opposite”, “around”, “and”, and “after”).

Original: Roughly, the decomposition of a SCAN statement resembles that of a mathematical expression with standard arithmetic operations.
Revised: Roughly, the decomposition of a SCAN statement resembles that of a mathematical expression with standard arithmetic operations.

Original: In practice, the decomposition for SCAN can be predicted by a language model using a simple prompt.
Revised: In practice, the decomposition for SCAN can be predicted by a language model using a simple prompt.

Original: CFQ and COGS sentences represent a richer subset of natural language, meaning the various components and their interactions involve grammatical features such as different parts of speech, grammatical voice, conjunctions, and pronouns.
Revised: CFQ and COGS sentences represent a richer subset of natural language, meaning the various components and their interactions involve grammatical features such as different parts of speech, grammatical voice, conjunctions, and pronouns.

Original: This makes decomposition much more challenging as it requires deep understanding of the underlying linguistic structures.
Revised: This makes decomposition much more challenging as it requires deep understanding of the underlying linguistic structures.

Original: Single Prompt Insufﬁcient to Represent Full Label Space
Revised: Single Prompt Insufﬁcient to Represent Full Label Space

Original: In the case of SCAN, the knowledge needed to translate a command into a sequence of actions is small enough that it can be captured with about a dozen examples.
Revised: In the case of SCAN, the knowledge needed to translate a command into a sequence of actions is small enough that it can be captured with about a dozen examples.

Original: This is not the case for more realistic semantic parsing problems.
Revised: This is not the case for more realistic semantic parsing problems.

Original: For example, CFQ uses more than 50 different Freebase types and relations and we cannot really expect the model to know the names of those relations without seeing them used in examples.
Revised: For example, CFQ uses more than 50 different Freebase types and relations, and we cannot really expect the model to know the names of those relations without seeing them used in examples.

Original: Similarly,
Revised: Similarly,

Original: COGS uses hundreds of verbs, and their translation depends on details such as whether or not a verb is unaccusative or unergative, which are difﬁcult or impossible to determine without seeing corresponding translation examples.
Revised: COGS uses hundreds of verbs, and their translation depends on details such as whether or not a verb is unaccusative or unergative, which are difﬁcult or impossible to determine without seeing corresponding translation examples.

Original: Constituent Translation is Context-Dependent Least-to-most prompting has only been applied to domains where constituents can be translated independent of their context.
Revised: Constituent Translation is Context-Dependent Least-to-most prompting has only been applied to domains where constituents can be translated independent of their context.

Original: The context-free nature of those tasks enables smooth derivation of a ﬁnal output from the solutions to the subproblems.
Revised: The context-free nature of those tasks enables smooth derivation of a ﬁnal output from the solutions to the subproblems.

Original: As an illustration of the context-free nature of SCAN, consider the expression “walk twice”, which always translates to “WALK WALK”.
Revised: As an illustration of the context-free nature of SCAN, consider the expression “walk twice”, which always translates to “WALK WALK”.

Original: The constituents in CFQ can not be translated independent of their context.
Revised: The constituents in CFQ cannot be translated independent of their context.

Original: This is exempliﬁed by the two sentences in Figure 1.
Revised: This is exempliﬁed by the two sentences in Figure 1.

Original: In one, the expression “a art director” translates to “?
Revised: In one, the expression “a art director” translates to “?

Original: x0 art directed M0”, but translates to “?
Revised: x0 art directed M0”, but translates to “?

Original: x1 a art director” in the other.
Revised: x1 a art director” in the other.

Original: As we will detail in the next section, this means that we cannot use the traditional approach for leastto-most prompting, where we ﬁrst ask the model to translate each subproblem in isolation.
Revised: As we will detail in the next section, this means that we cannot use the traditional approach for leastto-most prompting, where we ﬁrst ask the model to translate each subproblem in isolation.

Original: Instead, we need to make sure that the subproblems are provided to the model with enough context.
Revised: Instead, we need to make sure that the subproblems are provided to the model with enough context.

Original: What was produced by a art director that M1 and M2 employed and was directed by M
Revised: What was produced by a art director that M1 and M2 employed and was directed by M

Original: What was produced by (a art director) that (M1 and M2) employed and was directed by (M3)
Revised: What was produced by (a art director) that (M1 and M2) employed and was directed by (M3)

Original: Problem Reduction (Syntactic Parsing)
Revised: Problem Reduction (Syntactic Parsing)

Original: M1 and M2 employed
Revised: M1 and M2 employed

Original: Who edited a film that M1 and M2 produced
Revised: Who edited a film that M1 and M2 produced

Original: Retrieve using constituent from decomposition
Revised: Retrieve using constituent from decomposition

Original: Selected exemplar
Revised: Selected exemplar

Original: Dynamically Select Exemplars for Each Subproblem
Revised: Dynamically Select Exemplars for Each Subproblem

Original: Q: What was directed by M3 A: <
Revised: Q: What was directed by M3 A: <

Original: Predicted Answer> …
Revised: Predicted Answer> …

Original: Subproblems
Revised: Subproblems

Original: Q: What was produced by an art director that M1 and M2 employed and was directed by M
Revised: Q: What was produced by an art director that M1 and M2 employed and was directed by M

Original: Input
Revised: Input

Original: A: SELECT DISTINCT WHERE { ?x0 produced_by ?
Revised: A: SELECT DISTINCT WHERE { ?x0 produced_by ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 a art_director .
Revised: x1 a art_director .

Original: M1 employed ?
Revised: M1 employed ?

Original: x1 .
Revised: x1 .

Original: M2 employed ?
Revised: M2 employed ?

Original: x1 .
Revised: x1 .

Original: x0 directed_by M3 }
Revised: x0 directed_by M3 }

Original: Output
Revised: Output

Original: Q: Who edited a film that M1 and M2 produced A: <Exemplar Answer> …
Revised: Q: Who edited a film that M1 and M2 produced A: <Exemplar Answer> …

Original: Exemplars
Revised: Exemplars

Original: ExemplarPool
Revised: ExemplarPool

Original: Sequentially Solve Subproblems
Revised: Sequentially Solve Subproblems

Original: LM
Revised: LM

Original: LM
Revised: LM

Original: What was produced by N1 that N2 employed and was directed by N
Revised: What was produced by N1 that N2 employed and was directed by N

Original: What was produced by (N1 that (N2 employed)) and was directed by N3 LM
Revised: What was produced by (N1 that (N2 employed)) and was directed by N3 LM

Original:  D YNAMIC L EAST - TO - MOST P ROMPTING
Revised:  D YNAMIC L EAST - TO - MOST P ROMPTING

Original: In this section, we introduce dynamic least-to-most prompting , which is an extension of least to most prompting that allows us to overcome the challenges stated above and consequently apply least-to-most prompting to more realistic natural language tasks.
Revised: In this section, we introduce dynamic least-to-most prompting , which is an extension of least to most prompting that allows us to overcome the challenges stated above and consequently apply least-to-most prompting to more realistic natural language tasks.

Original: We start by giving a high-level summary of this approach, which is outlined in Figure 2.
Revised: We start by giving a high-level summary of this approach, which is outlined in Figure 2.

Original: Decomposition using LM based syntactic 
Revised: Decomposition using LM-based syntactic parsing.

Original: parsing.

Original: We use a series of prompts to teach the language model to perform a syntactic parse of all possible input sentences
Revised: We use a series of prompts to teach the language model to perform a syntactic parse of all possible input sentences.

Original: This provides us with a tree-based decomposition rather than a linear decomposition obtained
Revised: This provides us with a tree-based decomposition rather than a linear decomposition obtained

Original: by traditional least-to-most prompting.
Revised: by traditional least-to-most prompting.

Original: Dynamic selection of exemplars based on the decomposition.
Revised: Dynamic selection of exemplars based on the decomposition.

Original: We sample a small subset of the training set as a pool of candidate exemplars.
Revised: We sample a small subset of the training set as a pool of candidate exemplars.

Original: For each new input sentence to process, we dynamically select exemplars from this pool such that they collectively demonstrate relevant knowledge needed to translate the input sentences.
Revised: For each new input sentence to process, we dynamically select exemplars from this pool such that they collectively demonstrate relevant knowledge needed to translate the input sentences.

Original: This is done by matching the decomposition tree of the input against the decomposition tree of the candidatethe exemplars.
Revised: This is done by matching the decomposition tree of the input against the decomposition tree of the candidate exemplars.

Original: Sequential solution based on the decomposition.
Revised: Sequential solution based on the decomposition.

Original: We use the tree-based decomposition of the input sentence to generate a linear sequence of other relevant simpler sentences.
Revised: We use the tree-based decomposition of the input sentence to generate a linear sequence of other relevant simpler sentences.

Original: We then construct a prompt including the dynamically selected exemplars and use it to sequentially predict the solutions for the simpler sentences before generating the ﬁnal output.
Revised: We then construct a prompt including the dynamically selected exemplars and use it to sequentially predict the solutions for the simpler sentences before generating the ﬁnal output.

Original: 3. D ECOMPOSITION USING LM- BASED SYNTACTIC PARSING
Revised: 3. D ECOMPOSITION USING LM- BASED SYNTACTIC PARSING

Original: As discussed in Section 2.3, decomposition is more challenging for realistic tasks such as CFQ and COGS than for artiﬁcial tasks like SCAN.
Revised: As discussed in Section 2.3, decomposition is more challenging for realistic tasks such as CFQ and COGS than for artiﬁcial tasks like SCAN.

Original: Indeed, while decomposing SCAN commands is similar to decomposing mathematical expressions with standard arithmetic operations, decomposing sentences corresponding to more realistic subsets of natural language essentially becomes a problem of syntactic parsing.
Revised: Indeed, while decomposing SCAN commands is similar to decomposing mathematical expressions with standard arithmetic operations, decomposing sentences corresponding to more realistic subsets of natural language essentially becomes a problem of syntactic parsing.

Original: We ﬁnd it natural to decompose problems using a tree structure guided by that syntax (see Figure 3).
Revised: We ﬁnd it natural to decompose problems using a tree structure guided by that syntax (see Figure 3).

Revised: To teach LMs to perform decomposition, we divide the syntactic parsing task into multiple steps, such as subclause identiﬁcation, noun phrase identiﬁcation, verb phrase identiﬁcation, phrase an-

Original: What (was produced by ((a art director) that (M1 and M2 employed)))
Revised: What (was produced by ((a art director) that (M1 and M2 employed)))

Original: What (was produced by (a art director))
Revised: What (was produced by (a art director))

Original: What (was directed by M3)
Revised: What (was directed by M3)

Original: What (was produced by ((a art director) that (M1 and M2 employed)) and (was directed by M3))
Revised: What (was produced by ((a art director) that (M1 and M2 employed)) and (was directed by M3))

Original: What (was produced by ((a art director) that (M1 employed))) 
Revised: What (was produced by ((a art director) that (M1 employed))) notation, and verb normalization.

Original: To teach LMs to perform decomposition, we divide the syntactic parsing task into multiple steps, such as subclause identiﬁcation, noun phrase identiﬁcation, verb phrase identiﬁcation, phrase annotation, and verb normalization.

Original: For each step, we provide the LM with exemplars that illustrate the task to be performed.
Revised: For each step, we provide the LM with exemplars that illustrate the task to be performed.

Original: 1,2 See Appendix A.1 and Appendix B.1 for detailed description of the decomposition process and prompts used for CFQ and COGS, respectively.
Revised: 1,2 See Appendix A.1 and Appendix B.1 for detailed description of the decomposition process and prompts used for CFQ and COGS, respectively.

Original: 3. D YNAMIC EXEMPLAR SELECTION
Revised: 3. D YNAMIC EXEMPLAR SELECTION

Original: Prompt size is limited, so rather than attempt to represent all relevant knowledge in single prompt, for each input we dynamically select a set of relevant exemplars from a pre-selected exemplar pool.
Revised: Prompt size is limited, so rather than attempt to represent all relevant knowledge in single prompt, for each input we dynamically select a set of relevant exemplars from a pre-selected exemplar pool.

Original: Choosing the exemplar pool.
Revised: Choosing the exemplar pool.

Original: The exemplar pool is typically a small subset of the available training data.
Revised: The exemplar pool is typically a small subset of the available training data.

Original: The knowledge required for CFQ is rich and diverse, so we randomly sample 1000 exemplars from the training data for this purpose (separately for each split).
Revised: The knowledge required for CFQ is rich and diverse, so we randomly sample 1000 exemplars from the training data for this purpose (separately for each split).

Original: For COGS, including in the exemplar pool translations of as many different verbs and verb phrases is important.
Revised: For COGS, it is important to include in the exemplar pool translations of as many different verbs and verb phrases as possible.

Original: Therefore, we selected from the training data one exemplar for each unergative and unaccusative verb as well as 3 examples for each type of verb phrase (e.g., active, passive, with and without recipient, etc.).
Revised: Therefore, we selected from the training data one exemplar for each unergative and unaccusative verb, as well as 3 examples for each type of verb phrase (e.g., active, passive, with and without recipient, etc.).

Original: This resulted in a relatively small exemplar pool consisting of only 89 exemplars (which is used in addition to a static context consisting of 28 exemplars).
Revised: This resulted in a relatively small exemplar pool consisting of only 89 exemplars (which is used in addition to a static context consisting of 28 exemplars).

Original: Selecting exemplars.
Revised: Selecting exemplars.

Original: The goal of exemplar selection is to provide the LLM with the most relevant information needed to process a given input sentence.
Revised: The goal of exemplar selection is to provide the LLM with the most relevant information needed to process a given input sentence.

Original: We do this by making sure that as many nodes as possible of the decomposition tree of the input are covered by the decomposition trees of the selected exemplars.
Revised: We do this by making sure that as many nodes as possible of the decomposition tree of the input are covered by the decomposition trees of the selected exemplars.

Original: Speciﬁcally, we perform the following bottom-up and top-down matching
Revised: Speciﬁcally, we perform the following bottom-up and top-down matching.

Original: • Top-down matching: We begin by anonymizing the decomposition tree of the input.
Revised: • Top-down matching: We begin by anonymizing the decomposition tree of the input.

Original: For instance, the example “What was produced by a art director that M1 and M2 employed” is anonymized to “What V N that M and M V”, where V stands for verb, N for noun, and M for entity.
Revised: For instance, the example “What was produced by a art director that M1 and M2 employed” is anonymized to “What V N that M and M V”, where V stands for verb, N for noun, and M for entity.

Original: Starting at the top of the anonymized tree, we use a heuristic approach to ﬁnd exemplars such that all nodes are covered, prioritizing exemplars that match large subtrees.
Revised: Starting at the top of the anonymized tree, we use a heuristic approach to ﬁnd exemplars such that all nodes are covered, prioritizing exemplars that match large subtrees.

Original: • Bottom-up matching :
Revised: • Bottom-up matching :

Original: Then, we try to make sure that all leaf phrases are covered by an exemplar.
Revised: Then, we try to make sure that all leaf phrases are covered by an exemplar.

Original: If there is more than one exemplar for a certain phrase, we prefer exemplars where the phrase occurs within a similar anonymized subtree.
Revised: If there is more than one exemplar for a certain phrase, we prefer exemplars where the phrase occurs within a similar anonymized subtree.

Original: For instance, for the phrase “M and M2 employed” (which anonymizes to “M and M V”), we would prefer an exemplar containing “M1 and M2 produced” over both “art director employed” and “directed by M and M2”.
Revised: For instance, for the phrase “M and M2 employed” (which anonymizes to “M and M V”), we would prefer an exemplar containing “M1 and M2 produced” over both “art director employed” and “directed by M and M2”.

Original: Depending on its complexity, we select for each input between 4 and 35 exemplars for CFQ and between 1 and 3 exemplars for COGS.
Revised: Depending on its complexity, we select for each input between 4 and 35 exemplars for CFQ and between 1 and 3 exemplars for COGS.

Original: Full details of the anonymized tree and our heuristic matching algorithms are in Appendix A.2 for CFQ and Appendix B.2 for COGS.
Revised: Full details of the anonymized tree and our heuristic matching algorithms are in Appendix A.2 for CFQ and Appendix B.2 for COGS.

Original: speaks for ability of to perform syntactic parsing the is broken down into individual steps that are illustrated with appropriate prompts

Original: [Figure]
Revised: [Figure]

Original: 3. S EQUENTIAL SOLUTION
Revised: 3. S EQUENTIAL SOLUTION

Original: This is similar to the solution step for traditional least-to-most prompting.
Revised: This is similar to the solution step for traditional least-to-most prompting.

Original: The main difference is  we cannot translate the constituents in isolation because they might not be well-formed sentences and their translation is context-dependent.
Revised: The main difference is that we cannot translate the constituents in isolation because they might not be well-formed sentences and their translation is context-dependent.

Original: Instead, we linearize the decomposition tree into a sequence of increasingly complex subproblems and sequentially predict their solutions.
Revised: Instead, we linearize the decomposition tree into a sequence of increasingly complex subproblems and sequentially predict their solutions.

Original: In Figure
Revised: In Figure

Original: [Figure]
Revised: [Figure]

Original: we show a barebones snapshot of dynamic least-to-most prompting, immediately before predicting the ﬁnal output.
Revised: we show a barebones snapshot of dynamic least-to-most prompting, immediately before predicting the ﬁnal output.

Original: In previous steps, the model predicted via prompt the solution to the ﬁrst subproblem, “What was directed by M3”, then appended the prediction to the prompt before proceeding with, “What was produced by an art director”, etc.
Revised: In previous steps, the model predicted via prompting the solution to the ﬁrst subproblem, “What was directed by M3”, then appended the prediction to the prompt before proceeding with, “What was produced by an art director”, etc.

Original: Not displayed in the ﬁgure are dynamically selected exemplars, and a ﬁxed list of  exemplars  we prepend to each prompt.
Revised: Not displayed in the ﬁgure are dynamically selected exemplars and a ﬁxed list of partial exemplars that we prepend to each prompt.

Original:  B ASELINE : P ROMPTING WITHOUT D ECOMPOSITION
Revised:  B ASELINE : P ROMPTING WITHOUT D ECOMPOSITION

Original: To demonstrate the effectiveness of dynamic least-to-most prompting, we compare against a strong prompting baseline called chain-of-thought prompting (Wei et al., 2022b).
Revised: To demonstrate the effectiveness of dynamic least-to-most prompting, we compare against a strong prompting baseline called chain-of-thought prompting (Wei et al., 2022b).

Original: Chain-of-thought generates intermediate steps before predicting the ﬁnal answer and has been shown to improve reasoning capabilities of language models.
Revised: Chain-of-thought generates intermediate steps before predicting the ﬁnal answer and has been shown to improve reasoning capabilities of language models.

Original: Unlike least-to-most prompting, chain-of-thought does not have a decomposition step, potentially limiting its effectiveness for compositional generalization tasks.
Revised: Unlike least-to-most prompting, chain-of-thought does not have a decomposition step, potentially limiting its effectiveness for compositional generalization tasks.

Original: 4. C HAIN - OF -T HOUGHT P ROMPT D ESIGN
Revised: 4. C HAIN - OF -T HOUGHT P ROMPT D ESIGN

Original: Our chain-of-thought prompt is shown in Figure 4.
Revised: Our chain-of-thought prompt is shown in Figure 4.

Original: It ﬁrst categorizes the query, then generates quasi-alignments between the text and output statements.
Revised: It ﬁrst categorizes the query, then generates quasi-alignments between the text and output statements.

Original: This represents synchronicity in the same spirit as other higher performing semantic parsers do (Qiu et al., 2022a), but the intermediate constituents and clauses need not map exactly to what is seen in the input and output.
Revised: This represents synchronicity in the same spirit as other higher performing semantic parsers do (Qiu et al., 2022a), but the intermediate constituents and clauses need not map exactly to what is seen in the input and output.

Original: The ﬂexibility of chain-of-thought is convenient for data like CFQ where inclusion of variables is not compatible with synchronous grammar training and inference (Wong & Mooney, 2007).
Revised: The ﬂexibility of chain-of-thought is convenient for data like CFQ where inclusion of variables is not compatible with synchronous grammar training and inference (Wong & Mooney, 2007).

Original: 4. D YNAMICALLY SELECTING EXEMPLARS BASED ON LEXICAL SIMILARITY
Revised: 4. D YNAMICALLY SELECTING EXEMPLARS BASED ON LEXICAL SIMILARITY

Original: Since there is no decomposition with chain-of-thought, we rank exemplars by bag-of-words similarity with the current sentence.
Revised: Since there is no decomposition with chain-of-thought, we rank exemplars by bag-of-words similarity with the current sentence.

Original: To reduce redundancy, we select exemplars one by one, and at each iteration prioritize exemplars with relevant words that have not been found yet.
Revised: To reduce redundancy, we select exemplars one by one, and at each iteration prioritize exemplars with relevant words that have not been found yet.

Original: Thisis approach is not deterministic—selecting different exemplars in the ﬁrst iteration can alter results.
Revised: This approach is not deterministic—i.e., selecting different exemplars in the ﬁrst iteration can alter results.

Original: In practice, when using chain-of-thought we ﬁnd it is helpful to sample multiple exemplar lists, then use
Revised: In practice, when using chain-of-thought we ﬁnd it is helpful to sample multiple exemplar lists, then use

Original:  E XPERIMENTS AND R ESULTS 5. D ATA
Revised:  E XPERIMENTS AND R ESULTS 5. D ATA

Original: Datasets.
Revised: Datasets.

Original: We empirically measure the effectiveness of prompting on two semantic parsing datasets.
Revised: We empirically measure the effectiveness of prompting on two semantic parsing datasets.

Original: CFQ (Keysers et al., 2020) has three maximum compound divergence splits (MCD1,
Revised: CFQ (Keysers et al., 2020) has three maximum compound divergence splits (MCD1,

Original: MCD2, MCD3) for measuring compositional generalization, each with 95743/11968/11968 sentences in their train/validation/test splits.
Revised: MCD2, MCD3) for measuring compositional generalization, each with 95743/11968/11968 sentences in their train/validation/test splits.

Original: COGS (Kim & Linzen, 2020) has 24155/3000/ sentences in their train/validation/generalization splits.
Revised: COGS (Kim & Linzen, 2020) has 24155/3000/ sentences in their train/validation/generalization splits.

Original: Exemplar pools.
Revised: Exemplar pools.

Original: Also described in Section 3.2.
Revised: Also described in Section 3.2.

Original: For each CFQ split, we sampled 1000 training examples to use as potential exemplars (about 1% of the training data).
Revised: For each CFQ split, we sampled 1000 training examples to use as potential exemplars (about 1% of the training data).

Original: For COGS, we manually selected 89 training examples as potential exemplars (about 0.4% of the training data).
Revised: For COGS, we manually selected 89 training examples as potential exemplars (about 0.4% of the training data).

Original: Preprocessing.
Revised: Preprocessing.

Original: For CFQ we replace freebase identiﬁers with human-readable strings.
Revised: For CFQ we replace freebase identiﬁers with human-readable strings.

Original: 4 We also remove clauses that are redundant from a prediction perspective, which always appear alongside the same properties in both train and evaluation data.
Revised: 4 We also remove clauses that are redundant from a prediction perspective, which always appear alongside the same properties in both train and evaluation data.

Original: For COGS we use the variable-free and equivalent outputs introduced by Qiu et al.
Revised: For COGS we use the variable-free and equivalent outputs introduced by Qiu et al.

Original: (2022a).
Revised: (2022a).

Original: See Appendix D for details.
Revised: See Appendix D for details.

Original: Evaluation.
Revised: Evaluation.

Original: We measure accuracy using exact match (EM).
Revised: We measure accuracy using exact match (EM).

Original: This is computed as an exact string match between ground truth and predict labels, and no partial credit is assigned.
Revised: This is computed as an exact string match between ground truth and predict labels, and no partial credit is assigned.

Original: To make this metric interpretable for CFQ we apply normalization to outputs, including sorting properties and applying a deterministic argument ordering (additional details in Appendix D.1.2).
Revised: To make this metric interpretable for CFQ we apply normalization to outputs, including sorting properties and applying a deterministic argument ordering (additional details in Appendix D.1.2).

Original: For COGS we add any missing closing parentheses at the end of the output before computing EM.
Revised: For COGS we add any missing closing parentheses at the end of the output before computing EM.

Original: 5. E XPERIMENTAL D ESIGN C HOICES
Revised: 5. E XPERIMENTAL D ESIGN C HOICES

Original: The number of exemplars used in dynamic least-to-most varies based on the complexity of the decomposition tree, as does the number of subproblems.
Revised: The number of exemplars used in dynamic least-to-most varies based on the complexity of the decomposition tree, as does the number of subproblems.

Original: Predictions are made with greedy decoding.
Revised: Predictions are made with greedy decoding.

Original: Since there is a single ﬁnal output, no self-consistency is needed.
Revised: Since there is a single ﬁnal output, no self-consistency is needed.

Original: To improve performance with vanilla few-shot prompts (simple input/output exemplars) and chainof-thought prompts, we sample n = 4 different exemplar lists (each consisting  k = 15 exemplars), then sample s = 4 outputs per list using temperature-based decoding, yielding n · s = 16 outputs that are aggregated with self-consistency.
Revised: To improve performance with vanilla few-shot prompts (simple input/output exemplars) and chainof-thought prompts, we sample n = 4 different exemplar lists (each consisting of k = 15 exemplars), then sample s = 4 outputs per list using temperature-based decoding, yielding n · s =outputs that are aggregated with self-consistency.

Original: We use code-davinci-002 hosted by OpenAI for all experiments described in this paper.
Revised: We use code-davinci-002 hosted by OpenAI for all experiments described in this paper.

Original: This is a version of InstructGPT (Ouyang et al., 2022) ﬁnetuned on code, referred to as Codex.
Revised: This is a version of InstructGPT (Ouyang et al., 2022) ﬁnetuned on code, referred to as Codex.

Original: 7 Hyperparameters are summarized in Appendix D.3.
Revised: 7 Hyperparameters are summarized in Appendix D.3.

Original: 5. R ESULTS
Revised: 5. R ESULTS

Original: CFQ Our main results are on CFQ test splits and are reported in Table 1.
Revised: CFQ Our main results are on CFQ test splits and are reported in Table 1.

Original: Not only does this show that prompting enables compositional generalization on a realistic natural language task, dynamic least-to-most sets a new state of the art (95.0% accuracy) while only using about 1% of the training data, whereas traditional approaches are fully supervised
Revised: Not only does this show that prompting enables compositional generalization on a realistic natural language task, dynamic least-to-most sets a new state of the art (95.0% accuracy) while only using about 1% of the training data, whereas traditional approaches are fully supervised. MCD MCD MCD

Revised: Ave.

Revised: Fully Supervised

Original: [Table]
Revised: T5-base 58. T5-large 65. T5-3B 65. HPD 79.

Revised: T5-base + IR 85.

Revised: T5-large + IR 88.

Revised: T5-3B + IR 88.

Revised: LeAR 91.

Revised: Prompting (Ours) Dyn.

Revised: L2M 94.

Revised: [Figure]

Revised: Fully Supervised

Revised: LeAR (Liu et al., 2021)

Revised: [Figure]

Revised: T

Original: [Table]
Revised: [Figure] T [Figure] [Figure]

Revised: Prompting (Ours) Dynamic Least

Original: [Figure]
Revised: [Figure]

Revised: Most 99.

Original: -of-thought, we sample multiple exemplar lists and outputs with temperature-based decoding, then apply self-consistency
Revised: MCD Dynamic Least-to-Most 94. Using BoW Exemplars 92. Using Constant Exemplars

Original:  COGS We run experiments on COGS with minimal changes to our approach (Table 2) even though the output space is quite different from CFQ.
Revised: [Table] COGS We run experiments on COGS with minimal changes to our approach (Table 2) even though the output space is quite different from CFQ.

Original: Dynamic least-to-most prompting scores 99.2% accuracy (using 0.4% of the training data), reinforcing the generic nature of our approach.
Revised: Dynamic least-to-most prompting scores [Table] 2% accuracy (using 0.4% of the training data), reinforcing the generic nature of our approach.

Original:  D ISCUSSION AND A NALYSIS
Revised:  D ISCUSSION AND A NALYSIS

Original: Is dynamic least-to-most more effective than other prompting methods? In Figure 5, we compare dynamic least-to-most (L2M) with vanilla few-shot prompting and chain-of-thought prompting  enhanced with self-consistency.
Revised: Comparison of prompting methods In Figure 5, we compare dynamic least-to-most (L2M) with vanilla few-shot prompting and chain-of-thought prompting (CoT) enhanced with self-consistency.

Original: We measure performance on a random 500-sentence subset of CFQ’s MCD1 test split.
Revised: We measure performance on a random 500-sentence subset of CFQ’s MCD1 test split.

Original: Chain-of-thought (87.2%) outperforms vanilla (80.8%) and most baselines when extrapolating to the full data, but dynamic least-to-most (94.4%) achieves the best result. 
Revised: CoT (87.2%) outperforms vanilla (80.8%) and most baselines when extrapolating to the full data, but L2M (94.6%) achieves the best result. L2M is both more accurate and more efﬁcient than other prompting methods.

Original: Additionally, dynamic least-to-most is more than 2x as fast as chain-of-thought despite its sequential nature, since chain-of-thought uses self-consistency across multiple exemplar lists and outputs.
Revised: L2M performs favorably across different sizes of the exemplar pool. It is also 2x as fast as CoT despite its sequential nature, since CoT uses self-consistency across multiple exemplar lists and outputs.

Original: How many exemplars are needed in the pool?
Revised: Without selfconsistency, CoT accuracy drops from 87.2% to 75.4% and vanilla drops from 80.8% to 69.8%.

Original: Figure 5 shows performance as we decrease the size of the exemplar pool. Dynamic least-to-most outperforms other prompts, but performance degrades especially once the exemplar pool contains only 15 exemplars.
Revised: Exemplar selection and question decomposition In Table 3, we ablate conﬁgurations for dynamic least-to-most. We measure performance on a random 500-sentence subset of CFQ’s MCD test split.

Original: At this size, each input will basically have the same exemplars and many properties will not be represented at all.
Revised: Performance drops from 94.6% to 92.2% when selecting exemplars according to our bag-of-words (BoW) algorithm or to 76.4% when using a ﬁxed set of exemplars for all examples.

Original: The drop in performance is expected since exemplars are randomly chosen.
Revised: Both exemplar selection and question decomposition are crucial for the best L2M performance.

Original: We achieved high performance on COGS with a small exemplar pool (89 instances) by choosing them methodically.
Revised: L2M uses a different number of decomposition steps per example. If we reduce the amount of decomposition to half as many steps then performance drops to 88.8%, or to 83.2% when doing at most two steps.

Original: Why is Codex effective for semantic parsing?
Revised: If we skip decomposition entirely then performance drops further to 75.2%.

Original: We use Codex instead of text-based GPT-3 in our evaluation because of its better performance in our initial experiments, and this observation is also presented in prior works on semantic parsing (Shin et al., 2021; Shin & Van Durme, 2022).
Revised: See Appendix E.2 for more ablations together with a qualitative analysis of the associated errors.

Original:  For pretrained models, leakage of the test data is a potential concern (Krishna et al., 2020; Carlini et al., 2021).
Revised: Risk of memorization For pretrained models, leakage of the test data is a potential concern (Krishna et al., 2020; Carlini et al., 2021).

Original: However, given the inferior performance of vanilla few-shot prompting, we attribute success of prompting to in-context learning rather than memorization.
Revised: However, given the inferior performance of vanilla few-shot prompting, we attribute success of prompting to in-context learning rather than memorization.

Original: Furthermore, least-to-most prompting requires the LM to translate as intermediate steps new subproblems that are guaranteed to be unseen during training.
Revised: Furthermore, least-to-most prompting requires the LM to translate as intermediate steps new subproblems that are guaranteed to be unseen during training.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Compositional generalization Compositional generalization in machine learning has been a challenging problem that attracts attention across ﬁelds, including vision (Johnson et al., 2017;
Revised: Compositional generalization Compositional generalization in machine learning has been a challenging problem that attracts attention across ﬁelds, including vision (Johnson et al., 2017;

Original: Bahdanau et al., 2019;
Revised: Bahdanau et al., 2019;

Original: Ruis et al., 2020; Nikolaus et al., 2019) and language domains (Lake & Baroni, 2018;
Revised: Ruis et al., 2020; Nikolaus et al., 2019) and language domains (Lake & Baroni, 2018;

Original: Keysers et al., 2020; Kim & Linzen, 2020; Shaw et al., 2021;
Revised: Keysers et al., 2020; Kim & Linzen, 2020; Shaw et al., 2021;

Original: Yin et al., 2021; Gan et al., 2022).
Revised: Yin et al., 2021; Gan et al., 2022).

Original: A number of approaches have been proposed to improve compositional generalization on SCAN (Lake & Baroni, 2018;
Revised: A number of approaches have been proposed to improve compositional generalization on SCAN (Lake & Baroni, 2018;

Original: Loula et al., 2018), including specialized design of neural model architectures (Chen et al., 2020;
Revised: Loula et al., 2018), including specialized design of neural model architectures (Chen et al., 2020;

Original: Nye et al., 2020; Liu et al., 2020; Shaw et al., 2021; Russin et al., 2019; Li et al., 2019; Gordon et al., 2020; Herzig & Berant, 2021) and training algorithms (Lake, 2019; Kim, 2021), training data augmentation (Andreas, 2020; Aky¨urek et al., 2021), and prompting (Zhou et al., 2022).
Revised: Nye et al., 2020; Liu et al., 2020; Shaw et al., 2021; Russin et al., 2019; Li et al., 2019; Gordon et al., 2020; Herzig & Berant, 2021) and training algorithms (Lake, 2019; Kim, 2021), training data augmentation (Andreas, 2020; Aky¨urek et al., 2021), and prompting (Zhou et al., 2022).

Original: While 100% accuracy has been accomplished on SCAN (Chen et al., 2020; Nye et al., 2020; Liu et al., 2020; Shaw et al., 2021), good performance on SCAN does not necessarily transfer to more challenging compositional generalization problems (Furrer et al., 2020).
Revised: While 100% accuracy has been accomplished on SCAN (Chen et al., 2020; Nye et al., 2020; Liu et al., 2020; Shaw et al., 2021), good performance on SCAN does not necessarily transfer to more challenging compositional generalization problems (Furrer et al., 2020).

Original: Notably, although least-to-most prompting has achieved 99 .
Revised: Notably, although least-to-most prompting has achieved 99 .

Original: 7% accuracy on SCAN (Zhou et al., 2022), prior attempts on prompting for semantic parsing still demonstrate limited compositional generalization performance (Qiu et al., 2022b).
Revised: 7% accuracy on SCAN (Zhou et al., 2022), prior attempts on prompting for semantic parsing still demonstrate limited compositional generalization performance (Qiu et al., 2022b).

Original: In this work, we propose prompting schemes to bridge this gap.
Revised: In this work, we propose prompting schemes to bridge this gap.

Original: To improve compositional generalization for semantic parsing, recent works incorporate a latent syntactic component (Qiu et al., 2022a; Liu et al., 2021).
Revised: To improve compositional generalization for semantic parsing, recent works incorporate a latent syntactic component (Qiu et al., 2022a; Liu et al., 2021).

Original: Similarly to symbolic grammar learning techniques on SCAN, these approaches achieve impressive performance on several benchmarks, and represent the previous state of the art on CFQ (Keysers et al., 2020) and COGS (Kim & Linzen, 2020).
Revised: Similarly to symbolic grammar learning techniques on SCAN, these approaches achieve impressive performance on several benchmarks, and represent the previous state of the art on CFQ (Keysers et al., 2020) and COGS (Kim & Linzen, 2020).

Original: Other lines of work improve the performance on CFQ through specialized decoding algorithms, including graph decoding (Gai et al., 2021) and hierarchical poset decoding (Guo et al., 2020).
Revised: Other lines of work improve the performance on CFQ through specialized decoding algorithms, including graph decoding (Gai et al., 2021) and hierarchical poset decoding (Guo et al., 2020).

Original: Yet others exploit correlations between the input and output tokens, e.g. through loss functions with attention supervision (Yin et al., 2021), using a lexicon (Aky ¨ urek & Andreas, 2021), and reformulating the label space (Herzig et al., 2021).
Revised: Yet others exploit correlations between the input and output tokens, e.g. through loss functions with attention supervision (Yin et al., 2021), using a lexicon (Aky ¨ urek & Andreas, 2021), and reformulating the label space (Herzig et al., 2021).

Original: Without relying on specialized model architectures or training algorithms, our results demonstrate generic prompting schemes based on decomposition demonstrate strong results on CFQ and COGS, and achieve state-of-the-art results on CFQ.
Revised: Without relying on specialized model architectures or training algorithms, our results demonstrate generic prompting schemes based on decomposition demonstrate strong results on CFQ and COGS, and achieve state-of-the-art results on CFQ.

Original:  The most similar work to ours is SeqZero (Yang et al., 2022), but there are key differences.
Revised: Prompting The most similar work to ours is SeqZero (Yang et al., 2022), but there are key differences.

Original: SeqZero decomposes semantic parsing into generating three parts separately (SELECT,
Revised: SeqZero decomposes semantic parsing into generating three parts separately (SELECT,

Original: FROM, and WHERE parts of the output), and further decomposes WHERE into generating each clause separately.
Revised: FROM, and WHERE parts of the output), and further decomposes WHERE into generating each clause separately.

Original: This means that decomposition is conducted via a ﬁxed, rule-based system.
Revised: This means that decomposition is conducted via a ﬁxed, rule-based system.

Original: In contrast, our decomposition is automatically accomplished by prompting.
Revised: In contrast, our decomposition is automatically accomplished by prompting.

Original: We use the syntactic parse of the sentence to create different related sentences, such as by simplifying conjunctions or removing text fragments.
Revised: We use the syntactic parse of the sentence to create different related sentences, such as by simplifying conjunctions or removing text fragments.

Original: This is more general than SeqZero and it is readily applicable to many natural language tasks.
Revised: This is more general than SeqZero and it is readily applicable to many natural language tasks.

Original: For example, we successfully used our approach on COGS even though the output does not resemble SQL.
Revised: For example, we successfully used our approach on COGS even though the output does not resemble SQL.

Original: Furthermore, SeqZero is an ensemble of a ﬁnetuned BART and a zero-shot model while we use only prompting with large language models and forego ﬁnetuning entirely.
Revised: Furthermore, SeqZero is an ensemble of a ﬁnetuned BART and a zero-shot model, while we use only prompting with large language models and forego ﬁnetuning entirely.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Through dynamic least-to-most prompting, we demonstrate state-of-the-art performance on a difﬁcult natural language semantic parsing benchmark that measures compositional generalization.
Revised: Through dynamic least-to-most prompting, we demonstrate state-of-the-art performance on a difﬁcult natural language semantic parsing benchmark that measures compositional generalization.

Original: Our results are achieved using about 1% of the training data from traditional ﬁnetuning approaches.
Revised: Our results are achieved using about 1% of the training data from traditional ﬁnetuning approaches.

Original: Many machine learning models struggle with compositional generalization, and our ﬁndings should facilitate future research that enables this capability through task decomposition.
Revised: Many machine learning models struggle with compositional generalization, and our ﬁndings should facilitate future research that enables this capability through task decomposition.

Original: We expect dynamic least-to-most prompting to have immediate impact in a variety of settings.
Revised: We expect dynamic least-to-most prompting to have immediate impact in a variety of settings.

Original: It is ﬂexible and general purpose, enabling quick adaptation to new tasks and domains. Especially for knowledge-intensive applications of language models, where precise semantic parsing can be directly leveraged.
Revised: It is ﬂexible and general purpose, enabling quick adaptation to new tasks and domains, especially for knowledge-intensive applications of language models, where precise semantic parsing can be directly leveraged.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: Throughout our work we aim to provide exhaustive details about prompt design and exemplar selection, and we include all the prompts we use in the Appendix.
Revised: Throughout our work we aim to provide exhaustive details about prompt design and exemplar selection, and we include all the prompts we use in the Appendix.

Original: To ease future use, we further outline key details related to reproducibility in Appendix F
Revised: To ease future use, we further outline key details related to reproducibility in Appendix F

Revised: Appendix Table of Contents

Original: A L EAST - TO - MOST P ROMPTING FOR CFQ
Revised: A Least-to-most Prompting for CFQ

Revised: A.1 CFQ Decomposition: Details and Prompts . . . . . . . . . . . . . . . . . . . .

Revised: A.1.1 Step 1: Noun phrase identiﬁcation . . . . . . . . . . . . . . . . . . . .

Revised: A.1.2 Step 2: Subclause identiﬁcation . . . . . . . . . . . . . . . . . . . . . .

Revised: A.1.3 Step 3: Verb phrase identiﬁcation . . . . . . . . . . . . . . . . . . . . .

Revised: A.1.4 Step 4: Part of speech tagging . . . . . . . . . . . . . . . . . . . . . . .

Revised: A.1.5 Step 5: Verb normalization . . . . . . . . . . . . . . . . . . . . . . . .

Revised: A.2 CFQ Exemplar Selection: Details . . . . . . . . . . . . . . . . . . . . . . . . .

Revised: A.2.1 Top-down matching . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Revised: A.2.2 Bottom-up matching . . . . . . . . . . . . . . . . . . . . . . . . . . .

Revised: A.3 CFQ Solution: Details and Prompt . . . . . . . . . . . . . . . . . . . . . . . .

Revised: A.3.1 Part 1: Static prompt context. . . . . . . . . . . . . . . . . . . . . . . .

Revised: A.3.2 Part 2: Dynamically selected exemplars . . . . . . . . . . . . . . . . . .

Revised: A.3.3 Part 3: Sequential questions . . . . . . . . . . . . . . . . . . . . . . . .

Revised: [Table]

Revised: B

Revised: [Table]

Revised: Decomposition: Details and Prompts

Revised: B.1.

Revised: [Table]

Revised: Iterative subclause decomposition

Revised: [Table]

Revised: [Table]

Revised: Phrase identiﬁcation

Revised: [Table]

Revised: Iterative prepositional phrase decomposition and noun phrase annotation

Revised: [Table]

Revised: normalization

Revised: [Table]

Revised: Details

Revised: [Table]

Revised: pool

Revised: [Table]

Revised: [Table]

Revised: Exemplar matching

Revised: [Table]

Revised: Solution

Revised: [Table]

Revised: Details and Prompts

Revised: [Table]

Revised: Part 1: Static prompt context

Revised: [Table]

Revised: Part 2: Dynamically selected exemplars

Revised: [Table]

Revised: Part 3: Sequential subproblems

Revised: [Table]

Revised: [Table]

Revised: Bootstrap Prompt

Revised: C

Revised: [Table]

Revised: D Data Preparation, Evaluation, and Hyperparameters

Revised: D.

Revised: [Table]

Revised: Processing and Evaluation . . .

Revised: D.1.

Revised: [Table]

Revised: CFQ Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Revised: D.1.

Revised: [Table]

Revised: CFQ

Revised: [Table]

Revised: and Evaluation . .

Revised: D.

Revised: [Table]

Revised: COGS

Revised: [Table]

Revised: and Evaluation . .

Revised: D.

Revised: [Table]

Revised: Hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Revised: D.3.

Revised: [Table]

Revised: Prompt Hyperparameters . . . . . . . . . . . . . . . . . . . . . . . . .

Revised: D.3.

Revised: [Table]

Revised: Generation Hyperparameters . . . . . . . . . . . . . . . . . . . . . . .

Revised: E Additional Analysis

Revised: [Table]

Revised: Initial Prompting Attempts for CFQ

Revised: [Table]

Revised: [Table]

Revised: Least-to-most Prompting on CFQ: Ablations and

Revised: [Table]

Revised: Analysis

Revised: E.

Revised: 3 Fair Comparison against Other Prompting Techniques

Revised: F Reproducibility Summary

Revised: Which ﬁlm editor was inﬂuenced by a cinematographer that wrote M3 , M4 , and M5 and inﬂuenced by M1 ’s editor

Revised: SELECT DISTINCT ?

Revised: x0 WHERE { ?x0 a film editor .

Revised: x0 influenced by ?x1 . ?

Revised: x0 influenced by ?

Revised: x2 .

Revised: x1 edited M1 .

Revised: x2 a cinematographer . ?

Revised: x

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: A L EAST - TO - MOST P ROMPTING FOR CFQ

Original: In this section, we provide more details on the application of dynamic least-to-most prompting for
Revised: In this section, we provide more details on the application of dynamic least-to-most prompting for

Original: CFQ.
Revised: CFQ.

Original: In particular, we detail all the prompts and show how the example in Figure 6 is processed step by step.
Revised: In particular, we detail all the prompts and show how the example in Figure 6 is processed step by step.

Original: A. CFQ D ECOMPOSITION : D ETAILS AND P ROMPTS
Revised: A. CFQ D ECOMPOSITION : D ETAILS AND P ROMPTS

Original: As discussed in Section 3.1, we use prompting-based syntactic parsing to decompose CFQ questions.
Revised: As discussed in Section 3.1, we use prompting-based syntactic parsing to decompose CFQ questions.

Original: To teach LMs to perform this kind of decomposition, we divide the syntactic parsing task into the following steps:
Revised: To teach LMs to perform this kind of decomposition, we divide the syntactic parsing task into the following steps:

Original: Noun phrase identiﬁcation 2.
Revised: Noun phrase identiﬁcation 2.

Original: Subclause identiﬁcation 3.
Revised: Subclause identiﬁcation 3.

Original: Verb and other phrase identiﬁcation 4.
Revised: Verb and other phrase identiﬁcation 4.

Original: Part-of-speech tagging and phrase labeling 5.
Revised: Part-of-speech tagging and phrase labeling 5.

Original: Verb normalization Consider the question fromfrom Figure 6.
Revised: Verb normalization Consider the question from Figure 6.

Original: • Which ﬁlm editor was inﬂuenced by a cinematographer that wrote M3 , M4 , and M5 and inﬂuenced by M1 ’s editor
Revised: • Which ﬁlm editor was inﬂuenced by a cinematographer that wrote M3 , M4 , and M5 and inﬂuenced by M1 ’s editor

Original: The ﬁrst step, noun phrase identiﬁcation, yields:
Revised: The ﬁrst step, noun phrase identiﬁcation, yields:

Original: • Which (ﬁlm editor) was inﬂuenced by (a cinematographer) that wrote (M3 , M4 , and M5) and inﬂuenced by ( M ’s editor
Revised: • Which (ﬁlm editor) was inﬂuenced by (a cinematographer) that wrote (M3 , M4 , and M5) and inﬂuenced by (M1 ’s editor)

Original: Then, we replace the identiﬁed noun phrases with placeholders N1, N2, N3, and N4, which yields:
Revised: Then, we replace the identiﬁed noun phrases with placeholders N1, N2, N3, and N4, which yields:

Original: • Which N1 was inﬂuenced by N2 that wrote N3 and inﬂuenced by N
Revised: • Which N1 was inﬂuenced by N2 that wrote N3 and inﬂuenced by N

Original: The next step, subclause identiﬁcation, uses the form with placeholders and yields:
Revised: The next step, subclause identiﬁcation, uses the form with placeholders and yields:

Original: • Which N1 was inﬂuenced by ((N2)
Revised: • Which N1 was inﬂuenced by ((N2)

Original: that (wrote N3)) and inﬂuenced by N
Revised: that (wrote N3)) and inﬂuenced by N

Original: Then we have another round  applying placeholders, on the subclause this time:
Revised: Then we have another round of applying placeholders, on the subclause this time:

Original: • Which N1 was inﬂuenced by N5 and inﬂuenced by N
Revised: • Which N1 was inﬂuenced by N5 and inﬂuenced by N

Original: The third step is to identify verb phrases and other miscellaneous phrases, which yields:
Revised: The third step is to identify verb phrases and other miscellaneous phrases, which yields:

Original: • (Which) (N1) ((was inﬂuenced by N5) and (inﬂuenced by N4))
Revised: • (Which) (N1) ((was inﬂuenced by N5) and (inﬂuenced by N4))

Original: • that (wrote N3)
Revised: • that (wrote N3)

Original: As a fourth step, we perform various part-of-speech tagging and phrase labeling.
Revised: As a fourth step, we perform various part-of-speech tagging and phrase labeling.

Original: We start with the noun phrases, which yields:
Revised: We start with the noun phrases, which yields:

Original: • N=(ﬁlm editor) •
Revised: • N=(ﬁlm editor) •

Original: [a] N=(cinematographer) • M=(M3 , M4 , [and], M5) • M=(M1)
Revised: [a] N=(cinematographer) • M=(M3 , M4 , [and], M5) • M=(M1)

Original: P=(’s) N=(editor)
Revised: P=(’s) N=(editor)

Original: We continue with verb phrases, which yields:
Revised: We continue with verb phrases, which yields:

Original: • W=(Which) • V=([was] inﬂuenced by) (N5) • V=(inﬂuenced by) (N4) • V=(wrote) (N3)
Revised: • W=(Which) • V=([was] inﬂuenced by) (N5) • V=(inﬂuenced by) (N4) • V=(wrote) (N3)

Original: As a ﬁfth step, we normalize all the verbs: “was” yields “be”, “inﬂuenced” yields “inﬂuence”, and “wrote” yields “write”.
Revised: As a ﬁfth step, we normalize all the verbs: “was” yields “be”, “inﬂuenced” yields “inﬂuence”, and “wrote” yields “write”.

Original: Finally, we run a few lines of Python code that puts all these parts back together to obtain a parse tree.
Revised: Finally, we run a few lines of Python code that puts all these parts back together to obtain a parse tree.

Original: Note that the normalized verbs do not replace the original ones but are kept in addition, which allows us to obtain higher recall when selecting exemplars.
Revised: Note that the normalized verbs do not replace the original ones but are kept in addition, which allows us to obtain higher recall when selecting exemplars.

Original: This yields the following fully decomposed and annotated question:
Revised: This yields the following fully decomposed and annotated question:

Original: • W=(Which) N=(ﬁlm editor) ((V=([was] inﬂuenced by) ([a] N=(cinematographer) that (V=(wrote) N=(M3 , M4 , [and] M5)))) and (V=(inﬂuenced by) (M=(M1) P=(’s) N=(editor))))
Revised: • W=(Which) N=(ﬁlm editor) ((V=([was] inﬂuenced by) ([a] N=(cinematographer) that (V=(wrote) N=(M3 , M4 , [and] M5)))) and (V=(inﬂuenced by) (M=(M1) P=(’s) N=(editor))))

Original: Below, we present the exact prompt contexts used for each of the parsing steps.
Revised: Below, we present the exact prompt contexts used for each of the parsing steps.

Original: A.1. S TEP 1: N OUN 
Revised: A.1. S TEP 1: N OUN PHRASE IDENTIFICATION

Original: PHRASE IDENTIFICATION Q: Was M0 ’s producer a art director ’s sibling A:
Revised: Q: Was M0 ’s producer a art director ’s sibling A:

Original: Was (M0 ’s producer) (a art director ’s sibling) Q: Did M1 inﬂuence a French producer and editor ’s child ’s spouse A: Did (M1) inﬂuence (a French producer and editor ’s child ’s spouse) Q: Which female French costume designer , producer , and editor of M3 and M4 was M5 ’s parentA:Which (female French costume designer , producer , and editor of M3 and M4) was (M5 ’s parent) Q: Was a Dutch parent and spouse a editor , producer , and writer of a ﬁlm ’s prequel
Revised: Was (M0 ’s producer) (a art director ’s sibling) Q: Did M1 inﬂuence a French producer and editor ’s child ’s spouse A: Did (M1) inﬂuence (a French producer and editor ’s child ’s spouse) Q: Which female French costume designer , producer , and editor of M3 and M4 was M5 ’s parentA:Which (female French costume designer , producer , and editor of M3 and M4) was (M5 ’s parent) Q: Was a Dutch parent and spouse a editor , producer , and writer of a ﬁlm ’s prequel

Original: A:
Revised: A:

Original: Was (a Dutch parent and spouse) (a editor , producer , and writer of a ﬁlm ’s prequel) Q: Who employed , met , and was inﬂuenced by a female ﬁlm director ’s Spanish parent and married M A: Who employed , met , and was inﬂuenced by (a female ﬁlm director ’s Spanish parent) and married (M1) Q: Were M3 and M6 executive produced by , written by , and directed by a writer ’s spouse , friend , and employer , and produced by a child A: Were (M3 and M6) executive produced by , written by , and directed by (a writer ’s spouse , friend , and employer) , and produced by (a child) Q: What ﬁlm director and editor of M0 , M1 , and M2 married , divorced , and was kissed by a Spanish editor and producer of M A: What (ﬁlm director and editor of M0 , M1 , and M2) married , divorced , and was kissed by (a Spanish editor and producer of M5) 
Revised: Was (a Dutch parent and spouse) (a editor , producer , and writer of a ﬁlm ’s prequel) Q: Who employed , met , and was inﬂuenced by a female ﬁlm director ’s Spanish parent and married M A: Who employed , met , and was inﬂuenced by (a female ﬁlm director ’s Spanish parent) and married (M1) Q: Were M3 and M6 executive produced by , written by , and directed by a writer ’s spouse , friend , and employer , and produced by a child A: Were (M3 and M6) executive produced by , written by , and directed by (a writer ’s spouse , friend , and employer) , and produced by (a child) Q: What ﬁlm director and editor of M0 , M1 , and M2 married , divorced , and was kissed by a Spanish editor and producer of M A: What (ﬁlm director and editor of M0 , M1 , and M2) married , divorced , and was kissed by (a Spanish editor and producer of M5) Q: Which child of a production company did M0 acquire A: Which (child of a production company) did (M0) acquire

Original: Q: Whichchild of a production company did M0 acquire A: Which (child of a production company) did (M0) acquire Q: Which star of M5 was inﬂuenced by a person , inﬂuenced by M0 , M1 , M2 , and M3 , and inﬂuenced by M A: Which (star of M5) was inﬂuenced by (a person) , inﬂuenced by (M0 , M1 , M2 , and M3) , and inﬂuenced by (M4) Q: Was a screenwriter employed by M1 , M2 , and M3 and employed by M4 , M5 , and M M7 ’s spouse A:
Revised: Q: Which star of M5 was inﬂuenced by a person , inﬂuenced by M0 , M1 , M2 , and M3 , and inﬂuenced by M A: Which (star of M5) was inﬂuenced by (a person) , inﬂuenced by (M0 , M1 , M2 , and M3) , and inﬂuenced by (M4) Q: Was a screenwriter employed by M1 , M2 , and M3 and employed by M4 , M5 , and M M7 ’s spouse A:

Original: Was (a screenwriter) employed by (M1 , M2 , and M3) and employed by (M4 , M5 , and M6)
Revised: Was (a screenwriter) employed by (M1 , M2 , and M3) and employed by (M4 , M5 , and M6)

Original: (M ’s spouse) Q: Was M4 produced by a German screenwriter that M1 and spouse of M2 ’s sibling were inﬂuenced by A:
Revised: (M ’s spouse) Q: Was M4 produced by a German screenwriter that M1 and spouse of M2 ’s sibling were inﬂuenced by A:

Original: Was (M4) produced by (a German screenwriter) that (M1 and spouse of M2 ’s sibling) were inﬂuenced by Q: Was M0 M1 ’s sibling and spouse A: Was (M0) (M1 ’s sibling and spouse) Q: Which ﬁlm did M3 ’s employer ’s Mexican employee produce and M1 direct A: Which (ﬁlm) did (M3 ’s employer ’s Mexican employee) produce and (M1) direct Q: Did M3 ’s parent ’s employee inﬂuence M0 and M1 and inﬂuence M2 ’s founder and employeeA:Did(M3 ’s parent ’s employee) inﬂuence (M0 and M1) and inﬂuence (M2 ’s founder and employee) Q: What male director of M3 and M4 did M0 and
Revised: Was (M4) produced by (a German screenwriter) that (M1 and spouse of M2 ’s sibling) were inﬂuenced by Q: Was M0 M1 ’s sibling and spouse A: Was (M0) (M1 ’s sibling and spouse) Q: Which ﬁlm did M3 ’s employer ’s Mexican employee produce and M1 direct A: Which (ﬁlm) did (M3 ’s employer ’s Mexican employee) produce and (M1) direct Q: Did M3 ’s parent ’s employee inﬂuence M0 and M1 and inﬂuence M2 ’s founder and employeeA:Did(M3 ’s parent ’s employee) inﬂuence (M0 and M1) and inﬂuence (M2 ’s founder and employee) Q: What male director of M3 and M4 did M0 and

Original: M1 inﬂuence A: What (male director of M3 and M4) did (M0 and M1)
Revised: M1 inﬂuence A: What (male director of M3 and M4) did (M0 and M1)

Original: inﬂuence Q: Which female person whose sibling was inﬂuenced by M3 and was inﬂuenced by M4 and M5 directed M A: Which (female person) whose (sibling) was inﬂuenced by (M3) and was inﬂuenced by (M4 and M5) directed (M2) Q: Did M0 direct , produce , executive produce , edit , and write M1 , M2 , and M A: Did (M0) direct , produce , executive produce , edit , and write (M1 , M2 , and M3)
Revised: inﬂuence Q: Which female person whose sibling was inﬂuenced by M3 and was inﬂuenced by M4 and M5 directed M A: Which (female person) whose (sibling) was inﬂuenced by (M3) and was inﬂuenced by (M4 and M5) directed (M2) Q: Did M0 direct , produce , executive produce , edit , and write M1 , M2 , and M A: Did (M0) direct , produce , executive produce , edit , and write (M1 , M2 , and M3)

Original: Q: Was M0 executive produced by , edited by , written by , and directed by M1 , M2 , and M A:
Revised: Q: Was M0 executive produced by , edited by , written by , and directed by M1 , M2 , and M3A:

Original: Was (M0) executive produced by , edited by , written by , and directed by (M1 , M2 , and M3)
Revised: Was (M0) executive produced by , edited by , written by , and directed by (M1 , M2 , and M3)

Original: Q: Who inﬂuenced and was inﬂuenced by M1 ’s female actor ’s parent A: Who inﬂuenced and was inﬂuenced by (M1 ’s female actor ’s parent) Q: Did M0 ’s editor , costume designer , star , writer , and art director executive produce and produce M A: Did (M0 ’s editor , costume designer , star , writer , and art director) executive produce and produce (M1) Q: Which ﬁlm that was written by M1 M2 directed A: Which (ﬁlm) that was written by (M1) (M2) directed Q: Which director of M3 , M4 , and M5 was a Japanese screenwriter that M1 employed and was founded by A: Which (director of M3 , M4 , and M5) was (a Japanese screenwriter) that (M1) employed and was founded by 
Revised: Q: Who inﬂuenced and was inﬂuenced by M1 ’s female actor ’s parent A: Who inﬂuenced and was inﬂuenced by (M1 ’s female actor ’s parent) Q: Did M0 ’s editor , costume designer , star , writer , and art director executive produce and produce M A: Did (M0 ’s editor , costume designer , star , writer , and art director) executive produce and produce (M1) Q: Which ﬁlm that was written by M1 M2 directed A: Which (ﬁlm) that was written by (M1) (M2) directed Q: Which director of M3 , M4 , and M5 was a Japanese screenwriter that M1 employed and was founded by A: Which (director of M3 , M4 , and M5) was (a Japanese screenwriter) that (M1) employed and was founded by Q: Did M1 ’s executive producer employ M2 , edit M3 , employ a ﬁlm director , and employ M4A:

Original:  distributed 
Revised: Did (M1 ’s executive producer) employ (M2) , edit (M3) , employ (a ﬁlm director) , and employ (M4) Q: Was a ﬁlm whose star , writer , cinematographer , and executive producer executive produced , edited , and wrote M0 M A: Was (a ﬁlm) whose (star , writer , cinematographer , and executive producer) executive produced , edited , and wrote (M0) ( M1) Q: Was M1 a ﬁlm that M0 ’s editor distributed A: Was (M1)

Original:  distributed  child 
Revised: (a ﬁlm) that (M0 ’s editor) distributed Q: Was M1 a child of a production company ’s parent and child A: Was (M1)

Original: write A What did a director that 
Revised: (a child of a production company ’s parent and child) Q: What did a director that M1 inﬂuenced and M2 inﬂuenced write A:

Original: M write Q What was written by M0 ’s  executive producer A: What was written by (M0 ’s art director and executive producer) Q: Did M1 inﬂuence a production company , inﬂuence M2 , inﬂuence M3 , M4 , and M5 , and inﬂuence M A: Did (M1) inﬂuence (a production company) , inﬂuence (M2) , inﬂuence (M3 , M4 , and M5) , and inﬂuence (M6)
Revised: What did (a director) that (M1) inﬂuenced and (M2) inﬂuenced write Q: What was written by M0 ’s art director and executive producer A: What was written by (M0 ’s art director and executive producer) Q: Did M1 inﬂuence a production company , inﬂuence M2 , inﬂuence M3 , M4 , and M5 , and inﬂuence M A: Did (M1) inﬂuence (a production company) , inﬂuence (M2) , inﬂuence (M3 , M4 , and M5) , and inﬂuence (M6)

Original: A.1. S TEP 2: S UBCLAUSE IDENTIFICATION
Revised: A.1. S TEP 2: S UBCLAUSE IDENTIFICATION

Original: Q: What N1 that M0 was employed by directed M A: What ((N1) that (M0 was employed by)) directed M Q: What N1 was N2 that M2 inﬂuenced A: What N1 was ((N2) that (M2 inﬂuenced))
Revised: Q: What N1 that M0 was employed by directed M A: What ((N1) that (M0 was employed by)) directed M Q: What N1 was N2 that M2 inﬂuenced A: What N1 was ((N2) that (M2 inﬂuenced))

Original: Q: Did N1 that N2 married meet N A: Did ((N1) that (N2 married)) meet N Q: Did M2 marry N1 that M1 was edited by , directed by , and written by A: Did M2 marry ((N1) that (M1 was edited by , directed by , and written by))
Revised: Q: Did N1 that N2 married meet N A: Did ((N1) that (N2 married)) meet N Q: Did M2 marry N1 that M1 was edited by , directed by , and written by A: Did M2 marry ((N1) that (M1 was edited by , directed by , and written by))

Original: Q: Which N1 that was written by M1 M2 directed A: Which ((N1) that (was written by M1))
Revised: Q: Which N1 that was written by M1 M2 directed A: Which ((N1) that (was written by M1))

Original: M2 directed Q: Which N1 that N2 inﬂuenced was inﬂuenced by and married N A: Which ((N1) that (N2 inﬂuenced)) was inﬂuenced by and married N Q: Which director of N1 was N2 that M1 employed and was founded by A: Which director of N1 was ((N2) that (M1 employed and was founded by))
Revised: M2 directed Q: Which N1 that N2 inﬂuenced was inﬂuenced by and married N A: Which ((N1) that (N2 inﬂuenced)) was inﬂuenced by and married N Q: Which director of N1 was N2 that M1 employed and was founded by A: Which director of N1 was ((N2) that (M1 employed and was founded by))

Original: Q: Was N0 that M1 inﬂuenced and M2 was inﬂuenced by N A:
Revised: Q: Was N0 that M1 inﬂuenced and M2 was inﬂuenced by N A:

Original: Was ((N0) that (M1 inﬂuenced and M2 was inﬂuenced by))
Revised: Was ((N0) that (M1 inﬂuenced and M2 was inﬂuenced by))

Original: N Q: What N1 that N2 inﬂuenced and N3 was inﬂuenced by inﬂuenced M A:
Revised: N Q: What N1 that N2 inﬂuenced and N3 was inﬂuenced by inﬂuenced M A:

Original: What ((N1) that (N2 inﬂuenced and N3 was inﬂuenced by))
Revised: What ((N1) that (N2 inﬂuenced and N3 was inﬂuenced by))

Original: inﬂuenced M Q: Who was inﬂuenced by N1 that wrote N2 and inﬂuenced by N A:
Revised: inﬂuenced M Q: Who was inﬂuenced by N1 that wrote N2 and inﬂuenced by N A:

Original: Who was inﬂuenced by ((N1)
Revised: Who was inﬂuenced by ((N1)

Original: that (wrote N2)) and inﬂuenced by N Q: Was M4 produced by N1 that N2 were inﬂuenced by 
Revised: that (wrote N2)) and inﬂuenced by N Q: Was M4 produced by N1 that N2 were inﬂuenced by A:

Original: A: Was M4 produced by ((N1) that (N2 were inﬂuenced by))
Revised: Was M4 produced by ((N1) that (N2 were inﬂuenced by))

Original: Q: Was M0 N1 that M2 starred and was written byA:
Revised: Q: Was M0 N1 that M2 starred and was written by

Original:  Was M0 ((N1) that (M2 starred and was written by))
Revised: A: Was M0 ((N1) that (M2 starred and was written by))

Original: Q: Was N1 N2 that M2 employed A:
Revised: Q: Was N1 N2 that M2 employed A:

Original: Was N1 ((N2) that (M2 employed))
Revised: Was N1 ((N2) that (M2 employed))

Original: Q: Who was N0 that wrote N1 and edited N A:
Revised: Q: Who was N0 that wrote N1 and edited N A:

Original: Who was ((N0) that (wrote N1 and edited N2))
Revised: Who was ((N0) that (wrote N1 and edited N2))

Original: Q: Did N1 marry and divorce N2 whose N3 wrote M A: Did N1 marry and divorce ((N2) whose (N3 wrote M3))
Revised: Q: Did N1 marry and divorce N2 whose N3 wrote M A: Did N1 marry and divorce ((N2) whose (N3 wrote M3))

Original: Q: Did N1 whose N2 was employed by M2 and was employed by M3 direct and write M A: Did ((N1) whose (N2 was employed by M2 and was employed by M3))
Revised: Q: Did N1 whose N2 was employed by M2 and was employed by M3 direct and write M A: Did ((N1) whose (N2 was employed by M2 and was employed by M3))

Original: direct and write M Q: Did M3 found M4 and found N1 whose N2 wrote M A: Did M3 found M4 and found ((N1) whose (N2 wrote M1))
Revised: direct and write M Q: Did M3 found M4 and found N1 whose N2 wrote M A: Did M3 found M4 and found ((N1) whose (N2 wrote M1))

Original: Q: What N1 whose N2 was inﬂuenced by N3 played M A:
Revised: Q: What N1 whose N2 was inﬂuenced by N3 played M A:

Original: What ((N1) whose (N2 was inﬂuenced by N3))
Revised: What ((N1) whose (N2 was inﬂuenced by N3))

Original: played M Q: Which N1 whose N2 was inﬂuenced by M3 and was inﬂuenced by N3 directed M A:
Revised: played M Q: Which N1 whose N2 was inﬂuenced by M3 and was inﬂuenced by N3 directed M A:

Original: Which ((N1) whose (N2 was inﬂuenced by M3 and was inﬂuenced by N3))
Revised: Which ((N1) whose (N2 was inﬂuenced by M3 and was inﬂuenced by N3))

Original: directed M Q: Which N1 was inﬂuenced by N2 whose N3 edited M2 and employed by M A: Which N1 was inﬂuenced by ((N2) whose (N3 edited M2)) and employed by M Q: Was N1 whose N2 executive produced , edited , and wrote M0 M A: Was ((N1) whose (N2 executive produced , edited , and wrote M0))
Revised: directed M Q: Which N1 was inﬂuenced by N2 whose N3 edited M2 and employed by M A: Which N1 was inﬂuenced by ((N2) whose (N3 edited M2)) and employed by M Q: Was N1 whose N2 executive produced , edited , and wrote M0 M A: Was ((N1) whose (N2 executive produced , edited , and wrote M0))

Original: M Q: Was M1 N2 whose N3 directed , produced , and wrote N A:
Revised: M Q: Was M1 N2 whose N3 directed , produced , and wrote N A:

Original: Was M1 ((N2)
Revised: Was M1 ((N2)

Original: whose (N3 directed , produced , and wrote N4))
Revised: whose (N3 directed , produced , and wrote N4))

Original: Q: Was N1 that N2 directed N A:
Revised: Q: Was N1 that N2 directed N A:

Original: Was ((N1) that (N2 directed))
Revised: Was ((N1) that (N2 directed))

Original: N Q: Was N1 whose N2 executive produced M1 and edited M0 M A:
Revised: N Q: Was N1 whose N2 executive produced M1 and edited M0 M A:

Original: Was ((N1) whose (N2 executive produced M1 and edited M0))
Revised: Was ((N1) whose (N2 executive produced M1 and edited M0))

Original: M Q: What N1 was N2 whose N3 wrote N A: What N1 was ((N2) whose (N3 wrote N4))
Revised: M Q: What N1 was N2 whose N3 wrote N A: What N1 was ((N2) whose (N3 wrote N4))

Original: Q: Which N1 that N2 were written by and were produced by was N A: Which ((N1) that (N2 were written by and were produced by)) was N Q: Was M2 N2 whose N3 was employed by and founded M A:
Revised: Q: Which N1 that N2 were written by and were produced by was N A: Which ((N1) that (N2 were written by and were produced by)) was N Q: Was M2 N2 whose N3 was employed by and founded M A:

Original: Was M2 ((N2) whose (N3 was employed by and founded M1))
Revised: Was M2 ((N2) whose (N3 was employed by and founded M1))

Original: Q: Which N1 was N2 whose N3 produced and executive produced M A: Which N1 was ((N2) whose (N3 produced and executive produced M2))
Revised: Q: Which N1 was N2 whose N3 produced and executive produced M A: Which N1 was ((N2) whose (N3 produced and executive produced M2))

Original: Q: Was N0 that N1 were executive produced by , were edited by , were written by , and starred M A:
Revised: Q: Was N0 that N1 were executive produced by , were edited by , were written by , and starred M A:

Original: Was ((N0) that (N1 were executive produced by , were edited by , were written by , and starred M0))
Revised: Was ((N0) that (N1 were executive produced by , were edited by , were written by , and starred M0))

Original: Q: Which N0 that M2 employed executive produced M A: Which ((N0) that (M2 employed)) executive produced M Q: Who was N0 that M2 was inﬂuenced by and married A:
Revised: Q: Which N0 that M2 employed executive produced M A: Which ((N0) that (M2 employed)) executive produced M Q: Who was N0 that M2 was inﬂuenced by and married A:

Original: Who was ((N0) that (M2 was inﬂuenced by and married))
Revised: Who was ((N0) that (M2 was inﬂuenced by and married))

Original: Q: Which N0 that M1 was edited by did M2 inﬂuence A: Which ((N0) that (M1 was edited by)) did M2 inﬂuence Q: Were N0 directed by and produced by N1 that executive produced M A: Were N0 directed by and produced by ((N1) that (executive produced M1))
Revised: Q: Which N0 that M1 was edited by did M2 inﬂuence A: Which ((N0) that (M1 was edited by)) did M2 inﬂuence Q: Were N0 directed by and produced by N1 that executive produced M A: Were N0 directed by and produced by ((N1) that (executive produced M1))

Original: Q: What N0 that M1 was executive produced by and written by did N1 marry A: What ((N0) that (M1 was executive produced by and written by)) did N1 marry Q: Was N0 that M1 married N1 that directed and edited N A:
Revised: Q: What N0 that M1 was executive produced by and written by did N1 marry A: What ((N0) that (M1 was executive produced by and written by)) did N1 marry Q: Was N0 that M1 married N1 that directed and edited N A:

Original: Was ((N0) that (M1 married))
Revised: Was ((N0) that (M1 married))

Original: ((N1) that (directed and edited N2))
Revised: ((N1) that (directed and edited N2))

Original: Q: What N0 inﬂuenced by M1 and inﬂuenced by N1 that founded N2 edited M A:
Revised: Q: What N0 inﬂuenced by M1 and inﬂuenced by N1 that founded N2 edited M A:

Original: What N0 inﬂuenced by M1 and inﬂuenced by ((N1) that (founded N2)) edited M Q: Was N0 that N1 were edited , art directed , and produced by M A:
Revised: What N0 inﬂuenced by M1 and inﬂuenced by ((N1) that (founded N2)) edited M Q: Was N0 that N1 were edited , art directed , and produced by M A:

Original: Was ((N0) that (N1 were edited , art directed , and produced by))
Revised: Was ((N0) that (N1 were edited , art directed , and produced by))

Original: M
Revised: M

Original: A.1. S TEP 3: V ERB PHRASE IDENTIFICATION Verb phrase identiﬁcation in subclauses
Revised: A.1. S TEP 3: V ERB PHRASE IDENTIFICATION Verb phrase identiﬁcation in subclauses

Original: Q: that N1 inﬂuenced and N2 was inﬂuenced by A: that ((N1 inﬂuenced) and (N2 was inﬂuenced by))
Revised: Q: that N1 inﬂuenced and N2 was inﬂuenced by A: that ((N1 inﬂuenced) and (N2 was inﬂuenced by))

Original: Q: that was edited by N1 and starred N A: that ((was edited by N1) and (starred N2))
Revised: Q: that was edited by N1 and starred N A: that ((was edited by N1) and (starred N2))

Original: Q: whose N1 was inﬂuenced by M3 and was inﬂuenced by N A: whose (N1) ((was inﬂuenced by M3) and (was inﬂuenced by N2))
Revised: Q: whose N1 was inﬂuenced by M3 and was inﬂuenced by N A: whose (N1) ((was inﬂuenced by M3) and (was inﬂuenced by N2))

Original: Q: whose N1 produced and edited M A: whose (N1) (produced and edited) (M2) Q: whose N1 married M2 , met M3 , and played M A: whose (N1) ((married M2) , (met M3) , and (met M3))
Revised: Q: whose N1 produced and edited M A: whose (N1) (produced and edited) (M2) Q: whose N1 married M2 , met M3 , and played M A: whose (N1) ((married M2) , (met M3) , and (met M3))

Original: Q: that M2 was inﬂuenced by and M3 inﬂuenced A: that ((M2 was inﬂuenced by) and (M3 inﬂuenced))
Revised: Q: that M2 was inﬂuenced by and M3 inﬂuenced A: that ((M2 was inﬂuenced by) and (M3 inﬂuenced))

Original: Q: whose N1 executive produced , edited , and wrote M A: whose (N1) (executive produced , edited , and wrote) (M0) Q: whose N1 was inﬂuenced by N A: whose (N1) (was inﬂuenced by) (N2) Q: that N1 inﬂuenced A: that (N1) (inﬂuenced) Q: that M1 starred and was written by A: that (M1) (starred and was written by) Q: that M1 was edited by , directed by , and written by A: that (M1) (was edited by , directed by , and written by) Q: whose N1 was employed by M2 and was employed by M3 direct and write M 
Revised: Q: whose N1 executive produced , edited , and wrote M A: whose (N1) (executive produced , edited , and wrote) (M0) Q: whose N1 was inﬂuenced by N A: whose (N1) (was inﬂuenced by) (N2) Q: that N1 inﬂuenced A: that (N1) (inﬂuenced) Q: that M1 starred and was written by A: that (M1) (starred and was written by) Q: that M1 was edited by , directed by , and written by A: that (M1) (was edited by , directed by , and written by) Q: whose N1 was employed by M2 and was employed by M3 direct and write M A: whose (N1)

Original: A: whose (N1) ((was employed by M2) and (was employed by M3))
Revised: ((was employed by M2) and (was employed by M3))

Original: (direct and write) (M1) Q: whose N1 was employed by and founded M A: whose (N1) (was employed by and founded) (M1) Q: that M1 inﬂuenced A: that (M1) (inﬂuenced) Q: that directed and executive produced N A: that (directed and executive produced) (N1) Q: that M2 was inﬂuenced by and married A: that (M2) (was inﬂuenced by and married) Q: that N1 were inﬂuenced by , M3 was edited by , and M4 married A: that ((N1 were inﬂuenced by) , (M3 was edited by) , and (M4 married))
Revised: (direct and write) (M1) Q: whose N1 was employed by and founded M A: whose (N1) (was employed by and founded) (M1) Q: that M1 inﬂuenced A: that (M1) (inﬂuenced) Q: that directed and executive produced N A: that (directed and executive produced) (N1) Q: that M2 was inﬂuenced by and married A: that (M2) (was inﬂuenced by and married) Q: that N1 were inﬂuenced by , M3 was edited by , and M4 married A: that ((N1 were inﬂuenced by) , (M3 was edited by) , and (M4 married))

Original: Q: that N1 were written by A: that (N1) (were written by)
Revised: Q: that N1 were written by A: that (N1) (were written by)

Original: Q: that was founded by and employed N A: that (was founded by and employed)
Revised: Q: that was founded by and employed N A: that (was founded by and employed)

Original: (N2) Q: that was inﬂuenced by M2 and was employed by M3 and M A: that ((was inﬂuenced by M2) and (was employed by M3 and M4))
Revised: (N2) Q: that was inﬂuenced by M2 and was employed by M3 and M A: that ((was inﬂuenced by M2) and (was employed by M3 and M4))

Original: Q: that N1 married A: that (N1) (married) Q: that edited N1 and produced M A: that ((edited N1) and (produced M2))
Revised: Q: that N1 married A: that (N1) (married) Q: that edited N1 and produced M A: that ((edited N1) and (produced M2))

Original: Q: whose N1 edited M0 and executive produced N A: whose (N1) ((edited M0) and (executive produced N2))
Revised: Q: whose N1 edited M0 and executive produced N A: whose (N1) ((edited M0) and (executive produced N2))

Original: Q: whose N1 edited N A: whose (N1) (edited) (N2) Q: that M1 inﬂuenced and employed A: that (M1) (inﬂuenced and employed) Q: that wrote N1 and edited N A: that ((wrote N1) and (edited N2)) Q: that edited and wrote N A: that (edited and wrote) (N1) Q: whose N1 was employed by and founded M A: whose (N1) (was employed by and founded) (M1) Q: whose N1 married M2 and married N A: whose (N1) ((married M2) and (married N2))
Revised: Q: whose N1 edited N A: whose (N1) (edited) (N2) Q: that M1 inﬂuenced and employed A: that (M1) (inﬂuenced and employed) Q: that wrote N1 and edited N A: that ((wrote N1) and (edited N2)) Q: that edited and wrote N A: that (edited and wrote) (N1) Q: whose N1 was employed by and founded M A: whose (N1) (was employed by and founded) (M1) Q: whose N1 married M2 and married N A: whose (N1) ((married M2) and (married N2))

Original: Q: that played M2 , played M3 , and played M A: that ((played M2) , (played M3) , and (played M4))
Revised: Q: that played M2 , played M3 , and played M A: that ( [Table] and [Table]

Original: Q: that wrote , edited , executive produced , and directed N A: that (wrote , edited , executive produced , and directed) (N1)
Revised: Q: that wrote , edited , executive produced [Table] and directed N [Table]

Revised: : that (

Original:  Q: that N3 were written by and art directed by A: that (N3) (were written by and art directed by)
Revised: [Table] , edited , executive produced [Table] and directed) (N1) Q: that N3 were written by and art directed by A: that (N3) (were written by and art directed by)

Original: [Table]
Revised: [Table]

Original: Q: Was N1 N A: (Was) (N1) (N2) Q: Did M1 inﬂuence N A: (Did) (M1)
Revised: Q: Was N1 N A: (Was) (N1) (N2) Q: Did M1 inﬂuence N A: (Did) (M1)

Original: (inﬂuence) (N2) Q: Which N1 was N A: (Which) (N1) (was)
Revised: (inﬂuence) (N2) Q: Which N1 was N A: (Which) (N1) (was)

Original: (N2) Q: Who employed , met , and was inﬂuenced by N1 and married M A:
Revised: (N2) Q: Who employed , met , and was inﬂuenced by N1 and married M A:

Original: (Who) ((employed , met , and was inﬂuenced by N1) and (married M1))
Revised: (Who) ((employed , met , and was inﬂuenced by N1) and (married M1))

Original: Q: Were N1 executive produced by , written by , and directed by N2 , and produced by N A: (Were) (N1) ((executive produced by , written by , and directed by N2) , and (produced by N3))
Revised: Q: Were N1 executive produced by , written by , and directed by N2 , and produced by N A: (Were) (N1) ((executive produced by , written by , and directed by N2) , and (produced by N3))

Original: Q: What N1 married , divorced , and was kissed by N A: (What) (N1) (married , divorced , and was kissed by) (N2) Q: Which N1 did M0 acquire A: (Which) (N1) (did) (M0) (acquire) Q: Which N1 was inﬂuenced by N2 , inﬂuenced by N3 , and inﬂuenced by M A: (Which) (N1) ((was inﬂuenced by N2) , (inﬂuenced by N3) , and (inﬂuenced by M4))
Revised: Q: What N1 married , divorced , and was kissed by N A: (What) (N1) (married , divorced , and was kissed by) (N2) Q: Which N1 did M0 acquire A: (Which) (N1) (did) (M0) (acquire) Q: Which N1 was inﬂuenced by N2 , inﬂuenced by N3 , and inﬂuenced by M A: (Which) (N1) ((was inﬂuenced by N2) , (inﬂuenced by N3) , and (inﬂuenced by M4))

Original: Q: Was N1 employed by N2 and employed by M4 , M5 , and M6 N A: (Was) ((N1) ((employed by N2) and (employed by N3)))
Revised: Q: Was N1 employed by N2 and employed by M4 , M5 , and M6 N A: (Was) ((N1) ((employed by N2) and (employed by N3)))

Original: (N4) Q: Which N1 did N2 produce and M1 direct A: (Which) (N1) (did) ((N2 produce) and (M1 direct))
Revised: (N4) Q: Which N1 did N2 produce and M1 direct A: (Which) (N1) (did) ((N2 produce) and (M1 direct))

Original: Q: Did N1 inﬂuence N2 and inﬂuence N A:
Revised: Q: Did N1 inﬂuence N2 and inﬂuence N A:

Original: (Did) (N1) ((inﬂuence N2) and (inﬂuence N3))
Revised: (Did) (N1) ((inﬂuence N2) and (inﬂuence N3))

Original: Q: What N1 did N2 inﬂuence
Revised: Q: What N1 did N2 inﬂuence

Original: A: (What) (N1) (did) (N2)
Revised: A: (What) (N1) (did) (N2)

Original: (inﬂuence) Q: Did M0 direct , produce , executive produce , edit , and write N A: (Did) (M0) (direct , produce , executive produce , edit , and write) (N1) Q: Was M0 executive produced by , edited by , written by , and directed by N A: (Was) (M0)
Revised: (inﬂuence) Q: Did M0 direct , produce , executive produce , edit , and write N A: (Did) (M0) (direct , produce , executive produce , edit , and write) (N1) Q: Was M0 executive produced by , edited by , written by , and directed by N A: (Was) (M0)

Original: (executive produced by , edited by , written by , and directed by) (N1) Q: Who inﬂuenced and was inﬂuenced by N A: (Who) (inﬂuenced and was inﬂuenced by) (N1) Q: Did N1 executive produce and produce M A: (Did) (N1) (executive produce and produce) (M1) Q: Did N1 found N1 and found M A: (Did) (N1) ((found N1) and (found M2))Q: What was executive produced by N1 and executive produced by N A: (What) ((was executive produced by N1) and (executive produced by N2)) Q: Was N1 produced by and written by N A: (Was) (N1) (produced by and written by) (N2) Q: What did N1 produce and write A: (What) (did) (N1) (produce and write) Q:
Revised: (executive produced by , edited by , written by , and directed by) (N1) Q: Who inﬂuenced and was inﬂuenced by N A: (Who) (inﬂuenced and was inﬂuenced by) (N1) Q: Did N1 executive produce and produce M A: (Did) (N1) (executive produce and produce) (M1) Q: Did N1 found N1 and found M A: (Did) (N1) ((found N1) and (found M2))

Original: Was M1 produced byN A: (Was) (M1) (produced by) (N1) Q: Did N1edit , N2 direct , and N3 produce M A: (Did) ((N1 edit) , (N2 direct) , and (N3 produce)) M Q: Was N1 written by and executive produced by N2 M A: (Was) ((N1) (written by  and executive producedby) (N2)) (M1) Q: Which N1 was founded by N A: (Which) (N1) (was founded by)
Revised: Q: What was executive produced by N1 and executive produced by N A: (What) ((was executive produced by N1) and (executive produced by N2))

Original: (N2) Q: Which N1 was acquired by N2 and acquired N A: (Which) (N1) ((was acquired byN2) and(acquired N3)) Q: Was M2 N0 written byM4 and directed by N A: (Was) (M2) ((N0) ((written byM4) and (directed by N1))) Q: WhatN1 did N2 marry and inﬂuence A: (What) (N1) (did) (N2 marry and inﬂuence)
Revised: Q: Was N1 produced by and written by N A: (Was) (N1) (produced by and written by) (N2) Q: What did N1 produce and write

Original: Q: Did N1 inﬂuence and marry N A: (Did) (N1) (inﬂuence and marry) (N2) Q: Which N1 did M1 marry and M2 marry A: (Which) (N1) (did) ((M1 marry) and (M2 marry)) Q: What was directed by and edited by N A: (What) (was directed by and edited by) (N1) Q: What did N1 edit and M0 executive produce A: (What) (did) ((N1 edit) and (M0 executive produce)) Q: What N0 did N1 edit and produce A: (What) (N0) (did) (N1) (edit and produce) Q: Was N1 N2 edited by M A: (Was) (N1) (N2 (edited by) M0) Q: Did N1 write a ﬁlm and direct N A: (Did) (N1) ((write a ﬁlm) and (direct N2))
Revised: and direct N A: (Did) (N1) ((write a ﬁlm) and (direct N2))

Original: Q: Was N1 produced by M1 and executive produced by M A: (Was) (N1) ((produced by M1) and (executive produced by M2))
Revised: Q: Was N1 produced by M1 and executive produced by M A: (Was) (N1) ((produced by M1) and (executive produced by M2))

Original: Q: Were N1 written , executive produced , produced , and edited by N A: (Were) (N1) (written , executive produced , produced , and edited by) (N2) Q:
Revised: Q: Were N1 written , executive produced , produced , and edited by N A: (Were) (N1) (written , executive produced , produced , and edited by) (N2) Q:

Original: Was N0 inﬂuenced by N1 and inﬂuenced by M1 M A: (Was) ((N0) (inﬂuenced by N1) and (inﬂuenced by M1))
Revised: Was N0 inﬂuenced by N1 and inﬂuenced by M1 M A: (Was) ((N0) (inﬂuenced by N1) and (inﬂuenced by M1))

Original: (M1) Q: What was edited by M0 and executive produced by N A: (What) ((was edited by M0) and (executive produced by N0)) 
Revised: (M1) Q: What was edited by M0 and executive produced by N A: (What) ((was edited by M0) and (executive produced by N0)) Q: Did M2 star M3 and star N A: (Did) (M2) ((star M3) and (star N0))

Original: A.1. S TEP 4: P ART OF SPEECH TAGGING Part of speech tagging of noun phrases
Revised: A.1. S TEP 4: P ART OF SPEECH TAGGING Part of speech tagging of noun phrases

Original: A Q
Revised: Q:

Original: producer A

Original: M
Revised: M

Original: M P (’s
Revised: Q

Original: N
Revised: N

Original: producer) Q a art director ’s sibling A [a] (N art director) P
Revised: costume

Original: (’s N (sibling)) Q
Revised: nationality

Original: a French producer and editor ’s child ’s spouse A: [a] ((N (French /A, producer, [and] editor) P
Revised: Dutch

Original: (’s) N=(child)) P (’s) N (spouse))
Revised: =(of) ([a] (N=(ﬁlm) P=(’s) N=(prequel))))

Original: Q:  female French costume designer , producer , and editor of M3 and M A: N (female /A, French /A, costume designer, producer, [and] editor) P=(of) M=(M3, M4) Q: country of nationality of M A: N=(country of nationality) P=(of) M=(M3) Q: a Dutch parent and spouse  A:
Revised: Q: a female ﬁlm director ’s Spanish parent A: [a] ((female /A, ﬁlm director) P=(’s) N=(Spanish /A, parent)) Q: M3 and M A: M=(M3, [and] M6) Q: a writer ’s spouse , friend , and employer A:

Original: [a] N=(Dutch /A, parent, [and] spouse) Q a editor , producer [Table] and writer of a ﬁlm
Revised: [a] (N=(writer) P=(’s) N=(spouse, friend, [and] employer))

Original: [Table] prequel A: [a] (N=(editor, producer, [and] writer) P=(of) ([a] (N=(ﬁlm) P=(’s) N=(prequel)))) Q: a female ﬁlm director ’s Spanish  parent A: [a] ((female /A, ﬁlm director) P=(’s) N=(Spanish /A, parent)) Q: M3 and M A: M=(M3, [and] M6) Q: a writer ’s spouse , friend , and employer A:
Revised: Q: male Spanish spouse and parent A: N=(male /A, Spanish /A, spouse, [and] parent) Q: a person A:

Original: [a] (N=(writer) P=(’s) N=(spouse, friend, [and] employer))
Revised: [a] N=(person)

Original: Q: male Spanish spouse and parent A: N=(male /A, Spanish /A, spouse, [and] parent) Q: a person A: [a] N=(person) Q: M4 , M5 , M6 , and M A: M=(M4, M5, M6, [and] M7) Q: ﬁlm director and editor of M0 , M1 , and M A: N=(ﬁlm director, [and] editor) P=(of) M=(M0, M1, [and] M2) Q: a Spanish editor and producer of M A: [a] (N=(Spanish /A, editor, [and] producer) P=(of) M=(M5)) Q: child of a production company A: N=(child) P=(of) ([a] N=(production company)) Q: M1 ’s sibling and spouse A: M=(M P=(’s
Revised: Q

Original: N=(sibling, [and] spouse) Q: M3 ’s employer ’s Mexican employee A: (M=(
Revised: and M

Original: P=(’s
Revised: Q Spanish

Original: N=(employer))
Revised: employee

Original: P=(’s) N=(Mexican /A, employee) Q: M3 ’s parent ’s employee A: (M=(M3) P=(’s) N=(parent)) P=(’s) N=(employee) Q: M2 ’s founder and employee A: M=(M2) P=(’s) N=(founder, [and] employee) Q: director of M3 , M4 , and M A: N=(director) P=(of) M=(M3 , M4 , [and] M5) Q: male director of M3 and M A: N=(male /A, director) P=(of) M=(M3, [and
Revised: Q

Original: ] M4) Q: a child of M4 ’s male editor ’s sibling A: [a] (N=(child) P=(of) ((M=(M4) P=(’s) N=(male /A, editor))
Revised: employee A

Original: P=(’s) N=(sibling)))
Revised: (’s) N=(sibling))

Original: Q: spouse of M0 ’s sibling ’s employee A: N=(spouse) P=(of) ((M=(M0) P=(’s) N=(sibling))

Original: P=(’s) N=(employee))
Revised: P=(’s) N=(employee))

Original: Q: a actor ’s country of nationality A:
Revised: Q: a actor ’s country of nationality A:

Original: [a] (N=(actor) P=(’s) N=(country of nationality))
Revised: [a] (N=(actor) P=(’s) N=(country of nationality))

Original: Q: a female person ’s parent ’s sibling A:
Revised: Q: a female person ’s parent ’s sibling A:

Original: [a] ((N=(female /A, person) P=(’s) N=(parent))
Revised: [a] ((N=(female /A, person) P=(’s) N=(parent))

Original: P=(’s) N=(sibling))
Revised: P=(’s) N=(sibling))

Original: Q: a screenwriter ’s French parent A:
Revised: Q: a screenwriter ’s French parent A:

Original: [a] (N=(screenwriter) P=(’s) N=(French /A, parent))
Revised: [a] (N=(screenwriter) P=(’s) N=(French /A, parent))

Original: Q: sibling and friend of a director and actor A: N=(sibling, [and] friend) P=(of) ([a] (N=(director, [and] actor)))
Revised: Q: sibling and friend of a director and actor A: N=(sibling, [and] friend) P=(of) ([a] (N=(director, [and] actor)))

Original: Q: Mexican American parent A: N=(Mexican /A, American /A, parent)
Revised: Q: Mexican American parent A: N=(Mexican /A, American /A, parent)

Original: Part of speech tagging of other phrases
Revised: Part of speech tagging of other phrases

Original: Q: What A: W=(What) Q Which A
Revised: Q: What A: W=(What) Q: Which A: W=(Which)

Original: Which) Q Who A: W (Who) Q Did)
Revised: Q: Who A: W=(Who) Q: (Did) A: V=(Did) Q:

Original: A V (Did) Q Was A: V=(Was) Q: employed , met , and was inﬂuenced by N A: V=(employed, met, [and] [was] inﬂuenced by)(N1) Q:
Revised: Was A: V=(Was) Q: employed , met , and was inﬂuenced by N A: V=(employed, met, [and] [was] inﬂuenced by)

Original:  married M A: V=(married) M=(M1) Q: executive produced by , written by , and directed by N A: V=(executive produced by, V=written by, [and] V=directed by) (N1) Q: produced by N A: V=(produced by)
Revised: (N1) Q: married M A: V=(married) M=(M1) Q: executive produced by , written by , and directed by N A: V=(executive produced by, V=written by, [and] V=directed by) (N1) Q: produced by N A: V=(produced by)

Original: (N1) Q: M0 was employed by A: M=(M0)
Revised: (N1) Q: M0 was employed by A: M=(M0)

Original: V=([was] employed by)
Revised: V=([was] employed by)

Original: Q: N1 married A: (N1) V=(married) Q: M4 , M5 , M6 , and M A: M=(M4, M5, M6, [and] M7) Q: was written by M A: V=([was] written by)
Revised: Q: N1 married A: (N1) V=(married) Q: M4 , M5 , M6 , and M A: M=(M4, M5, M6, [and] M7) Q: was written by M A: V=([was] written by)

Original: M=(M1) Q: married , divorced , and was kissed by A: V=(married, divorced, [and] [was] kissed by) Q: N1 wrote M A: (N1)
Revised: M=(M1) Q: married , divorced , and was kissed by A: V=(married, divorced, [and] [was] kissed by) Q: N1 wrote M A: (N1)

Original: V=(wrote) M=(M3) Q: was inﬂuenced by N A: V=([was] inﬂuenced by) (N1) Q: inﬂuenced by N A: V=(inﬂuenced by) (N1) Q: (employed by M1 , M2 , and M3)
Revised: V=(wrote) M=(M3) Q: was inﬂuenced by N A: V=([was] inﬂuenced by) (N1) Q: inﬂuenced by N A: V=(inﬂuenced by) (N1) Q: (employed by M1 , M2 , and M3)

Original: A: V=(employed by) M=(M1, M2, [and] M3) Q: N1 produce A: (N1) V=(produce) Q: inﬂuence N A: V=(inﬂuence) (N1) Q: M1 employed and was founded by A: M=(M1) V=(employed, [and] [was] founded by))
Revised: A: V=(employed by) M=(M1, M2, [and] M3) Q: N1 produce A: (N1) V=(produce) Q: inﬂuence N A: V=(inﬂuence) (N1) Q: M1 employed and was founded by A: M=(M1) V=(employed, [and] [was] founded by))

Original: Q: direct , produce , executive produce , edit , and write 
Revised: Q: direct , produce , executive produce , edit , and write A:

Original: A: V=(direct, produce, executive produce, edit, [and] write)
Revised: =

Original: Q: executive produced by , edited by , written by , and directed by
Revised: Q: executive produced by , edited by , written by , and directed by

Original: A: V=(executive produced by, edited by, written by, [and] directed by)
Revised: A: V=(executive produced by, edited by, written by, [and] directed by)

Original: A.
Revised: A.1.

Original: S TEP 5: V ERB NORMALIZATION
Revised: S TEP 5: V ERB NORMALIZATION

Original: Q: and
Revised: Q: and

Original: A: and
Revised: A: and

Original: Q: was
Revised: Q: was

Original: A: be
Revised: A: be

Original: Q: Was
Revised: Q: Was

Original: A: be
Revised: A: be

Original: Q: were
Revised: Q: were

Original: A: be
Revised: A: be

Original: Q: Were
Revised: Q: Were

Original: A: be
Revised: A: be

Original: Q: did
Revised: Q: did

Original: A: do
Revised: A: do

Original: Q: Did
Revised: Q: Did

Original: A: do
Revised: A: do

Original: Q: directed
Revised: Q: directed

Original: A: direct
Revised: A: direct

Original: Q: edited
Revised: Q: edited

Original: A: edit
Revised: A: edit

Original: Q: met
Revised: Q: met

Original: A: meet
Revised: A: meet

Original: Q: found
Revised: Q: found

Original: A: found
Revised: A: found

Original: Q: edited by
Revised: Q: edited by

Original: A: edit by
Revised: A: edit by

Original: Q: written by
Revised: Q: written by

Original: A: write by
Revised: A: write by

Original: A. CFQ E XEMPLAR S ELECTION : D ETAILS
Revised: A. CFQ E XEMPLAR S ELECTION : D ETAILS

Original: For CFQ, we use a random subset consisting of 1000 examples from the training set as a pool of candidate exemplars.
Revised: For CFQ, we use a random subset consisting of 1000 examples from the training set as a pool of candidate exemplars.

Original: For each input question to process, we then dynamically select exemplars from this pool such that they collectively demonstrate relevant knowledge needed to translate the input sentences.
Revised: For each input question to process, we then dynamically select exemplars from this pool such that they collectively demonstrate relevant knowledge needed to translate the input sentences.

Original: This is done by making sure that as many nodes as possible of the decomposition tree of the input are covered by the decomposition trees of the selected exemplars.
Revised: This is done by making sure that as many nodes as possible of the decomposition tree of the input are covered by the decomposition trees of the selected exemplars.

Original: Depending on the complexity of the input and the similarity of the candidate exemplars in the pool, we select between 4 and 35 exemplars for any given input.
Revised: Depending on the complexity of the input and the similarity of the candidate exemplars in the pool, we select between 4 and 35 exemplars for any given input.

Original: We provide a general description of this process in Section 3.2 and add the CFQ-speciﬁc details here.
Revised: We provide a general description of this process in Section 3.2 and add the CFQ-speciﬁc details here.

Original: In Section A.3, we also show the set of all selected exemplars for the example in Figure 6.
Revised: In Section A.3, we also show the set of all selected exemplars for the example in Figure 6.

Original: A.2. T OP - DOWN MATCHING
Revised: A.2. T OP - DOWN MATCHING

Original: We want to select exemplars that cover the structure of the input question as well as possible.
Revised: We want to select exemplars that cover the structure of the input question as well as possible.

Original: We do this using the following process:
Revised: We do this using the following process:

Original: We ﬁrst convert the decomposition trees of all the candidate exemplars as well as the concrete input question into syntactic templates by anonymizing concrete leafs and just keeping their types.
Revised: We ﬁrst convert the decomposition trees of all the candidate exemplars as well as the concrete input question into syntactic templates by anonymizing concrete leaves and just keeping their types.

Original: For instance, the question shown in Figure 6 results in the syntactic template “Which N (V (N that (V (M , M , [and] M))) and (V (M ’s N))”.
Revised: For instance, the question shown in Figure 6 results in the syntactic template “Which N (V (N that (V (M , M , [and] M))) and (V (M ’s N))”.

Original: Then we try to ﬁnd exemplars that match the full template of the input question.
Revised: Then we try to ﬁnd exemplars that match the full template of the input question.

Original: If we succeed, we keep them.
Revised: If we succeed, we keep them.

Original: Otherwise, we reduce the templates by collapsing some of the nodes.
Revised: Otherwise, we reduce the templates by collapsing some of the nodes.

Original: For example, we can collapse the node “(M , M , [and] M)” in the above template and instead just use “M”.
Revised: For example, we can collapse the node “(M , M , [and] M)” in the above template and instead just use “M”.

Original: We again try to ﬁnd exemplars that match the reduced template, keep them if we succeed, and otherwise continue reducing the templates.
Revised: We again try to ﬁnd exemplars that match the reduced template, keep them if we succeed, and otherwise continue reducing the templates.

Original: We do this until we retrieve exemplars that collectively cover the input template as well as possible.
Revised: We do this until we retrieve exemplars that collectively cover the input template as well as possible.

Original: We wrote a small amount of Python code that implements a generic version of this heuristics and use it for both CFQ and COGS.
Revised: We wrote a small amount of Python code that implements a generic version of these heuristics and use it for both CFQ and COGS.

Original: For the example shown in Figure 6, the top-down matching yields exemplars such as the ones shown below.
Revised: For the example shown in Figure 6, the top-down matching yields exemplars such as the ones shown below.

Original: Note that to provide the LM with additional hints, we add to the original question parentheses that indicate the syntactic structure.
Revised: Note that to provide the LM with additional hints, we add to the original question parentheses that indicate the syntactic structure.

Original: Q: Which cinematographer founded (M1 ’s distributor)
Revised: Q: Which cinematographer founded (M1 ’s distributor)

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a cinematographer . ?
Revised: x0 WHERE { ?x0 a cinematographer . ?

Original: x0 founded ?x1 . ?
Revised: x0 founded ?x1 . ?

Original: x1 distributed M1 } Q: What (ﬁlm director ((inﬂuenced by M1) and (inﬂuenced by (a person that (founded (M and M5))))))
Revised: x1 distributed M1 } Q: What (ﬁlm director ((inﬂuenced by M1) and (inﬂuenced by (a person that (founded (M and M5))))))

Original: produced M A: SELECT DISTINCT ?
Revised: produced M A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a ﬁlm director . ?
Revised: x0 WHERE { ?x0 a ﬁlm director . ?

Original: x0 produced M2 . ?
Revised: x0 produced M2 . ?

Original: x0 inﬂuenced by ?
Revised: x0 inﬂuenced by ?

Original: x . ?
Revised: x . ?

Original: x0 inﬂuenced by M1 . ?
Revised: x0 inﬂuenced by M1 . ?

Original: x1 a person . ?
Revised: x1 a person . ?

Original: x1 founded M4 . ? 
Revised: x1 founded M4 . ?x

Original: x1 founded M5 } A.2. B OTTOM - UP MATCHING
Revised: founded M5 } A.2. B OTTOM - UP MATCHING

Original: We also want to select exemplars that collectively cover each of the unanonymized leafs.
Revised: We also want to select exemplars that collectively cover each of the unanonymized leaves.

Original: In our running example, this means that we want to cover the leafs “ﬁlm editor”, “inﬂuence”, “cinematographer”, “write”, and “editor”.
Revised: In our running example, this means that we want to cover the leaves “ﬁlm editor”, “inﬂuence”, “cinematographer”, “write”, and “editor”.

Original: In addition, we prefer exemplars where these leafs occur within a similar syntactic template as in the input question.
Revised: In addition, we prefer exemplars where these leaves occur within a similar syntactic template as in the input question.

Original: For each leaf, we do this by converting the decomposition trees into a form where everything but this leaf is anonymized.
Revised: For each leaf, we do this by converting the decomposition trees into a form where everything but this leaf is anonymized.

Original: For the leaf “editor”, this results in “Which N (V (N that (V (M , M , [and]
Revised: For the leaf “editor”, this results in “Which N (V (N that (V (M , M , [and]

Original: M)))
Revised: M)))

Original: and (V (M ’s editor))”.
Revised: and (V (M ’s editor))”.

Original: Then we try to ﬁnd exemplars that share as many subtrees containing “editor” as possible.
Revised: Then we try to ﬁnd exemplars that share as many subtrees containing “editor” as possible.

Original: This yields exemplars such as:
Revised: This yields exemplars such as:

Original: Q: Was a (Dutch ﬁlm editor)
Revised: Q: Was a (Dutch ﬁlm editor)

Original: M A: SELECT count(*) WHERE { M0 a ﬁlm editor .
Revised: M A: SELECT count(*) WHERE { M0 a ﬁlm editor .

Original: M0 has nationality Dutch } Q: Which actor (was inﬂuenced by and inﬂuenced) (M1 ’s editor)
Revised: M0 has nationality Dutch } Q: Which actor (was inﬂuenced by and inﬂuenced) (M1 ’s editor)

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a actor . ?
Revised: x0 WHERE { ?x0 a actor . ?

Original: x0 inﬂuenced ?
Revised: x0 inﬂuenced ?

Original: x1 . ?
Revised: x1 . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x edited M1 } Q: Was a (Chinese cinematographer)
Revised: x edited M1 } Q: Was a (Chinese cinematographer)

Original: M A: SELECT count(*) WHERE { M0 a cinematographer .
Revised: M A: SELECT count(*) WHERE { M0 a cinematographer .

Original: M0 has nationality Chinese } Q: What did (M0 ’s sibling) write A: SELECT DISTINCT ?
Revised: M0 has nationality Chinese } Q: What did (M0 ’s sibling) write A: SELECT DISTINCT ?

Original: x0 WHERE { ?
Revised: x0 WHERE { ?

Original: x0 written by ?
Revised: x0 written by ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 sibling of M0 } Q: Was M1 ((founded by (M0 ’s editor)) and (founded by M2))
Revised: x1 sibling of M0 } Q: Was M1 ((founded by (M0 ’s editor)) and (founded by M2))

Original: A: SELECT count(*) WHERE { ?x0 edited M0 .
Revised: A: SELECT count(*) WHERE { ?x0 edited M0 .

Original: M1 founded by ?
Revised: M1 founded by ?

Original: x0 . M1 founded by M2 }
Revised: x0 . M1 founded by M2 }

Original: A. CFQ S OLUTION : D ETAILS AND P ROMPT
Revised: A. CFQ S OLUTION : D ETAILS AND P ROMPT

Original: As we discussed in Section 3.3, one novelty of dynamic least-to-most prompting is that we cannot translate the constituents in isolation because they might not correspond to well-formed questions and their translation may depend on the context.
Revised: As we discussed in Section 3.3, one novelty of dynamic least-to-most prompting is that we cannot translate the constituents in isolation because they might not correspond to well-formed questions and their translation may depend on the context.

Original: Instead, we linearize the composition tree into a sequence of increasingly complex subquestions.
Revised: Instead, we linearize the composition tree into a sequence of increasingly complex subquestions.

Original: This linearization is performed by a walk over the parse tree.
Revised: This linearization is performed by a walk over the parse tree.

Original: We keep the most generic variant of each top-level node and then expand these nodes step by step to obtain a linear sequence of wellformed subquestions.
Revised: We keep the most generic variant of each top-level node and then expand these nodes step by step to obtain a linear sequence of wellformed subquestions.

Original: For each subquestion, we then query the language model using a prompt that consists of three parts.
Revised: For each subquestion, we then query the language model using a prompt that consists of three parts.

Original: Static prompt context for question grounding 2.
Revised: Static prompt context for question grounding 2.

Original: Dynamically selected exemplars as additional context 3.
Revised: Dynamically selected exemplars as additional context 3.

Original: Sequential questions
Revised: Sequential questions

Original: These parts are detailed throughout the rest of this section.
Revised: These parts are detailed throughout the rest of this section.

Original: A.3. P ART 1: S TATIC PROMPT CONTEXT .
Revised: A.3. P ART 1: S TATIC PROMPT CONTEXT .

Original: Because we cannot translate constituents in isolation, even the simplest subquestion may be too complex to be translated correctly based on the selected exemplars alone.
Revised: Because we cannot translate constituents in isolation, even the simplest subquestion may be too complex to be translated correctly based on the selected exemplars alone.

Original: This is especially the case if the exemplar pool does not contain similar questions.
Revised: This is especially the case if the exemplar pool does not contain similar questions.

Original: To teach the language model how to translate the simplest subquestions, we therefore provide it with a constant prompt context consisting of 12 grounding examples that illustrate these kinds of subquestions.
Revised: To teach the language model how to translate the simplest subquestions, we therefore provide it with a constant prompt context consisting of 12 grounding examples that illustrate these kinds of subquestions.

Original: In addition to the question and its translation, each of these grounding examples also provides a rationale that tells the model how the translation can be obtained (this resembles our chain-of-thought prompt).
Revised: In addition to the question and its translation, each of these grounding examples also provides a rationale that tells the model how the translation can be obtained (this resembles our chain-of-thought prompt).

Original: The static prompt context is provided below.
Revised: The static prompt context is provided below.

Original: Note that we we use a slightly different preﬁx (“Partial
Revised: Note that we we use a slightly different preﬁx (“Partial

Original: Q: ” instead of “Q:”) for these grounding questions.
Revised: Q: ” instead of “Q:”) for these grounding questions.

Original: This allows us to encourage the model to perform rationale-based reasoning when asking it to translate the simplest question of a sequence.
Revised: This allows us to encourage the model to perform rationale-based reasoning when asking it to translate the simplest question of a sequence.

Original: Also note that we again use parentheses to indicate the syntactic structure of the questions.
Revised: Also note that we again use parentheses to indicate the syntactic structure of the questions.

Original: Partial Q
Revised: Partial Q

Original: Was a costume designer ’s parent
Revised: Was a costume designer ’s parent

Original: M
Revised: M

Original: ’s
Revised: ’s

Original: editor)
Revised: editor)

Original: Rational: Was = {} , ”costume designer ’s parent” = { ?
Revised: Rational: Was = {} , ”costume designer ’s parent” = { ?

Original: x0 parent of ?x1 . ?
Revised: x0 parent of ?x1 . ?

Original: x1 a costume designer } , ”M0 ’s
Revised: x1 a costume designer } , ”M0 ’s

Original: editor”
Revised: editor”

Original: = { ?x0 edited M0 } == >
Revised: = { ?x0 edited M0 } == >

Original: A: SELECT count(*) WHERE { ?x0 parent of ?x1 . ?
Revised: A: SELECT count(*) WHERE { ?x0 parent of ?x1 . ?

Original: x1 a costume designer . ?
Revised: x1 a costume designer . ?

Original: x0 edited M0 }
Revised: x0 edited M0 }

Original: Partial Q: Was M0 a (screenwriter ’s spouse)
Revised: Partial Q: Was M0 a (screenwriter ’s spouse)

Original: Rationale:
Revised: Rationale:

Original: Was = {} , ”M0” replaces ?
Revised: Was = {} , ”M0” replaces ?

Original: x0, ”screenwriter ’s spouse” = { ?x0 married to ?x1 . ?
Revised: x0, ”screenwriter ’s spouse” = { ?x0 married to ?x1 . ?

Original: x1 a writer } == >
Revised: x1 a writer } == >

Original: A: SELECT count(*) WHERE { M0 married to ?
Revised: A: SELECT count(*) WHERE { M0 married to ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 a writer }
Revised: x1 a writer }

Original: Partial Q: Was a (sequel of M1)
Revised: Partial Q: Was a (sequel of M1)

Original: M
Revised: M

Original: Rationale:
Revised: Rationale:

Original: Was = {} , ”star of M1”
Revised: Was = {} , ”star of M1”

Original: = { ?x0 has prequel M1 } , ”M0” replaces ?
Revised: = { ?x0 has prequel M1 } , ”M0” replaces ?

Original: x0 =
Revised: x0 =

Original: = >
Revised: = >

Original: A: SELECT count(*) WHERE { M0 has prequel M1 }
Revised: A: SELECT count(*) WHERE { M0 has prequel M1 }

Original: Partial Q: Was M1 executive produced by a (sibling of a ﬁlm producer)
Revised: Partial Q: Was M1 executive produced by a (sibling of a ﬁlm producer)

Original: Rationale:
Revised: Rationale:

Original: Was = {} , ”M1” replaces ?
Revised: Was = {} , ”M1” replaces ?

Original: x0, ”executive produced by”
Revised: x0, ”executive produced by”

Original: = { ?x0 executive produced by ?
Revised: = { ?x0 executive produced by ?

Original: x } , ”sibling of a ﬁlm producer” = { ?
Revised: x } , ”sibling of a ﬁlm producer” = { ?

Original: x1 sibling of ?
Revised: x1 sibling of ?

Original: x2 . ?
Revised: x2 . ?

Original: x2 a ﬁlm producer } == >
Revised: x2 a ﬁlm producer } == >

Original: A: SELECT count(*)
Revised: A: SELECT count(*)

Original: WHERE { M1 executive produced by ?
Revised: WHERE { M1 executive produced by ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 sibling of ?
Revised: x1 sibling of ?

Original: x2 . ?
Revised: x2 . ?

Original: x2 a ﬁlm producer }
Revised: x2 a ﬁlm producer }

Original: Partial Q: Did a (ﬁlm ’s prequel) star M
Revised: Partial Q: Did a (ﬁlm ’s prequel) star M

Original: Rationale:
Revised: Rationale:

Original: Did = {} , ”ﬁlm ’s prequel”
Revised: Did = {} , ”ﬁlm ’s prequel”

Original: = { ?x0 has sequel ?
Revised: = { ?x0 has sequel ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 a ﬁlm } , ”star M1”
Revised: x1 a ﬁlm } , ”star M1”

Original: = { ?x0 starred
Revised: = { ?x0 starred

Original: M1 }
Revised: M1 }

Original: ==
Revised: ==

Original: >
Revised: >

Original: A: SELECT count(*) WHERE { ?x0 has sequel ?
Revised: A: SELECT count(*) WHERE { ?x0 has sequel ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 a ﬁlm .
Revised: x1 a ﬁlm .

Original: x0 starred M1 }
Revised: x0 starred M1 }

Original: Partial Q: Did M0 art direct M
Revised: Partial Q: Did M0 art direct M

Original: Rationale:
Revised: Rationale:

Original: Did = {} , ”M0” replaces ?
Revised: Did = {} , ”M0” replaces ?

Original: x0, art direct M

Revised: x0, art direct M

Original: = { ?x0 art directed M1 } =
Revised: = { ?x0 art directed M1 } =

Original: = >
Revised: = >

Original: A: SELECT count(*) WHERE { M0 art directed M1 }
Revised: A: SELECT count(*) WHERE { M0 art directed M1 }

Original: Partial Q: Which person did M1 star
Revised: Partial Q: Which person did M1 star

Original: Rationale: Which = {} , ”person” = { ?
Revised: Rationale: Which = {} , ”person” = { ?

Original: x0 a person } , ”did M1 star” = { M1 starred ?
Revised: x0 a person } , ”did M1 star” = { M1 starred ?

Original: x0 }
Revised: x0 }

Original: ==
Revised: ==

Original: >
Revised: >

Original: A:
Revised: A:

Original: SELECT DISTINCT ?
Revised: SELECT DISTINCT ?

Original: x0 WHERE { ?
Revised: x0 WHERE { ?

Original: x0 a person .
Revised: x0 a person .

Original: M1 starred ?
Revised: M1 starred ?

Original: x0 }
Revised: x0 }

Original: Partial Q: Which (parent of M0) was a (person ’s parent)
Revised: Partial Q: Which (parent of M0) was a (person ’s parent)

Original: Rationale: Which = {} , ”parent of M0”
Revised: Rationale: Which = {} , ”parent of M0”

Original: = { ?
Revised: = { ?

Original: x0 parent of M0 } , ”costume designer ’s parent” = { ?
Revised: x0 parent of M0 } , ”costume designer ’s parent” = { ?

Original: x parent of ?x1 . ?
Revised: x parent of ?x1 . ?

Original: x1 a costume designer } ==
Revised: x1 a costume designer } ==

Original: >
Revised: >

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 parent of
Revised: x0 WHERE { ?x0 parent of

Original: M0 . ?
Revised: M0 . ?

Original: x0 parent of ?x1 . ?
Revised: x0 parent of ?x1 . ?

Original: x1 a costume designer }
Revised: x1 a costume designer }

Original: Partial Q: What was a ﬁlm written by M
Revised: Partial Q: What was a ﬁlm written by M

Original: Rationale:
Revised: Rationale:

Original: What = {} , ”ﬁlm written by M1”
Revised: What = {} , ”ﬁlm written by M1”

Original: = { ?
Revised: = { ?

Original: x0 a ﬁlm . ?
Revised: x0 a ﬁlm . ?

Original: x0 written by M1 } =
Revised: x0 written by M1 } =

Original: = >
Revised: = >

Original: A: SELECT
Revised: A: SELECT

Original: DISTINCT ?
Revised: DISTINCT ?

Original: x0 WHERE { ?x0 a ﬁlm .
Revised: x0 WHERE { ?x0 a ﬁlm .

Original: x0 written by M1 }
Revised: x0 written by M1 }

Original: Partial Q: What (star of M1) was a (cinematographer ’s parent)”,
Revised: Partial Q: What (star of M1) was a (cinematographer ’s parent)”,

Original: Rationale:
Revised: Rationale:

Original: What = {} , ”star of M1”
Revised: What = {} , ”star of M1”

Original: = { ?x0 starred in M1 } , ”cinematographer ’s parent” = { ?
Revised: = { ?x0 starred in M1 } , ”cinematographer ’s parent” = { ?

Original: x parent of ?x1 . ?
Revised: x parent of ?x1 . ?

Original: x1 a cinematographer } ==
Revised: x1 a cinematographer } ==

Original: > A: SELECT DISTINCT ?
Revised: > A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 starred in
Revised: x0 WHERE { ?x0 starred in

Original: M1 .
Revised: M1 .

Original: x0 parent of ?x1 . ?
Revised: x0 parent of ?x1 . ?

Original: x1 a cinematographer }
Revised: x1 a cinematographer }

Original: Partial Q: Who was a (producer of M1)
Revised: Partial Q: Who was a (producer of M1)

Original: Rationale: ”Who” = { ?
Revised: Rationale: ”Who” = { ?

Original: x0 a person } , ”executive producer of M1”
Revised: x0 a person } , ”executive producer of M1”

Original: = { ?x0 produced M1 } == >
Revised: = { ?x0 produced M1 } == >

Original: SELECT DISTINCT ?
Revised: SELECT DISTINCT ?

Original: x0 WHERE { ?
Revised: x0 WHERE { ?

Original: x0 a person . ?
Revised: x0 a person . ?

Original: x0 produced M1 }
Revised: x0 produced M1 }

Original: Partial Q: Who employed a person inﬂuenced by M
Revised: Partial Q: Who employed a person inﬂuenced by M

Original: Rationale: ”Who” = { ?
Revised: Rationale: ”Who” = { ?

Original: x0 a person } , ”employed a person” = { ?
Revised: x0 a person } , ”employed a person” = { ?

Original: x0 employed ?
Revised: x0 employed ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 a person } , ”inﬂuenced by M0”
Revised: x1 a person } , ”inﬂuenced by M0”

Original: = {
Revised: = {

Original: ?x1 inﬂuenced by M0 } =
Revised: ?x1 inﬂuenced by M0 } =

Original: = >
Revised: = >

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?
Revised: x0 WHERE { ?

Original: x0 a person . ?
Revised: x0 a person . ?

Original: x0 employed ?
Revised: x0 employed ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 a person . ?
Revised: x1 a person . ?

Original: x1 inﬂuenced by M0 }
Revised: x1 inﬂuenced by M0 }

Original: A.3. P ART 2: D YNAMICALLY SELECTED EXEMPLARS
Revised: A.3. P ART 2: D YNAMICALLY SELECTED EXEMPLARS

Original: After the constant prompt context, we add as additional context the exemplars that we dynamically selected for the given input using the process described in Section A.2.
Revised: After the constant prompt context, we add as additional context the exemplars that we dynamically selected for the given input using the process described in Section A.2.

Original: For our example from Figure 6, this results in the following prompt context, which is appended to the static context shown above.
Revised: For our example from Figure 6, this results in the following prompt context, which is appended to the static context shown above.

Original: Q: Were (M2 and M3) produced by (a actor that (inﬂuenced (M1 ’s editor)))
Revised: Q: Were (M2 and M3) produced by (a actor that (inﬂuenced (M1 ’s editor)))

Original: A: SELECT count(*) WHERE { ?
Revised: A: SELECT count(*) WHERE { ?

Original: x0 a actor . ?
Revised: x0 a actor . ?

Original: x0 inﬂuenced ?
Revised: x0 inﬂuenced ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 edited M1 .
Revised: x1 edited M1 .

Original: M2 produced by ?
Revised: M2 produced by ?

Original: x0 .
Revised: x0 .

Original: M3 produced by ?
Revised: M3 produced by ?

Original: x0 } Q: Did M1 ((star a (editor of M0)) and (star M2))
Revised: x0 } Q: Did M1 ((star a (editor of M0)) and (star M2))

Original: A: SELECT count(*) WHERE { ?x0 edited M0 .
Revised: A: SELECT count(*) WHERE { ?x0 edited M0 .

Original: M1 starred ?
Revised: M1 starred ?

Original: x0 . M1 starred M2 } Q: Was M1 ((founded by (M0 ’s editor)) and (founded by M2))
Revised: x0 . M1 starred M2 } Q: Was M1 ((founded by (M0 ’s editor)) and (founded by M2))

Original: A: SELECT count(*) WHERE { ?x0 edited M0 .
Revised: A: SELECT count(*) WHERE { ?x0 edited M0 .

Original: M1 founded by ?
Revised: M1 founded by ?

Original: x0 . M1 founded by M2 } Q: Which actor ((was inﬂuenced by (M4 ’s (art director and writer))) , (inﬂuenced M1) , and (inﬂuenced (M2 and M3))
Revised: x0 . M1 founded by M2 } Q: Which actor ((was inﬂuenced by (M4 ’s (art director and writer))) , (inﬂuenced M1) , and (inﬂuenced (M2 and M3))

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a actor . ?
Revised: x0 WHERE { ?x0 a actor . ?

Original: x0 inﬂuenced M1 . ?
Revised: x0 inﬂuenced M1 . ?

Original: x0 inﬂuenced M2 . ?
Revised: x0 inﬂuenced M2 . ?

Original: x inﬂuenced M3 . ?
Revised: x inﬂuenced M3 . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x1 art directed M4 . ?
Revised: x1 art directed M4 . ?

Original: x1 wrote M4 } Q: Which actor ((married a ﬁlm editor) and (was inﬂuenced by (M1 ’s sibling)))
Revised: x1 wrote M4 } Q: Which actor ((married a ﬁlm editor) and (was inﬂuenced by (M1 ’s sibling)))

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a actor . ?
Revised: x0 WHERE { ?x0 a actor . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x0 married to ?
Revised: x0 married to ?

Original: x2 . ?
Revised: x2 . ?

Original: x sibling of M1 . ?
Revised: x sibling of M1 . ?

Original: x2 a ﬁlm editor } Q: Did a (ﬁlm
Revised: x2 a ﬁlm editor } Q: Did a (ﬁlm

Original: x0 } Q
Revised: x0 } Q:

Original:  M A
Revised: a actor of M edit M A

Original: x
Revised: SELECT x0 portrayed

Original: x
Revised: x

Revised: M0 } Q

Original: M
Revised: M

Original: Q

Original: M
Revised: M

Revised: M0 ’s employee)))

Original: A: SELECT count(*
Revised: A: SELECT count(*)

Original: x0 employed  M0 .x
Revised: x0 employed by M0 .

Original: M1 starred  M2 } Q Did M
Revised: M1 starred ? x0 . M1 starred M2 } Q: Did M

Original: executive produce (M1 , M2 , and produce))
Revised: executive produce (M1 , M2 , and produce))

Original: M A SELECT 
Revised: M A: SELECT count(*)

Original: M produced M1 .
Revised: M0 executive produced M1 .

Original: M produced M2 .
Revised: M0 executive produced M2 .

Original: M produced M Q: What
Revised: M produced M3 } Q: What

Original: ﬁlm  ((inﬂuenced by M1) inﬂuenced by 
Revised: ﬁlm director ((inﬂuenced by M1) and (inﬂuenced by (a person that (founded (M and M5)))))) produced M A: SELECT DISTINCT

Original: a
Revised: x

Original: that founded (M and M5)))))
Revised: x0 a ﬁlm

Original:  produced MA
Revised: x0 produced M

Original: SELECT
Revised: x0 inﬂuenced by

Original: x x
Revised: x .

Original: ﬁlm x0 produced
Revised: x0 inﬂuenced by M

Original: x{+x
+}
Revised: {+x
+}

Original: x . x
Revised: a

Original: x1 founded M4 .
Revised: x1 founded M4 .

Original: x1 founded M5 } Q: Which cinematographer founded (M1 ’s distributor)
Revised: x1 founded M5 } Q: Which cinematographer founded (M1 ’s distributor)

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a cinematographer . ?
Revised: x0 WHERE { ?x0 a cinematographer . ?

Original: x0 founded ?x1 . ?
Revised: x0 founded ?x1 . ?

Original: x1 distributed M1 } Q: Which character inﬂuenced a ﬁlm editor A: SELECT DISTINCT ?
Revised: x1 distributed M1 } Q: Which character inﬂuenced a ﬁlm editor A: SELECT DISTINCT ?

Original: [-x
-] WHERE 
Revised: [-x
-]x0 WHERE { ?

Original: x0 a ﬁctional character . ?
Revised: x0 a ﬁctional character . ?

Original: x0 inﬂuenced ?
Revised: x0 inﬂuenced ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 a ﬁlm editor } Q: Which ﬁlm editor founded (M1 ’s distributor)
Revised: x1 a ﬁlm editor } Q: Which ﬁlm editor founded (M1 ’s distributor)

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a ﬁlm editor . ?
Revised: x0 WHERE { ?x0 a ﬁlm editor . ?

Original: x0 founded ?x1 . ?
Revised: x0 founded ?x1 . ?

Original: x1 distributed M1 }
Revised: x1 distributed M1 }

Original: Q: Which character married a screenwriter
Revised: Q: Which character married a screenwriter

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?
Revised: x0 WHERE { ?

Original: x0 a ﬁctional character . ?
Revised: x0 a ﬁctional character . ?

Original: x0 married to ?x1 . ?
Revised: x0 married to ?x1 . ?

Original: x1 a writer }
Revised: x1 a writer }

Original: A.3. P ART 3: S EQUENTIAL QUESTIONS
Revised: A.3. P ART 3: S EQUENTIAL QUESTIONS

Original: After providing the static and dynamic prompt context, we start by grounding the simplest subquestion.
Revised: After providing the static and dynamic prompt context, we start by grounding the simplest subquestion.

Original: To encourage the model to use the rationale-based reasoning, we use the preﬁx “Partial Q:” rather than just “Q:” for this question.
Revised: To encourage the model to use the rationale-based reasoning, we use the preﬁx “Partial Q:” rather than just “Q:” for this question.

Original: After we obtain the translation from the model, we append it to the prompt and then again ask the same subquestion, but this time using the regular “Q:” preﬁx.
Revised: After we obtain the translation from the model, we append it to the prompt and then again ask the same subquestion, but this time using the regular “Q:” preﬁx.

Original: In addition, we tell the model that it can simply copy the answer from above by adding the comment “# Copy the answer from above”.
Revised: In addition, we tell the model that it can simply copy the answer from above by adding the comment “# Copy the answer from above”.

Original: For the example from Figure 6, this ﬁrst part of the prompt looks as follows.
Revised: For the example from Figure 6, this ﬁrst part of the prompt looks as follows.

Original: Note that the text that is not bolded corresponds to the answers provided by the model.
Revised: Note that the text that is not bolded corresponds to the answers provided by the model.

Original: Partial Q: Which ﬁlm editor was inﬂuenced by a cinematographer
Revised: Partial Q: Which ﬁlm editor was inﬂuenced by a cinematographer

Original: Rationale: Which = {} , ”ﬁlm editor” = { ?
Revised: Rationale: Which = {} , ”ﬁlm editor” = { ?

Original: x0 a ﬁlm editor } , ”was inﬂuenced by a cinematographer”
Revised: x0 a ﬁlm editor } , ”was inﬂuenced by a cinematographer”

Original: = { ?x0 inﬂuenced by ?
Revised: = { ?x0 inﬂuenced by ?

Original: x1 . ?
Revised: x1 . ?

Original: x1 a cinematographer } ==
Revised: x1 a cinematographer } ==

Original: > A: SELECT DISTINCT ?
Revised: > A: SELECT DISTINCT ?

Original: x0 WHERE { ?x a ﬁlm editor . ?
Revised: x0 WHERE { ?x a ﬁlm editor . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x1 a cinematographer } # Copy the answer from above.
Revised: x1 a cinematographer } # Copy the answer from above.

Original: Q: Which ﬁlm editor was inﬂuenced by a cinematographer
Revised: Q: Which ﬁlm editor was inﬂuenced by a cinematographer

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a ﬁlm editor . ?
Revised: x0 WHERE { ?x0 a ﬁlm editor . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x1 a cinematographer }
Revised: x1 a cinematographer }

Original: After grounding the simplest question, we continue with the next subquestions, which are presented to the model one after the other using the ordinary preﬁx “Q:”.
Revised: After grounding the simplest question, we continue with the next subquestions, which are presented to the model one after the other using the ordinary preﬁx “Q:”.

Original: Sometimes, a subquestion is a strict extension of the previous subquestion.
Revised: Sometimes, a subquestion is a strict extension of the previous subquestion.

Original: To make sure that the model does not miss this, we add a comment of the form “# Extend the answer anove.
Revised: To make sure that the model does not miss this, we add a comment of the form “# Extend the answer above.

Original: Add: ...”.
Revised: Add: ...”.

Original: which highlights the delta between the questions.
Revised: which highlights the delta between the questions.

Original: In other cases, a subquestion is almost the same as the previous question, except that one constituent is replaced by another.
Revised: In other cases, a subquestion is almost the same as the previous question, except that one constituent is replaced by another.

Original: For each of these subquestions, we perform a request to the language model and append the answer to the prompt before asking the next question.
Revised: For each of these subquestions, we perform a request to the language model and append the answer to the prompt before asking the next question.

Original: For our running example, this part of the prompt looks as follows.
Revised: For our running example, this part of the prompt looks as follows.

Original: Note that we obtain the correct answer to the original question at the end.
Revised: Note that we obtain the correct answer to the original question at the end.

Original: # Extend the answer above.
Revised: # Extend the answer above.

Original: Add: that (wrote M
Revised: Add: that (wrote M3)

Original: Q: Which ﬁlm editor (was inﬂuenced by (a cinematographer that (wrote M3)))
Revised: Q: Which ﬁlm editor (was inﬂuenced by (a cinematographer that (wrote M3)))

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a ﬁlm editor . ?
Revised: x0 WHERE { ?x0 a ﬁlm editor . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x1 a cinematographer . ?
Revised: x1 a cinematographer . ?

Original: x1 wrote M3 } # Extend the answer above.
Revised: x1 wrote M3 } # Extend the answer above.

Original: Add: M Q: Which ﬁlm editor (was inﬂuenced by (a cinematographer that (wrote (M3 and M5))))
Revised: Add: M Q: Which ﬁlm editor (was inﬂuenced by (a cinematographer that (wrote (M3 and M5))))

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a ﬁlm editor . ?
Revised: x0 WHERE { ?x0 a ﬁlm editor . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x1 a cinematographer . ?
Revised: x1 a cinematographer . ?

Original: x1 wrote M3 . ?
Revised: x1 wrote M3 . ?

Original: x1 wrote M5 } # Extend the answer above
Revised: x1 wrote M5 } # Extend the answer above.

Original: Add:
Revised: Add:

Original: M Q: Which ﬁlm editor (was inﬂuenced by (a cinematographer that (wrote (M3 , M
Revised: M Q: Which ﬁlm editor (was inﬂuenced by (a cinematographer that (wrote (M3 , M

Original: and M5))))
Revised: and M5))))

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a ﬁlm editor . ?
Revised: x0 WHERE { ?x0 a ﬁlm editor . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x1 a cinematographer . ?
Revised: x1 a cinematographer . ?

Original: x1 wrote M3 . ?
Revised: x1 wrote M3 . ?

Original: x1 wrote M4 . ?
Revised: x1 wrote M4 . ?

Original: x1 wrote M5 } Q: Which ﬁlm editor (inﬂuenced by (M1 ’s editor))
Revised: x1 wrote M5 } Q: Which ﬁlm editor (inﬂuenced by (M1 ’s editor))

Original: A: SELECT DISTINCT ?
Revised: A: SELECT DISTINCT ?

Original: x0 WHERE { ?x0 a ﬁlm editor . ?
Revised: x0 WHERE { ?x0 a ﬁlm editor . ?

Original: x0 inﬂuenced by ?x1 . ?
Revised: x0 inﬂuenced by ?x1 . ?

Original: x1 edited M1 } # Extend the answer above.
Revised: x1 edited M1 } # Extend the answer above.

Original: Add: was inﬂuenced by (a cinematographer that (wrote (M3 , M4 , and M5)))
Revised: Add: was inﬂuenced by (a cinematographer that (wrote (M3 , M4 , and M5)))

Original: Q: Which ﬁlm editor ((was inﬂuenced by (a cinematographer that (wrote (M3 , M4 , and M5)))) and (inﬂuenced by (
Revised: Q: Which ﬁlm editor ((was inﬂuenced by (a cinematographer that (wrote (M3 , M4 , and M5)))) and (inﬂuenced by (M editor

Revised: ) A: SELECT DISTINCT ?

Original: x0 WHERE { x0 a ﬁlm editor . 
Revised: x0 WHERE { ?x0 a ﬁlm editor . ?

Original: x0 inﬂuenced by  ?x
Revised: x0 inﬂuenced by ?x1 . ?

Original: x1 edited M1 . 
Revised: x1 edited M1 . ?

Original: x0 inﬂuenced by ?
Revised: x0 inﬂuenced by ?

Original: x2 . ?
Revised: x2 . ?

Original: x2 a cinematographer . ?
Revised: x2 a cinematographer . ?

Original: x2 wrote M3 . ?
Revised: x2 wrote M3 . ?

Original: x2 wrote M4 . ?
Revised: x2 wrote M4 . ?

Original: x2 wrote M
Revised: x2 wrote M

Original: B L EAST - TO - MOST P ROMPTING FOR COGS
Revised: B L EAST - TO - MOST P ROMPTING FOR COGS

Original: James said that a manager liked that Aiden appreciated that Emily believed that the girl was posted a cake beside a table by Olivia .
Revised: James said that a manager liked that Aiden appreciated that Emily believed that the girl was posted a cake beside a table by Olivia .

Original: PARSE: say ( agent =
Revised: PARSE: say ( agent =

Original: James , ccomp = like ( agent = manager , ccomp = appreciate ( agent =
Revised: James , ccomp = like ( agent = manager , ccomp = appreciate ( agent =

Original: Aiden , ccomp = believe ( agent =
Revised: Aiden , ccomp = believe ( agent =

Original: Emily , ccomp = post ( recipient = * girl , theme = cake ( nmod .
Revised: Emily , ccomp = post ( recipient = * girl , theme = cake ( nmod .

Original: beside = table ) , agent = Olivia ) ) ) ) ) DONE
Revised: beside = table ) , agent = Olivia ) ) ) ) ) DONE

Original: The boy shortened the donut beside the bed in the car in the garden in the can on the tree .
Revised: The boy shortened the donut beside the bed in the car in the garden in the can on the tree .

Original: PARSE: shorten ( agent =
Revised: PARSE: shorten ( agent =

Original: * boy , theme =
Revised: * boy , theme =

Original: * donut ( nmod .
Revised: * donut ( nmod .

Original: beside =
Revised: beside =

Original: * bed ( nmod .
Revised: * bed ( nmod .

Original: in = * car ( nmod . in = * garden ( nmod .
Revised: in = * car ( nmod . in = * garden ( nmod .

Original: in = * can ( nmod .
Revised: in = * can ( nmod .

Original: on = * tree ) ) ) ) ) ) DONE
Revised: on = * tree ) ) ) ) ) ) DONE

Original: In this section, we provide more details on the application of dynamic least-to-most prompting for
Revised: In this section, we provide more details on the application of dynamic least-to-most prompting for

Original: COGS.
Revised: COGS.

Original: In particular, we detail all the prompts and show how the example in Figure 7 is processed step by step.
Revised: In particular, we detail all the prompts and show how the example in Figure 7 is processed step by step.

Original: Note that this is a variation of the application of dynamic-least-to-most prompting for
Revised: Note that this is a variation of the application of dynamic-least-to-most prompting for

Original: CFQ, which is detailed in Section A.
Revised: CFQ, which is detailed in Section A.

Original: Therefore, we mostly focus on highlight the differences.
Revised: Therefore, we mostly focus on highlighting the differences.

Original: B. COGS D ECOMPOSITION : D ETAILS AND P ROMPTS
Revised: B. COGS D ECOMPOSITION : D ETAILS AND P ROMPTS

Original: As discussed in Section 3.1, we use prompting-based syntactic parsing to decompose COGS sentences.
Revised: As discussed in Section 3.1, we use prompting-based syntactic parsing to decompose COGS sentences.

Original: To teach LMs to perform this kind of decomposition, we divide the syntactic parsing task into the following steps
Revised: To teach LMs to perform this kind of decomposition, we divide the syntactic parsing task into the following steps

Original: Iterative subclause decomposition 2.
Revised: Iterative subclause decomposition 2.

Original: Phrase identiﬁcation 3.
Revised: Phrase identiﬁcation 3.

Original: Iterative prepositional phrase decomposition and noun phrase annotation 4.
Revised: Iterative prepositional phrase decomposition and noun phrase annotation 4.

Original: Verb phrase normalization
Revised: Verb phrase normalization

Original: This is quite similar to the steps used for CFQ (see Section A.1) but there are some important differences.
Revised: This is quite similar to the steps used for CFQ (see Section A.1), but there are some important differences.

Original: Since the CFQ dataset only contains limited nesting of subclauses and noun phrases, we were able to identify them in a single step.
Revised: Since the CFQ dataset only contains limited nesting of subclauses and noun phrases, we were able to identify them in a single step.

Original: As exempliﬁed by the ﬁrst example in Figure 7, the COGS dataset contains much deeper nesting of subclauses (up to level 12), which we address by performing subclause decomposition iteratively.
Revised: As exempliﬁed by the ﬁrst example in Figure 7, the COGS dataset contains much deeper nesting of subclauses (up to level 12), which we address by performing subclause decomposition iteratively.

Original: This means that the prompt for subclause decomposition (step 1) is designed such that it only extracts one subclause at a time and is applied iteratively until all subclauses are extracted.
Revised: This means that the prompt for subclause decomposition (step 1) is designed such that it only extracts one subclause at a time and is applied iteratively until all subclauses are extracted.

Original: As exempliﬁed by the second example in Figure 7, COGS also contains deep nesting of prepositional phrases (up to level 12).
Revised: As exempliﬁed by the second example in Figure 7, COGS also contains deep nesting of prepositional phrases (up to level 12).

Original: We again address this by performing prepositional phrase decomposition (step 3) iteratively and designed the prompt such that it only extracts one prepositional phrase at a time and is applied iteratively until all prepositional phrases are extracted.
Revised: We again address this by performing prepositional phrase decomposition (step 3) iteratively and designed the prompt such that it only extracts one prepositional phrase at a time and is applied iteratively until all prepositional phrases are extracted.

Original: Example 1.
Revised: Example 1.

Original: To illustrate this step-by-step process, we begin with the ﬁrst example from Figure 7, which is “James said that a manager liked that Aiden appreciated that Emily believed that the girl was posted a cake beside a table by Olivia .”.
Revised: To illustrate this step-by-step process, we begin with the ﬁrst example from Figure 7, which is “James said that a manager liked that Aiden appreciated that Emily believed that the girl was posted a cake beside a table by Olivia .”.

Original: The ﬁrst step decomposes the subclauses of this sentence via 4 iterative calls:
Revised: The ﬁrst step decomposes the subclauses of this sentence via 4 iterative calls:

Original: • P=(James) V=(said) that C=(a manager liked that Aiden appreciated that Emily believed that the girl was posted a cake beside a table by Olivia)
Revised: • P=(James) V=(said) that C=(a manager liked that Aiden appreciated that Emily believed that the girl was posted a cake beside a table by Olivia)

Original: • P=(a manager) V=(liked) that C=(Aiden appreciated that Emily believed that the girl was posted a cake beside a table by Olivia) • P=(Aiden) V=(appreciated) that C=(Emily believed that the girl was posted a cake beside a table by Olivia) • P=(Emily) V=(believed) that C=(the girl was posted a cake beside a table by Olivia)
Revised: • P=(a manager) V=(liked) that C=(Aiden appreciated that Emily believed that the girl was posted a cake beside a table by Olivia) • P=(Aiden) V=(appreciated) that C=(Emily believed that the girl was posted a cake beside a table by Olivia) • P=(Emily) V=(believed) that C=(the girl was posted a cake beside a table by Olivia)

Original: In the second step, we identify the different phrases for the ﬁnal subclause “the girl was posted a cake beside a table by Olivia”, which yields:
Revised: In the second step, we identify the different phrases for the ﬁnal subclause “the girl was posted a cake beside a table by Olivia”, which yields:

Original: • P=(the girl)
Revised: • P=(the girl)

Original: V=(was posted) P=(a cake beside a table) by P=(Olivia)
Revised: V=(was posted) P=(a cake beside a table) by P=(Olivia)

Original: In the third step, we decompose the propositional phrase, which is not nested and therefore requires only one iteration:
Revised: In the third step, we decompose the propositional phrase, which is not nested and therefore requires only one iteration:

Original: • (a cake) beside P=(a table)
Revised: • (a cake) beside P=(a table)

Original: Note that the same prompt is also used to annotate noun phrases by marking the use of the deﬁnite article with a star.
Revised: Note that the same prompt is also used to annotate noun phrases by marking the use of the deﬁnite article with a star.

Original: We therefore also apply it to all other noun phrases, which yields:
Revised: We therefore also apply it to all other noun phrases, which yields:

Original: • James • a manager • Aiden •
Revised: • James • a manager • Aiden •

Original: Emily • the * girl • Olivia • a table
Revised: Emily • the * girl • Olivia • a table

Original: In the fourth step, we normalize all the verbs, which means that “said” yields “say”, “liked” yields “like”, “appreciated” yields “appreciate”, “believed” yields “believe”, and “was posted” yields “post”.
Revised: In the fourth step, we normalize all the verbs, which means that “said” yields “say”, “liked” yields “like”, “appreciated” yields “appreciate”, “believed” yields “believe”, and “was posted” yields “post”.

Original: Finally, we run a few lines of Python code that puts all these parts back together to obtain a parse tree.
Revised: Finally, we run a few lines of Python code that puts all these parts back together to obtain a parse tree.

Original: Note that the normalized verbs do not replace the original ones but are kept in addition, which allows us to obtain higher recall when selecting exemplars.
Revised: Note that the normalized verbs do not replace the original ones but are kept in addition, which allows us to obtain higher recall when selecting exemplars.

Original: This yields the following fully decomposed sentence:
Revised: This yields the following fully decomposed sentence:

Original: • (James) (said [say]) that ((a manager) (liked [like]) that ((Aiden) (appreciated [appreciate]) that ((Emily) (believed [believe]) that ((the * girl) (was posted [post]) ((a cake) (beside) (a table)) by (Olivia)))))
Revised: • (James) (said [say]) that ((a manager) (liked [like]) that ((Aiden) (appreciated [appreciate]) that ((Emily) (believed [believe]) that ((the * girl) (was posted [post]) ((a cake) (beside) (a table)) by (Olivia)))))

Original: Example 2.
Revised: Example 2.

Original: We now walk through the step-by-step process using the second example from Figure 7, which is “The boy shortened the donut beside the bed in the car in the garden in the can on the tree .”.
Revised: We now walk through the step-by-step process using the second example from Figure 7, which is “The boy shortened the donut beside the bed in the car in the garden in the can on the tree .”.

Original: Since this example does not contain any suubclauses, the ﬁrst step does not apply.
Revised: Since this example does not contain any suubclauses, the ﬁrst step does not apply.

Original: In the second step, we identify the different phrases, which yields:
Revised: In the second step, we identify the different phrases, which yields:

Original: • P=(The boy) V=(shortened)
Revised: • P=(The boy) V=(shortened)

Original: P=(the donut beside the bed in the car in the garden in the can on the tree)
Revised: P=(the donut beside the bed in the car in the garden in the can on the tree)

Original: In the third step, we decompose the propositional phrase, which happens to be nested and therefore requires 5 iterations:
Revised: In the third step, we decompose the prepositional phrase, which happens to be nested and therefore requires 5 iterations:

Original: • (the * donut) beside P=(the bed in the car in the garden in the can on the tree) • (the * bed) in P=(the car in the garden in the can on the tree) • (the * car) in P=(the garden in the can on the tree) • (the * garden) in P=(the can on the tree) • (the * can) on P=(the tree)
Revised: • (the * donut) beside P=(the bed in the car in the garden in the can on the tree) • (the * bed) in P=(the car in the garden in the can on the tree) • (the * car) in P=(the garden in the can on the tree) • (the * garden) in P=(the can on the tree) • (the * can) on P=(the tree)

Original: Since the same prompt is also used to annotate noun phrases by marking the use of the deﬁnite article with a star, we also apply it to all other noun phrases, which yields:
Revised: Since the same prompt is also used to annotate noun phrases by marking the use of the deﬁnite article with a star, we also apply it to all other noun phrases, which yields:

Original: • The * boy • the * tree
Revised: • The * boy • the * tree

Original: In the fourth step, we normalize all the verbs, which means that “shortened” yields “shorten”.
Revised: In the fourth step, we normalize all the verbs, which means that “shortened” yields “shorten”.

Original: This yields the following fully decomposed sentence:
Revised: This yields the following fully decomposed sentence:

Original: • (The * boy) (shortened [shorten]) ((the * donut) (beside) ((the * bed) (in) ((the * car) (in) ((the * garden) (in) ((the * can) (on) (the * tree))))))
Revised: • (The * boy) (shortened [shorten]) ((the * donut) (beside) ((the * bed) (in) ((the * car) (in) ((the * garden) (in) ((the * can) (on) (the * tree))))))

Original: Below, we present the exact prompt contexts used for each of the parsing steps.
Revised: Below, we present the exact prompt contexts used for each of the parsing steps.

Original: B.1. I TERATIVE SUBCLAUSE DECOMPOSITION
Revised: B.1. I TERATIVE SUBCLAUSE DECOMPOSITION

Original: Q
Revised: Q

Original: : The girl expected that
Revised: : The girl expected that

Original: Daniel liked that a weapon was liked
Revised: Daniel liked that a weapon was liked

Original: A: P=(The girl) V=(expected) that C=(Daniel liked that a weapon was liked) Q: The girl hoped that a donut was returned to a cat by a monkey
Revised: A: P=(The girl) V=(expected) that C=(Daniel liked that a weapon was liked) Q: The girl hoped that a donut was returned to a cat by a monkey

Original: A: P=(The girl) V=(hoped) that C=(a donut was returned to a cat by a monkey) Q: a teacher thought that a girl liked that a cookie was valued by Emma A: P=(a teacher) V=(thought) that C=(a girl liked that a cookie was valued by Emma) Q: Benjamin supported that Avery loaned a donut in a cup to a frog A: P=(Benjamin) V=(supported)
Revised: A: P=(The girl) V=(hoped) that C=(a donut was returned to a cat by a monkey) Q: a teacher thought that a girl liked that a cookie was valued by Emma A: P=(a teacher) V=(thought) that C=(a girl liked that a cookie was valued by Emma) Q: Benjamin supported that Avery loaned a donut in a cup to a frog A: P=(Benjamin) V=(supported)

Original: C=(that Avery loaned a donut in a cup to a frog) Q: A girl liked that a frog liked that Eleanor gave a brush to the professor A: P=(A girl) V=(liked) that C=(a frog liked that Eleanor gave a brush to the professor) Q: Charlotte expected that Emma liked that the patient lended a box on a table to Nora A: P=(Charlotte) V=(expected) that C=(Emma liked that the patient lended a box on a table to Nora) Q: Aiden believed that a mouse hoped that Olivia examined a glue beside the stage beside a duckA:P=(Aiden) V=(believed) that C=(a mouse hoped that Olivia examined a glue beside the stage beside a duck) Q: a girl declared that the boy was offered the watch beside the table A: P=(a girl) V=(declared) that C=(the boy was offered the watch beside the table)
Revised: C=(that Avery loaned a donut in a cup to a frog) Q: A girl liked that a frog liked that Eleanor gave a brush to the professor A: P=(A girl) V=(liked) that C=(a frog liked that Eleanor gave a brush to the professor) Q: Charlotte expected that Emma liked that the patient lended a box on a table to Nora A: P=(Charlotte) V=(expected) that C=(Emma liked that the patient lended a box on a table to Nora) Q: Aiden believed that a mouse hoped that Olivia examined a glue beside the stage beside a duckA:P=(Aiden) V=(believed) that C=(a mouse hoped that Olivia examined a glue beside the stage beside a duck) Q: a girl declared that the boy was offered the watch beside the table A: P=(a girl) V=(declared) that C=(the boy was offered the watch beside the table)

Original: Q: the teacher declared that a donut was given to the student by Noah A: P=(the teacher) V=(declared) that C=(a donut was given to the student by Noah) Q: A dog respected that Emma rented the biscuit in a pot beside a nest to the boy A: P=(A dog) V=(respected) that C=(Emma rented the biscuit in a pot beside a nest to the boy) Q: The teacher liked that a driver hoped that the girl meant to eat A: P=(The teacher) V=(liked) that C=(a driver hoped that the girl meant to eat) Q: the giraffe liked that Olivia noticed that the butterﬂy was given the drink in the garden on a tableA:
Revised: Q: the teacher declared that a donut was given to the student by Noah A: P=(the teacher) V=(declared) that C=(a donut was given to the student by Noah) Q: A dog respected that Emma rented the biscuit in a pot beside a nest to the boy A: P=(A dog) V=(respected) that C=(Emma rented the biscuit in a pot beside a nest to the boy) Q: The teacher liked that a driver hoped that the girl meant to eat A: P=(The teacher) V=(liked) that C=(a driver hoped that the girl meant to eat) Q: the giraffe liked that Olivia noticed that the butterﬂy was given the drink in the garden on a tableA:

Original: [Table]
Revised: [Table]

Original: =
Revised: =

Original: [Table]
Revised: [Table]

Original: the
Revised: the

Original: [Table]
Revised: [Table]

Original: [Table]

Original: =
Revised: =

Original: [Table]
Revised: [Table]

Original: that
Revised: that

Original: [Table]
Revised: [Table]

Original: =(
Revised: =

Original: Olivia noticed that the butterﬂy was given the drink in the garden on a table)
Revised: [Table] noticed that the butterﬂy was given the drink in the garden on a table)

Original: Q: A cat in a box on a table tolerated that a professor beside the stage liked a dog A: P=(A cat in a box on a table) V=(tolerated) that C=(a professor beside the stage liked a dog)
Revised: Q: A cat in a box on a table tolerated that a professor beside the stage liked a dog A: P=(A cat in a box on a table) V=(tolerated) that C=(a professor beside the stage liked a dog)

Original: Q: the professor on a chair beside a bench in a room expected that a mouse was given a present by a doctor on the stage in a house
Revised: Q: the professor on a chair beside a bench in a room expected that a mouse was given a present by a doctor on the stage in a house

Original: A: P=(the professor on a chair beside a bench in a room) V=(expected) that C=(a mouse was given a present by a doctor on the stage in a house) Q: Joe liked that Fred thought that a girl confessed that the chicken meant that Freda liked that Peter hoped that Elizabeth said that a professor whished that Anna hoped that Lisa declared that the creature tolerated that a teacher liked that Lily was given the clock beside a guitar beside the table by Isabella A: P=(Joe) V=(liked) that C=(Fred thought that a girl confessed that the chicken meant that Freda liked that Peter hoped that Elizabeth said that a professor whished that Anna hoped that Lisa declared that the creature tolerated that a teacher liked that Lily was given the clock beside a guitar beside the table by Isabella)
Revised: A: P=(the professor on a chair beside a bench in a room) V=(expected) that C=(a mouse was given a present by a doctor on the stage in a house) Q: Joe liked that Fred thought that a girl confessed that the chicken meant that Freda liked that Peter hoped that Elizabeth said that a professor whished that Anna hoped that Lisa declared that the creature tolerated that a teacher liked that Lily was given the clock beside a guitar beside the table by Isabella A: P=(Joe) V=(liked) that C=(Fred thought that a girl confessed that the chicken meant that Freda liked that Peter hoped that Elizabeth said that a professor whished that Anna hoped that Lisa declared that the creature tolerated that a teacher liked that Lily was given the clock beside a guitar beside the table by Isabella)

Original: B.1. P HRASE IDENTIFICATION
Revised: B.1. P HRASE IDENTIFICATION

Original: Q: a boy meant to talk A: P=(a boy) V=(meant) (to talk) Q: Camila rolled a lamb beside a ﬂower A: P=(Camila) V=(rolled) P=(a lamb beside a ﬂower))
Revised: Q: a boy meant to talk A: P=(a boy) V=(meant) (to talk) Q: Camila rolled a lamb beside a ﬂower A: P=(Camila) V=(rolled) P=(a lamb beside a ﬂower))

Original: Q: the hen appreciated the journalist on the piano A: P=(the hen) V=(appreciated)
Revised: Q: the hen appreciated the journalist on the piano A: P=(the hen) V=(appreciated)

Original: P=(the journalist on the piano) Q: The chicken on a box needed to eat A: P=(The chicken on a box) V=(needed) (to eat) Q: a king awarded Joe Fred A: P=(a king) V=(awarded) P=(Joe) P=(Fred) Q: The drink was snapped by Ava A: P=(The drink) V=(was snapped) by P=(Ava) Q: the teacher passed Joe a pen
Revised: P=(the journalist on the piano) Q: The chicken on a box needed to eat A: P=(The chicken on a box) V=(needed) (to eat) Q: a king awarded Joe Fred A: P=(a king) V=(awarded) P=(Joe) P=(Fred) Q: The drink was snapped by Ava A: P=(The drink) V=(was snapped) by P=(Ava) Q: the teacher passed Joe a pen

Original: A: P=(the teacher) V=(passed) P=(Joe)
Revised: A: P=(the teacher) V=(passed) P=(Joe)

Original: P=(a pen) Q: the girl offered Fred to James A: P=(the girl) V=(offered) P=(Fred) to P=(James
Revised: P=(a pen) Q: the girl offered Fred to James A: P=(the girl) V=(offered) P=(Fred) to P=(James

Original: ) Q: A teacher beside the table wanted to hunt A: P
Revised: ) Q: A teacher beside the table wanted to hunt A: P

Original: [Table]
Revised: [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table][Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: monkey

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Q: Avery loaned a donut in a cup to a frog A
Revised: Q: Avery loaned a donut in a cup to a frog A

Original: P
Revised: P

Original: Avery
Revised: Avery

Original: V
Revised: V

Original: loaned
Revised: loaned

Original: donut
Revised: donut

Original: a
Revised: a

Original: to P
Revised: to P

Original: (a frog) Q: the patient lended a box on a table to Nora A: P=(the patient) V=(lended) P=(a box on a table) to P=(Nora) Q: The butterﬂy was given the drink in the garden on a table A: P=(The butterﬂy)
Revised: (a frog) Q: the patient lended a box on a table to Nora A: P=(the patient) V=(lended) P=(a box on a table) to P=(Nora) Q: The butterﬂy was given the drink in the garden on a table A: P=(The butterﬂy)

Original: V=(was given)
Revised: V=(was given)

Original: P=(the drink in the garden on a table) Q: Luke gave the cake on the windowsill beside a bed under a lamp to a teacher on a pony A: P=(Luke) V=(gave)
Revised: P=(the drink in the garden on a table) Q: Luke gave the cake on the windowsill beside a bed under a lamp to a teacher on a pony A: P=(Luke) V=(gave)

Original: P=(the cake on the windowsill beside a bed under a lamp) to P=(a teacher on a pony) Q: The teacher received a cat in a box on a table on a carpet beside a window from Joe A: P=(The teacher) V=(received)
Revised: P=(the cake on the windowsill beside a bed under a lamp) to P=(a teacher on a pony) Q: The teacher received a cat in a box on a table on a carpet beside a window from Joe A: P=(The teacher) V=(received)

Original: P=(a cat in a box on a table on a carpet beside a window) (from P=(Joe))
Revised: P=(a cat in a box on a table on a carpet beside a window) (from P=(Joe))

Original: Q: A cat in a cage on a table gave a present in a box to a dog on the ﬂoor A: P=(A cat in a cage on a table) V=(gave)
Revised: Q: A cat in a cage on a table gave a present in a box to a dog on the ﬂoor A: P=(A cat in a cage on a table) V=(gave)

Original: P=(a present in a box) to P=(a dog on the ﬂoor) Q: A ring on a pillow in a wrapper was presented to the professor in a room by a priest on a chairA:P=(a ring on a pillow in a wrapper)
Revised: P=(a present in a box) to P=(a dog on the ﬂoor) Q: A ring on a pillow in a wrapper was presented to the professor in a room by a priest on a chairA:P=(a ring on a pillow in a wrapper)

Original: V=(was presented) to P=(the professor in a room) by P=(a priest on a chair)
Revised: V=(was presented) to P=(the professor in a room) by P=(a priest on a chair)

Original: Q: Charles posted the pony in a barn the drink in a pear on the ﬂoor beside the book in the iron on a brick in the robe in a pipe A: P=(Charles) V=(posted)
Revised: Q: Charles posted the pony in a barn the drink in a pear on the ﬂoor beside the book in the iron on a brick in the robe in a pipe A: P=(Charles) V=(posted)

Original: P=(the pony in a barn)
Revised: P=(the pony in a barn)

Original: P=(the drink in a pear on the ﬂoor beside the book in the iron on a brick in the robe in a pipe) Q: a frog was given the apple in a school in a town on a stool on a
Revised: P=(the drink in a pear on the ﬂoor beside the book in the iron on a brick in the robe in a pipe) Q: a frog was given the apple in a school in a town on a stool on a

Original: [Table]
Revised: [Table]

Original: Q: Joe A: Joe
Revised: Q: Joe A: Joe

Original: Q: Liam
Revised: Q: Liam

Original: A: Liam
Revised: A: Liam

Original: Q: Isabelle
Revised: Q: Isabelle

Original: A: Isabelle
Revised: A: Isabelle

Original: Q: Benedict
Revised: Q: Benedict

Original: A: Benedict
Revised: A: Benedict

Original: Q: A atom A: A atom
Revised: Q: A atom A: A atom

Original: Q: a helmet
Revised: Q: a helmet

Original: A: a helmet
Revised: A: a helmet

Original: Q: A camera
Revised: Q: A camera

Original: A: A camera
Revised: A: A camera

Original: Q: a helmet
Revised: Q: a helmet

Original: A: a helmet
Revised: A: a helmet

Original: Q: the atom
Revised: Q: the atom

Original: A: the * atom
Revised: A: the * atom

Original: Q: The helmet
Revised: Q: The helmet

Original: A: The * helmet
Revised: A: The * helmet

Original: Q: the camera
Revised: Q: the camera

Original: A: the * camera
Revised: A: the * camera

Original: Q: The song
Revised: Q: The song

Original: A: The * song
Revised: A: The * song

Original: Q: the rino
Revised: Q: the rino

Original: A: the * rino
Revised: A: the * rino

Original: Q: The lizzard
Revised: Q: The lizzard

Original: A: The * lizzard
Revised: A: The * lizzard

Original: Q: the plane
Revised: Q: the plane

Original: A: the * plane
Revised: A: the * plane

Original: Q: The gnome
Revised: Q: The gnome

Original: A: The * gnome
Revised: A: The * gnome

Original: Q: the nose
Revised: Q: the nose

Original: A: the * nose
Revised: A: the * nose

Original: Q: The adventure
Revised: Q: The adventure

Original: A: The * adventure
Revised: A: The * adventure

Original: Q: the undertaking
Revised: Q: the undertaking

Original: A: the * undertaking
Revised: A: the * undertaking

Original: Q: The camera
Revised: Q: The camera

Original: A: The * camera
Revised: A: The * camera

Original: Q: the sky
Revised: Q: the sky

Original: A: the * sky
Revised: A: the * sky

Original: Q: The trainer
Revised: Q: The trainer

Original: A: The * trainer
Revised: A: The * trainer

Original: Q: the underdog
Revised: Q: the underdog

Original: A: the * underdog
Revised: A: the * underdog

Original: Q: The mole
Revised: Q: The mole

Original: A: The * mole
Revised: A: The * mole

Original: Q: the ﬂoor
Revised: Q: the ﬂoor

Original: A: the * ﬂoor
Revised: A: the * ﬂoor

Original: Q: The stone
Revised: Q: The stone

Original: A: The * stone
Revised: A: The * stone

Original: Q: the smartphone
Revised: Q: the smartphone

Original: ) Q: A
Revised: ) Q: A

Original: in a trainer on a leg on a helmet beside a window A:
Revised: in a trainer on a leg on a helmet beside a window A:

Original: P
Revised: P

Original: trainer on a leg on
Revised: trainer on a leg on

Original: helmet beside a window) Q: a rifﬂe on the ﬂoor in the rino on the leg beside the stone on a spear beside the rino on a sofaA:(a rifﬂe) (on) P=(the ﬂoor in the rino on the leg beside the stone on a spear beside the rino on a sofa) 
Revised: helmet beside a window) Q: a rifﬂe on the ﬂoor in the rino on the leg beside the stone on a spear beside the rino on a sofaA:(a rifﬂe) (on) P=(the ﬂoor in the rino on the leg beside the stone on a spear beside the rino on a sofa) Q

Original: [Table]
Revised: [Table]

Original: The referee in a rino beside a ofﬁcial on a smartphone in the camera beside the trainer in
Revised: The referee in a rino beside a ofﬁcial on a smartphone in the camera beside the trainer in

Original: a trainer under the spider on the shingle A: (The * referee) (in) P=(a rino beside a ofﬁcial on a smartphone in the camera beside the trainer in a trainer under the spider on the shingle) Q: Joe in a helmet in a vessel on the water beside a harbor in a village beside city in a country on a continent on earth A:
Revised: a trainer under the spider on the shingle A: (The * referee) (in) P=(a rino beside a ofﬁcial on a smartphone in the camera beside the trainer in a trainer under the spider on the shingle) Q: Joe in a helmet in a vessel on the water beside a harbor in a village beside city in a country on a continent on earth A:

Original: (Joe) (in) P=(a helmet in a vessel on the water beside a harbor in a village beside city in a country on a continent on earth) Q: a underdog beside a foreigner on a smartphone in a truck in a microwave beside a rocket on a computer on a stool on the surface A: (a underdog) (beside) P=(a foreigner on a smartphone in a truck in a microwave beside a rocket on a computer on a stool on the surface) Q: the rooster over a computer on a board on the spear on a leg on the ﬂoor beside a rocker under a window on a shingle beside a rino under the clouds on the sky A: (the * rooster) (over) P=(a computer on a board on the spear on a leg on the ﬂoor beside a rocker under a window on a shingle beside a rino under the clouds on the sky) Q: A phone on the helmet in the rino on the wallet beside the knive on a ﬂoor beside the lizzard on a underdog in a wardrobe beside the rocket in the bus beside a hut in the village A: (A phone)
Revised: (Joe) (in) P=(a helmet in a vessel on the water beside a harbor in a village beside city in a country on a continent on earth) Q: a underdog beside a foreigner on a smartphone in a truck in a microwave beside a rocket on a computer on a stool on the surface A: (a underdog) (beside) P=(a foreigner on a smartphone in a truck in a microwave beside a rocket on a computer on a stool on the surface) Q: the rooster over a computer on a board on the spear on a leg on the ﬂoor beside a rocker under a window on a shingle beside a rino under the clouds on the sky A: (the * rooster) (over) P=(a computer on a board on the spear on a leg on the ﬂoor beside a rocker under a window on a shingle beside a rino under the clouds on the sky) Q: A phone on the helmet in the rino on the wallet beside the knive on a ﬂoor beside the lizzard on a underdog in a wardrobe beside the rocket in the bus beside a hut in the village A: (A phone)

Original: (on) P=(the helmet in the rino on the wallet beside the knive on a ﬂoor beside the lizzard on a underdog in a wardrobe beside the rocket in the bus beside a hut in the village) Q: the country in a school in a town on a stool on a sheet on a leg on a rocker on the ﬂoor in a camera beside a building on a street in a city in the world in the universe A: (the * country) (in) P=(a school in a town on a stool on a sheet on a leg on a rocker on the ﬂoor in a camera beside a building on a street in a city in the world in the universe)
Revised: (on) P=(the helmet in the rino on the wallet beside the knive on a ﬂoor beside the lizzard on a underdog in a wardrobe beside the rocket in the bus beside a hut in the village) Q: the country in a school in a town on a stool on a sheet on a leg on a rocker on the ﬂoor in a camera beside a building on a street in a city in the world in the universe A: (the * country) (in) P=(a school in a town on a stool on a sheet on a leg on a rocker on the ﬂoor in a camera beside a building on a street in a city in the world in the universe)

Original: B.1. V ERB
Revised: B.1. V ERB

Original: [Table]
Revised: [Table]

Original: A: return
Revised: A: return

Original: Q: was lended
Revised: Q: was lended

Original: A: lend
Revised: A: lend

Original: Q: was brought
Revised: Q: was brought

Original: A: bring
Revised: A: bring

Original: B
Revised: B

Original: COGS E XEMPLAR S ELECTION : D ETAILS
Revised: COGS E XEMPLAR S ELECTION : D ETAILS

Original: We provide a general description of the exemplar selection process in Section 3.2 and add the CFQspeciﬁc details in Section A.2.
Revised: We provide a general description of the exemplar selection process in Section 3.2 and add the CFQspeciﬁc details in Section A.2.

Original: Since COGS is using the same heuristics with slightly different parameters, we just highlight the differences here.
Revised: Since COGS is using the same heuristics with slightly different parameters, we just highlight the differences here.

Original: B.2. E XEMPLAR POOL
Revised: B.2. E XEMPLAR POOL

Original: For CFQ, we use a random subset consisting of 1000 examples from the training set as a pool of candidate exemplars.
Revised: For CFQ, we use a random subset consisting of 1000 examples from the training set as a pool of candidate exemplars.

Original: For COGS, we went a different route and manually selected a set of exemplars from the training data.
Revised: For COGS, we went a different route and manually selected a set of exemplars from the training data.

Original: The main reason for this is that COGS contains many different verbs and since unergative and unaccusative are translated differently, it is important to include as many of them in the exemplar pool as  it would otherwise be hard for the model to ﬁgure out how a certain verb should be treated.
Revised: The main reason for this is that COGS contains many different verbs and since unergative and unaccusative are translated differently, it is important to include as many of them in the exemplar pool as possible, as it would otherwise be hard for the model to ﬁgure out how a certain verb should be treated.

Original: Also, since some of those verbs are quite rare in the training data, some of them would not be included in a reasonably sized random sample.
Revised: Also, since some of those verbs are quite rare in the training data, some of them would not be included in a reasonably sized random sample.

Original: Concretely, we include in the exemplar pool for COGS the following 62 exemplars containing all unegative and unaccuastive verbs that occur in the training data.
Revised: Concretely, we include in the exemplar pool for COGS the following 62 exemplars containing all unergative and unaccusative verbs that occur in the training data.

Original: As for CFQ, we use for all exemplars a parsed version of the natural language sentence, which contains additional parentheses and other annotations that make it easier for the model to perform the translation.
Revised: As for CFQ, we use for all exemplars a parsed version of the natural language sentence, which contains additional parentheses and other annotations that make it easier for the model to perform the translation.

Original: (The * goose) (baked [bake]) PARSE: bake ( agent = * goose ) DONE (A bell) (broke [break]) PARSE: break ( theme = bell ) DONE (A molecule) (burned [burn]) PARSE: burn ( theme = molecule ) DONE (A cat) (called [call]) PARSE: call ( agent = cat ) DONE (A cake) (changed [change]) PARSE: change ( theme = cake ) DONE (A student) (cleaned [clean]) PARSE: clean ( agent = student ) DONE (A pickle)(collapsed
Revised: (The * goose) (baked [bake]) PARSE: bake ( agent = * goose ) DONE (A bell) (broke [break]) PARSE: break ( theme = bell ) DONE (A molecule) (burned [burn]) PARSE: burn ( theme = molecule ) DONE (A cat) (called [call]) PARSE: call ( agent = cat ) DONE (A cake) (changed [change]) PARSE: change ( theme = cake ) DONE (A student) (cleaned [clean]) PARSE: clean ( agent = student ) DONE (A pickle)

Original:  [collapse])
Revised: (collapsed [collapse])

Original: PARSE: collapse ( theme = pickle ) DONE (A baby)
Revised: PARSE: collapse ( theme = pickle ) DONE (A baby)

Original: (cooked [cook])
Revised: (cooked [cook])

Original: PARSE: cook ( agent = baby ) DONE (A dog) (crumpled [crumple])
Revised: PARSE: cook ( agent = baby ) DONE (A dog) ( crumpled

Original:  PARSE: crumple ( theme = dog ) DONE (A pig) (cried [cry])
Revised: [crumple]) PARSE: crumple ( theme = dog ) DONE (A pig cried

Original:  PARSE: cry ( agent = pig ) DONE (A chicken)
Revised: [cry]) PARSE: cry ( agent = pig ) DONE (A chicken)

Original: (danced [dance])
Revised: (danced [ dance

Original: PARSE: dance ( agent = chicken ) DONE (A buyer) (decomposed [decompose])
Revised: PARSE: dance ( agent = chicken ) DONE (A buyer) (decomposed [decompose])

Original: PARSE: decompose ( theme = buyer ) DONE (A mandarin) (disintegrated [disintegrate])
Revised: PARSE: decompose ( theme = buyer ) DONE (A mandarin) (disintegrated [disintegrate])

Original: PARSE: disintegrate ( theme = mandarin ) DONE (A bowl) (doubled [double])
Revised: PARSE: disintegrate ( theme = mandarin ) DONE (A bowl) (doubled [double])

Original: PARSE: double ( theme = bowl ) DONE (A cat) (drew [draw])
Revised: PARSE: double ( theme = bowl ) DONE (A cat) (drew [draw])

Original: PARSE: draw ( agent = cat ) DONE (A monster) (dusted [dust])
Revised: PARSE: draw ( agent = cat ) DONE (A monster) (dusted [dust])

Original: PARSE: dust ( agent = monster ) DONE (A boy) (ate [eat])
Revised: PARSE: dust ( agent = monster ) DONE (A boy) (ate [eat])

Original: PARSE: eat ( agent = boy ) DONE (A boy) (enlarged [enlarge])
Revised: PARSE: eat ( agent = boy ) DONE (A boy) (enlarged [enlarge])

Original: PARSE: enlarge ( theme = boy ) DONE (A boy) (examined [examine])
Revised: PARSE: enlarge ( theme = boy ) DONE (A boy) (examined [examine])

Original: PARSE: examine ( agent = boy ) DONE (A girl) (ﬂoated [ﬂoat])
Revised: PARSE: examine ( agent = boy ) DONE (A girl) (ﬂoated [ﬂoat])

Original: PARSE: ﬂoat ( theme = girl ) DONE (A melon) (froze [freeze]) PARSE: freeze ( theme = melon ) DONE (A resident) (frowned [frown])
Revised: PARSE: ﬂoat ( theme = girl ) DONE (A melon) (froze [freeze]) PARSE: freeze ( theme = melon ) DONE (A resident) (frowned [frown])

Original: PARSE: frown ( agent = resident ) DONE (A dog) (gasped [gasp])
Revised: PARSE: frown ( agent = resident ) DONE (A dog) (gasped [gasp])

Original: PARSE: gasp ( agent = dog ) DONE (A chicken)
Revised: PARSE: gasp ( agent = dog ) DONE (A chicken)

Original: (giggled [giggle])
Revised: (giggled [giggle])

Original: PARSE: giggle ( agent = chicken ) DONE (A dog) (grew [grow])
Revised: PARSE: giggle ( agent = chicken ) DONE (A dog) (grew [grow])

Original: PARSE: grow ( theme = dog ) DONE (A butterﬂy) (heard [hear])
Revised: PARSE: grow ( theme = dog ) DONE (A butterﬂy) (heard [hear])

Original: PARSE: hear ( agent = butterﬂy ) DONE (A child)
Revised: PARSE: hear ( agent = butterﬂy ) DONE (A child)

Original: (hunted [hunt])
Revised: (hunted [hunt])

Original: PARSE: hunt ( agent = child ) DONE (A cat) (improved [improve])
Revised: PARSE: hunt ( agent = child ) DONE (A cat) (improved [improve])

Original: PARSE: improve ( theme = cat ) DONE (A boy) (inﬂated [inﬂate])
Revised: PARSE: improve ( theme = cat ) DONE (A boy) (inﬂated [inﬂate])

Original: PARSE: inﬂate ( theme = boy ) DONE (A cat) (investigated [investigate])
Revised: PARSE: inﬂate ( theme = boy ) DONE (A cat) (investigated [investigate])

Original: PARSE: investigate ( agent = cat ) DONE
Revised: PARSE: investigate ( agent = cat ) DONE

Original: (A girl) (jogged [jog])
Revised: (A girl) (jogged [jog])

Original: PARSE: jog ( agent = girl ) DONE (A chicken)
Revised: PARSE: jog ( agent = girl ) DONE (A chicken)

Original: (juggled [juggle])
Revised: (juggled [juggle])

Original: PARSE: juggle ( agent = chicken ) DONE (A penguin) (knew [know])
Revised: PARSE: juggle ( agent = chicken ) DONE (A penguin) (knew [know])

Original: PARSE: know ( agent = penguin ) DONE (A mouse) (laughed [laugh])
Revised: PARSE: know ( agent = penguin ) DONE (A mouse) (laughed [laugh])

Original: PARSE: laugh ( agent = mouse ) DONE (A mouse) (napped [nap])
Revised: PARSE: laugh ( agent = mouse ) DONE (A mouse) (napped [nap])

Original: PARSE: nap ( agent = mouse ) DONE (A professor) (noticed [notice])
Revised: PARSE: nap ( agent = mouse ) DONE (A professor) (noticed [notice])

Original: PARSE: notice ( agent = professor ) DONE (A bee) (nursed [nurse])
Revised: PARSE: notice ( agent = professor ) DONE (A bee) (nursed [nurse])

Original: PARSE: nurse ( agent = bee ) DONE (A dog) (observed [observe])
Revised: PARSE: nurse ( agent = bee ) DONE (A dog) (observed [observe])

Original: PARSE: observe ( agent = dog ) DONE (A frog) (packed [pack])
Revised: PARSE: observe ( agent = dog ) DONE (A frog) (packed [pack])

Original: PARSE: pack ( agent = frog ) DONE (A kid) (painted [paint])
Revised: PARSE: pack ( agent = frog ) DONE (A kid) (painted [paint])

Original: PARSE: paint ( agent = kid ) DONE (A dog) (reddened [redden])
Revised: PARSE: paint ( agent = kid ) DONE (A dog) (reddened [redden])

Original: PARSE: redden ( theme = dog ) DONE (A pig) (rolled [roll])
Revised: PARSE: redden ( theme = dog ) DONE (A pig) (rolled [roll])

Original: PARSE: roll ( theme = pig ) DONE (A queen) (ran [run])
Revised: PARSE: roll ( theme = pig ) DONE (A queen) (ran [run])

Original: PARSE: run ( agent = queen ) DONE (A butterﬂy) (scoffed [scoff])
Revised: PARSE: run ( agent = queen ) DONE (A butterﬂy) (scoffed [scoff])

Original: PARSE: scoff ( agent = butterﬂy ) DONE (A dog) (screamed [scream])
Revised: PARSE: scoff ( agent = butterﬂy ) DONE (A dog) (screamed [scream])

Original: PARSE: scream ( agent = dog ) DONE (A boy) (saw [see])
Revised: PARSE: scream ( agent = dog ) DONE (A boy) (saw [see])

Original: PARSE: see ( agent = boy ) DONE (Julian) (shattered [shatter])
Revised: PARSE: see ( agent = boy ) DONE (Julian) (shattered [shatter])

Original: PARSE: shatter ( theme = Julian ) DONE (A cookie) (shortened [shorten])
Revised: PARSE: shatter ( theme = Julian ) DONE (A cookie) (shortened [shorten])

Original: PARSE: shorten ( theme = cookie ) DONE (A boy) (sketched [sketch])
Revised: PARSE: shorten ( theme = cookie ) DONE (A boy) (sketched [sketch])

Original: PARSE: sketch ( agent = boy ) DONE (A soldier) (slept [sleep])
Revised: PARSE: sketch ( agent = boy ) DONE (A soldier) (slept [sleep])

Original: PARSE: sleep ( agent = soldier ) DONE
Revised: PARSE: sleep ( agent = soldier ) DONE

Original: (A
Revised: (A

Original: [Table]
Revised: [Table]

Original: ) (slid
Revised: ) (slid

Original: [Table]
Revised: [Table]

Original: PARSE: slide ( theme = raisin ) DONE (A champion) (smiled [smile])
Revised: PARSE: slide ( theme = raisin ) DONE (A champion) (smiled [smile])

Original: PARSE: smile ( agent = champion ) DONE (A chicken)
Revised: PARSE: smile ( agent = champion ) DONE (A chicken)

Original: (smirked [smirk])
Revised: (smirked [smirk])

Original: PARSE: smirk ( agent = chicken ) DONE (A donut) (snapped [snap])
Revised: PARSE: smirk ( agent = chicken ) DONE (A donut) (snapped [snap])

Original: PARSE: snap ( theme = donut ) DONE (A tenant) (sneezed [sneeze])
Revised: PARSE: snap ( theme = donut ) DONE (A tenant) (sneezed [sneeze])

Original: PARSE: sneeze ( agent = tenant ) DONE (A student) (snoozed [snooze])
Revised: PARSE: sneeze ( agent = tenant ) DONE (A student) (snoozed [snooze])

Original: PARSE: snooze ( agent = student ) DONE (A girl) (snored [snore])
Revised: PARSE: snooze ( agent = student ) DONE (A girl) (snored [snore])

Original: PARSE: snore ( agent = girl ) DONE (A boy) (split [split])
Revised: PARSE: snore ( agent = girl ) DONE (A boy) (split [split])