Paper: Skills-in-Context Prompting:  Unlocking Compositionality in Large Language Models (ICLR-2024)

Reviewer comment (Experiments):
The paper is written around the premise that the prompting method unlocks compositionality, but the way the authors use the term is unlike existing literature. Systematic compositionality is the ability to understand and produce novel combinations from known parts, whereas the paper claims that the SKiC prompting method teaches LLMs to generalize to harder problems and to problems requiring innovative compositions of existing knowledge. However, the authors do not investigate at all whether the LLMs tested are trained on the tasks they present.

Author response:
First, in this work, we follow the previous work [1,2,3] and focus on the easy-to-hard generalization. Specifically, the in-distribution/seen tasks here means the testing samples are sampled from the same problem size with what have been shown in the context [3]. For example, we demonstrate examples of 2-digit addition, and then test it over samples that are also from 2-digit addition. In contrast, the out-of-distribution/unseen tasks here are defined to be the harder unseen variants of the problem which are with different complexity compared to examples provided in the context.  For example, the testing samples of 5-digit additions are the harder variant of the problem that are not seen in the context examples (2-digit additions) [3]. And we propose the SKiC prompts to improve such easy-to-hard compositional generalization [3] and complex reasoning tasks compared to previous methods.  In some of the tasks, our SKiC actually achieves near-perfect performances on harder cases, which are not demonstrated in the context. 

In terms of the investigation on whether the LLMs have been pre-trained on some of the tasks, even if some of the tasks could be crawled into the pretraining corpus, we believe that they are mostly general and simple examples (e.g., last letters of 4 or 5 words)  rather than the harder cases that we tested on (e.g., last letters of 12 words). This is also demonstrated in the zero-shot performances on the harder cases: for example, the zero-shot performances of ChatGPT on last-letter, addition, multiplication and dynamic programming are quite low (lower than 50% in most of the cases)). With our SKiC prompting, the easy-to-hard generalization capability is significantly boosted to even near-perfect generalization, while other strong prompting methods such CoT and Least-to-Most cannot do so.

Reviewer comment (Experiments):
The authors call the last-letter concatenation task of concatenating the last letter of 4 words or more "out-of-distribution," but a simple Google search reveals an example for 5 words (https://tutorialspoint.com/print-last-character-of-each-word-in-a-string). This makes it likely that more examples exist in the pre-training data of these models, bringing into question whether the tasks are truly out-of-distribution.

Author response:
Second, we understand and value the reviewer’s feedback on examining the potential leakage of task data into the pretraining corpus of these LLMs. Unfortunately, the pretraining data for most of the LLMs are not publicly available even for the opensource models like LLAMA and LLAMA-2. We are also very eager to perform such analysis if the data are available. Due to such constraints, our best strategy to is try our best effort to ensure the harder testing cases (e.g., last letters of 12 words, 7 digits addition, 5 digits multiplication, finding max subsequence over 8 numbers) are not seen during pretraining. Generally, if a training data has been seen in LLMs, their zero-shot performance would be very high. And if the harder testing cases (e.g., 5 digits multiplication) exhibit zero performance, it is very unlikely that these testing cases have been seen in pretraining data. It might be possible that the easy versions of the problem (e.g., last letter of 2-3 words) are crawled into the pretraining corpus. However, it is very unlikely that the training process has seen the harder versions of the testing cases (e.g., concatenating last letters of 12 words, performing 5 digits multiplication, finding max subsequence over 8 numbers). In fact, compositional generalization allows  the model to see the easy versions of the examples (e.g., last letter concatenation of 2-3 words), as long as they have not seen the harder testing cases (e.g., last letter of 12 words). And all previous methods [1,4,5,6] have been shown to be struggling in solving these harder problems that require the composition of simple problems/basic skills, even though their easy variations might be seen in the pre-training stage. Compared to previous work, our method can drastically improve over that in the easy-to-hard testing cases. Further mitigate such a concern, we add a new experiment result where we test the last letter of 50 and 100 random strings with ChatGPT, which are reported below. It is very unlikely that such data are included in the pretraining data.
| ChatGPT | Last Letter (50 words)  |Last Letter (100 words)  |
|---------|-------|-------|
| COT     | 62.0 |56.0|
| SKiC    |100 |100|

Reviewer comment (Evaluation):
The nature of systematic generalisation requires near-perfect generalisation. For models that achieve less than 50% accuracy on some of the tasks, it cannot be said that they are generalising systematically, because they are not.

Author response:
On the tasks which do not reach near-perfect accuracy (i.e., multiplication), they are mainly constrained by the basic skills and the length of context windows. For example, our error analysis on the multiplication tasks  reveals that most of the errors in SKiC prompting are caused by missing the multi-digit addition capability, which can be incorporated as a basic skill in the prompting context. 

However, it would require an extra context window (over the limit when we perform the 4,5-digit multiplication) to calculate the addition following the adding SKiC prompts. So we did not include them which resulted in 48.5 performances with ChatGPT on 5-digits multiplication. 
 
When equipped with such extra skills, the most recent GPT-4 (128k context window) with SKiC can achieve near-perfect generalization. We further check the error cases and find that they are mainly caused by copying errors (copy wrong numbers from previous steps).

| GPT4      | 2     | 3     | 4    | 5    |
|-----------|-------|-------|------|------|
| Zero-shot | 99.0  | 59.0  | 4.0  | 0    |
| SKiC (without addition)      | 100.0 | 92.0 | 88.0 | 68.0 |
| SKiC (with addition as basic skills)     | 100.0 | 100.0 | 99.0 | 93.0 |

Reviewer comment (Novelty):
Other aspects of the paper suggest that the authors have a different definition of systematic generalisation than most of the literature. For example, they claim that Nye et al. (2021) developed the scratchpad method to unlock compositional generalization capabilities, but there is nothing in that paper to indicate this. Scratchpad prompting was developed to allow intermediate computational steps and improve performance on multi-step computation problems.

Author response:
First of all, we kindly disagree that our work does not contribute to compositional generalization. Of course, we never claim that we have completely solved the compositional generalization problem. Instead, our contribution is developing a prompting strategy that can effectively boost the composition generalization performance. Compositional generalization refers to the ability to generate solutions to complex problems by recombining known foundational skills in a novel way. The tasks considered in this paper are all designed to measure the model capability in this regard, which were commonly used in related literature. Specifically, we follow previous work [1], where models need to use known skills (from in-distribution/easy problems) and generalize to unseen/complex/out-of-distribution problems. On these tasks, our proposed SKiC prompting can greatly boost the easy-to-hard performance compared to the previous state-of-the-art methods. For example, SKiC achieves near-perfect generalization on many tasks (e.g.,  last letters, addition, multiplication and dynamic programming) with huge improvement margin in the much harder testing cases, which was never observed before in earlier LLMs works. For tasks that are still not near-perfect, our improvement margin is still also huge in the regime of harder testing cases (for example, on the multihop QA task with ~15% improvements over [4]). Therefore, given the great performance improvement by SKiC over these compositional benchmarks, we believe our work should be regarded as a solid contribution towards improving easy-to-hard compositional generalization for LLMs.

Reviewer comment (Evaluation):
Without knowing whether LLMs have been trained on the tasks labeled "out-of-distribution," claims of systematic generalisation to unseen tasks cannot be substantiated. When models do not reach near-perfect accuracy, it should not be called systematic generalisation. The paper should be reframed to better reflect what it actually does, which is developing a prompting method empirically shown to be superior to past prompting methods.

Author response:
In terms of the investigation on whether the LLMs have been pre-trained on some of the tasks, even if some of the tasks could be crawled into the pretraining corpus, we believe that they are mostly general and simple examples (e.g., last letters of 4 or 5 words)  rather than the harder cases that we tested on (e.g., last letters of 12 words). This is also demonstrated in the zero-shot performances on the harder cases: for example, the zero-shot performances of ChatGPT on last-letter, addition, multiplication and dynamic programming are quite low (lower than 50% in most of the cases)). With our SKiC prompting, the easy-to-hard generalization capability is significantly boosted to even near-perfect generalization, while other strong prompting methods such CoT and Least-to-Most cannot do so.

Reviewer comment (Writing):
Question: How do you define "out-of-distribution"?

Author response:
First, in this work, we follow the previous work [1,2,3] and focus on the easy-to-hard generalization. Specifically, the in-distribution/seen tasks here means the testing samples are sampled from the same problem size with what have been shown in the context [3]. For example, we demonstrate examples of 2-digit addition, and then test it over samples that are also from 2-digit addition. In contrast, the out-of-distribution/unseen tasks here are defined to be the harder unseen variants of the problem which are with different complexity compared to examples provided in the context.  For example, the testing samples of 5-digit additions are the harder variant of the problem that are not seen in the context examples (2-digit additions) [3]. And we propose the SKiC prompts to improve such easy-to-hard compositional generalization [3] and complex reasoning tasks compared to previous methods.

Reviewer comment (Experiments):
Can this be used for semantic parsing tasks where an LM is prompted to discover skills?

Author response:
## Can this be used for semantic parsing tasks?

Yes, our SKiC could be easily used for semantic parsing tasks. For example, we further design prompts and perform experiments on SCAN dataset [1]. Specifically our skills include <decompose_commands> and <command_parse>:

```

Skill <command_parse>: Parse a simple command into structured output.

For example,

‘Walk’ is parsed to ‘WALK’

‘Look’ is parsed to ‘LOOK’

‘Run’ is parsed to ‘RUN’

‘Jump’ is parsed to ‘JUMP’’

‘turn left’ is parsed to ‘LTURN’’

‘turn right’ is parsed to ‘RTURN’’

‘Turn opposite left’ is parsed to ‘LTURN LTURN’

‘Turn opposite right’ is parsed to ‘RTURN RTURN

  

Skill <decompose_commands>: decompose the complex commands into basic units.

For example,

‘Look right’ can be decomposed to ‘look’ and ‘turn right’

‘Look twice’ can be decomposed to ‘look’ and ‘look’

‘Look right after look twice’ can be decompose to ‘look right’ and ‘look twice’

‘Jump opposite right thrice and walk’ can be decomposed to ‘jump opposite right thrice’, ‘walk’

```

One example of composing the above skills is also shown below

```

Example: run and look twice.

Answer: 1. Using the Skill <decompose_commands>, ‘run and look twice’ can be decomposed to ‘run’, and ‘look twice’.

2. For ‘run’, using the Skill <command_parse>, ‘run’ is parsed to ‘RUN’

3. For ‘look twice’, using the Skill <decompose_commands>, ‘look twice’ can be decomposed to ‘look’ and ‘look’

      i. For ‘look’, using the Skill <command_parse>, ‘look’ is parsed to ‘LOOK’

      ii. For ‘look’, using the Skill <command_parse>, ‘look’ is parsed to ‘LOOK’

4. The output is ‘RUN’+’LOOK’+’LOOK’ = ‘RUN’ + ‘LOOK’*2

```

And the performance with ChatGPT is shown below:

| ChatGPT | SCAN |
|---------|-------|
| COT | 72.5 |
| SKiC | 100.0 |

  

[1] Compositional generalization via neural-symbolic stack machines

Reviewer comment (Experiments):
Can this be applied to general NLU tasks (e.g. textual entailment) by prompting an LM to discover skills?

Author response:
## Can LM discover skills and be applied to general NLU tasks textual entailment?

Yes, LM can also be prompted to discover skills for general tasks.

First, in our second setting, we have shown that the skills can be directly distilled from LLM by prompting ‘what are some basic skills in algebra’, which derive skills shown in the Figure 17.

Additionally, we further provide experiments to show that some of the skills in our first setting can also be summarized by LLMs and our SKiC could also be applied to general NLU settings like NLI tasks.

### Discover skills for last letters.
We provide ChatGPT with the chain-of-thought prompts for last-letter tasks and ask it to summarize basic skill to solve the tasks:
    
```
Summarize the basic skills to solve last-letter tasks from the examples below:


Q: “think, machine”

A: The last letter of “think” is “k”. The last letter of “machine” is “e”. Concatenating “k”, “e” leads to “ke”. So, “think, machine” outputs “ke”.

Q: “learning, reasoning, generalization”

A: The last letter of “learning” is “g”. The last letter of “reasoning” is “g”. The last letter of “generalization” is “n”. Concatenating “g”, “g”, “n” leads to “ggn”. So, “learning, reasoning, generalization” outputs “ggn”.

```

The LLMs (ChatGPT) would further summarize the needed basic skills:

```

To solve last-letter tasks based on the given examples:

-   Identify Words: Recognize the words provided in the task.
    
-   Determine Last Letters: Find the last letter of each identified word.
    
-   Concatenate Last Letters: Combine the last letters in the specified order.
    
-   Form New Sequence: The result is a new sequence created by concatenating the last letters.
    

In the provided examples:

-   For "think, machine": Identify "k" (last letter of "think") and "e" (last letter of "machine"), then concatenate to form "ke."
    
-   For "learning, reasoning, generalization": Identify "g" (last letter of "learning") and "g" (last letter of "reasoning"), and "n" (last letter of "generalization"), then concatenate to form "ggn."
    

```

The discovered skills include ‘Identify Words’, ‘Determine Last Letters’, ‘Concatenate Last Letters’, ‘Form New Sequence’, which are similar to what we have shown in Figure 6. With such skills, we could further construct the SKiC prompts by adding these basic skills in the context and grounding reasoning steps onto these basic skills. This gives the similar performance compared to what we constructed manually.

| ChatGPT | Last Letter (12) |
|---------------------------------|------------------|
| COT | 72.5 |
| SKiC | 100.0 |
| SKiC (Skills discovered by LLM) | 100.0 |

Reviewer comment (Theory):
The paper currently lacks answers to many technical questions related to compositional skill utilization in LMs.

Author response:
### Discover skills for NLI (we use RTE as an example). 
We first randomly sample two examples from RTE training dataset and generate the predictions.
    
```

Q: Please identify whether the premise entails the hypothesis. 
premise: Dana Reeve, the widow of the actor Christopher Reeve, has died of lung cancer at age 44, according to the Christopher Reeve Foundation. 
hypothesis: Christopher Reeve had an accident

ChatGPT: The premise and hypothesis are separate statements, and the premise does not entail the hypothesis. The premise provides information about Dana Reeve, the widow of Christopher Reeve, and her cause of death, which is lung cancer. The hypothesis, on the other hand, is about Christopher Reeve having an accident, and it is not directly related to the information provided in the premise.

  
Q: Please identify whether the premise entails the hypothesis. 
premise: Security forces were on high alert after an election campaign in which more than 1,000 people, including seven election candidates, have been killed. 
hypothesis: Security forces were on high alert after a campaign marred by violence

ChatGPT: Yes, the hypothesis is consistent with the premise. The premise states that security forces were on high alert after an election campaign in which more than 1,000 people, including seven election candidates, have been killed. The hypothesis aligns with this by stating that security forces were on high alert after a campaign marred by violence, which corresponds to the information provided in the premise.

```

Then we prompt ChatGPT to discover the skills:

```
Discover the skills from above examples to do the NLI tasks:

```

Which result in the following output:

```

The basic skills to solve NLI (Natural Language Inference) tasks involve understanding the relationship between the premise and hypothesis statements. Here are the key skills demonstrated in the examples:

-   Context Understanding: Understanding the context and contextually relevant information is key. In both examples, comprehending the context provided in the premises is essential to evaluate the relationship with the hypotheses. This includes understanding details about individuals, events, and the cause-and-effect relationships presented in the text.
    
-   Inference Evaluation: The skill to evaluate whether the hypothesis is supported, contradicted, or unrelated to the information in the premise. In the first example, the inference evaluation involves determining that the premise about Dana Reeve's death does not support the hypothesis about Christopher Reeve's accident. In the second example, it involves recognizing that the premise supports the hypothesis about security forces being on high alert due to campaign violence.
    

These skills collectively enable a reader or a model to make accurate assessments of the logical connections between statements in NLI tasks.

  

```

We could summarize basic skills from the LLMs: Context Understanding, Inference Evaluation. And we can further form the SKiC prompts:

```

Skill <Context Understanding>: Understanding the context and contextually relevant information.

For example,

‘Dana Reeve, the widow of the actor Christopher Reeve, has died of lung cancer at age 44, according to the Christopher Reeve Foundation.’

The sentence provides information about Dana Reeve, the widow of Christopher Reeve, and her cause of death, which is lung cancer.

‘Christopher Reeve had an accident’

The sentence provides information about Christopher Reeve having an accident

  

Skill <Inference Evaluation> The skill to evaluate whether the hypothesis is supported, contradicted, or unrelated to the information.

For example,

Premise: Dana Reeve

Hypothesis: Christopher Reeve having an accident

Using the skill <Inference Evaluation>, the hypothesis is not related to premise.


  

Compose the above skills to solve the NLI tasks.

Example #1:

premise: Dana Reeve, the widow of the actor Christopher Reeve, has died of lung cancer at age 44, according to the Christopher Reeve Foundation.

hypothesis: Christopher Reeve had an accident

Answer: 1. Using the skill <Context Understanding>, The premise provides information about Dana Reeve, the widow of Christopher Reeve, and her cause of death, which is lung cancer.

2. Using the skill <Context Understanding>, the hypothesis is about Christopher Reeve having an accident.

3.  Using the skill <Inference Evaluation>, the information in hypothesis is not directly related to the information provided in the premise. So the answer is Not entail.
    

```

  

The SKiC prompts constructed by utilizing the skills discovered from LLMs could achieve better performances on RTE compared to COT baselines.

| ChatGPT | RTE |
|---------------------------------|------|
| COT | 85.2 |
| SKiC (Skills discovered by LLM) | 89.8 |

With the extra experiments in semantic parsing and discovering skills for lastletter and NLI, we believe that SKiC is generally applicable.

Reviewer comment (Experiments):
It is not clear if this approach is generally applicable.

Author response:
### Discover skills for NLI (we use RTE as an example). 
We first randomly sample two examples from RTE training dataset and generate the predictions.
    
```

Q: Please identify whether the premise entails the hypothesis. 
premise: Dana Reeve, the widow of the actor Christopher Reeve, has died of lung cancer at age 44, according to the Christopher Reeve Foundation. 
hypothesis: Christopher Reeve had an accident

ChatGPT: The premise and hypothesis are separate statements, and the premise does not entail the hypothesis. The premise provides information about Dana Reeve, the widow of Christopher Reeve, and her cause of death, which is lung cancer. The hypothesis, on the other hand, is about Christopher Reeve having an accident, and it is not directly related to the information provided in the premise.

  
Q: Please identify whether the premise entails the hypothesis. 
premise: Security forces were on high alert after an election campaign in which more than 1,000 people, including seven election candidates, have been killed. 
hypothesis: Security forces were on high alert after a campaign marred by violence

ChatGPT: Yes, the hypothesis is consistent with the premise. The premise states that security forces were on high alert after an election campaign in which more than 1,000 people, including seven election candidates, have been killed. The hypothesis aligns with this by stating that security forces were on high alert after a campaign marred by violence, which corresponds to the information provided in the premise.

```

Then we prompt ChatGPT to discover the skills:

```
Discover the skills from above examples to do the NLI tasks:

```

Which result in the following output:

```

The basic skills to solve NLI (Natural Language Inference) tasks involve understanding the relationship between the premise and hypothesis statements. Here are the key skills demonstrated in the examples:

-   Context Understanding: Understanding the context and contextually relevant information is key. In both examples, comprehending the context provided in the premises is essential to evaluate the relationship with the hypotheses. This includes understanding details about individuals, events, and the cause-and-effect relationships presented in the text.
    
-   Inference Evaluation: The skill to evaluate whether the hypothesis is supported, contradicted, or unrelated to the information in the premise. In the first example, the inference evaluation involves determining that the premise about Dana Reeve's death does not support the hypothesis about Christopher Reeve's accident. In the second example, it involves recognizing that the premise supports the hypothesis about security forces being on high alert due to campaign violence.
    

These skills collectively enable a reader or a model to make accurate assessments of the logical connections between statements in NLI tasks.

  

```

We could summarize basic skills from the LLMs: Context Understanding, Inference Evaluation. And we can further form the SKiC prompts:

```

Skill <Context Understanding>: Understanding the context and contextually relevant information.

For example,

‘Dana Reeve, the widow of the actor Christopher Reeve, has died of lung cancer at age 44, according to the Christopher Reeve Foundation.’

The sentence provides information about Dana Reeve, the widow of Christopher Reeve, and her cause of death, which is lung cancer.

‘Christopher Reeve had an accident’

The sentence provides information about Christopher Reeve having an accident

  

Skill <Inference Evaluation> The skill to evaluate whether the hypothesis is supported, contradicted, or unrelated to the information.

For example,

Premise: Dana Reeve

Hypothesis: Christopher Reeve having an accident

Using the skill <Inference Evaluation>, the hypothesis is not related to premise.


  

Compose the above skills to solve the NLI tasks.

Example #1:

premise: Dana Reeve, the widow of the actor Christopher Reeve, has died of lung cancer at age 44, according to the Christopher Reeve Foundation.

hypothesis: Christopher Reeve had an accident

Answer: 1. Using the skill <Context Understanding>, The premise provides information about Dana Reeve, the widow of Christopher Reeve, and her cause of death, which is lung cancer.

2. Using the skill <Context Understanding>, the hypothesis is about Christopher Reeve having an accident.

3.  Using the skill <Inference Evaluation>, the information in hypothesis is not directly related to the information provided in the premise. So the answer is Not entail.
    

```

  

The SKiC prompts constructed by utilizing the skills discovered from LLMs could achieve better performances on RTE compared to COT baselines.

| ChatGPT | RTE |
|---------------------------------|------|
| COT | 85.2 |
| SKiC (Skills discovered by LLM) | 89.8 |

With the extra experiments in semantic parsing and discovering skills for lastletter and NLI, we believe that SKiC is generally applicable.

Reviewer comment (Novelty):
I believe the idea of including skills in context has been proposed in many previous works, which made me unsure about its novelty and whether the contribution of this work is significant. For example, [1] also focuses on the challenge of large number addition/multiplication, and the authors also proposed to include basic skills and their composition in the context. They have also shown significant improvement with in-context skills prompting when there are more digits involved. I'd love to hear from the authors what the novelty of this work is given [1] was proposed and publicly available nearly one year ago.

Author response:
Thanks for your insightful comments. We humbly think that some concerns might be caused by misunderstanding, which we will clarify in detail below. We hope that our response can clarify the misunderstandings and you can consider our work more favorably.

## On the difference from previous work “Teaching Algorithmic Reasoning via In-context Learning”

The major goal in [1] is to design algorithm-style prompts to first teach basic math skills like adding two numbers and then decompose hard problems into linear subproblems which could be solved by the usage of previously taught math operations such as multiplication through adding. This is actually similar to [2] which has been discussed in our Section 2.1.

Our work is different and better than [1] in several aspects:

(i) First, beyond algorithmic operation, we emphasize the explicit grounding of general basic skills and teaching models to compose multiple basic and grounded skills to improve the easy-to-hard generalization and the performances on complex tasks. For example, our SKiC for adding two number teach models to compose extract_digits skills, list_length skills, add_two_single_digit_number skills, sub_10 skills and compare_10 skills compare to the algorithmic style prompt in [1] and achieve better performances on adding tasks with ChatGPT as shown below:

| Add | 2 | 3 | 4 | 5 | 6 | 7 |
|---------------|-----|-----|------|------|------|------|
| COT | 100 | 100 | 98.5 | 90.0 | 87.5 | 80.0 |
| Algorithm [1] | 100 | 100 | 98.0 | 94.5 | 91.5 | 90.0 |
| SKiC | 100 | 100 | 99.0 | 98.0 | 99.0 | 98.5 |

  

(ii) Second, not all the problems can be decomposed linearly and follow [1] to solve. Even though [1] formalize multiplication as a composition of multiple adds (e,g, 11*3 = (11+11) + 11), it is impossible and slow to call multiple adds when it comes to larger numbers like 128*128 (doing 128 additions is infeasible in terms of error propagation and computation cost (significantly longer context window or significant amount of extra calls of basic skills). Instead, our SKiC encourages models to perform multi-digits multiplication by composing single-digit multiplication skill and adding skill in only one stage, which also achieves significantly better performance with one single calls of the foundation model (e.g., ChatGPT) as shown below (when dealing with multiplication between two 3/4/5 digits number, linearly decomposition like [1] would result in exceeding context length.):

| Mul | 2 | 3 | 4 | 5 |
|---------------|-----|------|------|------|
| COT | 99 | 54.5 | 13.0 | 2.0 |
| Algorithm [1] | 100 | 68.0 | 20.0 | 0 |
| SKiC | 100 | 82.0 | 72.0 | 48.5 |

(iii) Third, our SKiC also encourages LLMs to utilize their internal skills beyond what have been specified in the context to solve complex math reasoning tasks like GSM8K and MATH, which can not be achieved by [1] which are limited to the usage of provided skills. Our proposed SKiC actually enables LLMs to utilize a wider range of internal skills to solve problems as shown in Figure 27, 28, 29, 30, 31, 32, 33 via automatically generating skills from LLM itself and grounding to both mentioned skills and unmentioned skills in examples. The overall performance on GSM8K with ChatGPT also demonstrated the powerfulness of our SKiC compared to algorithm calls in [1].

| ChatGPT | COT | Algo [1] | SKiC |
|---------|------|----------|------|
| GSM8K-addition-only SubSet | 78.8 | 84.8 | 90.5 |
| GSM8K | 74.4 | 79.5 | 87.2 |

  
  
  

[1] Teaching Algorithmic Reasoning via In-context Learning

[2] Decomposed Prompting: A MODULAR APPROACH FOR SOLVING COMPLEX TASKS

Reviewer comment (Novelty):
Many recent works have found using tools can significantly improve performance in math reasoning tasks (e.g., GSM8k and MATH), such as PAL [2] and Program-of-Thought [3], and they have shown that tool-using skills (rather than natural or symbolic language skills) can bring more gain to math reasoning tasks. I understand the main focus of this work is not about tool-using, but the method proposed here is a bit less appealing to me considering the current SoTA can reach 80+ (see [4]), especially when considering [1] already discussed the tool-using case.

Author response:
## On the comparison to tool-using work

The major contribution of our work is to understand and unlock the inherent composition abilities (easy-to-hard generalization) in LLMs themselves. The line of tool-using work is actually complementary with our proposed work and can be easily integrated to substitute several basic skills to further improve the performances.

First, external tools like programs might bring in extra latency during inferences as LLMs need to call multiple external functions when dealing with complex problems. As a result, we mainly focus on whether LLM itself could solve tasks in one-stage through our SKiC.

Second, it is hard/impossible to enumerate all the needed external skills (external calls) in the context for complex tasks, which would lower down the generalization abilities if the models are taught to rely on provided external calls. So, our SKiC also encourages models to utilize their internal skills which are not provided in the context to solve complex tasks.

What’s more, tool-using cases are more focused on math-related reasonings. Not all the tasks can be improved by external tools (e.g., QA in our Table 3). Our SKiC is more general to different types of tasks, and the tool-use is actually one basic skill that could be integrated into SKiC. We leave this as a future work.

In terms for the performance on the MATH dataset, 80+ achieved in [1] are mainly due to the usage of GPT4-code-interpreter (a stronger backbone model api which could achieve 69.69 by itself compared to 42.20 from GPT4 and 31.1 from ChatGPT which are our backbone model) and majority voting (majority voting over 16, 73.54 without majority voting). We utilize the code-based-verification in [1] without majority voting on MATH with ChatGPT and it actually achieves worse performances compared to our SKiC:

| ChatGPT | COT | CSV [1] | SKiC |
|---------|------|---------|------|
| MATH | 31.1 | 34.7 | 40.6 |

[1] Solving challenging math word problems using gpt-4 code interpreter with code-based self-verification

Reviewer comment (Experiments):
My main concern is that the skills used in the SKiC are somehow mined from the test set (e.g., you create basic addition/multiplication skills and the compositional examples for GSM8k because you pre-know it is a math reasoning task and preview some of its questions), which might not be the case in practice. This raises the question of how the framework's performance would be affected if the skills were taken from a different set of tasks. For example, will I see any improvement in BBH object counting or penguins on a table task (both are about numeric reasoning) with the SKiC skills you designed for GSM8k? In other words, I guess there is a pre-condition of the success of SKiC, which you can somehow preview the test set of the target tasks, and then you can prepare the needed skills beforehand (either from human annotation or from LLM prompting), which I don't believe is the true definition of "solving unseen tasks".

Author response:
## On the designed skills

In our work, we come up with two ways to construct the skills in SKiC. And in both ways, we are not using any of the testing data to mine the skills: (i) In the first way where we construct the skill set manually, for last letters, addition, multiplication, dynamic programming, we randomly generating 2 cases and summarize the potential skills, and for QA and GSM8K, we randomly sample 4 cases from their training set to summarize the skills. (ii) In the second way where we prompt LLMs to generate skills, we did not utilize any training or testing samples. We only prompt the model to generate the skills (e.g. prompting the LLM with “what are the basic skills in algebra?”).

Dating back to GPT2 and GPT3, the knowledge/preview of the target tasks is also the essential component to design few-shot prompts in all the previous work [1,2,3,4,5], where all the examples in designed prompts are actually task-specific. Additionally, the number of examples we used are also limited (2 to 4 examples).

We still observe performances improvements even if the skills in the context are taken from different tasks because our SKiC encourage the models to utilize and compose skills beyond what are included in the context: We use the skills and prompts designed for GSM8K and directly apply them on MATH (math reasoning), FOLIO (logical inference) which are unseen tasks as shown below.

| ChatGPT | COT for GSM8K | SKiC for GSM8K |
|---------|---------------|----------------|
| MATH | 28.2 | 31.34 |
| FOLIO | 68.8 | 72.5 |

  

[1] Teaching Algorithmic Reasoning via In-context Learning

[2] Decomposed Prompting: A MODULAR APPROACH FOR SOLVING COMPLEX TASKS

[3] Solving challenging math word problems using gpt-4 code interpreter with code-based self-verification

[4] Least-to-Most Prompting Enables Complex Reasoning in Large Language Models

[5] Complexity-based prompting for multi-step reasoning

Reviewer comment (Experiments):
While the paper does not explicitly address this issue, it is possible that the performance of the framework could be affected if the skills were taken from a different set of tasks. This is because the skills used in the framework are specific to the domain of the target task test set, and may not be applicable to other domains. For example, if the skills were taken from the Last Letter Concatenation and tested on the GSM8K/MATH tasks, I don't think SKiC can lead to optimal performance.

Author response:
## On the definition of ‘solving unseen tasks’

To make clarifications, the ‘solving unseen tasks’ in our work refers to easy-to-hard generalization following the convention used in previous works [1,2,3], where models see the simple examples but need to generalize to harder examples for the same tasks (e.g., seeing examples of 2-digits adding but inferring how to add 5 digits numbers).

However, in our extra experiments, we also show that our SKiC which teach the model how to compose skills can also help the performances even if the provided prompts are designed for different tasks: We use the skills and prompts designed for GSM8K and directly apply them on MATH (math reasoning), FOLIO (logical inference) which are unseen tasks as shown below.

| ChatGPT | COT for GSM8K | SKiC for GSM8K |
|---------|---------------|----------------|
| MATH | 28.2 | 31.34 |
| FOLIO | 68.8 | 72.5 |

  
  
  

[1]Decomposed Prompting: A MODULAR APPROACH FOR SOLVING COMPLEX TASKS

[2]Least-to-Most Prompting Enables Complex Reasoning in Large Language Models

[3]  Faith and Fate: Limits of Transformers on Compositionality

Reviewer comment (Experiments):
How did you pick the two/N examples in your prompts? I believe the performance will be heavily affected by the choice of these examples. Did you run experiments or have a discussion on their impact?

Author response:
## On the questions about how to pick two/N examples in prompts

We randomly pick the examples. For instance, the examples for the last-letter task are randomly sampled (we randomly set the words in the example like the last letter of ‘apple, pie’). The performance improvements are consistent even if we perturb the examples in the prompts. The results on last-letter tasks with ChatGPT with the use of different choices of few-shot examplars in the prompts are shown below. It shows the robustness of our proposed SKiC prompt to the selection of the few-shot exemplars.

| ChatGPT | COT | SKiC |
|-----------------------------------|------|-------|
| 'apple, banana'; 'apple, pie' | 91.4 | 100.0 |
| 'math, code'; 'science, computer' | 92.5 | 100.0 |
| 'ashc, edhoh'; 'shbod, wojois' | 90.8 | 100.0 |

  

## On the order of the examples

  

The order of the examples also does not matter a lot because we randomly sample a limited number of examples (2 examples in most of the cases) to design SKiC. We shuffle the order in our prompts (consisting of 4 examples) for GSM8K and the performances are shown below:

  

| ChatGPT | COT | SKiC |
|---------|------|------|
| order 1 | 74.4 | 87.2 |
| order 2 | 73.8 | 86.9 |
| order 3 | 73.0 | 87.8 |

Reviewer comment (Experiments):
Does the order of these examples matter? Telling from [1] I think the order is another factor that will affect the performance, but the work does not discuss it.

Author response:
## On the questions about how to pick two/N examples in prompts

We randomly pick the examples. For instance, the examples for the last-letter task are randomly sampled (we randomly set the words in the example like the last letter of ‘apple, pie’). The performance improvements are consistent even if we perturb the examples in the prompts. The results on last-letter tasks with ChatGPT with the use of different choices of few-shot examplars in the prompts are shown below. It shows the robustness of our proposed SKiC prompt to the selection of the few-shot exemplars.

| ChatGPT | COT | SKiC |
|-----------------------------------|------|-------|
| 'apple, banana'; 'apple, pie' | 91.4 | 100.0 |
| 'math, code'; 'science, computer' | 92.5 | 100.0 |
| 'ashc, edhoh'; 'shbod, wojois' | 90.8 | 100.0 |

  

## On the order of the examples

  

The order of the examples also does not matter a lot because we randomly sample a limited number of examples (2 examples in most of the cases) to design SKiC. We shuffle the order in our prompts (consisting of 4 examples) for GSM8K and the performances are shown below:

  

| ChatGPT | COT | SKiC |
|---------|------|------|
| order 1 | 74.4 | 87.2 |
| order 2 | 73.8 | 86.9 |
| order 3 | 73.0 | 87.8 |

Reviewer comment (Presentation):
Tables 1 and 2 could be better organized in the page.

Author response:
### The organization of Table 1 and 2

We will better distribute Table 1 and 2 in the revised version.

Reviewer comment (Experiments):
In the beyond in-context skills setting, does the LLM use skills that are not specified in Figure 16?

Author response:
## On the question about the use of skills in the beyond in-context skills setting

  

Yes, in the beyond in-context skills setting, our methods would encourage models to use skills in their pre-trained knowledge that are not provided in the provided context.

  

For GSM8K, Figures 27 and 28 show the cases where models utilize skills that are not specified in Figure 16 (e.g., “round” and “compare”)

  

For MATH, Figure 29, 30, 31, 32, 33 show the examples where models utilize skills that are not specified in context (Figure 17). Also, Table 12 in the Appendix shows the most frequent skills that are from Figure 16 and the most frequent skills that are not specified in Figure 17.

Reviewer comment (Reproducibility):
Is it possible to replace some of these skills with function calls to Python, for example, to decrease computation errors?

Author response:
## The possibility to replace some of the skills with function calls

  

Yes, it is possible to replace several skills with function calls like python which could be used to decrease some computation errors. This is complementary to our proposed methods. But in this work, the major goal is to explore the inherent capabilities of the language model itself. That is, we would like to examine and unlock the inherent abilities of LLMs to compose different skills for complex tasks.