Complex Few-shot Reasoning
with LLMs
Aman Madaan
https://madaan.github.io/

Part 5/7 of the ACL 2023 Tutorial
Complex Reasoning in Natural Language
1Questions @ Rocket Chat

Complex Few-shot Reasoning withLLMs
Goal Today:● Try to cover the details of all possible exciting recent works● Provide a high-level summary of a class of few-shot
prompting techniques
I will deﬁnitely miss interesting works:● Please reach out! We will add it to our website
2

Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration

Instructions

Tool Augmentation

Structured GenerationFeedback

Memory3

Quick Detour: Few-shot Prompting
Q: Shawn has 5 toys. For Christmas, he got 2 toys each from his momand dad. How many toys does he have now?

A: Theanswer is 9toys

4Fine-tuning
Q: If there are 3 cars in the parking lot
and 2 more cars arrive, how many cars are in
the parking lot?

θA: The answer is 5cars.

θ

A: The answer is 39pieces.

Q: Leah had 32 chocolates and her sister had
42. If they ate 35, how many pieces do they
have left in total?

Train/Fine-tune

Q: Shawn has five toys. For Christmas, hegot two toys each from his mom and dad. How
many toys does he have now?

θ

A: The answer is 9 toys

Test5Few-shot prompting
Q: If there are 3 cars in the parking lot and 2
more cars arrive, how many cars are in the parking
lot?
A: The answer is 5 cars.

PromptLLM
The answer is 9 toys

Q: Shawn has five toys. For Christmas, he got two toys eachfrom his mom and dad. How many toys does he have now?
A:Test Example

6Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration

Instructions

Tool Augmentation

Structured Generation

Feedback

Memory7Complex Few-shot Reasoning with LLMs: Key Techniques
HelpsGenerate

Instructions
Reasoning Elaboration

Tool Augmentation

Requires
Techniques overlap in practice – focus on
the main contribution

Structured Generation

Stored inFeedback

Helps Generate

Memory8Rest of the slides●General idea of the technique●Representative Work●

Goal/Hope:○

A checklist of techniques for complex reasoning with LLMs

Progress Bar
9Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration

Instructions

Tool Augmentation

Structured Generation

Feedback

Memory10Few-shot prompting
Q: If there are 3 cars in the parking lot and 2
more cars arrive, how many cars are in the parking
lot?
A: The answer is 5 cars.

Prompt

LLM
The answer is 9 toys

Q: Shawn has five toys. For Christmas, he got two toys eachfrom his mom and dad. How many toys does he have now?
A:

Test Example

11

Few-shot prompting
Q: If there are 3 cars in the parking lot and 2 morecars arrive, how many cars are in the parking lot?
DirectPrompt

A: The answer is 5 cars.

12

Chain-of-thought Prompting (Wei et al. 2022)
Q: If there are 3 cars in the parking lot and 2 morecars arrive, how many cars are in the parking lot?
DirectPrompt

A: The answer is 5 cars.
Q: If there are 3 cars in the parking lot and 2 morecars arrive, how many cars are in the parking lot?
Chain-of-Thought
PromptThought (T): There are originally 3 cars. 2 more cars
arrive. 3 + 2 = 5.
A: The answer is 5 cars.
13

Chain-of-thought prompting is extremely effective

14

Chain-of-thought Prompting (CoT)●General idea:○○

Standard prompt:■Q→A
Chain-of-thought prompt:■
Q → Reasoning Process, A●Many existing prompting techniques can be seen as
an improvement over the general CoT strategy●Similar ideas:○

Program Induction by Rationale Generation: Learning toSolve and Explain Algebraic Word Problems (Ling et al.
2017)

○

Think about it! Improving defeasible reasoning by firstmodeling the question scenario (Madaan et al. 2021).

○

Show your work: Scratchpads for Intermediate Computation
with Language Models (Nye et al. 2021)

Q: If there are 3 cars in the parking lot and2 more cars arrive, how many cars are in theparking lot?
Thought (T): There are originally 3 cars. 2
more cars arrive. 3 + 2 = 5.
A: The answer is 5 cars.

15

Least-to-Most Prompting (Zhou et al. 2022)●

Breakdown the reasoning process into steps○

Decompose the problem into simpler sub-problems○

Solve simpler sub-problems

16

Least-to-Most Prompting
Question

Q: If there are 3 cars in the parking lot and 2 more cars arrive, howmany cars are in the parking lot?

Question
Decomposition

Q: How many cars are in the parking lot after the first car arrives?
Q: How many cars are in the parking lot after the second car arrives?

Q: How many cars are in the parking lot after the firstcar arrives?
A: There are originally 3 cars. After the first car
Subproblem
Solvingarrives, we have 3 + 1 = 4 cars.
Q: How many cars are in the parking lot after thesecond car arrives?
A: After the first car arrives, we have 3 + 1 = 4 cars.
After the second car arrives, we have 4 + 1 = 5 cars.
17

Least-to-Most Prompting

When do you need more steps? Complexity-Based Prompting forMulti-Step Reasoning, Fu et al. 2023
Decomposed Prompting: A Modular Approach for Solving Complex Tasks

18

Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration: Spell out
the reasoning process before
generating the answer

Instructions

Tool Augmentation

Structured Generation

Feedback

Memory19Instructions●●

Many ways to communicate the same intent:●

Write a summary of the history of computer science●

Summarize the history of computer science, highlighting key developments.

●

Provide a summary of the history of computer science in this format:❖ Inventors and innovations❖ Evolution of programming languages❖ Impact of internet●Summarize the history of computer science in no more than 200 words, keeping the language
simple and easy to understandDifferent levels of abstract, details, requirements

20Reframing Instructional Prompts to GPTk’s Language (Mishra et al., 2022)

21

Instructions

Mishra, Swaroop, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. "Reframing
Instructional Prompts to GPTk's Language." ACL 2022 Findings.

22

Instructions●Explicit is better than implicit●Be aware of instances whereyou might be expecting themodel to read your mind!

Mishra, Swaroop, Daniel Khashabi, Chitta Baral, Yejin Choi,
and Hannaneh Hajishirzi. "Reframing Instructional Prompts
to GPTk's Language." ACL 2022 Findings.

23

Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration: Spell out
the reasoning process before
generating the answer

Instructions: Provide explicit
instructions to the LLM, capture all
expectations

Tool Augmentation

Structured Generation

Feedback

Memory24

Augmenting LLMs with Tools●Typical language generation setup: LLMs handle everything○ There are originally 3 cars. 2 more cars arrive. 3 + 2 = 5.
■■

What has to be done?
●
Add 3 + 2
How to do it?
●
Perform the calculation and produce the answer = 5●

Leverage external tools to separate 'what' needs to be done from 'how' to do it.

●

Calculator as a tool (T):
○
○

○
○
○

There are originally 3 cars.
2 more cars arrive.
Number of cars after the first car comes = calculator(3, 1) = 4.
Number of cars after the second car comes = calculator(4, 1) = 5.
The answer is 5.

25

React (Yao et al. 2022)

26

React (Yao et al. 2022)

27

Augmenting LLMs with Tools●Obvious use cases
○
○
○

●

Real-time information: Stock market updates, Temperature monitoring, Traffic reports.
Specialization: Solving Ordinary Differential Equations (ODEs)
Multimodal: Performing clicks, Generating image captions.

Future Directions
○
○
○
○
○

Combine multiple tools within the same language model framework for increased functionality
and adaptability.
[2304.09842] Chameleon: Plug-and-Play Compositional Reasoning with Large Language
Models[2304.08354] Tool Learning with Foundation Models[2302.04761] Toolformer: Language Models Can Teach Themselves to Use Tools[2303.09014] ART: Automatic multi-step reasoning and tool-use for large language models

28Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration: Spell out
the reasoning process before
generating the answer

Instructions Provide explicit
instructions to the LLM, capture all
expectations

Structured Generation

Feedback

Tool Augmentation: Enhance
LLMs for specialized tasks

Memory29

Structured Commonsense Reasoning● Natural language input (e.g., scenario)● Structured output (e.g., plan graph, reasoning graph)
Find recipe

Gather Ingredients

Bake a cake

Preheat oven at 375 F

Mix ingredients

Put the cake batter into oven

Take the cake out after 30 min
https://proscript.allenai.org/30

Leveraging Language Models for Structured commonsense Reasoning● Need to generate a graph but … language models can only generate strings● Workaround○ Flatten the graph as a string○ Train a seq2seq model

Goal: Bake a cake"ﬁnd recipe" -> "gather ingredients”;"gather ingredients" -> "mix ingredients”;"gather ingredients" -> "preheat oven at 375F”;"ﬁnd recipe" -> "preheat oven at 375F”;"preheat oven at 375F" -> "put the cake batter into oven";"mix ingredients" -> "put the cake batter into oven";"put the cake batter into oven" -> "take the cake out after 30
min"Intermediate Representation

Recovered Graph31

Leveraging Language Models for Structured commonsense Reasoning● Issues with the workaround"find recipe" → "gather ingredients";“gather ingredients" → "mix ingredients”;"find recipe" → "preheat oven at 375F”;"preheat oven at 375F" → "put the cake batter into oven";"mix ingredients" → "put the cake batter into oven";"put the cake batter into oven" → "take the cake out after 30
min"

Are the two
mix ingredients the same?

● We want structures, not strings
32

Language Models of Code Are Few-shot Commonsense Learners (Madaan et al. 2022)

33

Language Models of Code Are Few-shot Commonsense Learners (Madaan et al. 2022)

Goal: Bakea cake

Recovered Graph

Intermediate Representation
34

Script Generation Results on ProScript

35

PAL: Program-aided Language Models (Gao*, Madaan*, Zhou* et al. 2023)● Olivia has $23. She bought five bagels for $3 each. How much money does she haveleft?
def solution():money_initial = 23

Olivia had 23 dollars. 5 bagels for 3
dollars each will be dollars. So she hasdollars left.

bagels = 5
bagel_cost = 3
money_spent = bagels * bagel_cost
money_left = money_initial - money_spent
result = money_left
return result

CoTPaL

Comparison with CoT:●The language model is responsible for generating a high-level plan that is●The results are obtained after running the program

executed to derive the answer
36

Improves Solve Rate for Multiple Maths Reasoning Tasks

Why should code help?

38

Structured Generation

More on future directions

39

Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration: Spell out
the reasoning process before
generating the answer

Instructions Provide explicit
instructions to the LLM, capture all
expectations

Structured Generation: Use structure
to guide model

Feedback

Tool Augmentation: Enhance
LLMs for specialized tasks

Memory40

Feedback●Humans typically generate content iteratively:○●

Generate → Feedback → Refine

LLMs, with the right prompt, have recently shown promise at mimicking thisprocess○○○

Generate an output
Gather feedback
Improve output with feedback, repeat41

Self-Refine (Madaan et al. 2023)

GPT-3.5/4 is good!
But often requires afew rounds of talking
through the problem

42

Let LLM Self-Reﬁne
Generate Acronym for: Politeness transfer: Atag and generate approach

POLTAG
GENERATE

PTATAGAREFINEFEEDBACKFEEDBACK
Relevance of PTATAGA: The acronym includes the ﬁrst letters of eachmajor word in the title but doesn't fully capture the meaning of the title.
<3/5>
Memorability of PTATAGA: The acronym is not the most memorable.
<2/5>
Pronunciation of PTATAGA: The acronym is pronounceable but notvery ﬂuid. <3/5>

Relevance of POLTAG: A tag and generate approach:
The acronym POLTAG is a good representation of the title
as it combines the ﬁrst few letters of Politeness and Tag.
<4/5>
Memorability of POLTAG: The acronym POLTAG is easy
to remember as it is short, simple, and has a clear
structure. <4/5>

STOP

Pronunciation of POLTAG: The acronym is easilypronounced as it follows the common structure ofacronyms. <5/5>43

Self-Reﬁne Results●●Larger gains where feedback is obvious + easy
Lots of room for improvement!
44

Future Directions: Feedback + Planning● Monte-Carlo Tree Search +Self-Reﬁne○ Extend Self-Reﬁne Beyond
Linear Search● Other exciting work:○

Language Models can Solve Computer
Tasks (Kim et al. 2023)○

Tree-of-thought (Yao et al. 2023)○

Reﬂexion (Shinn et al. 2023)○Reﬁner (Paul et al. 2023)
45

Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration: Spell out
the reasoning process before
generating the answer

Instructions Provide explicit
instructions to the LLM, capture allexpectations

Structured Generation: Use structure
to guide model

Feedback: Refine model outputs
during inference

Tool Augmentation: Enhance
LLMs for specialized tasks

Memory46

Models Repeat Mistakes

GPT-NBoulder

What is like Bolder?

GPT-NKnew
What is like new?

By like I mean synonym!

47

Memory●●Standard few-shot prompting setup○○Prompt P: [Xi, Yi]
Test examples X1, X2■ P + X1 → Y’1■ P + X2 → Y’2○

What if user provides a feedback on X1 → Y’1 ?

○Can we improve the output generated for X2 without re-training?

Solution: maintain a memory of examples seen so far, and any feedback○Update the prompt P dynamically○Stateful inferences

48

MemPrompt

49

MemPrompt: Workﬂow● Step 1: User asks a question● Step 2: Check if the same question has been asked
before, and a clarification is present in memory○ Step 2.1: If a clarification is present, add question +clarification to the prompt○ Step 2.2: If not, just ask a question● Step 3: Model generates an answer● Step 4: Take clarification on answer if needed, add
clariﬁcation to memory

50

MemPrompt: Personalization
Queries in Punjabi

51

Memory for Few-shot Prompting●Simpler but effective variants○What Makes Good In-Context Examples for GPT-3? (Liu et al. 2021): Store the training set in a database,retrieve most relevant examples on the fly

52Complex Few-shot Reasoning with LLMs: Key Techniques

Reasoning Elaboration: Spell out
the reasoning process before
generating the answer

Structured Generation: Use structure
to guide model

Instructions: Provide explicit
instructions to the LLM, don’t
expect mind reading

Tool Augmentation: Blend tools
to leverage LLMs for specializedtasks

Feedback: Refine model outputs
during inference

Memory: Maintain a history ofinteractions with LLM

Future Directions● LLMs are getting better at following
instructions, exciting new possibilities●○

Planning, Search – resurgence of classicalAI techniques○

Generating actionable feedback from tools

Appendix: Why does few-shot
prompting work?

54Complex Few-shot Reasoning with LLMs: Key Techniques

Questions @ Rocket Chat

Appendix

56

But Why Does it Work?

Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangohelps

TePr shelp

Paper57

What makes the chain of thought prompting so effective?
●

The thought makes the model think about the problem?

●

The thought helps the model learn better●The thought serves as an additional example of the task●

The thought helps the model remind of the task●

The thought helps extract relevant information for solving the task

58

What makes chain of thought prompting so effective
Text: Extracting common sense, copy mechanism

TeSbsPr s

Patterns: Task understanding, final answer generation

59

Approach
Q: If there are 3 cars in the parking lot and 2 more cars arrive, how manycars are in the parking lot?
T: There are originally 3 cars. 2 more cars arrive. 3 + 2 = 5.

A: The answer is 5 cars.

●Symbols

Patterns

Text

Counterfactual prompting:■

Change one knob at a time (symbol, patterns, text)60

What if? prompting (counterfactual prompting)
Q: If there are α cars in the parking lot
and β more cars arrive, how many cars are

What if we don’t
have actualnumbers?
Q: If there are 3 cars inin the parking lot?
Thought (T): There are originally α cars. βmore cars arrive. α + β = λ.

the parking lot and 2 morecars arrive, how many cars
A: The answer is λ cars.

are in the parking lot?
Thought (T): There are

Q: If there are 3 cars in the parking lot

originally 3 cars. 2 moreand 2 more cars arrive, how many cars arecars arrive. 3 + 2 = 5.

in the parking lot?

A: The answer is 5 cars.

What if the prompt
is misleading?

Thought (T): There are originally 3 cars.
2 more cars arrive. 3 + 2 = 7.

A: The answer is 5 cars.

61Performance with Counterfactual Prompts

Performance does not change!
62

Other Tasks

63

The Search Engine Analogy

QueryDatabase

Query IntentResults

Query ≅ Prompt

Query Intent ≅ Task Understanding

Database ≅ Weights

Also see: https://ai.stanford.edu/blog/understanding-incontext/

64