Noisy Exemplars Make Large Language Models More Robust: A Domain-Agnostic Behavioral Analysis
Authors: Hongyi Zheng
Abulhair Saparov
Venue: EMNLP 2023 Main (2023)
Keywords: Large Language Models
Robustness
Perturbation Analysis
Few-shot Prompting
Chain-of-thought

Abstract: Recent advances in prompt engineering enable large language models (LLMs) to solve multi-hop logical reasoning problems with impressive accuracy. However, there is little existing work investigating the robustness of LLMs with few-shot prompting techniques. Therefore, we introduce a systematic approach to test the robustness of LLMs in multi-hop reasoning tasks via domain-agnostic perturbations. We include perturbations at multiple levels of abstractions (e.g. lexical perturbations such as typos, and semantic perturbations such as the inclusion of intermediate reasoning steps in the questions) to conduct behavioral analysis on the LLMs. Throughout our experiments, we find that models are more sensitive to certain perturbations such as replacing words with their synonyms. We also demonstrate that increasing the proportion of perturbed exemplars in the prompts improves the robustness of few-shot prompting methods.

Noisy Exemplars Make Large Language Models More Robust: A Domain-Agnosti Behavioral Analysis
Hongyi Zheng
Abulhair Saparov New York University as #s17582@nyu. edu
New York University hz221 20222..uu.... edu
Abstract
Original
Natalla clips April. Then sold many cloo May. many clips sell ahogether??
Recent advances in prompt engineering enable large language models (LLMs) to solve multi- hop logical reasoning problems with impres- sive accuracy. However, there is little existing work investigating the robustness of LLMs with few-shot prompting techniques. Therefore, introduce systematic approach to test ro- bustness of LLMs in multi-hop reasoning tasks via domain- domain-agnostic agnostic perturbations. We include perturbations at multiple levels of abstractions (e.g. lexical perturbations such as typos, and semantic perturbations such as the inclusion of intermediate reasoning steps in the questions) to conduct behavioral analysis on the LLMs. Throughout our experiments, els are more sensitive to certain perturbations such as replacing words with their synonyms. We also demonstrate that increasing the pro- portion of perturbed exemplars in the prompts improves the robustness of few-shot prompting methods.
clips altogether
Perturoed
Natalia sold cips April. Then she sold half as many clos May. Then she sold many clips in May. How many
She sold 48 2 2 clpps
May, so she sell 48 clipa alttogither
Natalia
x
altogethar?
Figure I: A simple repetition may result in LLM producing incorrect solution.
very small amount of common lexical perturbations such as word replacement and deletion could result in drastic change in machine translation results.
find that mod-
There are few existing studies investigating the robustness of these prompting schemes through be- havioral experiments on perturbed examples. Thus in research, we create selection of domain- agnostic tests to investigate the robustness of state- of-the-art prompting methods. Our two main goals are: (1) to compare and contrast the performance of prompting methods with respect to various per- turbations, and (2) to explore empirical approaches that may improve their robustness.!
1 Introduction
Large language models (LLMs) achieve human- like performance on many natural language pro- cessing tasks after few-shot learning due increas- ing scale (Kaplan al., 2020). However, they often struggle in conducting multi-hop reasoning tasks after standard prompting (Rae et al., 2021). Re- cently, multiple prompt engineering methods such as thain-offthouglee prompting (Wei et al., 2023), zero-shot prompting (Kojima et al., 2023) and least- -most-prompping (Zhou et al., 2023) have led to significant empirical improvements in these tasks. Despite these signs of progress, there is an impor- tant drawback in recent studies: the datasets used in these experiments are often idcalized, noise-free, and rather distinct from examples that LLMs en- counter in real applications, which put the general- izability of these prompting methods into question since applications of LLMs in practice are often noisy, containing errors, redundant or irrelevant sentences, utilizing out-of-distributiio vocabulary, etc. For instance, Cheng et al. (2018) show that
Related Work
2.1 Prompt Engineering
Contemporary prompt engineering methods that aim to improve LLMs' reasoning performance stem from Sfain-of-thought prompting (CoT) proposed by Wei et al. (2023), which draws inspiration from the earlier work of Ling et al. (2017) with the key idea of augmenting standard few-shot prompting with a chain-of-thought (i.e. a description the reasoning steps that lead to the answer). CoT im- proves LLM's performance in a wide range of rea- soning tasks. Nevertheless, recent research shows that it may lead inconsistent reasoning steps
'Our code is open source and available at https://gith ub. cwm/Hiroki39/////ocoocco sxs.-yeepparrr-Maaeee.... ageeModels- Mooodees-More--- Robust

---

Original
Typo
Synonym
Repetition
Shortcut
Elsa has 5 cars. Anna has 2 more cars than Elsa. How many cars do they have together?
Elsa has 5 cars. Anna has 2 more cras than Elsa. How many cars do thye have together?
Elsa has 5 cars. Anna has 2 more vehicles than Elsa. How many automobiles do they have together?
Elsa has cars. Anna has 2 more cars than Elsa. Elsa has 5 cars. How many cars do they have together?
Elsa has 5 cars. Anna
has more cars than
Elsa. Anna has 7 cars.
How many cars do they have together?
Figure 2: Examples for each type of perturbation test. A more detailed example shown in Table 1.
and thus worse performance under certain circum- stances (Ye and Durrett, 2022a), which highlights the necessity to conduct further behavioral analy- ses to identify such circumstances and find ways to mitigate this issue.
reasoning tasks (Ye and Durrett, 2022b). Some tests investigate LLM consistency via replacing real concept nouns with fictional ones (Saparov and He, 2023) and adding semantically related yet logically irrelevant distractor sentences (Saparov et al., 2023). While those studies mainly fo- cus on domain-specific perturbations, or on syn- thetic settings, our work evaluates the prompt engineering methods' robustness against domain- agnostic perturbations applied to more realistic data, across more levels of abstraction, which are more widespread in the practical use of LLMs.
COT inspired a few subsequent prompting tech- niques. Zero-shot prompting (0CoT) proposed by Kojima et al. (2023) requires significantly less hu- man engineering to generate prompts compared with the original approach. Least-to-most prompt- ing (LTM) proposed by Zhou et al. (2023) decom- poses complex reasoning tasks into easier subprob- lems to improve the model performance. Selection- Inference prompting (Creswell al., 2023), LAM- BADA (Kazemi et al., 2023), and tree-of-thought prompting ( (Yao al., 2023) further break down the problem so that the LLM is queried for each step of the reasoning. Lastly, self-consistency prompting (Wang et al., 2023) uses sampling and aggrega- tion techniques to diversify reasoning paths and increase the chance of deriving correct answers.
3 Method
We aim to conduct domain-agnostic analysis with tests that span multiple levels of abstraction, are easy to automate, while still closely resembling examples in real applications. Thus, we use the following four perturbation tests:
Typo. We test whether the model's output is sen- sitive to typing errors in its input by introducing typos. To be robust to this perturbation, LLMs cannot rely on copying problem sentences with re- peated words to produce reasoning chains. Typos are introduced by randomly swapping one charac- ter with its adjacent character within a token with probability 0.1 given that the token has more than one character and is not numeric.
These methods have been shown to be effective in increasing model accuracy under noise-free free en- vironments. Our work, on the other hand, focuses on investigating the robustness of these methods when a variety of perturbations are present.
2.2 Behavioral Testing
The concept of behavioral testing (also known as black-box testing) is first proposed by Beizer and Wiley (1996) as an effective approach to probing large software or computer systems. Ribeiro et al. (2020) brought many of these insights to the test- ing of NLP models and proposed LIST, a comprehensive domain-agnostic methodology that embraces the benefits of challenge sets such as systematic control of test examples (Belinkov and Glass, 2019) while avoiding their drawbacks such the fact that challenge sets are artificial and do not resemble real data.
Synonym. We test whether models could recognize semantically similar tokens that refer to the same object of interest by replacing some words with synonyms. To be robust to this perturbation, LLMs should not exploit lexical spurious correlations. Op- erationally, every noun and verb token is replaced by one of its synonyms within its WordNet (Miller, 1992) synsets with probability 0.2.²
Repetition. We also test whether models are ro- bust to relevant but redundant information by dupli- cating sentence in the input. To be robust to this perturbation, the model must ignore the redundant sentence, or utilize to complete the COT. Opera- tionally, we randomly choose a sentence within the problem text other than the question sentence (i.e.
Recent behavioral analysis of LLMs has revealed their deficiencies when handling domain-specific perturbations. LLMS are shown to be sensitive to domain-specific perturbations in reading com- prehension (Jia and Liang, 2017), text classifi- cation (Gan and Mori, 2023), as well as logical
"higher than Typo probability to ensure the number of perturhed tokens is comparable with Typo test

---

GSMBK GPT-3.5-Turbe
GSMBK LLAMA2-78
GSMBK LLAMA2-138
Accuracy
0.2
0.6
0.1
OCoT
OoT
LTM
0C&T
OCoT
StrategyQA] GPT-3.5-Turbo
StrategyQA LLAMA2-7B
StrategyQA LLAMA2-13B
Accuraloy
0.50
0.50
0.5
0.26
0.25
OCoT
LTM
No Perturbation
Typo
Synonym
Repetitinn
Shortcut
Figure 3: Perturbations the test question accuracy under various combinations of dataastts and prompting approaches. 95% confidence intervals are shown.
the last sentence), and insert copy of it before the last sentence. This minimizes the risk of breaking coreference links (Jia and Liang, 2017).
solution, followed by the test problem. The exem- plars are created from the problems within GSM8K either with or without perturbations to allow us bet- ter understand how perturbing the exemplars would affect the performance of the LLMs in answering the test problem. For OCoT, the test question is rectly presented without exemplars but instead fol- lowed by "A: Let's think step by step". To facilitate answer extraction, a high-level instruction, "End your response with 'The answer is <answer> is prepended to every prompt. We do not apply LTM prompting for Strategy as it does not pro- vide high-quality decompositions for each question, which are required by LTM prompting.
Shortcut. We test whether model behavior is affected if an intermediate result is given in the question description. The LLM may take advan- tage of this extra information to expedite reasoning. Conversely, the LLM could also be confused, as would serve as redundant information if the LLM first ignored this directly given intermediate result but later derived Operationally, we extract the first reasoning step for the problem (first hop for COT and 0CoT; first sub-probe -roblloblem and its corre- sponding answer for LTM) and insert it before the question sentence.
Results
4
Experimental Setup
We carry out two experiments. In the first ex- periment, we evaluate the effect of perturbations of the test question on LLM performance under different prompting methods. In the second ex- periment, we perturb exemplar questions and vary the proportion of perturbed exemplars for fhw-shot prompting techniques to investigate whether an in- creasing share of perturbed exemplars would lead to better robustness against the perturbations in the test question.
4.1 Model and Dataset
The GSM8K (Cobbe et al., 2021) and Strate- gyQA (Geva et al., 2021) datasets are used for all experiments. The relatively simple problem state- ments in these datasets facilitate the creation of adversarial examples for testing. The two datasets provide the intermediate reasoning steps required by the few-shot prompting methods. For the choice of LLMs, GPT-3.5-Turbh (Brown et al., 2020), LLaMA2 7B and LLLMA22 13B models used in our experiments. All models are open source with weights available for public use.
5.1 Do Perturbations in the Test Question Affect Prompting Performance?
In this experiment, we investigate the effect of perturbations on prompting methods with original unperturbed exemplars. For CoT and LTM trials, unperturbed question and its associated reason- ing process are provided as the exemplar, followed by the perturbed test question, whereas exem- plar is provided for OCoT trials. The results are illustrated in Figure 3. We draw a few conclusions from the results:
4.2 Prompting Methods
We experiment with COT, 0CoT, and LTM. Our prompt design closely follows Zhou et al. (2023), Kojima et al. (2023), and Shi et al. (2023). For few- shot prompting techniques (i.e. CoT and LrM), each input prompt includes either one or eight ex- emplar problems as well as their corresponding

---

LLAMA2-7B Typo
LLAMA2-TB Syrenym
LLAMA2-7B Repetition
LLAMA2-7B Shoricul
04
LLAMAG-133 Type
LLAMA2-138] Synonym
LLAMAR-13R
LLAMAR-13R (hhonout
0.25
0.20
Accurecy
1.15
0.10
GPT-3.5 Turbo Typo
GPT-3.5 Turbo| Synonym
GPT-3.5 Turbo Repotiion
GPT-3.5 Turbe| Shontout
0.76
0.70
0.65
0880
OxT
LTM
CST
LTM
CUT
LTM
CxT
LTM
Figure 4: Number of perturbed in-context exemplars vs accuracy for GSM8K trials. Note that the total number of in-context exemplars is always 8, Dashed red lines indicate the accuracy of 0CoT in subsection 5.1. 95% confidence intervals shown.
In general, prompting methods are most sus- ceptible to Synonym replacement. When conduct- ing the Synonym tests, the accuracy of the LLM falls below the baseline performance across trials. For instance, for GPT-3.5-Turbo on the GSM8K dataset, Synonym perturbation lowers the accuracy by 0.097 0.122 (green bars in the top-left sub- plot of Figure 3) as compared to the baseline (blue bars), which are larger decreases as compared to the Typo (0.034 0.044, orange bars) and Repetition tests (0.046 ~ 0.062, red bars). This might result from low-frequency phrases and sen- tences created by the Synonym perturbation, as ev- ery token is substituted with synonym regardless of the adjacent tokens. Many substitutions would likely create grammatically correct yet infrequent sentences and phrases (e.g. "She eats three repasts per day"") which might lead worse performance.
experiment for two few-shot prompting methods CoT and LTM. each trial, 8 exemplars are pre- sented the I.L,, with proportion of them (0, 1, 2, 4, or
of 8) being perturbed. Then, the perturbed test question is presented to the LLM. The results shown in Figure 4 and Figure 5. We find the following key observations:
Inereasing the proportion of perturbed ex- emplars improves few-shot prompting perfor- mance, except for the Typo perturbation. From the results, the accuracy for both few-shot prompt- ing methods trends upwards when the number of perturbed exemplars presented to the LLM in- creases aside from LLaMA2-13B on the shortcut perturbation, demonstrating the evidence that the LLM able to adapt to the perturbations through the few-shot learning process. Take GPT-3.5-Turbo on GSM8K dataset as an example, if excluding tri- als with the Typo perturbation, increasing the num- ber of perturbed exemplars from 8 (lightest bars versus darkest bars in the third row of Fig- ure 4) results in an average of 0.035 increase in accuracy, with the minimum increase of 0.01 and the maximum of 0. 0.4. The performance of the LLM in many trials surpasses the benchmark set by the OCoT method in subsection 5.1 with the max-
5.2 Do Perturbations in the Exemplars Affect Few-shot Prompting Performance?
Following the previous experiment, question of interest naturally arises: would perturbing the exemplars during the few-shot learning process im- proves the robustness few-shot prompting meth- ods? To answer this question, we conduct another

---

LLAMA2-7B
LLAMA2-138
GPT-3.5-Turbo
0.8
Accuracy
0.6
0.4
0.0
Synonym
Repetition
Shortout
Typo
Synonym
Repetiton
Shortout
Typo
Synonym
Repetition
Shortout
# Perturbed Exempiars
Figure 5: Number of perturbed in-context exemplars accuracy for Strategy trials. Note that do apply LTM on StrategyQA QA and all bars represent COT results. Dashed red lines indicate the accuracy of OCOT in subsection 5.1. 95% confidence intervals are shown.
imum advantage of 0.048 achieved by CoT after few-shot learning with all eight exemplars with the Synonym perturbation. Typo trials turn out to be the exception, where increasing the number of per- turbed exemplars does not lead to improvement in accuracy. We speculate that this results from the fact that typos are much more common in the pre- training dataset as compared to other perturbations, which offsets the benefits of few-shot learning on perturbed exemplars.
There are a number directions for future work available: the first to explore the effect of more comprehensive perturbations, such as presenting the LLM with problem statement containing mul- tiple possible lines of reasoning that each lead to the correct conclusion, or adding some semantically related but logically irrelevant extra information to the problem statement (Shi et al., 2023). Another direction is to explore the influence of several other variables on the robustness of the prompting meth- ods, such as the model size, the number of hops required to answer the problem, whether the prob- lem counterfactual, etc. Lastly, we observe that in some cases, though the LLM able to produce the correct answer under perturbation, the model is more likely to produce sentences with errors shown in Table (e.g. "Janet lay 16 eggs per day"). Further studying the relationship between pertur- bations in the model's s input and the errors in the output would be illuminating.
This experiment suggests that perturbing in- context exemplars may serve as more efficient alternative to augmenting pretraining with pertur- bations. is much cheaper and easier to introduce perturbed exemplars at inference time rather than pretraining.
6 Discussion and Future Work
In conclusion, through two experiments, we con- ducted an investigation of the robustness of state-of- the-art prompting schemes via series of domain- agnostic perturbation tests. Our first experiment revealed the robustness of OCoT prompting versus few-shot prompting methods when the few-shot exemplars are unperturbed. Our second experiment demonstrated that perturbing few-shot exemplars led to notable improvements in robustness to per- turbations in the test question, which is valuable in real applications. Suppose a user knows ad- vance that the questions of interest will be subject some certain kind of perturbation (e.g. mathe- matics problems scraped from the online forums which are subject typos and uneliminated HTML tags, questions asked by beginner-level English speakers that may contain grammatical errors). The user can then prepend exemplars perturbed in sim- ilar fashion to improve the robustness of the LLM on their test examples. This can easily be applied in real-world use cases as all perturbations discussed above are easily automated.
Limitations
We acknowledge several limitations of our work. Firstly, due to the constraints in computational power, available time, and budget, we only ex- periment with relatively small set of LLMs and datasets, and our results could benefit from ex- periments
increased number of models and datasets with more diverse tasks. Secondly, some perturbations might introduce unintended side ef- fects. For instance, the Repetition test, adding repeated information might not always be idempo- tent, as repeating sentences like man walked into the bar might alter the correct answer to the prob- lem. These nuances and subtleties might require more detailed manual inspection.
Reproducibility Statement
In our experiments, fixed random seed (42) is applied over all trials to ensure the reproducibil- ity of perturbed questions and exemplars, and the

---

sampling temperature is set to 0 so that the model behavior is as deterministic as possible. The exper- iment output files well as the analysis codes are available in our GitHub repository (www.gth thub. om/Hiroki 39/No9 - Mars-MMke-Laad Large More
Robin Jia and Percy Liang. 2017. Adversarial exam- ples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference Empiri- cal Methods in Natural Language Processing, pages 2021-2031, Copenhagen, Denmark. Association for Computational Linguistics.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray. Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models.
References
B. Beizer and Wiley. 1996. Black box testing: Tech- niques for functional testing of software and systems. IEEE Software, 13(5):98-.
Mehran Kazemi, Najoung Kim, Deepti Bhatia, Xin Xu, and Deepak Ramachandran. 2023. Lambada: Back- ward chaining for automated reasoning in natural language.
Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, -22
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2023. Large lan- guage models are zero-shot reasoners.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Berbert-Vosss, Gretchen Krueger, Tom Hemighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Fric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. CoRR, 0bs/2005.14165.
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blun- som. 2017. Program induction by rationale genera- tion: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics (Volume I: Long Papers). pages 158-167, Vancouver, Canada. Association for Computational Linguistics.
George A. Miller, 1992 WordNet: lexical database for English. In Speech and Natural Language: Pro- ceedings of a Workshop Held at Harriman, New York, February 23-26, 1992.
Yong Cheng. Zhaopeng Tu, Fandong Meng. Junjie Zhai, and Yang Liu. 2018 Towards robust neural machine translation. In Proceedings of the 56th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1756-1766, Mel- bourne, Australia. Association for Computational Linguistics.
Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, H. Francis Song, John Aslanides. Sarah Henderson, Roman Ring, Susan- nah Young, Eliza Rutherford, Tom Hennigan, Ja- cob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Mari- beth Rauh, Po-Sen Huang, Amelia Glaese, Jo- hannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Sid- dhant M. Jayakumar, Elena Buchatskaya, David Bud- den, Esme Sutherland, Karen Simonyan, Michela Pa- ganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadch, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou. Arthur Mensch, Jean -ran-Btptistee Lespiau, Maria Tsim- poukelli, Nikolai Grigorev, Doug Fritz, Thibault Sot- tiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d' Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy. Chris Jones, James Bradbury, Matthew Johnson. Blake A. Hechtman, Laura Weidinger, lason Gabriel, William S. Isaac. Edward Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals. Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hass- abis, Koray Kavukcuoglu, and Geoffrey Irving. 2021. Scaling language models: Methods, analysis & in- sights from training gopher. CoRR, abs/21 12. 1446.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word prob- lems.
Antonia Creswell, Murray Shanahan, and Irina Higgins. 2023. iineeiooniifference:: Exploiting large language models for interpretable logical reasoning. In The Eleventh International Conference on Learning Rep- resentations.
Chengguang Gan and Tatsunori Mori. 2023. Sensitivity and robustness of large language models prompt template in japanese text classification tasks.
Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant, 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346- 361.

---

Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin. and Sameer Singh. 2020. Beyond accuracy: Be- havioral testing of NLP models with Ciecklist. In Proceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 4902-
4912, Online. Association for Computational Lin- guistics.
Abulhair Saparov and He He. 2023. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought.
Abulhair Saparov, Richard Yuanzhe Pang. Vishakh Pad- makumar, Nitish Joshi, Seyed Mehran Kazemi, Na- joung Kim, and He He. 2023. Testing the general deductive reasoning capacity of large language mod- els using ood examples.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. 2023, Large language models can be easily distracted by irrelevant context.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning language models. In International Conference on Learning Representa- tions.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Chi, Quoc Lc, and Denny Zhou. 2023. Chain-of-thought prompting clic- its reasoning in large language models.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models.
Xi Ye and Greg Durrett. 2022a. The unreliability of ex- planations in few-shot prompting for textual reason- ing. In Advances in Neural Information Processing Systems.
Xi Ye and Greg Durrett. 2022b. The unreliability of explanations in few-shot prompting for textual rea- soning.
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang. Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoe Le, and Ed Chi. 2023. Least-to-most prompting enables complex soning large language models.

---

Appendices
A
Test Question and Model Output Example
Model Output
Input Question
COT
0CoT
LTM
Original
Janet's 's ducks lay 16 eggs per day. She cats three for break- fast every morning and bakes muffins for her friends every day with four. She sells the re- mainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day the farmers' market?
Janet uses 3+4=7 eggs ev- cry day for breakfast and muffins. So she has 16- 7=9 eggs left sell at the farmers' market. She makes 9+2=$18 every day at the farmers' market. The answer is 18.
Janet has 16 eges per day. She
How many eggs does Janet use for breakfast and muffins every day? Janet ery day. How many eggs does she have left to sell? So she has 16-7=9 eggs left to sell every day. How much money does she make selling the eggs? Selling each egg $2, she makes 9*2-S18 every day farmers' market. The answer 18.
for breakfast and uses 4 for muffins, which leaves her with 9 eggs. She sells these 9 eggs at S2 per egg, which means she makes $18 per day at the farmers' mar- ket. The answer is $18.
3+4=7 eggs ev-
Typo
Janct's is ducks aly 16 eggs per day. She eats three for break- fast every omrning and bakes muffnis for her friends every day with four. She sells the re- mainder at the farmers' market daily ofr $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
Janet uses eggs for
Jante has 16 eggs per day.
How many eggs does Janet use for breakfast every day? Janet she eats 3 breakfast and 4 for muffins.
breakfast and she uses 3+4=7 eggs per day. That means she has 16-7-9 eggs left to sell at the farmers' market. Selling each egg for $2, she makes 9*2=$18 every day at the farmers' market. The answer is 18.
muffins,
eggs for breakfast every day. How many eggs does Janet use for muttins every day? Janet uses 4 eggs for muffins every day. How many eggs does Janet have left to sell? Janet has 16 = eggs left to sell. How much money does Janet make ev- ery day the farmers' market? Janet makes 9 $2 = $18 every day at the farmers' market. The answer is S18.
which leaves her with eggs. She sells these 9 eggs S2 per egg, which means she makes $18 ev-
ery day at the farmers' market. The answer S18.
Synonym
Janet's ducks lay 16 eggs per day. She eats three for break- fast every morning and bakes muffins for her friends every day with four. She sells the residue at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every sidereal day at the farmers' market?
Janet lays 16 eggs per day, and she eats 3 4 7 eggs per day. So, she has 16 7 = 9 eggs left to sell at the farmers' mar- ket. She makes 9 $2 = every day at the farm- ers' market. The answer 18.
Janet has 16 duck eggs per day. She eats 3 for breakfast and bakes muftins with 4, which leaves her with 9 eggs. She sells the remaining 9 eggs at the farmers' mar- ket for $2 each, which means she makes $18 per day. The answer is $18.
How many orchids does Janet have left after breakfast and bak- ing muffins? Janet has 16 3- 4 = 9 orchids left. How much money does she make from sell- ing these orchids? She makes 9 x $2 = $18 from selling the or- chids. How much money does Janet make every day the farm- ers' market? So Janet makes $ $18 every day at the farmers' market. The answer is 18.
Repetition
Janet's ducks lay 16 eggs per day. She cats three for break- fast every morning and bakes muffins for her friends every day with four, She sells the remainder at the farmers' mar- ket daily for per fresh duck egg. Janet's ducks lay 16 eggs per day. How much dollars does she make every day at the farmers' market?
Janet uses 3+4=7 eggs ev- ery day for breakfast and muffins. So she has 16- 7=9 eggs left to sell at the farmers' market. She makes 9*2=518 every day the farmers' market. The answer 18.
Janet has 16 eggs per day.
How many eggs does Janet use for breakfast and mutfins every day? Janet uses 3+4=7 eggs ev- cry day. How many eggs does she have left to sell? So she has 16-7=9 eggs left to sell every day. How much money does she make selling the eggs? Selling each egg for $2, she makes 9+2=$18 every day at the farmers' market. The answer is 18.
She cats 3 for breakfast and uses 4 for muffins, which leaves her with 9 eggs. She sells these eggs at S2 per egg, which means she makes $18 ev- ery day the farmers' market. The answer S18.

---

Shortcut
Janct's ducks lay eggs per day. She eats three for break- fast every morning and bakes muffins for her friends every day with four. She sells the re- mainder at the farmers' market daily $2 per fresh duck egg. duck eggs day. How much in dol- lars does she make every day at the farmers' market?
Janet sells duck eggs day, she makes 9 2 =18 every day the farmers' market. The an- swer is S18.
sells duck eggs day, she makes 9 $2 $18 every day at the farmers' market. The an- swer is $18.
How many eggs does Janet sell? Janet sells 9 duck eggs day. How much in dollars does she make every day at the farmers' market? She makes 9 2 = every day at the farmers' market. The answer
Janet sells
Table 1: An example of different perturbations and corresponding model outputs. The first column illustrates the original perturbed example test question next three columns show model outputs with CoT, 0CoT, and LrM respectively.


Reviews:
[{"review_id": "bxpCd89CVu", "reviewer": "Reviewer_5nja", "summary": "", "questions": "1. The author claim that one of the goal is `to explore empirical approaches that may improve their robustness.`, but I did not see much discussion on this. Could you provide more information on it?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Rebuttal\n\nWe thank the reviewer for their helpful and constructive feedback. We appreciate any additional comments or concerns.\n\nThe main concern of the reviewer is the limited generalizability of our results, since our experiments were limited to a single language model (gpt-3.5-turbo) and a single dataset (GSM8K). We address this concern by performing additional experiments on LLaMA-2 7B and 13B, as well as on StrategyQA [1], a commonsense question-answering dataset. We chose StrategyQA as it is distinct from GSM8K, and tests different reasoning abilities of LLMs. The experiment on the open model LLaMA-2 also helps to facilitate reproducibility.\n\nThe results for the experiment where 8 unperturbed few-shot examples are given for each perturbed test example (Fig 3 in the paper) are shown below:\n\n| GSM8K 0CoT   | ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:| \n| no perturbation |     0.78       |           0.11        |         0.08        | \n| typo                  |     0.75       |          0.10         |         0.04        | \n| synonym         |      0.69      |           0.11        |          0.04       | \n| repetition         |      0.74      |           0.09        |         0.05        | \n| shortcut          |     0.79       |           0.19       |          0.02        | \n\n| GSM8K   CoT   | ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |   0.78         |          0.24         |       0.16          |\n| typo                 |    0.75        |           0.24        |        0.19         |\n| synonym         |    0.70        |           0.24        |        0.15         |\n| repetition         |    0.75        |           0.26        |        0.14         |\n| shortcut           |    0.79       |            0.30        |        0.18         |\n\n| GSM8K   LTM  |  ChatGPT  |   LLaMA-2 13B |    LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n|no perturbation  |    0.77        |        0.29        |         0.15          |\n|typo                   |     0.74       |        0.26        |         0.19          |\n|synonym           |     0.70       |        0.24        |         0.14          |\n|repetition          |     0.75        |        0.29        |         0.22          |\n|shortcut            |     0.59        |        0.28        |         0.17          |\n\n| StratQA 0CoT  | ChatGPT   |  LLaMA-2 13B   |  LLaMA-2 7B  |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |    0.25       |        0.07            |    0.10          |\n| typo                  |    0.22       |         0.06           |     0.07          |\n| synonym          |     0.20      |         0.07           |     0.06          |\n| repetition          |    0.25       |         0.07           |     0.10          |\n| shortcut           |     0.27      |          0.07         |      0.16          |\n\n| StratQA  CoT  |  ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |   0.67       |            0.57     |           0.54      |\n| typo                |      0.64      |              0.54   |             0.53   |\n| synonym        |     0.63        |            0.52      |          0.53     |\n| repetition        |     0.67        |            0.57      |          0.54    |\n| shortcut         |     0.76         |           0.62       |          0.61     |\n\nNote that in the camera-ready version, all results mentioned here will be rendered as bar plots with confidence intervals, similar to Figs 3 and 4, making use of the extra page allowance.\n\nThe results for the experiment where the proportion of perturbed few-shot examples is varied (Fig 4 in the paper) is shown below:\n\nGSM8K Typo CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.74 | 0.75 | 0.75 | 0.75 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.29 | 0.28 | 0.24 |\n| LLaMA-2 7B                      | 0.20 | 0.20 | 0.21 | 0.20 | 0.17 |\n\nGSM8K Synonym CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.70 | 0.71 | 0.71 | 0.72 | 0.74 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.25 | 0.24 | 0.26 |\n| LLaMA-2 7B                      | 0.15 | 0.17 | 0.16 | 0.19 | 0.16 |\n\nGSM8K Repetition CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.77 | 0.76 | 0.77 | 0.76 |\n| LLaMA-2 13B                    | 0.26 | 0.26 | 0.26 | 0.27 | 0.26 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.17 | 0.18 | 0.17 |\n\nGSM8K Shortcut CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.79 | 0.80 | 0.79 | 0.79 | 0.80 |\n| LLaMA-2 13B                    | 0.30 | 0.30 | 0.28 | 0.31 | 0.31 |\n| LLaMA-2 7B                      | 0.20 | 0.20 | 0.20 | 0.21 | 0.22 |\n\nGSM8K Typo LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.74 | 0.73 | 0.73 | 0.72 | 0.74 |\n| LLaMA-2 13B                    | 0.26 | 0.26 | 0.25 | 0.28 | 0.26 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.17 | 0.17 | 0.15 |\n\nGSM8K Synonym LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.70 | 0.69 | 0.70 | 0.70 | 0.71 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.26 | 0.26 | 0.27 |\n| LLaMA-2 7B                      | 0.15 | 0.15 | 0.14 | 0.18 | 0.17 |\n\nGSM8K Repetition LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.75 | 0.76 | 0.76 | 0.77 |\n| LLaMA-2 13B                    | 0.29 | 0.29 | 0.28 | 0.28 | 0.30 |\n| LLaMA-2 7B                      | 0.18 | 0.18 | 0.19 | 0.20 | 0.20 |\n\nGSM8K Shortcut LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.59 | 0.67 | 0.70 | 0.74 | 0.70 |\n| LLaMA-2 13B                    | 0.28 | 0.28 | 0.26 | 0.26 | 0.24 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.18 | 0.20 | 0.23 |\n\nStrategy QA Typo CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.64 | 0.64 | 0.66 | 0.66 | 0.67 |\n| LLaMA-2 13B                    | 0.54 | 0.54 | 0.57 | 0.58 | 0.52 |\n| LLaMA-2 7B                      | 0.53 | 0.53 | 0.55 | 0.49 | 0.52 |\n\nStrategy QA Synonym CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.63 | 0.61 | 0.62 | 0.62 | 0.64 |\n| LLaMA-2 13B                    | 0.52 | 0.51 | 0.51 | 0.53 | 0.53 |\n| LLaMA-2 7B                      | 0.53 | 0.48 | 0.53 | 0.54 | 0.56 |\n\nStrategy QA Repetition CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.67 | 0.67 | 0.67 | 0.68 | 0.67 |\n| LLaMA-2 13B                    | 0.57 | 0.57 | 0.57 | 0.57 | 0.57 |\n| LLaMA-2 7B                      | 0.54 | 0.54 | 0.54 | 0.54 | 0.54 |\n\nStrategy QA Shortcut CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.76 | 0.76 | 0.76 | 0.77 | 0.76 |\n| LLaMA-2 13B                    | 0.62 | 0.61 | 0.63 | 0.61 | 0.61 |\n| LLaMA-2 7B                      | 0.61 | 0.61 | 0.60 | 0.58 | 0.57 |\n\nConsistent with the previous results on ChatGPT, LLaMA 7B generally seems to favor having more perturbed examples than fewer. Also consistent with earlier results, CoT is more robust against perturbations than LTM prompting. We propose to replace Figs 3 and 4 with the data from the new experiments shown above (rendered as bar plots) and add additional text to Sec 5 to better contextualize the results, mentioning that the models are more sensitive to perturbations on GSM8K than StrategyQA. We will also qualify the claims in the Introduction and Discussion sections accordingly.\n\n[1] Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, & Jonathan Berant (2021). Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies. TACL.\n\n## Answer to the Reviewer's Question\n\nQ: The author claims that one of the goals is to explore empirical approaches that may improve their robustness, but I did not see much discussion on this. Could you provide more information on it?\n\nA: We are happy to clarify. The goal of the experiment where we vary the number of perturbed examples is to test whether including perturbations in the in-context prompt can make the model more robust against those perturbations. Indeed we find that for some perturbations (notably Shortcut with LTM prompting), perturbed in-context examples can significantly improve the robustness of the model. We propose to include this in Sec 6.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is well-motivated and clearly written. \n2. Although the experiment is not comprehensive, this paper investigate the effect of several perturbation techniques on arithmetic reasoning tasks.", "reasons_to_reject": "1. Experiments are limited: Due to the computational power, only using gpt-3.5-turbo is acceptable, but at least experiment on two datasets which would make the conclusion more reliable. Also, if more domain can be tested (commonsense reasoning, symbolic reasoning) would be better.\n2. Setting is limited: To analyse the robustness of LLMs, it is expected to see more (e.g. written style perturbations, order perturbations)", "questions_to_authors": "1. The author claim that one of the goal is `to explore empirical approaches that may improve their robustness.`, but I did not see much discussion on this. Could you provide more information on it?", "paper_topic_and_main_contributions": "This paper investigates the robustness of large language models (LLMs) in multi-hop arithmetic reasoning tasks using domain-agnostic perturbations. The authors introduce a systematic approach (Typo, Synonym, Repetition Shortcut) to test the robustness of LLMs and compare the performance of different prompting methods under various perturbations.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LmWQWHO29p", "reviewer": "Reviewer_Scw6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful and constructive feedback. We appreciate any additional comments or concerns.\n\nThe main concern of the reviewer is the limited generalizability of our results, since our experiments were limited to a single language model (gpt-3.5-turbo) and a single dataset (GSM8K). We address this concern by performing additional experiments on LLaMA-2 7B and 13B, as well as on StrategyQA [1], a commonsense question-answering dataset. We chose StrategyQA as it is distinct from GSM8K, and tests different reasoning abilities of LLMs. The experiment on the open model LLaMA-2 also helps to facilitate reproducibility.\n\nThe results for the experiment where 8 unperturbed few-shot examples are given for each perturbed test example (Fig 3 in the paper) are shown below:\n\n| GSM8K 0CoT   | ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:| \n| no perturbation |     0.78       |           0.11        |         0.08        | \n| typo                  |     0.75       |          0.10         |         0.04        | \n| synonym         |      0.69      |           0.11        |          0.04       | \n| repetition         |      0.74      |           0.09        |         0.05        | \n| shortcut          |     0.79       |           0.19       |          0.02        | \n\n| GSM8K   CoT   | ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |   0.78         |          0.24         |       0.16          |\n| typo                 |    0.75        |           0.24        |        0.19         |\n| synonym         |    0.70        |           0.24        |        0.15         |\n| repetition         |    0.75        |           0.26        |        0.14         |\n| shortcut           |    0.79       |            0.30        |        0.18         |\n\n| GSM8K   LTM  |  ChatGPT  |   LLaMA-2 13B |    LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n|no perturbation  |    0.77        |        0.29        |         0.15          |\n|typo                   |     0.74       |        0.26        |         0.19          |\n|synonym           |     0.70       |        0.24        |         0.14          |\n|repetition          |     0.75        |        0.29        |         0.22          |\n|shortcut            |     0.59        |        0.28        |         0.17          |\n\n| StratQA 0CoT  | ChatGPT   |  LLaMA-2 13B   |  LLaMA-2 7B  |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |    0.25       |        0.07            |    0.10          |\n| typo                  |    0.22       |         0.06           |     0.07          |\n| synonym          |     0.20      |         0.07           |     0.06          |\n| repetition          |    0.25       |         0.07           |     0.10          |\n| shortcut           |     0.27      |          0.07         |      0.16          |\n\n| StratQA  CoT  |  ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |   0.67       |            0.57     |           0.54      |\n| typo                |      0.64      |              0.54   |             0.53   |\n| synonym        |     0.63        |            0.52      |          0.53     |\n| repetition        |     0.67        |            0.57      |          0.54    |\n| shortcut         |     0.76         |           0.62       |          0.61     |\n\nNote that in the camera-ready version, all results mentioned here will be rendered as bar plots with confidence intervals, similar to Figs 3 and 4, making use of the extra page allowance.\n\nThe results for the experiment where the proportion of perturbed few-shot examples is varied (Fig 4 in the paper) is shown below:\n\nGSM8K Typo CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.74 | 0.75 | 0.75 | 0.75 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.29 | 0.28 | 0.24 |\n| LLaMA-2 7B                      | 0.20 | 0.20 | 0.21 | 0.20 | 0.17 |\n\nGSM8K Synonym CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.70 | 0.71 | 0.71 | 0.72 | 0.74 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.25 | 0.24 | 0.26 |\n| LLaMA-2 7B                      | 0.15 | 0.17 | 0.16 | 0.19 | 0.16 |\n\nGSM8K Repetition CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.77 | 0.76 | 0.77 | 0.76 |\n| LLaMA-2 13B                    | 0.26 | 0.26 | 0.26 | 0.27 | 0.26 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.17 | 0.18 | 0.17 |\n\nGSM8K Shortcut CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.79 | 0.80 | 0.79 | 0.79 | 0.80 |\n| LLaMA-2 13B                    | 0.30 | 0.30 | 0.28 | 0.31 | 0.31 |\n| LLaMA-2 7B                      | 0.20 | 0.20 | 0.20 | 0.21 | 0.22 |\n\nGSM8K Typo LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.74 | 0.73 | 0.73 | 0.72 | 0.74 |\n| LLaMA-2 13B                    | 0.26 | 0.26 | 0.25 | 0.28 | 0.26 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.17 | 0.17 | 0.15 |\n\nGSM8K Synonym LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.70 | 0.69 | 0.70 | 0.70 | 0.71 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.26 | 0.26 | 0.27 |\n| LLaMA-2 7B                      | 0.15 | 0.15 | 0.14 | 0.18 | 0.17 |\n\nGSM8K Repetition LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.75 | 0.76 | 0.76 | 0.77 |\n| LLaMA-2 13B                    | 0.29 | 0.29 | 0.28 | 0.28 | 0.30 |\n| LLaMA-2 7B                      | 0.18 | 0.18 | 0.19 | 0.20 | 0.20 |\n\nGSM8K Shortcut LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.59 | 0.67 | 0.70 | 0.74 | 0.70 |\n| LLaMA-2 13B                    | 0.28 | 0.28 | 0.26 | 0.26 | 0.24 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.18 | 0.20 | 0.23 |\n\nStrategy QA Typo CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.64 | 0.64 | 0.66 | 0.66 | 0.67 |\n| LLaMA-2 13B                    | 0.54 | 0.54 | 0.57 | 0.58 | 0.52 |\n| LLaMA-2 7B                      | 0.53 | 0.53 | 0.55 | 0.49 | 0.52 |\n\nStrategy QA Synonym CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.63 | 0.61 | 0.62 | 0.62 | 0.64 |\n| LLaMA-2 13B                    | 0.52 | 0.51 | 0.51 | 0.53 | 0.53 |\n| LLaMA-2 7B                      | 0.53 | 0.48 | 0.53 | 0.54 | 0.56 |\n\nStrategy QA Repetition CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.67 | 0.67 | 0.67 | 0.68 | 0.67 |\n| LLaMA-2 13B                    | 0.57 | 0.57 | 0.57 | 0.57 | 0.57 |\n| LLaMA-2 7B                      | 0.54 | 0.54 | 0.54 | 0.54 | 0.54 |\n\nStrategy QA Shortcut CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.76 | 0.76 | 0.76 | 0.77 | 0.76 |\n| LLaMA-2 13B                    | 0.62 | 0.61 | 0.63 | 0.61 | 0.61 |\n| LLaMA-2 7B                      | 0.61 | 0.61 | 0.60 | 0.58 | 0.57 |\n\nConsistent with the previous results on ChatGPT, LLaMA 7B generally seems to favor having more perturbed examples than fewer. Also consistent with earlier results, CoT is more robust against perturbations than LTM prompting. We propose to replace Figs 3 and 4 with the data from the new experiments shown above (rendered as bar plots) and add additional text to Sec 5 to better contextualize the results, mentioning that the models are more sensitive to perturbations on GSM8K than StrategyQA. We will also qualify the claims in the Introduction and Discussion sections accordingly.\n\n[1] Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, & Jonathan Berant (2021). Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies. TACL.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Systematic Approach: This paper introduces a systematic approach to test the robustness of large language models (LLMs) in multi-hop arithmetic reasoning tasks. It includes domain-agnostic perturbations at multiple levels of abstractions, such as lexical and semantic perturbations, to conduct behavioral analysis on LLMs.\n\n2. Comparison of Prompting Methods: The paper compares and contrasts the performance of different prompting methods, including chain-of-thought prompting (COT), zero-shot prompting, and least-to-most prompting (LTM), with respect to various perturbations. This provides valuable insights into the strengths and weaknesses of these methods.\n\n3. Empirical Approaches to Improve Robustness: The paper explores empirical approaches to improve the robustness of prompting methods. It demonstrates that increasing the proportion of perturbed exemplars in the prompts improves the robustness of few-shot prompting methods.", "reasons_to_reject": "1. Limited Generalizability: The paper acknowledges that the experiments were only conducted with GPT-3 and GSM8K dataset, which limits the generalizability of the conclusions. This raises concerns about the applicability of the findings to other language models and datasets.\n\n2. Unintended Side Effects: The paper mentions that some perturbations, such as adding repeated information, might introduce unintended side effects. This could potentially alter the correct answer to the problem and affect the reliability of the model's output.\n\n3. Need for Manual Inspection: The nuances and subtleties of the perturbations used in the experiments might require more detailed manual inspection. This suggests that the automated evaluation alone may not capture all the potential issues and limitations of the proposed methods.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on the robustness of large language models (LLMs) in multi-hop arithmetic reasoning tasks using few-shot prompting techniques. The main contributions of the paper include introducing a systematic approach to test the robustness of LLMs through domain-agnostic perturbations, conducting behavioral analysis on LLMs using perturbations at multiple levels of abstractions, and comparing the performance of different prompting methods with respect to various perturbations. The paper also explores empirical approaches to improve the robustness of LLMs in these tasks.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "izh43HGBdo", "reviewer": "Reviewer_dKHZ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their helpful and constructive feedback. We appreciate any additional comments or concerns.\n\nThe main concern of the reviewer is the limited generalizability of our results, since our experiments were limited to a single language model (gpt-3.5-turbo) and a single dataset (GSM8K). We address this concern by performing additional experiments on LLaMA-2 7B and 13B, as well as on StrategyQA [1], a commonsense question-answering dataset. We chose StrategyQA as it is distinct from GSM8K, and tests different reasoning abilities of LLMs. The experiment on the open model LLaMA-2 also helps to facilitate reproducibility.\n\nThe results for the experiment where 8 unperturbed few-shot examples are given for each perturbed test example (Fig 3 in the paper) are shown below:\n\n| GSM8K 0CoT   | ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:| \n| no perturbation |     0.78       |           0.11        |         0.08        | \n| typo                  |     0.75       |          0.10         |         0.04        | \n| synonym         |      0.69      |           0.11        |          0.04       | \n| repetition         |      0.74      |           0.09        |         0.05        | \n| shortcut          |     0.79       |           0.19       |          0.02        | \n\n| GSM8K   CoT   | ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |   0.78         |          0.24         |       0.16          |\n| typo                 |    0.75        |           0.24        |        0.19         |\n| synonym         |    0.70        |           0.24        |        0.15         |\n| repetition         |    0.75        |           0.26        |        0.14         |\n| shortcut           |    0.79       |            0.30        |        0.18         |\n\n| GSM8K   LTM  |  ChatGPT  |   LLaMA-2 13B |    LLaMA-2 7B |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n|no perturbation  |    0.77        |        0.29        |         0.15          |\n|typo                   |     0.74       |        0.26        |         0.19          |\n|synonym           |     0.70       |        0.24        |         0.14          |\n|repetition          |     0.75        |        0.29        |         0.22          |\n|shortcut            |     0.59        |        0.28        |         0.17          |\n\n| StratQA 0CoT  | ChatGPT   |  LLaMA-2 13B   |  LLaMA-2 7B  |\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |    0.25       |        0.07            |    0.10          |\n| typo                  |    0.22       |         0.06           |     0.07          |\n| synonym          |     0.20      |         0.07           |     0.06          |\n| repetition          |    0.25       |         0.07           |     0.10          |\n| shortcut           |     0.27      |          0.07         |      0.16          |\n\n| StratQA  CoT  |  ChatGPT  |   LLaMA-2 13B  |   LLaMA-2 7B\n|:-------------------|:---------------:|:--------------------:|:-------------------:|\n| no perturbation |   0.67       |            0.57     |           0.54      |\n| typo                |      0.64      |              0.54   |             0.53   |\n| synonym        |     0.63        |            0.52      |          0.53     |\n| repetition        |     0.67        |            0.57      |          0.54    |\n| shortcut         |     0.76         |           0.62       |          0.61     |\n\nNote that in the camera-ready version, all results mentioned here will be rendered as bar plots with confidence intervals, similar to Figs 3 and 4, making use of the extra page allowance.\n\nThe results for the experiment where the proportion of perturbed few-shot examples is varied (Fig 4 in the paper) is shown below:\n\nGSM8K Typo CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.74 | 0.75 | 0.75 | 0.75 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.29 | 0.28 | 0.24 |\n| LLaMA-2 7B                      | 0.20 | 0.20 | 0.21 | 0.20 | 0.17 |\n\nGSM8K Synonym CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.70 | 0.71 | 0.71 | 0.72 | 0.74 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.25 | 0.24 | 0.26 |\n| LLaMA-2 7B                      | 0.15 | 0.17 | 0.16 | 0.19 | 0.16 |\n\nGSM8K Repetition CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.77 | 0.76 | 0.77 | 0.76 |\n| LLaMA-2 13B                    | 0.26 | 0.26 | 0.26 | 0.27 | 0.26 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.17 | 0.18 | 0.17 |\n\nGSM8K Shortcut CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.79 | 0.80 | 0.79 | 0.79 | 0.80 |\n| LLaMA-2 13B                    | 0.30 | 0.30 | 0.28 | 0.31 | 0.31 |\n| LLaMA-2 7B                      | 0.20 | 0.20 | 0.20 | 0.21 | 0.22 |\n\nGSM8K Typo LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.74 | 0.73 | 0.73 | 0.72 | 0.74 |\n| LLaMA-2 13B                    | 0.26 | 0.26 | 0.25 | 0.28 | 0.26 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.17 | 0.17 | 0.15 |\n\nGSM8K Synonym LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.70 | 0.69 | 0.70 | 0.70 | 0.71 |\n| LLaMA-2 13B                    | 0.24 | 0.24 | 0.26 | 0.26 | 0.27 |\n| LLaMA-2 7B                      | 0.15 | 0.15 | 0.14 | 0.18 | 0.17 |\n\nGSM8K Repetition LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.75 | 0.75 | 0.76 | 0.76 | 0.77 |\n| LLaMA-2 13B                    | 0.29 | 0.29 | 0.28 | 0.28 | 0.30 |\n| LLaMA-2 7B                      | 0.18 | 0.18 | 0.19 | 0.20 | 0.20 |\n\nGSM8K Shortcut LTM:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.59 | 0.67 | 0.70 | 0.74 | 0.70 |\n| LLaMA-2 13B                    | 0.28 | 0.28 | 0.26 | 0.26 | 0.24 |\n| LLaMA-2 7B                      | 0.16 | 0.16 | 0.18 | 0.20 | 0.23 |\n\nStrategy QA Typo CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.64 | 0.64 | 0.66 | 0.66 | 0.67 |\n| LLaMA-2 13B                    | 0.54 | 0.54 | 0.57 | 0.58 | 0.52 |\n| LLaMA-2 7B                      | 0.53 | 0.53 | 0.55 | 0.49 | 0.52 |\n\nStrategy QA Synonym CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.63 | 0.61 | 0.62 | 0.62 | 0.64 |\n| LLaMA-2 13B                    | 0.52 | 0.51 | 0.51 | 0.53 | 0.53 |\n| LLaMA-2 7B                      | 0.53 | 0.48 | 0.53 | 0.54 | 0.56 |\n\nStrategy QA Repetition CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.67 | 0.67 | 0.67 | 0.68 | 0.67 |\n| LLaMA-2 13B                    | 0.57 | 0.57 | 0.57 | 0.57 | 0.57 |\n| LLaMA-2 7B                      | 0.54 | 0.54 | 0.54 | 0.54 | 0.54 |\n\nStrategy QA Shortcut CoT:\n| num perturbed exemplars |    0   |    1   |   2   |    4   |    8    |\n|:----------------------------------|:------:|:------:|:-----:|:------:|:------:|\n| ChatGPT                           | 0.76 | 0.76 | 0.76 | 0.77 | 0.76 |\n| LLaMA-2 13B                    | 0.62 | 0.61 | 0.63 | 0.61 | 0.61 |\n| LLaMA-2 7B                      | 0.61 | 0.61 | 0.60 | 0.58 | 0.57 |\n\nConsistent with the previous results on ChatGPT, LLaMA 7B generally seems to favor having more perturbed examples than fewer. Also consistent with earlier results, CoT is more robust against perturbations than LTM prompting. We propose to replace Figs 3 and 4 with the data from the new experiments shown above (rendered as bar plots) and add additional text to Sec 5 to better contextualize the results, mentioning that the models are more sensitive to perturbations on GSM8K than StrategyQA. We will also qualify the claims in the Introduction and Discussion sections accordingly.\n\n[1] Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, & Jonathan Berant (2021). Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies. TACL.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The authors present a novel research direction that might be of great value in the LLM research space as their adaptation in zero-shot settings increases.", "reasons_to_reject": "- The scope of the paper is quite limited, no conclusions can be made using the data the authors provide since they test preliminary perturbations only on a single language model.\n- The authors make some conclusions based on the results they see on a single dataset using a single model, first, the performance variations don't look very significant to me, second, although the variation is significant I would argue that the evidence is too modest to make any conclusions.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper introduces an approach to assess the robustness of large language models (LLMs) in multi-hop arithmetic reasoning tasks using domain-agnostic perturbations. The authors apply perturbations at various levels of abstraction including lexical and semantic changes (typos, repetitions, shortcuts and synonyms) to conduct an in-depth analysis of the LLM behavior. The authors mention that although recent advances in prompt engineering have shown impressive accuracy in multi-hop logical reasoning, their paper is one of the first to explore the robustness of LLMs with few-shot prompting techniques. The experiments in the paper demonstrate that increasing the proportion of perturbed examples in the prompts enhances the robustness of few-shot prompting methods thus helping attain greater resilience in LLMs' performance on multi-hop reasoning tasks.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: To assess the robustness of Large Language Models on multi-hop arithmetic reasoning tasks, the manuscript proposes a systematic approach that encompasses both lexical and semantic perturbations. All reviewers agree that the comparative analysis of distinct prompting methods, coupled with various proposed perturbations, offers significant insights. To further improve the quality of current draft, reviewers suggest 1) expanding to other reasoning task for wider generalization, 2) investigating different perturbations such as styles and orders, and 3) incorporating human evaluations as a supplement to the automated assessments. As some of these points are addressed during rebuttal, it is encouraged to properly integrate them into the revised manuscript.
Decision: Accept-Main