# The Impact of Prompting Techniques on the Security of the LLMs and the Systems to Which They Belong
## 2 Domain Overview
### ReAct: Synergizing Reasoning and Acting in Language Models
To test the performance of ReAct, the authors compare it for baseline performance against standard prompting, chain-of-thought prompting [2] and acting-only prompting, which resembles how WebGPT [27] interacts with the internet. The paper notes that ReAct is more accurate and anchored compared to CoT, which is much better at structuring but suffers from hallucinations. They come up with a method that uses both approaches. The model decides when to use one of the two based on two heuristics. The LLM should switch from ReAct to CoT when ReAct fails to return an answer within a fixed number of steps and the other way around when most of the answers among \(n\ CoT-SC\) samples arise less than \(n/2\) times (i.e., the internal knowledge is not sufficient to solve the task, and the responses are not consistent). The results of the tests seem promising. The paper notes a few important observations. ReAct outperforms Act consistently, which demonstrates the fact that reasoning is necessary to guide acting. Speaking of ReAct vs. CoT, ReAct outperforms CoT on FEAVER (Fact Extraction and VERification, which consists of 185,445 claims generated by altering sentences extracted from Wikipedia and subsequently verified without knowledge of the sentence they were derived from. The claims are classified as Supported, Refuted or NotEnoughInfo. For the first two classes, the annotators also recorded the sentence(s) forming the necessary evidence for their judgment [26]) (60.9 vs. 56.3) but lags behind CoT on the HotpotQA dataset (27.4 vs. 29.4). The difference between SUPPORTS/REFUTES is a subtle one, so the ability to have access to up-to-date information is vital. We already mentioned that CoT suffers from hallucinations quite often. This translates to a bigger rate of false positives in success mode, and it is the main reason for its failures. On the other hand, interwoven reasoning, action and observation steps improve ReAct's groundedness and reliability, functioning as a structural constraint. On the other hand, it reduces its adaptability in formulating reasoning traces, which results in a greater number of logic errors than CoT. The authors identify one recurrent pattern specific to ReAct prompting. The model ends up in a loop of thoughts and actions, which stops the model from finding the solution. Another important observation is the fact that for ReAct, fetching quality knowledge through search is mandatory. Almost one quarter of the error cases come from non-informative searching, which derails its reasoning. Most of the time, it is not able to recover and reformulate thoughts, highlighting the trade-off between factuality and flexibility. Fortunately, the combination of the two performs the best and seems the best choice for prompting LLMs. \(ReAct\to CoT-SC\) and \(CoT-SC\to ReAct\) outperform the competition. The decision-making ability of ReAct is tested using ALFWorld [25] and WebShop [28]. As both of these benchmarks require simulated interaction with the outside world, ReAct is tested only against Act. The results show again the dominance of ReAct over Act; even the worst performance of ReAct outplays Act at its peak. The reasoning component of ReAct seems to help it split the goal into smaller subgoals and remain focused on the current conditions of the environment. ReAct shows the huge potential of the LLMs' reasoning abilities especially when put together with other tools. Unfortunately, just as in the case of PaL, all the benefits come with extra complexity, especially in our case with security concerns. Could we trust a model to interact with APIs and query a database? The attack surface increases once again with each new ability the LLM gains. Thus, we must find a way to guarantee that our model cannot be corrupted by someone with malicious intents and be used to hack our system.