Boosting Language Models Reasoning with Chain-of-Knowledge Prompting (2024)

Abstract: Recently, Chain-of-Thought (CoT) prompting has delivered success on complex reasoning tasks, which aims at designing a simple prompt like “Let’s think step by step” or multiple in-context exemplars with well-designed rationales to elicit Large Language Models (LLMs) to generate intermediate reasoning steps. However, the generated rationales often come with hallucinations, making unfactual and unfaithful reasoning chains. To mitigate this brittleness, we propose a novel Chain-of-Knowledge (CoK) prompting, where we aim at eliciting LLMs to generate explicit pieces of knowledge evidence in the form of structure triple. This is inspired by our human behaviors, i.e., we can draw a mind map or knowledge map as the reasoning evidence in the brain before answering a complex question. Benefiting from CoK, we additionally introduce an F2-Verification method to estimate the reliability of the reasoning chains in terms of factuality and faithfulness. For the unreliable response, the wrong evidence can be indicated to prompt the LLM to rethink. Extensive experiments demonstrate that our method can further improve the performance of commonsense, factual, symbolic, and arithmetic reasoning tasks.

## introduction large language models (llms) have succeeded in advancing the state-of-the-arts for many natural language processing (nlp) tasks @xcite @xcite @xcite @xcite , benefiting from the ultra-large-scale training corpora and computation resources. to unleash the llms' power of adaptation on unseen tasks without any parameter updates, in-context learning (icl) has become one of the flourishing research topics, aiming at generating the prediction by conditioning on a few labeled exemplars (figure 1 (a) ) @xcite @xcite @xcite . a series of recent works have explored that llms can spontaneously decompose the complex multi-step problem into intermediate reasoning chains , elicited by a simple prompt like "let's think step by step" or well-designed demonstrations with human-annotated rationales, which are called chain-of-thought (cot) prompting (figure 1 (b)) @xcite @xcite @xcite . this finding is intriguing and has been sensational because cot may mainly specify an output space/format that regularizes the model generation to look step-by-step while being in order and relevant to the query @xcite . despite impressive performances, current llms are susceptible to generating hallucination @xcite , along with providing unfactual or unfaithful reasoning chains that inevitably lead to a wrong conclusion @xcite . take figure 1 as an example. given a query "is the following sentence plausible 'derrick white backhanded a shot.'" from strate-gyqa @xcite , the standard icl and cot make a wrong answer. one of the reasoning steps "derrick white is most likely a hockey player" is fake (in fact, derrick white is a basketball player), making the unfactual inference towards the question. in addition, the response may be unfaithful when the llm generates logically sound reasoning chains while still providing an incorrect answer. to address these concerns, we propose a novel chain-of-knowledge (cok) prompting method to boost the llm's reasoning capability by a series of exemplars that combine explicit structure knowledge evidence with textual explanations. to elaborate, cok prompting consists of two compositions (figure 1 (c )), i.e., evidence triples (cok-et) and explanation hints (cok-eh), where cok-et is a list of structure triples can reflect the overall reasoning evidence from the query towards the answer and cok-eh is the explanation of this evidence. to construct in-context exemplars with the cok prompt, we first sample k labeled examples and each of them can be concatenated with a simple hint "let's think step by step" to prompt the llm to generate reasoning chains. then, we retrieve some structure triples from the external knowledge base (kb) and judiciously manually annotate evidence triples to obtain a well-designed cok prompt. like standard icl and cot, the cok prompt can be perceived as a rule that regularizes the output space/format and urges llms to generate explicit evidence instead of only attempting to generate vague textual reasoning chains. furthermore, we also propose an f 2 -verification strategy to estimate the reliability of the reasoning chains in terms of factuality and faithfulness, where factuality is the quantification of the matching degree between reasoning evidence and ground-truth knowledge, and faithfulness is the consistency degree between reasoning evidence and the textual explanation with the final answer. particularly for the unreliable response, the wrong pieces of evidence can be indicated to prompt the llm to rethink the problem. we design a rethinking algorithm to reach this goal. we have conducted empirical evaluations across various reasoning tasks (e.g., commonsense, fac-tual, arithmetic, and symbolic), showing that cok prompting with f 2 -verification can significantly outperform standard icl and cot prompting. we also integrate cok prompting with some prevailing strategies, such as self-consistency. the results indicate that such cok can serve as a plug-and-play module to further improve reasoning ability. ## related work prompting for llms with in-context learning. in-context learning (icl) is the task of causal language modeling, allowing llms to perform zero/few-shot learning with a well-designed textbased prompt @xcite @xcite @xcite . icl can bypass the model parameter update and achieve the salient performance by conditioning on a few labeled examples. previous works have explored some impact facets of icl. for example, the input-output mapping and the template format @xcite @xcite , the different selection and permutations of the exemplars @xcite . to improve icl's effectiveness, some novel methods have been proposed, involving meta-learning @xcite , prompt and exemplars engineering @xcite . ## methodology the generated reasoning chains elicited by cot prompting sometimes come with mistakes, ultimately leading to hallucinated answers. we attribute this challenge to the textual reasoning chain: llms may forcibly generate a textual rationale that conforms to the prompt format of cot but is logically ambiguous and reaches the wrong answer. to address this challenge, we provide our specific solution on how to boost llm's reasoning ability in two corners: the reasoning elicitation format of the prompt and the knowledge-enhanced postverification. the overview of the framework is shown in figure 2 . ## conclusion we propose chain-of-knowledge prompting, which aims to decompose the reasoning chains derived from the llms into multiple evidence triples and explanation hints, to further improve the reasoning capabilities. based on the chain-of-knowledge prompt, we introduce f 2 -verification and fully exploit external knowledge bases to perform postverification for the generated reasoning chains in terms of factuality and faithfulness. a rethinking process then be used to inject knowledge to correct the false evidence triples and elicit the llm to regenerate the answer. our extensive results show that it outperforms other prompt methods over multiple reasoning tasks. in the future, we will 1) further improve the performance of other scale llms, 2) extend the kb to search engines to realize realtime verification, and 3) perform interpretability analysis on llms' reasoning.