ERA - C o T : Improving Chain-of-Thought through Entity Relationship Analysis (2024)

Abstract: Large language models (LLMs) have achieved commendable accomplishments in various natural language processing tasks. However, LLMs still encounter significant challenges when dealing with complex scenarios involving multiple entities. These challenges arise from the presence of implicit relationships that demand multi-step reasoning. In this paper, we propose a novel approach ERA-CoT, which aids LLMs in understanding context by capturing relationships between entities and supports the reasoning of diverse tasks through Chain-of-Thoughts (CoT).Experimental results show that ERA-CoT demonstrates the superior performance of our proposed method compared to current CoT prompting methods, achieving a significant improvement of an average of 5.1% on GPT3.5 compared to previous SOTA baselines. Our analysis indicates that ERA-CoT increases the LLM’s understanding of entity relationships, significantly improves the accuracy of question answering, and enhances the reasoning ability of LLMs.

## introduction large language models (llms) @xcite @xcite have shown remarkable in-context learning capabilities in various natural language processing (nlp) tasks, including machine translation @xcite , question answering @xcite @xcite , and named entity extraction @xcite , etc. recently, prompting strategies like chain-of-thought (cot) @xcite have garnered attention due to their capacity to significantly enhance llms reasoning capabilities. considering the ability of cot to guide llms in breaking down complex reasoning processes into simple steps, it stands out compared to standard zero-shot and fewshot methods. however, due to the presence of numerous entities such as characters, locations, etc., and the multitude of implicit relationships among them in certain scenarios, cot still faces significant challenges in handling these situations. named entity recognition (ner) has typically been employed when addressing these tasks. ner is a sequence labeling task in nature, where the model needs to assign an entity-type label to each token within a sentence @xcite . relation extraction is a category of methods for handling entity relationships within text passages. various studies @xcite have also investigated the performance of llms in zero-shot relation extraction. however, without additional prompts, llms have limited entity and relation extraction capabilities. considering the importance of contextual content in answering questions, addressing knowledge-intensive tasks also requires a comprehensive analysis of entity relationships. in this paper, we propose entity relationship analysis with chain-of-thought (era-cot), a novel framework to better address reasoning tasks in complex entity scenarios. first, we extract all the entities involved in the text; second, we extract the directly mentioned explicit relationships between entities based on the text; then, we infer the indirect implicit relationships between entities based on these explicit relationships and the hidden information in the text; after that, we let the model score the implicit relationships based on the reliability of the relationships, set a threshold for judging the reliability of the relationships, and eliminate the implicit relationships that are lower than the threshold; finally, answer the questions based on the previously extracted entities and the obtained implicit and explicit relationships. we conducted experiments on six widely adopted datasets and compared with four baseline methods. the results show that era-cot outperforms baselines on nearly all benchmarks, achieving a significant improvement of about 5.1% on average. from the performance results, our method outperforms on all three types of reasoning problems: commonsense reasoning, mathematical reasoning, and logical reasoning. this indicates that enhancing the model's understanding of entity relationships can significantly boost the reasoning abilities and accuracy in answering questions of llms. our main contributions can be summarized as follows. ## methodology as shown in figure 1 , we introduce era-cot, a novel framework to enhance the model's understanding and reasoning of entity relationships in various nlp tasks. it consists of five stages, each involving different degrees of enhanced understanding of entity relations. simultaneously, leveraging the model's capabilities, it can jointly learn explicit and implicit relationships in the context throughout this progressive process, filtering out relevant knowledge to enhance the capability and performance of question reasoning. problem formulation. given an input sequence x, the cot method is to predict the answer via: ## conclusions in this paper, we introduce the method of era-cot to address open-domain question answering and knowledge reasoning tasks. by leveraging inference and detection of entity relationships, this method exhibits remarkable performance in tasks with lengthy texts or those involving numerous and complex entity relationships. extensive experiments demonstrate the superiority of the proposed method in various reasoning categories. we hope that our method can be applied to enhance the performance of llms in diverse domains.