Exploring Faithful and Informative Commonsense Reasoning and Moral Understanding in Children’s Stories (2024)

Abstract: “Commonsense reasoning and moral understanding are crucial tasks in artificial intelligence (AI) and natural language processing (NLP). However, existing research often falls short in terms of faithfulness and informativeness during the reasoning process. We propose a novel framework for performing commonsense reasoning and moral understanding using large language models (LLMs), involving constructing guided prompts by incorporating relevant knowledge for commonsense reasoning and extracting facts from stories for moral understanding. We conduct extensive experiments on the Commonsense Reasoning and Moral Understanding in Children’s Stories (CRMUS) dataset with widely recognised LLMs under both zero-shot and fine-tuning settings, demonstrating the effectiveness of our proposed method. Furthermore, we analyse the adaptability of different LLMs in extracting facts for moral understanding performance.”

## introduction proficiency in acquiring reasoning abilities, such as arithmetic, commonsense, and symbolic reasoning, plays an essential role in artificial intelligence (ai) and natural language processing (nlp) @xcite . unlike arithmetic reasoning, which involves manipulating numbers, and symbolic reasoning, which involves interpreting logic and symbols, commonsense reasoning encompasses counterfactual, abductive, and monotonic reasoning @xcite . it is crucial for language understanding and enables humans to navigate daily situations seamlessly @xcite . applications of commonsense reasoning include text classification @xcite , question answering @xcite , and natural language generation @xcite . commonsense reasoning is typically framed as a multiple-choice format, where the goal is to determine the plausibility of candidate answers. this approach mirrors how people often consider several plausible choices based on a given situation and their thought processes (figure 1 ) @xcite . previous research has focused primarily on utilising pre-trained language models (plms) and conducting the reasoning process based on factual time and space information @xcite , human behaviours @xcite , and story texts @xcite . recently, with the development of large language models (llms) that have shown remarkable performance in a range of natural language understanding and reasoning tasks @xcite , they have also been leveraged to enhance the reasoning process @xcite @xcite . similar to commonsense reasoning, moral understanding is the process of comprehending the moral of the given context from multiple candidates. despite the progress made, existing research in commonsense reasoning and moral understanding faces significant challenges, particularly regarding the unfaithfulness and uninformativeness of the reasoning process. current llm-based methods primarily follow the in-context learning (icl) paradigm motivated by this phenomenon, we design a novel framework to conduct commonsense reasoning and moral understanding, making the reasoning process more faithful and informative. unlike the previous work that utilises external knowledge, such as knowledge bases @xcite , search engines @xcite , and the knowledge generated by llms @xcite , we construct guided prompts for the two tasks, as shown in figure 1 . for commonsense reasoning, we incorporate the related knowledge concerning the story and the question into the prompt, and for moral understanding, we extract the facts contained in the stories as additional supervision. we conduct extensive experiments on the commonsense reasoning and moral understanding in children's stories (crmus) dataset with the widely recognised llms: glm-3, glm-4, moonshot, and yi-34b for zero-shot prompting and chatglm3-6b @xcite , internlm2-7b @xcite , qwen1.5-7b @xcite , and yi-6b @xcite for fine-tuning. experimental results demonstrate the effectiveness of guided prompts for both commonsense reasoning and moral understanding. among the models, glm-4 and internlm2-7b achieve the best performance in zero-shot and fine-tuning settings, respectively. furthermore, we conduct additional experiments to analyse the adaptability of extracted facts from different llms for moral understanding performance. the key contributions of this work are summarised as follows: ## background commonsense reasoning has received considerable attention over the past decade. recent research highlights the substantial improvements in this area by incorporating additional knowledge, broadly falling c c l 2 0 2 4 end-to-end prompt for commonsense reasoning: 根据故事回答下面的单项选择题，只给出答案即可： [translation: please answer the following multiple-choice question based on the story, providing only the answer:] 故事[story]：{story_text} 问题[question]：{question} 选项[options]：{options} 答案[answer]： end-to-end prompt for moral understanding: 根据故事从给定选项中选择最符合故事说明的寓意的选项，只给出答案即可： [translation: please select the option that best matches the moral from the given choices based on the story, providing only the answer:] 故事[story]：{story_text} 选项[options]：{options} 答案[answer]： the second approach focuses on leveraging the internal knowledge of llms, which are trained on massive datasets to generate task-specific knowledge. for instance, @xcite employs self-talk procedures @xcite and inquiry-based discovery learning to generate implicit commonsense before response generation. similarly, @xcite and @xcite generate plausible explanations for commonsense reasoning by incorporating future context in decoding algorithms and using posterior regularisation for constraint enforcement. additionally, @xcite prompts gpt2-xl @xcite for inference using generated contrastive explanations. furthermore, studies like those discussed by @xcite and @xcite emphasise the improvements in commonsense reasoning, even in zero-shot scenarios, through the incorporation of llm-generated knowledge. unlike the previous work, we construct guided prompts with knowledge highly related to the stories, which are more faithful to the story contexts and have higher generalisability. ## system overview following the overall framework illustrated in figure 1 , in this section, we describe the design of the system to conduct faithful and informative commonsense reasoning and moral understanding in detail. ## conclusion and future work we introduced a novel framework for commonsense reasoning and moral understanding with llms, aiming to ensure a faithful and informative reasoning process. specifically, we developed guided prompts that integrate relevant knowledge for commonsense reasoning and the facts that happened in the stories extracted by llms for moral understanding. we conducted extensive experiments on the crmus c c l 2 0 2 4 dataset with widely recognised llms under both zero-shot and fine-tuning settings and demonstrated the effectiveness of our proposed method. we further analysed the adaptability of extracted facts of different llms on moral understanding. in the future, we will make the guided prompts more diverse, incorporating more useful features to guide the reasoning process. we will also transfer our method on more reasoning tasks to test the generalisability of our proposed method.