Subjective Topic meets LLM s: Unleashing Comprehensive, Reflective and Creative Thinking through the Negation of Negation (2024)

Abstract: Large language models (LLMs) exhibit powerful reasoning capacity, as evidenced by prior studies focusing on objective topics that with unique standard answers such as arithmetic and commonsense reasoning. However, the reasoning to definite answers emphasizes more on logical thinking, and falls short in effectively reflecting the comprehensive, reflective, and creative thinking that is also critical for the overall reasoning prowess of LLMs. In light of this, we build a dataset SJTP comprising diverse SubJective ToPics with free responses, as well as three evaluation indicators to fully explore LLM’s reasoning ability. We observe that a sole emphasis on logical thinking falls short in effectively tackling subjective challenges. Therefore, we introduce a framework grounded in the principle of the Negation of Negation (NeoN) to unleash the potential comprehensive, reflective, and creative thinking abilities of LLMs. Comprehensive experiments on SJTP demonstrate the efficacy of NeoN, and the enhanced performance on various objective reasoning tasks unequivocally underscores the benefits of stimulating LLM’s subjective thinking in augmenting overall reasoning capabilities.

## introduction large language models (llms) have achieved remarkable performance in recent years @xcite @xcite and have displayed formidable reasoning ability that validated on various tasks, including arithmetic reasoning, symbolic reasoning, commonsense reasoning @xcite despite the prominent reasoning capabilities, their evaluation still lacks comprehensiveness. previous works primarily investigate llms based on objective topics with unique standard answers and the logical reasoning path is clear-out (e.g."3-2=1", "cat is herbivorous?") @xcite . it is evident that reasoning on these topics heavily relies on logical thinking. the success of chain-of-thought (cot) @xcite serves as compelling evidence, as it elicits logical thinking through prompts with a series of reasoning steps, instantly leading to significant improvement in solving objective questions. however, the ideal reasoning ability transcends mere logical thinking, comprehensive, reflective, and creative thinking are indispensable when tackling complex tasks, which cannot be well reflected on objective topics, as shown in figure 1 . consequently, the lack and difficulty in evaluating these abilities pose challenges for approaching human-like thinking ability of current llms. to enable analysis of llms in terms of comprehensiveness, reflection, and creativity and further enhance the overall reasoning ability, we first develop a benchmark sjtp that consists of subjective topics with free responses. respondents need to engage in open-ended discussions topic type example scr 1.viewpoint discourse what do you think about the impact of social media on mental health? 76.5 (comprehensive views) (①comprehensiveness: ⋆ ⋆ ⋆, ②reflection: ⋆⋆, ③creativity:⋆⋆ ) 2.binary dialectics should school wear uniforms? 69.9 (in-depth analysis) (①comprehensiveness: ⋆⋆, ②reflection: ⋆ ⋆ ⋆, ③creativity:⋆⋆ ) 3.practical analysis how can the preservation of cultural heritage contribute to the promotion of social cohesion? 80.2 (constructive opinions) (①comprehensiveness: ⋆⋆, ②reflection: ⋆⋆, ③creativity:⋆ ⋆ ⋆) topic filed ① social and ethics, ② history and civilization, ③ literature and arts, ④ technology and education, ⑤ environment and health, ⑥ economy and politics, ⑦ law and human rights, ⑧ psychology and emotions evaluation [basic]: ① clarity of viewpoint, ② logicality of argumentation, ③ correctness and fidelity dimensions [advanced]: ④ comprehensiveness&diversity, ⑤ depth&essentiality, ⑥ innovation&uniqueness on these dimensions, we design three evaluation indicators for free responses and briefly evaluate gpt-3.5 armed with the promising cot prompts, which show plain performance. as shown in figure 1 , the reasoning pathway does not deliberately consider the full-sided perspectives and excavate the intrinsic causes or potential solutions about "the impact of social media on mental health", resulting in a lack of depth and unbalanced analysis. to elicit the llms' comprehensive, reflective, and creative thinking abilities, we propose a structured framework neon inspired by the principle of the negation of negation, which is a philosophical principle proposed by engels @xcite . the core insight is that the development and completion of things must go through negation and transcendence of themselves, which also aligns with hegel's "abstraction-negation-concreteness" principle @xcite . through constant negation, we can emphasize the multifaceted and complex nature of problems, and break established thinking patterns, thereby achieving refinement to the defects in initial response. please note that if the current response is impeccable, further negation will be futile and lead to repetitive output, as negation should be performed under the correct premise. specifically, the proposed neon comprises three stages: direct answer, iterative negation, and unification reasoning. we first allow the llm to generate answers directly with strict logical thinking to guarantee fundamental reasoning ability. then we cast the llm as a negator instead of a logical reasoner. it will construct a negation link to the previous responses under the correct premise, and then determine whether to proceed with negation based on the similarity between the current and previous responses. therefore, by negating existing responses from any potential aspects, the llm will be encouraged to explore unconsidered perspectives, excavate profound insights, and inspire innovative ideas, which supplements and surpasses the previous responses in a spiral upward manner. finally, we enable the llm to generate a refinement response by considering all these answers. note that the whole workflow is built on top of unified zero-shot prompting, without carefully designed examples that are specific to different tasks, making it highly convenient and versatile. we employ both api-based and open-source llms including gpt (openai, 2023), chat-gpt (openai, 2022), llama @xcite and mistral @xcite , to validate the efficacy of our framework. experimental results show that neon leads to significant and consistent improvements on both subjective and objective topics, underscoring the necessity and effectiveness of unleashing the comprehensive, reflective and creative thinking for better reasoning ability. ## subjective topic dataset we construct a sjtp benchmark to explore the comprehensiveness, reflection, and creativity of llms, which includes diverse subjective topics along with reasonable scoring points and solutions. besides, we elaborately develop three evaluation indicators to assess the quality of free responses. the overall illustration is shown in fig. 2 . ## neon framework to unleash the comprehensive, reflective, and creative cognitive faculties of llms, we introduce the neon framework grounded on the principle of the negation of negation, allowing the model to continuously improve and transcend itself. the overall framework is schematically illustrated in figure 2 . firstly, neon generates an initial solution leveraging llm's inherent reasoning ability. then, it embarks on a process of negation, critiquing its ## conclusion we introduce sjtp, a subjective topic benchmark to explore the comprehensive, reflective, and creative thinking abilities of llms. we empirically observed that current methods that only focus on logical thinking fall short in solving subjective tasks. to alleviate this gap, we introduce neon, a framework based on the principle of negation of negation, allowing models to constantly improve and surpass previous responses. experiments on both subjective and objective tasks across models show neon leads to significant and consistent improvement, indicating the impact of negation and the significance for unleashing the subjective thinking abilities, and shedding light on new directions for enhancing overall reasoning abilities of llms.