Perspective Transition of Large Language Models for Solving Subjective Tasks
Authors: Xiaolong Wang, Yuanchi Zhang, Ziyue Wang, Yuzhuang Xu, Fuwen Luo, Yile Wang, Peng Li, Yang Liu
Accepted at: acl25

Perspective Transition of Large Language Models for Solving Subjective Tasks

Xiaolong Wang*1,3, Yuanchi Zhang*7,†, Ziyue Wang1, Yuzhuang Xu4, Fuwen Luo1
Yile Wang5,6, Peng Li5,2, Yang Liu1,2,6
1Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China
2Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China
3Jiuquan Satellite Launch Center (JSLC), Gansu, China
4 Harbin Institute of Technology, Harbin, China
5College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China
6 Jiangsu Collaborative Innovation Center for Language Competence, Jiangsu, China
7 Tencent Inc, China
wangxl22@mails.tsinghua.edu.cn, wangyile@szu.edu.cn
lipeng@air.tsinghua.edu.cn, liuyang2011@tsinghua.edu.cn

Abstract
Large language models (LLMs) have revolutionized the field of natural language processing, enabling remarkable progress in various tasks. Different from objective tasks such as commonsense reasoning and arithmetic question-answering, the performance of LLMs on subjective tasks is still limited, where the perspective on the specific problem plays crucial roles for better interpreting the context and giving proper response. For example, in certain scenarios, LLMs may perform better when answering from an expert role perspective, potentially eliciting their relevant domain knowledge. In contrast, in some scenarios, LLMs may provide more accurate responses when answering from a third-person standpoint, enabling a more comprehensive understanding of the problem and potentially mitigating inherent biases. In this paper, we propose Reasoning through Perspective Transition (RPT), a method based on in-context learning that enables LLMs to dynamically select among direct, role, and third-person perspectives for the best way to solve corresponding subjective problem. Through extensive experiments on totally 12 subjective tasks by using both closed-source and open-source LLMs including GPT-4, GPT-3.5, Llama-3, and Qwen-2, our method outperforms widely used single fixed perspective based methods such as chain-of-thought prompting and expert prompting, highlights the intricate ways that LLMs can adapt their perspectives to provide nuanced and contextually appropriate responses for different problems.

1 Introduction
Large language models (LLMs) have exhibited substantial advancements (Brown et al., 2020; OpenAI, 2023, 2022; Touvron et al., 2023; Jiang et al., 2023) in recent years, demonstrating remarkable performance across various tasks such as mathematical reasoning (Luo et al., 2023; Yang et al., 2023), code generation (Chen et al., 2021; Roziere et al., 2023), and commonsense question answering (Talmor et al., 2019). Meanwhile, research in the realm of subjective tasks remains relatively nascent (Rottger et al., 2022; Kanclerz et al., 2023; Sun et al., 2023). Unlike objective tasks, which are typically well-defined and directly solvable, subjective tasks such as metaphor recognition (Mohler et al., 2016) and dark humor detection (Meaney et al., 2021) require an understanding of context, linguistic subtleties, and varying individual perspectives. Although advanced chain-of-thought (CoT) style methods have been widely used to elicit the reasoning ability of LLMs (Wei et al., 2022; Kojima et al., 2022; Zhang et al., 2023; Yao et al., 2023), these approaches primarily focus on “how to think deeper in its own perspective” (OpenAI, 2024; DeepSeek-AI et al., 2025), while overlooking “from which perspective to think”.

Example 1
Identify whether the sentence “her smile was a blinding sun” is a paraphrase of the metaphoric sentence “her smile was wonderful and fulfilling to see”.

Chain-of-Thought Reasoning
While both sentences express admiration for her smile, the second sentence lacks the vivid imagery and specific emotional intensity that the first sentence provides. Thus, the second sentence is not an accurate paraphrase of the first. The answer is False.

Example 2
Identify whether the following examples are intended to be a joke (with dark humor) or not “I have a fish that can breakdance! Only for 20 seconds though, and only once”.

Chain-of-Thought Reasoning
The phrase is light-hearted and whimsical about a fish that can breakdance, suggesting a humorous intent rather than a dark theme. Therefore, it does not seem to fit the category of dark humor. The answer is Not.

Figure 1: Two examples for showing challenges of solving subjective tasks using chain-of-thought prompting. Refer to Appendix B.6 for more examples.
Question
Identify whether the following examples are intended to be a joke (with dark humor). "Give a man a match, and he'll be warm for a few hours. Set a man on fire, and he will be warm for the rest of his life."

Let's think step by step.

Step 1: Analyze the content of the examples.
- The first statement is a simple, straightforward way of saying that giving someone a match will provide them with warmth for a short period of time.
- The second statement suggests that setting someone on fire will give them even warmer for an extended period, which is not a typical or safe way to go.

Step 2: Consider the tone and context.
- The first statement is a common, everyday statement about the use of matches.
- The second statement, while it could be interpreted as a literal description of the act of setting someone on fire, is phrased in a way that could be seen as dark humor.

Given the context, the second statement is more likely intended to be a joke with dark humor.

You are a humorist with a keen sense of irony and a deep understanding of the nuances of dark humor. Your expertise lies in recognizing the subtleties of humor, including the use of irony, the unexpected, and the unexpected. You have a strong background in literature and philosophy, allowing you to analyze and appreciate the complexities of humor. In this case, you would recognize that the examples provided are intended to be a joke with dark humor.

As a humorist with a keen sense of irony and a deep understanding of the nuances of dark humor, the first statement is a straightforward description of what happens when you give someone a match. The second statement is a play on words, both in a literal sense (he'll be warm due to the fire) and also in a figurative sense (he'll be warm for the rest of his life).

Hey, I came across these two statements. They seem a bit unusual. Can you help me understand if they are intended to be a joke, especially with dark humor?

Sure, let's break it down. The first statement is a straightforward statement. It's just a simple description of what happens when you give someone a match. The second statement is a play on words, both in a literal sense (he'll be warm due to the fire) and also in a figurative sense (he'll be warm for the rest of his life).

So, this statement is more likely to be interpreted as a dark joke because it implies a grim outcome, which is not usually the intention when giving someone a match.

From the conversation, we know that the second statement is more likely to be a joke (with dark humor).

(a) Direct Perspective
(b) Role Perspective
(c) Third-Person Perspective

Figure 2: An example of solving dark humor detection task by different perspectives. (a) direct perspective: the model give the answer according to its analysis (Kojima et al., 2022). (b) role perspective: the model gives the answer by setting as a role related to the question (Xu et al., 2023). (c) third-person perspective: the model gives the answer as a third-person based on a simulated dialogue (Wang et al., 2024c).

This leads to modern LLMs performing poorly on subjective tasks challenging to be quantified or measured objectively (Jentzsch and Kersting, 2023; Wachowiak and Gromann, 2023; Mao et al., 2024). Moreover, due to the nature of the subjective tasks, identifying a chain-of-thought pathway similar to that in conventional reasoning tasks is difficult (Wang et al., 2024c). For instance, the generated reasoning pathways can even mislead the model to provide incorrect answers, as examples shown in Figure 1. Therefore, directly exploring CoT prompting techniques may not be practical for subjective tasks, motivating us to propose a general method to enhance the ability of LLMs to solve various subjective tasks.

Inspired by the theory of mind (Premack and Woodruff, 1978; Wellman et al., 2001), which refers to the ability to attribute mental states to oneself and others and to understand that these mental states can influence behavior, we leverage different perspectives to better address the aforementioned challenging subjective tasks rather than directly answering the questions based on the LLMs’ own direct perspective (e.g., zero-shot (Brown et al., 2020) or zero-shot-CoT reasoning (Kojima et al., 2022)). As shown in Figure 2, we categorize reasoning methods of LLMs into three perspectives: 1) direct perspective, which involves the model directly answering questions or tasks based on its internal understanding without considering external factors or alternative viewpoints; 2) role perspective, which focuses on assigning specific roles to the model, simulating different viewpoints or expertise within a given context or scenario; 3) third-person perspective, which involves the model considering external viewpoints or perspectives beyond its own, similar to how a third party or observer might view a situation.

Based on the intuition that LLMs perform optimally when their operational parameters align with confidence levels in specific contexts, we introduce Reasoning through Perspective Transition (RPT) method to dynamically select suitable perspectives to solve specific problems. To enable dynamic perspective transitions in reasoning, we adopt in-context learning (Brown et al., 2020), providing templates for multi-perspective answers. The model then evaluates confidence levels (Li et al., 2024; de Vries and Thierens, 2024; Bank et al., 2019) and selects the perspective with the highest confidence. This adaptability allows it to handle diverse subjective tasks more effectively than static methods.

We conduct experiments on four LLMs (including two closed-source models GPT-3.5 (OpenAI, 2022)/GPT-4 (OpenAI, 2023), and two open-source models Llama-3 (Dubey et al., 2024)/Qwen-2 (Yang et al., 2024) across 12 subjective tasks. Extensive experimental results demonstrates that, compared to previous methods based on a single perspective or some simple ensemble-based methods, our approach can improve the performance consistently.

2 Related Work

Subjective Tasks in NLP. Compared with objective tasks such as commonsense reasoning (Talmor et al., 2019) and arithmetic question-answering (Cobbe et al., 2021), research on LLMs in subjective tasks (e.g., metaphor recognition and dark humor detection) (Rottger et al., 2022; Kanclerz et al., 2023; Sun et al., 2023) is still underexplored. Different from objective tasks that can often be clearly defined and solved, subjective tasks involve the capability to perceive context, language nuances, and emotions, which cannot be easily quantified or objectively measured, thereby posing challenges for current LLMs (Jentzsch and Kersting, 2023; Wachowiak and Gromann, 2023; Mao et al., 2024). For example, as shown in results of BigBench(bench authors, 2023), the zero-shot accuracy of PaLM-535B
(Chowdhery et al., 2023) model on metaphor recognition, dark humor detection, and sarcasm detection tasks does not exceed 50%.

In-Context Learning of LLMs. As model parameters increase, LLMs gain stronger in-context learning (Brown et al., 2020), enhancing zero-shot and few-shot reasoning without fine-tuning. Chain-of-thought (CoT) prompting (Wei et al., 2022; Kojima et al., 2022) is widely used to elicit reasoning by adding explicit reasoning steps, emphasizing “how to think deeper” (OpenAI, 2024; DeepSeek-AI et al., 2025) while neglecting “who should think.” However, recent research (Sprague et al., 2024) suggests that such reasoning pathways are primarily effective for mathematical and symbolic reasoning. Our work also relies on in-context learning; however, we propose a method based on dynamic perspective transition to elicit knowledge from the different perspective of LLMs, which does not rely on a single reasoning pathway and achieve better results on a wider range of subjective tasks. Our work also relates to LLM-based multi-agents (Xi et al., 2023; Wang et al., 2024a), aiming to enhance LLMs’ ability to understand context, analyze problems, and generate solutions beyond a single fixed perspective.

Perspective Transition of LLMs. There are various ways to use LLMs currently that are based on different perspectives: 1) Direct prompting methods (Brown et al., 2020; Wei et al., 2022; Kojima et al., 2022) let the model to provide answers based on the factual knowledge or reasoning ability by LLMs themselves directly, without setting specific roles. 2) By assigning roles (Xu et al., 2023; Wang et al., 2024e; Wilf et al., 2024) such as experts and engaging in role-playing dialogue, the internal knowledge of LLMs on specific roles can be elicited. 3) By constructing scenarios through multi-agent cooperation (Wang et al., 2024f,b), debates (Du et al., 2024), or dialogues (Wang et al., 2024c), and then providing answers from a third-person perspective by incorporating contextualized information by the constructed agents. The previous methods only consider a fixed perspective and validate the effectiveness in certain problems. In contrast, RPT is orthogonal to improvements such as deep long-chain CoT (DeepSeek-AI et al., 2025), enabling LLMs to adaptively select the most suitable perspective to solve various subjective tasks, which has not been studied in previous research.

3 Method

The overall pipeline of the proposed RPT is structured into three steps, as shown in Figure 3. Firstly, we input the task description and a specific question, prompting the model to select the most appropriate perspective for answering the question. Secondly, the model evaluates and ranks these perspectives based on their confidence levels in addressing the question. Thirdly, the model adopts the perspective with the highest confidence to formulate and deliver the definitive answer.

Formally, given a subjective task with a description

Illustration of the Unified Prompting and Response

/* Input */

/* Perspective-based Instructions */
Here are some examples of perspectives you can choose:
Direct Perspective (answer the question directly).
……
Role Perspective (assume you are some roles (e.g., expert) and answer the question).
……
Third-person Perspective (simulate some agents and discussions, then summarize and give the answer).(D)
……
/* Input Question */
Metaphor Recognition: Identify whether the sentence “Adam did not understand the root of the crisis” is a paraphrase of the metaphoric sentence “Adam did not understand the solution to the crisis” (Q)

Firstly, analyzing the question from diverse perspectives, and selecting some available perspectives based on the question. (\( T_1 \))
Secondly, ranking all the perspectives according to the confidence level in solving the question. [Note: The confidence indicates how likely you think your answer is true]. (\( T_2 \))
Finally, choosing the method with the highest confidence to solve the question, reason and give the final answer. Select from True or False. (\( T_3 \))

/* Response of \( \mathcal{M} \) */
Perspective and Confidence: (\( \mathcal{P}, C \))
Third-person Perspective, 85%
Role Perspective, 70%
Direct Perspective, 60%

Selected Perspective Reasoning: (\( \mathcal{R} \))
Tom: Adam did not understand the root of the crisis.
Jerry: So, does that mean Adam did not understand the solution to the crisis?
Tom: Not exactly. Understanding the root of the crisis doesn’t necessarily mean understanding the solution. Understanding the root is figuring out what causes the problem, while a solution refers to ways of solving the problem. They can be related but are not synonymous.
Answer: False

Table 1: The example of a metaphor recognition task, a specific question, the unified prompt, and the corresponding response.

\( D \) and a specific question \( Q \), our goal is to let the LLM \( \mathcal{M} \) solve the question \( Q \).

Step 1: Exploring Diverse Available Perspectives.
We first let LLM \( \mathcal{M} \) explore diverse perspectives \( \mathcal{P} \) according to the description \( D \) and question \( Q \), promoting more comprehensive reasoning. Specifically, we define:

\[
\mathcal{P} = \{p_1, p_2, ..., p_n\} = \mathcal{M}(D \oplus Q \oplus T_1),
\]

where \( n \) (the number of perspectives) is usually 3, \( \oplus \) denotes concatenation operation. \( T_1 \) is a prompt serving as a trigger sentence, for example, we can set \( T_1 \) as “Firstly, analyzing the question from diverse perspec-
Figure 3: An overview of RPT pipeline. For each input question, RPT explores the available perspectives and then ranks them based on confidence. Accordingly, the input question is reasoned using the selected perspective.

tives, and selecting some available perspectives based on the question”.

Step 2: Ranking Perspectives by Confidence Level. Then, based on the generated perspectives, we allow the LLM \( \mathcal{M} \) to rank them according to their confidence levels \( \mathcal{C} \) in solving the question.

\[
\mathcal{P}, \mathcal{C} = \mathcal{M}(\mathcal{D} \oplus \mathcal{Q} \oplus \mathcal{T}_2),
\]

where \( \mathcal{T}_2 \) is a prompt for ranking the confidence level of all the available perspectives. For example, we can set \( \mathcal{T}_2 \) as “Secondly, Ranking all the methods according to the confidence level in solving the question. [Note: The confidence indicates how likely you think your answer is true.]”.

Step 3: Selected Perspective Reasoning. Finally, we take the original task description \( \mathcal{D} \), question \( \mathcal{Q} \), and the ranked confidence level perspective \( \mathcal{P} \) as the input, letting LLM \( \mathcal{M} \) give the final response \( \mathcal{R} \):

\[
\mathcal{R} = \mathcal{M}(\mathcal{D} \oplus \mathcal{Q} \oplus \mathcal{P} \oplus \mathcal{T}_3),
\]

where \( \mathcal{T}_3 \) is the last prompt leading to the final answer which can be set as “Finally, Choosing the perspective with the highest confidence to solve the question, and give the final answer”.

Combine All Steps through Unified Prompting. In practice, we find that the three aforementioned steps can be combined and accomplished through a single prompt \( \mathcal{T} \). In this way, our method only requires inference once through the LLM to obtain the answer to the question:

\[
\begin{align*}
\mathcal{T} &= \mathcal{T}_1 \oplus \mathcal{T}_2 \oplus \mathcal{T}_3, \\
\mathcal{P}, \mathcal{C}, \mathcal{R} &= \mathcal{M}(\mathcal{D} \oplus \mathcal{Q} \oplus \mathcal{T}),
\end{align*}
\]

where an example of the unified prompt and response is shown in Table 1.

4 Experiments

4.1 Settings

Datasets. We evaluate the effectiveness of our method on twelve subjective reasoning datasets, which can be categorized into five types, as shown in Table 2. Notably, for SemEval and cultural-related datasets which contain training sets, we evaluate in both zero-shot and few-shot settings. For the other tasks, we utilize corresponding test sets from BigBench1 (Srivastava et al., 2022) and only evaluate in zero-shot settings.

Baselines. We compare our method with 11 baselines including different single perspective methods and ensemble-based methods as follows.

Single Direct Perspective. Directly Prompt (Brown et al., 2020) directly use the question as input in zero-shot or few-shot manners. ICL (Brown et al., 2020) (in-context learning) uses examples and labels as few input demonstrations. Few-shot-CoT (Wei et al., 2022) uses manually created external reasoning pathways as demonstrations. Zero-Shot-CoT (Kojima et al., 2022) does not rely on demonstrations and elicits the reasoning ability by using “Let’s think step by step.” as external input. Self-Ask (Press et al., 2023) actively proposes and solves subquestions before generating the final answer.

Single Role Perspective. ExpertPrompt (Xu et al., 2023) introduces the expert identities and customizes information descriptions for LLMs before generating responses. Role-Play Prompting (Kong et al., 2024) also lets models simulate complex human-like interactions and behaviors for zero-shot reasoning.

Single Third-Person Perspective. SPP (Wang et al., 2024f) (solo performance prompting) proposes solo performance prompting by involving multi-turn collaboration with multi-persona. RiC (Wang et al., 2024c)

<table>
  <tr>
    <th>Dataset (names in short)</th>
    <th>Subjective Tasks</th>
    <th>#Train/Dev/Test</th>
  </tr>
  <tr>
    <td>(Linguistic Rhetoric)</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Metaphor (Mobler et al., 2016)</td>
    <td>Metaphor Understanding</td>
    <td>J-0880</td>
  </tr>
  <tr>
    <td>SNARKS (Khodak et al., 2018)</td>
    <td>Sarcasm Detection</td>
    <td>J-1581</td>
  </tr>
  <tr>
    <td>Humor (Huang et al., 2022)</td>
    <td>Dish Humor Detection</td>
    <td>J-0980</td>
  </tr>
  <tr>
    <td>(Disambiguation QA)</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Pronoun (Rudinger et al., 2018)</td>
    <td>Pronoun Resolution</td>
    <td>J-2258</td>
  </tr>
  <tr>
    <td>Anachronism (Geva et al., 2021)</td>
    <td>Identifying Anachronisms</td>
    <td>J-2210</td>
  </tr>
  <tr>
    <td>(Source Determination)</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>SEQ (Hendrycks et al., 2021)</td>
    <td>Simple Ethical Questions</td>
    <td>J-4115</td>
  </tr>
  <tr>
    <td>SemEval (Mohammad et al., 2016)</td>
    <td>Opinion Analysis</td>
    <td>2,194/921/707</td>
  </tr>
  <tr>
    <td>(Cultural-Related)</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>SocNorm (Cheng et al., 2023)</td>
    <td>Sociocultural Norm NLI</td>
    <td>2,301/300/768</td>
  </tr>
  <tr>
    <td>eSocNorm (CH-Wang et al., 2023)</td>
    <td>Sociocultural Norm NLI</td>
    <td>2,301/300/768</td>
  </tr>
  <tr>
    <td>CALI (Huang and Yang, 2021)</td>
    <td>Culturally Aware NLI</td>
    <td>1,777 / - / 440</td>
  </tr>
  <tr>
    <td>(Traditional NLI)</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Entailment (Srivastava et al., 2022)</td>
    <td>Analytic Entailment</td>
    <td>J-0716</td>
  </tr>
  <tr>
    <td>IPA (Williams et al., 2018)</td>
    <td>NLI in the International Phonetic Alphabet</td>
    <td>J-0126</td>
  </tr>
</table>

Table 2: Statistics and resources of datasets.
Table 3: Zero-shot results. To demonstrate generalizability, we repeat each set of experiments separated by dashed lines on four LLMs independently. Random represents the result of random prediction with uniform probability, and Majority represents the result of predicting the label with the highest proportion. S1: single direct perspective, S2: single role perspective, S3: single third-person perspective, E: ensemble-based method. D: dynamic perspective. For each dataset, the best result is in bold and the second-best result is underlined.

(reason in conversation) first lets model generating dialogues between simulated roles, and then summarize conversations and give final answers according to the additional information from conversations.

Ensemble-based Methods. Ensemble (Messuti et al., 2024; Agrawal et al., 2024) involves combining multiple model generation to enhance prediction accuracy and robustness (refer to Appendix B.3). Reranking (Farinhas et al., 2024; Kim et al., 2024) reorder different generation options based on requirements and select the optimal result. CoT-SC (Wang et al., 2023) improves performance by generating diverse chain-of-thought reasoning paths and selecting the most self-consistent answer, thereby enhancing the robustness and reliability.

Models. We evaluate our method on both closed-source models including GPT-4 (OpenAI, 2023) and GPT-3.5 (OpenAI, 2022), and open-source Llama-3 (Dubey et al., 2024) and Qwen-2 (Yang et al., 2024) models. In particular, we use the released API versions of gpt-4-0613 and gpt-3.5-turbo-1106 by OpenAI, and open-source Llama-3-8b-instruct and qwen-2-7b-instruct models released in Huggingface hub. We set the decoding temperature to 0 to ensure deterministic outputs and maintain the reproducibility of the responses generated by LLMs.

4.2 Zero-shot Results

In Table 3, we shows the experimental results of the baselines and our RPT method in zero-shot settings. From the experimental results, we can observe that:

RPT method consistently outperforms the baselines in most settings. Due to its ability to rank perspectives and select different perspectives to suit various subjective scenarios, our method achieves an average improvement of 3.27 points on all subjective tasks using the open-source model Llama-3 compared to the best-performing baseline. Similarly, on the closed-source GPT-3.5 model, our method achieves an average improvement of 4.56 points. Since subjective tasks vary widely, RPT achieves optimal performance through dynamic selection. For instance, on the Metaphor dataset, which requires complex contextual subjective understanding, our method, using Llama-3, outperforms the RIC method, which focuses on dialogue understanding, by 5.44 points.

Compared to baseline methods, our RPT method exhibits greater robustness. Although baselines introduce different perspectives to adapt to subjective tasks, they are typically effective only in specific domains. For example, using Llama-3, the Zero-Shot-CoT baseline achieves good performance on the Linguistic Rhetoric task, reaching the highest 70.72 accuracy
<table>
  <tr>
    <th>Type</th>
    <th>Method</th>
    <th>SemEval</th>
    <th>SocNorm</th>
    <th>e-SocNorm</th>
    <th>CALI</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td colspan="7">(Llama-3 + S3)</td>
  </tr>
  <tr>
    <td>S1</td>
    <td>ICL (Brown et al., 2020)</td>
    <td>70.71</td>
    <td>47.82</td>
    <td>57.73</td>
    <td>47.27</td>
    <td>55.88</td>
  </tr>
  <tr>
    <td>S2</td>
    <td>Few-Shot-CoT (Brown et al., 2020)</td>
    <td>76.45</td>
    <td>48.37</td>
    <td>57.77</td>
    <td>48.41</td>
    <td>57.75</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>Self-Ask (Press et al., 2023)</td>
    <td>74.16</td>
    <td>51.16</td>
    <td>63.51</td>
    <td>51.41</td>
    <td>60.02</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>ExpertPrompt (Xu et al., 2023)</td>
    <td>75.08</td>
    <td>47.46</td>
    <td>61.85</td>
    <td>49.50</td>
    <td>58.10</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>RiC (Wang et al., 2024c)</td>
    <td>74.18</td>
    <td>51.03</td>
    <td>63.76</td>
    <td>51.03</td>
    <td>61.05</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>Ensemble (Agrawal et al., 2024)</td>
    <td>77.48</td>
    <td>52.54</td>
    <td>66.60</td>
    <td>50.23</td>
    <td>61.71</td>
  </tr>
  <tr>
    <td>E</td>
    <td>Resampling (Furthaus et al., 2024)</td>
    <td>76.23</td>
    <td>45.53</td>
    <td>67.31</td>
    <td>51.14</td>
    <td>60.05</td>
  </tr>
  <tr>
    <td>E</td>
    <td>CoT-SC (Wang et al., 2023)</td>
    <td>79.34</td>
    <td>40.92</td>
    <td>75.80</td>
    <td>51.59</td>
    <td>61.01</td>
  </tr>
  <tr>
    <td>RPT (Ours)</td>
    <td></td>
    <td>80.71</td>
    <td>42.42</td>
    <td>77.20</td>
    <td>51.70</td>
    <td>61.80</td>
  </tr>
  <tr>
    <td colspan="7">S1</td>
  </tr>
  <tr>
    <td>S1</td>
    <td>ICL (Brown et al., 2020)</td>
    <td>70.81</td>
    <td>35.97</td>
    <td>54.52</td>
    <td>52.27</td>
    <td>53.40</td>
  </tr>
  <tr>
    <td>S2</td>
    <td>Few-Shot-CoT (Brown et al., 2020)</td>
    <td>76.16</td>
    <td>47.09</td>
    <td>63.51</td>
    <td>51.41</td>
    <td>60.02</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>Self-Ask (Press et al., 2023)</td>
    <td>74.10</td>
    <td>47.89</td>
    <td>62.60</td>
    <td>52.07</td>
    <td>60.55</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>ExpertPrompt (Xu et al., 2023)</td>
    <td>72.65</td>
    <td>54.56</td>
    <td>62.70</td>
    <td>52.27</td>
    <td>60.55</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>RiC (Wang et al., 2024c)</td>
    <td>72.45</td>
    <td>54.56</td>
    <td>62.70</td>
    <td>52.27</td>
    <td>60.55</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>Ensemble (Agrawal et al., 2024)</td>
    <td>76.37</td>
    <td>55.02</td>
    <td>68.12</td>
    <td>55.91</td>
    <td>64.02</td>
  </tr>
  <tr>
    <td>E</td>
    <td>Resampling (Furthaus et al., 2024)</td>
    <td>77.06</td>
    <td>51.32</td>
    <td>67.25</td>
    <td>55.75</td>
    <td>64.05</td>
  </tr>
  <tr>
    <td>E</td>
    <td>CoT-SC (Wang et al., 2023)</td>
    <td>72.55</td>
    <td>39.08</td>
    <td>54.25</td>
    <td>58.64</td>
    <td>54.09</td>
  </tr>
  <tr>
    <td>RPT (Ours)</td>
    <td></td>
    <td>74.18</td>
    <td>48.73</td>
    <td>72.80</td>
    <td>54.09</td>
    <td>64.09</td>
  </tr>
  <tr>
    <td colspan="7">S1</td>
  </tr>
  <tr>
    <td>S1</td>
    <td>ICL (Brown et al., 2020)</td>
    <td>72.02</td>
    <td>52.95</td>
    <td>55.60</td>
    <td>54.77</td>
    <td>58.84</td>
  </tr>
  <tr>
    <td>S2</td>
    <td>Few-Shot-CoT (Brown et al., 2020)</td>
    <td>77.45</td>
    <td>54.85</td>
    <td>61.35</td>
    <td>54.25</td>
    <td>60.80</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>Self-Ask (Press et al., 2023)</td>
    <td>73.04</td>
    <td>53.94</td>
    <td>67.81</td>
    <td>57.27</td>
    <td>60.52</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>ExpertPrompt (Xu et al., 2023)</td>
    <td>72.72</td>
    <td>56.06</td>
    <td>65.29</td>
    <td>55.60</td>
    <td>60.61</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>RiC (Wang et al., 2024c)</td>
    <td>77.22</td>
    <td>51.92</td>
    <td>67.08</td>
    <td>53.61</td>
    <td>60.81</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>Ensemble (Agrawal et al., 2024)</td>
    <td>78.72</td>
    <td>57.70</td>
    <td>72.78</td>
    <td>60.00</td>
    <td>67.17</td>
  </tr>
  <tr>
    <td>E</td>
    <td>Resampling (Furthaus et al., 2024)</td>
    <td>74.18</td>
    <td>51.03</td>
    <td>67.31</td>
    <td>51.14</td>
    <td>60.05</td>
  </tr>
  <tr>
    <td>E</td>
    <td>CoT-SC (Wang et al., 2023)</td>
    <td>68.73</td>
    <td>50.90</td>
    <td>74.15</td>
    <td>54.32</td>
    <td>62.10</td>
  </tr>
  <tr>
    <td>RPT (Ours)</td>
    <td></td>
    <td>80.78</td>
    <td>62.70</td>
    <td>74.80</td>
    <td>60.00</td>
    <td>69.52</td>
  </tr>
  <tr>
    <td colspan="7">S1</td>
  </tr>
  <tr>
    <td>S1</td>
    <td>ICL (Brown et al., 2020)</td>
    <td>77.72</td>
    <td>54.75</td>
    <td>61.41</td>
    <td>62.29</td>
    <td>63.88</td>
  </tr>
  <tr>
    <td>S2</td>
    <td>Few-Shot-CoT (Brown et al., 2020)</td>
    <td>77.45</td>
    <td>54.85</td>
    <td>61.35</td>
    <td>54.25</td>
    <td>60.80</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>Self-Ask (Press et al., 2023)</td>
    <td>73.52</td>
    <td>56.74</td>
    <td>64.62</td>
    <td>65.45</td>
    <td>65.08</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>ExpertPrompt (Xu et al., 2023)</td>
    <td>77.56</td>
    <td>56.74</td>
    <td>64.62</td>
    <td>65.45</td>
    <td>65.08</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>RiC (Wang et al., 2024c)</td>
    <td>78.72</td>
    <td>57.74</td>
    <td>65.04</td>
    <td>54.32</td>
    <td>63.96</td>
  </tr>
  <tr>
    <td>S3</td>
    <td>Ensemble (Agrawal et al., 2024)</td>
    <td>80.95</td>
    <td>61.05</td>
    <td>74.45</td>
    <td>67.88</td>
    <td>71.05</td>
  </tr>
  <tr>
    <td>E</td>
    <td>Resampling (Furthaus et al., 2024)</td>
    <td>80.95</td>
    <td>61.05</td>
    <td>74.45</td>
    <td>67.88</td>
    <td>71.05</td>
  </tr>
  <tr>
    <td>E</td>
    <td>CoT-SC (Wang et al., 2023)</td>
    <td>66.61</td>
    <td>60.52</td>
    <td>72.45</td>
    <td>62.05</td>
    <td>65.41</td>
  </tr>
  <tr>
    <td>RPT (Ours)</td>
    <td></td>
    <td>80.11</td>
    <td>66.79</td>
    <td>79.89</td>
    <td>66.59</td>
    <td>73.95</td>
  </tr>
</table>

Table 4: Main results of baselines and our RPT method in few-shot settings. S1: single direct perspective, S2: single role perspective, S3: single third-person perspective, E: ensemble-based method. D: dynamic perspective. We select the same 3-shot demonstrations from the training sets to each method for fair comparison.

on the SNARKS dataset, but performs poorly on tasks requiring complex contexts and cultural backgrounds, such as stance detection and culturally related datasets. For example, it only achieves 40.07 F1 score on SocNorm, the lowest among all baselines. Conversely, the RiC baseline, which employs role-playing for dialogue simulation, performs well in culturally relevant scenarios, achieving the highest F1 score of 64.05 on the e-SocNorm dataset, but struggles in the Linguistic Rhetoric task. Overall, different baselines that simulate distinct perspectives excel in specific domains but exhibit poor generalizability. In contrast, our method demonstrates consistent improvements across various subjective tasks, making it more robust.

Introducing more diverse perspectives into LLM and switching dynamically among them improves subjective reasoning performance. The RPT method achieves ensemble through exploring diverse perspectives and ranking perspectives by confidence level. In various settings, baselines that utilize multiple perspectives (e.g., RiC) outperform those that employ a single perspective (e.g., CoT), with scores of 60.85 vs. 58.77 on Llama-3 and 65.10 vs. 62.12 on Qwen. RPT takes this further by proposing dynamic perspective shifts, which offer high generalization and scalability, resulting in optimal performance through dynamic ensemble of all the baselines mentioned above.

Subjective reasoning is highly challenging and per-

![Line chart showing the impact of different perspectives on the RPT method.](page_1012_120_382_246.png)

Figure 4: The impact of different perspectives on the RPT method. DP: direct perspective, RP: role perspective, TP: third-person perspective.

forms better on powerful LLMs capable of deep reasoning. RPT almost always achieves the best performance on powerful models capable of deep reasoning, such as GPT-3.5 and GPT-4, while being more unstable on weaker open-source models. This suggests that rather than merely enhancing the ability to think from its own perspective, considering different perspectives can further improve LLM performance, especially for well-trained LLMs capable of long-chain deep reasoning. The performance gap between close-source models and open-source models narrows when using the RPT method, given that LLMs still possess the knowledge required for subjective reasoning and not been effectively elicited during training. RPT selects the method best suited to the model’s strengths, effectively eliciting LLM capabilities in subjective tasks, and compensating for the lack of subjective task data during LLM training.

4.3 Few-shot Results

In Table 4, we present the 3-shot results (refer to Section 5.3 for other settings). Similar to the zero-shot results, RPT method achieves the best average performance across different models. For instance, on Llama-3, RPT surpasses CoT-SC, the best-performing baseline, by 2.06 points.

A possible explanation is that providing a few examples in the prompt generally benefits LLM performance by providing context. However, subjective tasks are not well-defined and directly solvable, leading to significant differences between examples. As a result, LLMs exhibit varying confidence across examples, limiting the performance gains from examples and sometimes introducing noise or bias. For instance, using 3-shot examples in the RiC baseline lead to an average performance drop of 6.50 points on GPT-4. In contrast, RPT choose among perspectives and evaluates confidence for each input and method, providing finer-grained supervision signals and resulting in an average performance gain of 1.67 points.
Figure 5: The relationship between the performance and prediction lengths of the 3-shot experiments on GPT-4.

Figure 6: The inference cost of RPT when using different numbers of perspectives. RPT does not significantly increase inference costs on most datasets.

Figure 7: The performance of baselines and our RPT method by using different numbers of demonstrations (\( d = 1, 2, 3, 4 \)) in few-shot settings.

5 Analyses and Discussion

5.1 Ablation Study
As shown in Figure 4, we further investigate the impact of every perspective on the RPT method. The full RPT method achieves the best performance across all datasets. From the results of the ablation study, we can observe the following:

Removing any single perspective results in an average performance drop of 1.32–2.53 points, indicating that direct perspective, role perspective, and third-person perspective each have unique and irreplaceable contributions to subjective reasoning tasks. Specifically, removing the third-person component has the greatest impact, followed by role perspective and direct perspective, suggesting that the flexibility of switching between perspectives benefits overall performance.

Removing any two perspectives results in an even greater average performance drop of 5.15-6.48 points, and removing all perspectives (i.e., performing simple reasoning) leads to the highest performance drop of 7.60 points. As the number of perspectives removed increases, the range of dynamic switching decreases, causing a corresponding decline in RPT performance. This highlights the crucial role of switching between different perspectives in the RPT method.

In summary, all perspectives involved in RPT and the ability to flexibly switch between them are essential for achieving optimal performance. Thus, every component of our method is effective. The detailed ablation experiment results are presented in Appendix A.

5.2 Analysis on Performance-Inference Cost
Unlike methods such as ensemble and oversampling, which multiply inference costs, RPT aims to improve performance while reducing inference costs.

To estimate the inference cost, we follow Wang et al. (2024c) by representing the inference cost by the length of the response during the reasoning process before producing the final answer. In Figure 5, using the 3-shot GPT-4 as an example as detailed in Appendix B.4, we plot the length-performance relationship for RPT and the baselines. It can be observed that compared to most baselines, RPT achieves the best performance with a smaller inference cost, demonstrating the efficiency of the dynamic perspective selection approach. As shown in Figure 6, we compare the inference cost of the RPT method with its variants that only allow the use of one or two perspectives. The results indicate that the RPT method incurs a cost similar to that of the variants allowing only one or two perspectives in most datasets, without increasing the inference length, and thus the additional cost is minimal. This is because RPT performs method selection and perspective switching, avoiding the substantial inference cost caused by ensembling multiple perspectives.

5.3 Analysis on the Number of Shots
As shown in Figure 7, we specify the number of shots and study the performance difference compared to the
Figure 8: Case of SemEval task. We use GPT-4 to analyze attitudes toward Donald Trump. Our RPT method effectively guides the model in selecting appropriate perspectives for stance detection.

original RPT on SemEval. We observe that performance is lower when fewer shots are selected, as the model is unfamiliar with the task and method. As the number of shots increases, performance improves. However, in some circumstances when the number of shots reaches three or more, performance declines.

On one hand, LLMs exhibit greater flexibility when autonomously evaluating confidence and planning the number of shots during reasoning, allowing them to adapt to unique subjective tasks. On the other hand, providing too many examples may lead to increasing the inference cost, raising the risk of over-fitting, and challenging the instruction-following ability of LLMs. Overall, under the majority of settings for each dataset, RPT achieves the best performance, demonstrating its generalization ability and versatility (See Appendix B.8 for full results and analysis).

5.4 Case Study

In Figure 8, we showcase an example from the SemEval stance detection dataset to highlight the effectiveness of the RPT method in subjective reasoning tasks. Unlike baselines such as CoT and Role-playing, which sometimes emphasize skepticism or negative sentiment without fully accounting for context, RPT evaluates multiple perspectives, including direct and third-person analyses. For example, CoT and RiC interpret the phrase “Get the truth from Trump!” as reflecting skepticism or disbelief, leading to an “AGAINST” prediction. In contrast, RPT dynamically selects the most confident perspective, reasoning that the exclamation mark and phrase suggest advocacy or favor toward Trump. This ability to transition between and rank perspectives makes RPT more adaptable and effective in subjective reasoning tasks compared to single-perspective baselines (See Appendix B.2 for more cases).

5.5 Analysis on Keyword Statistics

As shown in Figure 9, to further investigate the characteristics of different perspectives in the RPT pipeline during reasoning, we conduct a keyword frequency analysis for the three perspectives in the RPT pipeline. After removing stopwords and irrelevant prompt words, we can observe the following reasoning characteristics for each perspective: the direct perspective tends to perform straightforward reasoning; the role perspective leans towards adopting different expert roles and contexts; and the third perspective excels in discussions and dialogues. The uniqueness of each perspective underscores the necessity of RPT’s dynamic perspective selection.

6 Conclusion

In this paper, we introduce RPT, a novel method that achieves multi-perspective reasoning and integration by exploring diverse perspectives and ranking them based
on confidence. Comprehensive experiments conducted on GPT-4, GPT-3.5, Llama-3, and Qwen-2 demonstrate that RPT effectively integrates various perspectives, enhancing the subjective task-solving capabilities of LLMs without significantly increasing inference costs. This work highlights how LLMs can better handle the fluidity of subjective reasoning, even in the absence of nuanced understanding of perspectives or personal biases. Future research directions include incorporating more reasoning perspectives, refining adaptive perspective taxonomies, and expanding applications.

Limitations

First, in designing the RPT pipeline, we categorize perspectives into three types based on related works. Although RPT and many inference paradigms involved in the baselines are orthogonal and combinable, this taxonomy could still be further refined, for example, by adopting alternative categorization methods or employing a more fine-grained division. Second, RPT directly selects perspectives rather than methods. We consider perspectives as a meta-method, meaning that RPT can be combined with other methods to achieve better performance. Thirdly, RPT operates within a single round of dialogue, without accounting for multi-turn conversations or result feedback. In the future, exploring multi-turn dialogue or multi-agent perspective writing could be a promising direction.

Ethics Statement

This paper uses widely available datasets, including stance detection, sarcasm detection, and cultural comparison, along with LLM-generated responses, solely to validate the proposed method without reflecting any stance or bias from the authors.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 62306161, 62276152).

References

Aakriti Agrawal, Mucong Ding, Zora Che, Chenghao Deng, Anirudh Satheesh, John Langford, and Furong Huang. 2024. Ensemw2s: Can an ensemble of llms be leveraged to obtain a stronger llm? ArXiv preprint, abs/2410.04571.

Dor Bank, Daniel Greenfeld, and Gal Hyams. 2019. Improved training for self training by confidence assessments. In Intelligent Computing, pages 163–173. Cham. Springer International Publishing.

BIG bench authors. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. TMLR.

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.

Sky CH-Wang, Arkadiy Saakyan, Oliver Li, Zhou Yu, and Smaranda Muresan. 2023. Sociocultural norm similarities and differences via situational alignment and explainable textual entailment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3548–3564, Singapore. Association for Computational Linguistics.

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. ArXiv preprint, abs/2107.03374.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Germann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. ArXiv preprint, abs/2110.14168.

Sjoerd de Vries and Dirk Thierens. 2024. Learning with confidence: Training better classifiers from soft labels. ArXiv preprint, abs/2409.16071.

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihaou Zhu, Shiron Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhabin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guangting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan
Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuan Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaoqin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxian Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. R. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhenan Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zhihu Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.

Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. Improving factuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning.

Abhimanyu Dubey, Abhinav Jauhari, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. ArXiv preprint, abs/2407.21783.

António Farinhais, Haau-Sing Li, and André F. T. Martins. 2024. Reranking laws for language generation: A communication-theoretic perspective. ArXiv preprint, abs/2409.07131.

Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346–361.

Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. Aligning AI with shared human values. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. ArXiv preprint, abs/2203.15556.

Jing Huang and Diyi Yang. 2023. Culturally aware natural language inference. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7591–7609, Singapore. Association for Computational Linguistics.

Sophie Jentsch and Kristian Kersting. 2023. ChatGPT is fun, but it is not funny! humor is still challenging large language models. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 325–340, Toronto, Canada. Association for Computational Linguistics.

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. ArXiv preprint, abs/2310.06825.

Kamil Kancerz, Konrad Karanowski, Julita Bielaniewicz, Marcin Gruza, Piotr Mikowski, Jan Kocon, and Przemyslaw Kazienko. 2023. PALS: Personalized active learning for subjective tasks in NLP. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13326–13341, Singapore. Association for Computational Linguistics.

Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2018. A large self-annotated corpus for sarcasm. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).

Tongyoung Kim, Soojin Yoon, Seongku Kang, Jinyoung Yeo, and Dongha Lee. 2024. Sc-rec: Enhancing generative retrieval with self-consistent reranking for sequential recommendation. ArXiv preprint, abs/2408.08686.

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.

Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruqi Sun, Xin Zhou, Enzhi Wang, and Xiaohang Dong. 2024. Better zero-shot reasoning with role-play prompting. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4099–4113, Mexico City, Mexico. Association for Computational Linguistics.
Jia Li, Yuqi Zhu, Yongmin Li, Ge Li, and Zhi Jin. 2024. Showing llm-generated code selectively based on confidence of llms. ArXiv preprint, abs/2410.03234.

Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. ArXiv preprint, abs/2308.09583.

Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin, and Erik Cambria. 2024. GPTEval: A survey on assessments of ChatGPT and GPT-4. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 7844–7866, Torino, Italia. ELRA and ICCL.

J. A. Meaney, Steven Wilson, Luis Chiruzzo, Adam Lopez, and Walid Magdy. 2021. SemEval 2021 task 7: HaHackathon, detecting and rating humor and offense. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 105–119, Online. Association for Computational Linguistics.

Giovanni Messuti, ortensia Amoroso, Ferdinando Napolitano, Mariarosaria Falanga, Paolo Capuano, and Silvia Scarpetta. 2024. Uncertainty estimation via ensembles of deep learning models and dropout layers for seismic traces. ArXiv preprint, abs/2410.06120.

Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. SemEval-2016 task 6: Detecting stance in tweets. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 31–41, San Diego, California. Association for Computational Linguistics.

Michael Mohler, Mary Brunson, Bryan Rink, and Marc Tomlinson. 2016. Introducing the LCC metaphor datasets. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 4221–4227, Portorož, Slovenia. European Language Resources Association (ELRA).

OpenAI. 2022. ChatGPT.

OpenAI. 2023. GPT-4 technical report. ArXiv preprint, abs/2303.08774.

OpenAI. 2024. Learning to reason with llms.

David Premack and Guy Woodruff. 1978. Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, 1(4):515–526.

Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711, Singapore. Association for Computational Linguistics.

Paul Rottger, Bertie Vidgen, Dirk Hovy, and Janet Pierrehumbert. 2022. Two contrasting data annotation paradigms for subjective NLP tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 175–190, Seattle, United States. Association for Computational Linguistics.

Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. 2023. Code llama: Open foundation models for code. ArXiv preprint, abs/2308.12950.

Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 8–14, New Orleans, Louisiana. Association for Computational Linguistics.

Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. 2024. To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. ArXiv preprint, abs/2409.12183.

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. ArXiv preprint, abs/2206.04615.

Huaman Sun, Jiaxin Pei, Minje Choi, and David Jurgens. 2023. Aligning with whom? large language models have gender and racial biases in subjective nlp tasks. ArXiv preprint, abs/2311.09730.

Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4149–4158, Minneapolis, Minnesota. Association for Computational Linguistics.

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. ArXiv preprint, abs/2302.13971.

Lennart Wachowiak and Dagmar Gromann. 2023. Does GPT-3 grasp metaphors? identifying metaphor mappings with generative language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1018–1032, Toronto, Canada. Association for Computational Linguistics.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024a. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345.

Xiaolong Wang, Yile Wang, Sijie Cheng, Peng Li, and Yang Liu. 2024b. DEEM: Dynamic experienced expert modeling for stance detection. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 4530–4541, Torino, Italia. ELRA and ICCL.

Xiaolong Wang, Yile Wang, Yuanchi Zhang, Fuwen Luo, Peng Li, Maosong Sun, and Yang Liu. 2024c. Reasoning in conversation: Solving subjective tasks through dialogue simulation for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15880–15893, Bangkok, Thailand. Association for Computational Linguistics.

Xiaolong Wang, Yile Wang, Yuanchi Zhang, Fuwen Luo, Peng Li, Maosong Sun, and Yang Liu. 2024d. Reasoning in conversation: Solving subjective tasks through dialogue simulation for large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15880–15893, Bangkok, Thailand. Association for Computational Linguistics.

Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao. 2024e. InCharacter: Evaluating personality fidelity in role-playing agents through psychological interviews. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1840–1873, Bangkok, Thailand. Association for Computational Linguistics.

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.

Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. 2024f. Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 257–279, Mexico City, Mexico. Association for Computational Linguistics.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.

Henry M. Wellman, David Cross, and Julanne Watson. 2001. Meta-analysis of theory-of-mind development: The truth about false belief. Child Development, 72(3):655–684.

Alex Wilf, Sihyun Lee, Paul Pu Liang, and Louis-Philippe Morency. 2024. Think twice: Perspective-taking improves large language models’ theory-of-mind capabilities. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8292–8308, Bangkok, Thailand. Association for Computational Linguistics.

Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.

Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023. The rise and potential of large language model based agents: A survey. ArXiv preprint, abs/2309.07864.

Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. 2023. ExpertPrompting: Instructing large language models to be distinguished experts. ArXiv preprint, abs/2305.14688.

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. ArXiv preprint, abs/2407.10671.

Zhen Yang, Ming Ding, Qingsong Lv, Zhihuan Jiang, Zehai He, Yuyi Guo, Jinfeng Bai, and Jie Tang. 2023. Gpt can solve mathematical problems without a calculator. ArXiv preprint, abs/2309.03241.

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.

Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. 2023. Exchange-of-thought: Enhancing large language model capabilities through cross-model communication. In Proceedings of the 2023 Conference
on Empirical Methods in Natural Language Processing, pages 15135–15153, Singapore. Association for Computational Linguistics.

Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2023. Automatic chain of thought prompting in large language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.

A  Details of Ablation Study

As shown in Table 5, we present the complete and detailed results of the ablation experiments. By removing one, two, and all three perspectives, we demonstrate the effectiveness of RPT. Based on the number of perspectives removed, we divide the ablation experiments into three groups. It can be seen that all the perspectives involved in RPT are beneficial. Meanwhile, restricting the range of perspective selection also results in performance degradation.

<table>
  <tr>
    <th>Method</th>
    <th>SemEval</th>
    <th>SocNorm</th>
    <th>e-SocNorm</th>
    <th>CALI</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td>RPT (Ours)</td>
    <td>81.43</td>
    <td>59.81</td>
    <td>77.57</td>
    <td>61.13</td>
    <td>69.99</td>
  </tr>
  <tr>
    <td colspan="6">(removing 1 perspective)</td>
  </tr>
  <tr>
    <td>w/o DP</td>
    <td>0.83</td>
    <td>1.85</td>
    <td>1.44</td>
    <td>1.14</td>
    <td>1.32</td>
  </tr>
  <tr>
    <td>w/o RP</td>
    <td>1.24</td>
    <td>3.35</td>
    <td>1.71</td>
    <td>1.82</td>
    <td>2.03</td>
  </tr>
  <tr>
    <td>w/o TP</td>
    <td>1.44</td>
    <td>3.77</td>
    <td>1.88</td>
    <td>3.05</td>
    <td>2.53</td>
  </tr>
  <tr>
    <td colspan="6">(removing 2 perspectives)</td>
  </tr>
  <tr>
    <td>w/o DP&RP</td>
    <td>2.14</td>
    <td>8.41</td>
    <td>5.05</td>
    <td>5.00</td>
    <td>5.15</td>
  </tr>
  <tr>
    <td>w/o DP&TP</td>
    <td>2.14</td>
    <td>8.45</td>
    <td>1.78</td>
    <td>5.14</td>
    <td>5.60</td>
  </tr>
  <tr>
    <td>w/o RP&TP</td>
    <td>2.78</td>
    <td>8.82</td>
    <td>9.27</td>
    <td>5.05</td>
    <td>6.48</td>
  </tr>
  <tr>
    <td colspan="6">(removing 3 perspectives)</td>
  </tr>
  <tr>
    <td>w/o DP&RP&TP</td>
    <td>3.93</td>
    <td>9.36</td>
    <td>9.76</td>
    <td>7.37</td>
    <td>7.60</td>
  </tr>
</table>

Table 5: Detailed results of ablation study of our proposed RPT method with GPT-3.5 in zero-shot settings. DP: Direct perspective. RP: Role Perspective. TP: Third-Person Perspectives.

B  More Analysis

RPT ranks different perspectives based on confidence levels without relying on external information. In this section, using the zero-shot GPT-3.5 experiment as an example, we force the LLM to select the perspective with the second highest confidence, the lowest confidence, and a randomly chosen perspective during RPT inference.

As shown in Table 6, the lower the confidence of the selected perspective, the poorer the performance of LLM. When randomly selecting perspectives, the performance of the LLM is also worse than that of the perspective with the highest confidence. This shows that ranked perspectives based on confidence levels are effective, explaining the underlying mechanism by which RPT improves performance.

B.1  Analysis on the Correlation between Confidence and Accuracy

In RPT, we use LLM itself to judge the confidence of perspectives for a given input, allowing the model to rank and switch among perspectives accordingly. As shown in Figure 10, using the GPT-3.5 model as an example, we analyze the relationship between predicted confidence and the actual accuracy. We can observe that when confidence exceeds the threshold of approximately 70%, the accuracy of the chosen perspective is significantly higher. This indicates that LLMs are capable of ranking the confidence of perspective for a specific input based on confidence levels.

Using GPT-3.5 as an example, we report in Figure 11 the average confidence for each dataset. Figure 12 shows the human evaluation consistency when estimating the confidence. We find that when evaluating confidence, the estimation of the LLM are highly correlated with those of human experts, indicating that the LLM has the ability to evaluate confidence and select perspectives.

Moreover, RPT generally performs better within high-confidence perspectives, indicating that confidence-based perspective ranking is efficient when choosing among perspectives. In Figure 13, we present the proportion of different perspectives used on each dataset, showing that different datasets have different perspective biases. This suggests that, compared to a single perspective, RPT offers perspective flexibility, which helps RPT achieve optimal performance.

![Bar chart showing the correlation between confidence and accuracy for different datasets](page_1012_120_377_246.png)

Figure 10: Analysis on the correlation between confidence and accuracy.

![Bar chart showing the averaged confidence level by our RPT method in different datasets](page_1012_370_377_246.png)

Figure 11: The averaged confidence level by our RPT method in different datasets.
<table>
  <tr>
    <th rowspan="2">Type</th>
    <th rowspan="2">Method</th>
    <th colspan="3">Linguistic Rhetoric</th>
    <th colspan="2">Disambiguation QA</th>
    <th colspan="2">Stance Detection</th>
    <th colspan="2">Cultural-Related</th>
    <th colspan="2">Traditional NLI</th>
    <th rowspan="2">Avg.</th>
  </tr>
  <tr>
    <th>Metaphor (Acc.)</th>
    <th>SNARKS (Acc.)</th>
    <th>Humor (Acc.)</th>
    <th>Pronoun (Acc.)</th>
    <th>Anach. (Acc.)</th>
    <th>SEQ (Acc.)</th>
    <th>SemEval (F1)</th>
    <th>SocNorm (Acc.)</th>
    <th>e-SocNorm (F1)</th>
    <th>CALI (Acc.)</th>
    <th>Entail. (Acc.)</th>
    <th>IPA (Acc.)</th>
  </tr>
  <tr>
    <td>D</td>
    <td>RPT (Ours)</td>
    <td>91.76</td>
    <td>87.29</td>
    <td>70.00</td>
    <td>65.12</td>
    <td>73.48</td>
    <td>99.13</td>
    <td>81.43</td>
    <td>59.81</td>
    <td>77.57</td>
    <td>61.13</td>
    <td>88.57</td>
    <td>80.00</td>
    <td>77.94</td>
  </tr>
  <tr>
    <td>D</td>
    <td>RPT (second)</td>
    <td>85.82</td>
    <td>81.77</td>
    <td>56.50</td>
    <td>56.20</td>
    <td>72.61</td>
    <td>98.26</td>
    <td>80.61</td>
    <td>45.52</td>
    <td>61.53</td>
    <td>51.36</td>
    <td>77.14</td>
    <td>54.20</td>
    <td>68.98</td>
  </tr>
  <tr>
    <td>D</td>
    <td>RPT (lowest)</td>
    <td>79.12</td>
    <td>79.01</td>
    <td>37.98</td>
    <td>38.76</td>
    <td>69.13</td>
    <td>96.52</td>
    <td>78.89</td>
    <td>38.03</td>
    <td>51.54</td>
    <td>48.64</td>
    <td>57.14</td>
    <td>56.35</td>
    <td>60.93</td>
  </tr>
  <tr>
    <td>D</td>
    <td>RPT (random)</td>
    <td>85.88</td>
    <td>80.11</td>
    <td>53.49</td>
    <td>58.91</td>
    <td>70.43</td>
    <td>97.39</td>
    <td>80.56</td>
    <td>45.15</td>
    <td>67.99</td>
    <td>54.09</td>
    <td>67.14</td>
    <td>67.46</td>
    <td>69.05</td>
  </tr>
</table>

Table 6: Analysis on confidence-based perspective selection. \( RPT \) (\( random \)) represents the result of random prediction with uniform probability, \( RPT \) (\( second \)) represents selecting the perspective with the second highest confidence, and \( RPT \) (\( lowest \)) means choosing the most unconfident perspective.

![Bar chart showing human evaluation consistency on confidence.](page_186_370_563_246.png)

Figure 12: Human evaluation consistency on confidence.

![Pie charts showing the proportion of selected perspectives for different input questions in each dataset.](page_186_627_563_246.png)

Figure 13: The proportion of selected perspectives for different input questions in each dataset.

B.2 More Cases of RPT

In Figure 15 and Figure 14, we present several examples from a culturally related NLI task SocNorm and a stance detection dataset SemEval. Take the second case in SemEval task as an example, baseline methods captures strong negative emotions in the input through words like “joke”, “fool” and “betray” and makes judgments about the speaker’s attitude toward Trump based on these cues, overlooking the potential underlying implications of the text. However, RPT evaluates and selects the third-person perspective, providing the correct analysis by simulating some agents and discussions, illustrating the effectiveness of the RPT in subjective reasoning tasks.

B.3 Comparison with Ensemble-based Methods

Instead of oversampling responses and simply making a selection using model confidence, the core idea of RPT is that LLMs can leverage their internal priors and the given prompt to directly select the perspective with the highest posterior accuracy. Unlike conventional ensembling approaches, RPT assigns a single optimal perspective to each query without attempting inferencing on multiple perspectives per instance before making a decision. This means that:

• RPT does not ensemble multiple responses.

• RPT does not significantly increase the generation window like ensembling does.

• RPT does not rely on oversampling responses before selection.

Instead, RPT possesses the following features:

• RPT selects a single perspective after a brief internal deliberation based on the input prompt and then conducts inference solely from that perspective.

• The computational cost of RPT is comparable to using a single perspective.

• Despite its simplicity, RPT achieves performance comparable to or better than ensemble methods.

For example, given \( k \) candidate perspectives, an ensemble method incurs a response length/inference cost of \( k \times \) cost, with a performance of \( p \). In contrast, RPT has a response length/inference cost of approximately \( 1 \times \) cost, while achieving performance \( \geq p \). We believe that LLMs’ internal priors can be leveraged to preemptively plan and evaluate reasoning chains across different perspectives, allowing them to anticipate ensemble outcomes before actually performing ensembling, thus eliminating unnecessary computational overhead. This potential has not been explored before. Analogous to Mixture of Experts (MoE), RPT can be regarded as a form of “Mixture of Perspectives”, where exactly one perspective is activated per inference. Thus, we believe that RPT is a highly novel approach.

Therefore, instead of simply selecting perspective, we propose the dynamic process of shifting between different perspectives when handling various tasks, rather
Instruction:
Does the sentence "Getting married is the first step in settling down and starting a family." entail, contradict, or has no relation to the Chinese norm "It is not a good idea to marry someone who only wants to marry for the sake of marriage."?
Please answer between "Entailment", "Contradiction" or "No Relation" and explain your decision.

Model Predictions:
The U.S. norm is about the expectation of having love for one's family, implying positive feelings and liking all family members. The Chinese norm talks about it being inappropriate to express disliking you own siblings, implies a negative feeling but discourages expressing it. Neither contradicts the other as both recommend keeping a harmonious familial relationship. The US one doesn't exactly entail the Chinese one.
Both norms deal with the notion of familial relationships and expectations within their cultural contexts.
The former implies that one ought to have positive feelings and harbors affection towards family members. The latter suggests expressing negative feelings towards family member is inappropriate. This doesn't mean the individual cannot harbor negative feelings ... they reflect different aspects of familial expressing ...
The keyword in the US norm is 'love' and in the Chinese norm is 'dislike'. The former encourages positive feelings towards the family. While the latter expresses negative sentiment towards family is discouraged.
Both advise maintaining a positive relationship with the family...However, it's unclear if loving your family (the US) means you must express approval of your family in an appropriate way as suggested by the Chinese...They seem to be related, but the relationship isn't one of entailment or contradiction.

Selected Perspective Analysis: From a Direct Perspective, implied in the US norm is the belief that one should have positive feelings towards their family, which does not contradict the Chinese norm against expressing dislike for a family member. However, the US norm does not suggest that expressing dislike would be against the expectation of love. Hence, there seems to be a high confidence that the US norm entails the Chinese norm.

Model Predictions:
The US norm focuses on marriage as a significant life milestone for settling down and starting a family, while the Chinese norm emphasizes the importance of marrying for genuine reasons, not just for marriage. These two norms discuss different aspects of marriage, one highlighting its role in personal life stages and the other focusing on the motivation behind marriage, indicating a "no relation"
The US norm focuses on marriage as a significant life milestone for settling down and starting a family, while the Chinese norm emphasizes the importance of marrying for genuine reasons, not just for marriage. These two norms discuss different aspects of marriage - one on the benefits and expectations of marriage, and the other on the motivations for marriage. Thus, they are not related.
The US norm suggests marriage as a significant step towards settling down and family formation, while the Chinese norm warns against marrying solely for marriage. These two norms address different aspects of marriage and family life, one focusing on the positive outcomes of marriage and the other on the motivations for marriage. They are of no relation.

Selected Perspective Analysis: Direct Perspective: The US norm focuses on marriage as a significant life milestone for settling down and starting a family. The Chinese norm emphasizes the importance of marrying for genuine reasons, suggesting a different perspective on the purpose of marriage. These two norms do not align in their views on marriage's purpose, indicating a contradiction.

Figure 14: Cases of SocNorm task. We provide results of two widely used models, GPT-4 and Qwen-2. In each of the cases, our proposed method successfully spots the actual meaning behind these norms and leads to the correct answer.
Instruction:
Question: What is the attitude toward “Donald Trump” in the tweet “Get the truth from Trump! @realDonaldTrump #SemST”?
Options: (A) FAVOR (B) AGAINST (C) NEUTRAL

Model Predictions:
The sentence is a call to action for getting the truth from Trump, but this does not inherently indicate favor or against. However, demanding truth usually indicates a level of skepticism or disbelief, which usually tends towards negative sentiment.
Answer and Confidence (0-100): B, 65%

The phrase ‘Get the truth from Trump!’ can be interpreted in two ways. It might indicate that the person trusts Donald Trump to provide the truth, signaling a FAVOR sentiment. On the other hand, it could also be a call-to-action urging Trump to be truthful, which might suggest skepticism or critique. ...
Answer and Confidence (0-100): A, 80%

The scenario here could be someone asking for more transparency or truthful information from Donald Trump it is not expressing a view against or in favor of him directly, however it does show a certain level of skepticism or needs for validation about the truthfulness of Mr. Trump. Therefore, it is not simply favor or against him, but rather calling for transparency... it’s potentially speaking in opposition or against.
Answer and Confidence (0-100): B, 55%

Perspectives with Confidence Level:
Selected Perspective Analysis:
As an expert in political science, it seems very apparent that the text is promoting Donald Trump as a source of truth. The exclamation mark signifies excitement or advocacy, further demonstrating a positive association or attitude towards Trump.

Instruction:
Question: What is the attitude toward “Donald Trump” in the tweet “@Greta trump a joke? Ur guest is a fool. Better a joke than traitor RINOS. It’s trump or no one for me. Reps betrayed us. #SemST”?
Options: (A) FAVOR (B) AGAINST (C) NEUTRAL

Model Predictions:
The sentence contains several derogatory terms and expressions directed against Greta, implying a negative attitude towards her. However, the focus is on the target Donald Trump. The user states a preference for “Trump or no one”, indicating strong support for him. The phrase “Better a joke than traitor RINOS” also indicates a negative attitude towards Trump, but it is less direct than the previous examples, so the attitude towards Donald Trump is positive or favorable.
Answer and Confidence (0-100): (A), 95%

The sentence contains several derogatory terms directed towards Greta, implying the speaker’s strong disapproval. The phrase “Better a joke than traitor RINOS” suggests a preference for someone who is seen as entertaining over those considered disloyal. The statement “It’s trump or no one for me” clearly indicates a strong support for Donald Trump ... Answer and Confidence (0-100): (B), 85%

The sentence contains several phrases that express negative opinions towards the target ‘Donald Trump’. The phrases ‘a joke’, ‘a fool’ ... all indicate a strong negative attitude. The phrase ‘Better a joke than traitor RINOS’ further emphasizes the negative view ... The hashtag #SemST could be interpreted as a form of support, but the overall tone of the sentence is clearly negative. Answer and Confidence (0-100): (B), 75%

Perspectives with Confidence Level:
Selected Perspective Analysis:
This tweet sentence directly expresses an opinion towards Donald Trump. The words “better a joke than traitor” and “It’s trump or no one for me” indicate a strong positive stance towards Trump, suggesting the speaker views Trump even if he’s seen as a joke compared to others who might be considered traitors or RINOS.

Instruction:
Question: What is the attitude toward “Donald Trump” in the tweet “Really respect @realDonaldTrump for giving up #TheApprentice to make way for his campaign #AmericaNeedsThis #GodBlessedAmerica #SemST”?
Options: (A) FAVOR (B) AGAINST (C) NEUTRAL

Model Predictions:
The sentence uses words like “Really respect” and “#GodBlessedAmerica” which convey a positive tone. The sentence also mentions @realDonaldTrump, ... “#AmericaNeedsThis” and “#SemST” also suggest a positive and supportive attitude towards Trump’s campaign. Answer and Confidence (0-100): B, 20%

The sentence expresses admiration and respect towards Donald Trump, using phrases like “Really respect” and “#GodBlessedAmerica”. The fact that he gave up “The Apprentice” to focus on his campaign also implies that the speaker views this as a noble decision. Answer and Confidence (0-100): B, 20%

The keywords extracted from the question are “respect”, “give up”, “make way”, “campaign”, “America”, and “#GodBlessedAmerica”. Considering scenarios, we can imagine a conversation where someone is discussing Donald Trump’s decision to give up The Apprentice to focus on his presidential campaign. The tone is positive and supportive. Reasoning via scenarios, it is likely that the attitude towards Donald Trump is FAVOR. Answer and Confidence: A, 90%

Perspectives with Confidence Level:
Selected Perspective Analysis:
From the Third-person Perspective, we can analyze the sentence by simulating some agents and discussions. For example:
Sarah: I really respect @realDonaldTrump for giving up #TheApprentice to make way for his campaigns
Amy: Because it shows he’s willing to put his country first.
This conversation suggests that the sentence expresses a positive attitude towards Donald Trump, implying that he’s an authentic leader

Figure 15: Cases of SemEval task. We provide detailed responses of three models, GPT-4, Qwen-2, and Llama-3, regarding the attitude towards Donald Trump. Our method prompts models to successfully selects suitable perspectives to solve stance detection problems.
Figure 16: The performance of baselines and our RPT method by using different numbers of demonstrations (\( d = 1, 2, 3, 4 \)) in few-shot settings.

than merely selecting a single perspective. It highlights the following two aspects:

• Dynamism: The model autonomously transitions from one perspective to another, rather than making a static selection. This process is an inherent decision made by the model itself.

• Variability: In practice, this transition occurs through mechanisms such as Chain-of-Thought (CoT), role-playing, multi-agent discussions, and other discourse-based approaches, facilitating perspective shifts. Compared to perspective selection, we emphasize the dynamic and evolving nature of this process.

B.4 Details on Shot Numbers

In Section 5.3, we provide experimental results across multiple shot settings, demonstrating that the 3-shot setting yields the best performance. Moreover, the datasets used in this study primarily consist of binary or ternary classification tasks. Conventionally, 3-shot is a common setting for practical inference. Additionally, both the baseline and our proposed method achieve the best overall performance under this setting, making it a reasonable choice for our primary study on inference cost. Therefore, we choose 3-shot setting to conduct analysis on performance-inference cost experiments.

B.5 Estimation of Inference Cost

The purpose of Section 5.2 is to examine the trade-off between inference cost and performance when using different methods with LLMs. Counting the number of output tokens is the most common approach for estimating inference time, as LLMs process input tokens in parallel with relatively low overhead, whereas output generation is auto-regressive and thus dominates inference latency and cost. For common LLMs, the cost of input tokens is lower than output tokens. Including both input and output tokens in the performance-cost scatter plot would introduce bias and deviate from standard practices in the LLM community.

Therefore, we chose to plot a two-dimensional scatter plot of output token count versus performance. Introducing a third variable would unnecessarily complicate the experiment and make it more difficult to draw clear conclusions.
B.6 More Examples on Subjective Tasks
Perspective in language is essentially a matter of deixis, how entities are positioned relative to the speaker (user), the addressee (LLM), and a potential third party.

• First-person refers directly to the speaker’s own viewpoint, emphasizing subjectivity and experiential immediacy. It forms the core of self-reference and establishes speaker agency. This is the most common LLM reasoning mode, as it activates the model’s cognitive framing, directly connecting it to the question and reasoning process.

• Second-person focuses on the listener in an interaction, creating direct engagement. It reduces the narrative distance through dialogic framing, commonly used in imperative or persuasive contexts. LLMs can leverage second-person narration to enhance reasoning in ambiguous scenarios by simulating conversational exchange.

• Third-person provides an external, often omniscient view of the situation or entities involved. In LLM reasoning, this resembles a form of distancing effect, offering broader perspective through heterodiegetic narration.

As shown in the experiment results, our experiments reveal that for subjective tasks, Zero-shot-CoT does not exhibit a significant advantage over Direct Prompting on GPT-3.5 and GPT-4-0613, and even performs slightly worse on Qwen2-7B-Instruct (Direct Prompt: 62.70, Zero-shot-CoT: 61.42), as detailed in Table 3. This suggests that unlike objective tasks, subjective tasks may not allow for a universally applicable reasoning pathway. This observation aligns with findings by Wang et al. (2024d), which systematically studies the performance of LLMs on subjective tasks.

Example 1:
• Question: Identify whether the sentence “her smile was a blinding sun” is a paraphrase of the metaphoric sentence “her smile was wonderful and fulfilling to see”.

• Chain-of-Thought Reasoning: While both sentences express admiration for her smile, the second sentence lacks the vivid imagery and specific emotional intensity that the first sentence provides. Thus, the second sentence is not an accurate paraphrase of the first. The answer is False.

Example 2:
• Question: Identify whether the following examples are intended to be a joke (with dark humor) or not “I have a fish that can breakdance! Only for 20 seconds though, and only once”.

• Chain-of-Thought Reasoning: The phrase is light-hearted and whimsical about a fish that can breakdance, suggesting a humorous intent rather than a dark theme. Therefore, it does not seem to fit the category of dark humor. The answer is Not.

Example 3:
• Question: The sentence “Joseph is very kind” is a paraphrase of the metaphoric sentence “Joseph has the heart of a lion”.

• RiC Reasoning: The second sentence suggests that Joseph possesses qualities associated with a lion, while the first sentence only mentions kindness. The answer is False.

B.7 Discussion on Overconfidence and Underconfidence
Regarding the inherent overconfidence and underconfidence phenomena in LLMs, previous research (Yin et al., 2023) and our experiments (as shown in Figure 4) indicate that the higher the confidence of an LLM’s response, the more likely it is to be correct. We have further investigated the impact of different confidence levels on data selection (as shown in Table 6) by explicitly providing confidence scores for different perspectives and selecting perspectives based on these confidence levels.

Our experiments also show that automatically selecting the perspective with the highest confidence achieves an average performance of 77.94, significantly outperforming other baseline methods. In contrast, selecting the second-highest confidence perspective results in an average performance of 68.98, which is similar to random selection (69.05). The lowest-confidence selection, however, performs notably worse, with an average performance of only 60.93, indicating that choosing perspectives with the lowest confidence leads to poor outcomes.

In Figure 12, by comparing human evaluation scores with LLM-generated confidence scores after normalization, we find that LLMs’ confidence predictions are generally accurate. As shown in Figure 10, a per-instance comparison demonstrates a high correlation between LLM-predicted confidence and actual accuracy. In future research, providing additional supervision signals to models based on dataset characteristics and input prompt designs may further mitigate overconfidence and underconfidence issues.

B.8 Full Results of Analysis on the Number of Shots
The full results of analysis on the number of shots is shown in Figure 16. The results are consistent across the four datasets. It can also be observed that compared to zero-shot settings, few-shot settings are not always helpful, which is consistent with the findings by DeepSeek-AI et al. (2025). The following factors may contribute to the observed trends:

• As shown in Figure 7, increasing the number of shots does not always enhance LLM performance.
<table>
  <tr>
    <th>Hyperparameters</th>
    <th>Accuracy (%)</th>
    <th>F1-Score (%)</th>
  </tr>
  <tr>
    <td>epoch: 3, lr: 2e-5</td>
    <td>36</td>
    <td>23</td>
  </tr>
  <tr>
    <td>epoch: 4, lr: 2e-5</td>
    <td>33</td>
    <td>32</td>
  </tr>
  <tr>
    <td>epoch: 4, lr: 1e-5</td>
    <td>30</td>
    <td>28</td>
  </tr>
  <tr>
    <td>epoch: 4, lr: 2e-4</td>
    <td>47</td>
    <td>35</td>
  </tr>
  <tr>
    <td>epoch: 4, lr: 5e-4</td>
    <td>48</td>
    <td>31</td>
  </tr>
  <tr>
    <td>epoch: 8, lr: 2e-5</td>
    <td>46</td>
    <td>35</td>
  </tr>
</table>

Table 7: Performance of small model classifier with different training hyperparameters.

While providing a few examples can help the model handle unfamiliar tasks and follow instructions, excessive shots can lead to diminishing returns or performance degradation. We chose the 3-shot setting because most datasets consist of binary or ternary classification tasks.

• The length of the input prompt can impact LLM inference. Excessive in-context examples may distract the model, reducing its ability to follow instructions precisely, which can negatively impact performance.

• The added examples may introduce noise, affecting reasoning performance. For instance, gpt-3.5-turbo-1106 tends to overfit the provided cases, often favoring the answer from the last example presented in the prompt.

• The inherent openness and uncertainty of subjective tasks may contribute to this phenomenon. Subjective task responses are often ambiguous, and in few-shot settings, methods like Ensemble, Reranking, and CoT-SC may be influenced by the provided examples, potentially introducing biases and limiting generalization. In contrast, zero-shot performance relies on pre-trained knowledge, making it more stable. Ensemble methods are susceptible to errors from individual models, reranking depends on the quality of initial generations, and CoT-SC may reduce reasoning diversity, diminishing its advantage in maintaining consistency.

B.9 Using Small Models as Classifiers

We experiment with using small discriminative models, such as bert-base-uncased, as classifiers. However, due to the complexity of the datasets involved in RPT and the lack of distinctive content features, the classification performance was generally poor. Specifically, we use a large model to run each of the different perspective methods on the training set. For samples where the answer was correct, we labeled them according to a priority order: 1 (Direct), 2 (Role-Play), 3 (Third-Person), and then split the data into training and test sets with an 8:2 ratio. The results are shown in Table 7, with the highest accuracy reaching only 48%.

These results suggest that using small models is not a feasible approach. Moreover, introducing additional parameters for perspective ranking increases inference overhead. In contrast, RPT leverages the LLM’s own capabilities to assess the suitability of different perspectives on a given input without requiring additional training or parameters, achieving performance gains more efficiently.

Reviewer paper summaries: The authors present a prompt-based method to ensemble different strategies and select one based on confidence. They show their method leads to some improvements in subjective tasks.
Summary of strengths: The focus on subjective tasks is interesting and, in this reviewer's experience, relatively under-explored in the literature compared to "objective" tasks. The authors present thorough ablations.
Summary of weaknesses: The method presented amounts to ensembling three responses elicited with different prompts and selecting one using the model's confidence. The only difference is that the authors do this with one single prompt, thanks to the long generation window allowed by current LLMs. The idea of oversampling responses and making a selection using some form of model confidence is hardly novel, and I have reservations on whether this paper makes a strong enough contribution.
Comments, suggestions and typos: - Table 3 seems unclear: what do the different groups of results refer to? I assume they were computed with different base LLMs, but this information is not given in the table.

- There are a few typos and imprecisions in the text that the authors may want to double-check.




*Comments after Author Discussion: I have read the authors' response. I remain nonetheless of the opinion that this method is a form of confidence-based ensembling of the responses and not particularly groundbreaking. There is also the issue that for the first two models in the main table (Llama and Qwen according to the authors' response), RPT is the best performing method in only ~50% of cases. The relative lack of novelty and weakness of the results give me reservations on this paper. I maintain my score on this occasion.*
Overall assessment scores: 2.5
Metareview score: 2
Metareview: This paper introduces a method that enables LLM to automatically select a perspective for answering subjective questions.
Summary of reasons to publish: The strengths of this paper include:
1.) this paper is easy-to-follow;
2.) the experimental part of this paper is comprehensive;
3.) the proposed method is straightforward and easy-to-understandable.